Clinical Evaluation of the V5-LU01 AI Software in Thoracic CT for V5med Inc.

NCT ID: NCT06514248

Last Updated: 2024-07-23

Basic Information

• Recruitment Status: COMPLETED
• Clinical Phase: NA
• Total Enrollment: 16 participants
• Study Classification: INTERVENTIONAL
• Study Start Date: 2023-12-18
• Study Completion Date: 2024-05-17

Brief Summary

STUDY DESIGN:

This is a two-arm retrospective, multi-reader, multi-case, (MRMC) randomized reader study.

OBJECTIVE:

Primary: The primary objective of this clinical study is to prove that radiologist's performance aided with V5med Lung AI is superior to the unaided for detecting qualified lung nodules.

Secondary: The secondary objective of this clinical study is to prove that the radiologist's reading time is not significantly increased when aided with V5med Lung AI.

Addition Objectives: To prove that the agreement (e.g., in kappa correlation coefficient) between experts and radiologist's Lung-RADS score aided with V5med Lung AI is superior to the unaided.

NUMBER OF SUBJECTS:

Retrospective CT studies from approximately 350 patients will be included in the study with approximately 170 true positive cases and 180 normal cases.

PRIMARY ENDPOINTS:

Scores given by the radiologists with and without V5med Lung AI will be recorded and compared to the true status of the study-cases. The frequency of the scores for each method (Aided, Unaided) will be tabulated and LROC curves constructed along with sensitivity, specificity, PPV, NPV and clinical actions. Additionally, machine nodule detection rate and false positives per patient on normal cases will be measured.

PATIENT POPULATION:

The study will target approximately two hundred (200) patients whose CT lung nodules were shown to be cancer and one hundred and eighty (150) patients who have no lung nodule greater than 4mm. The patient population will be consistent with the national lung cancer screening protocols.

Detailed Description

Overview of the Study:

A reader study with sixteen participating radiologists will be conducted. A location-specific receiver operating characteristic (LROC) curve will be used to evaluate radiologists' detection performance in detecting qualified lung nodules on chest CT scans with and without the use of the V5med Lung AI system. The reading time for each case will also be recorded.

The 16 study radiologists will be split into two groups, X and Y. The 350 chest CT scans will be split into two subsets, A and B. Group X radiologists will interpret subset A without AI and subset B with V5med Lung AI. Group Y radiologist will interpret subset B without AI and subset A with AI. After a minimum of one month, each radiologist group will interpret the same images again but in the reverse order with regard to study modes (without AI and with AI). Please see Figures 1 and 2 for detailed arrangement of the radiologist interpretation sessions.

During the baseline reading session the radiologist will mark each location of qualified nodules and assign a score. If at least one suspicious nodule is observed, the radiologist will also indicate a Lung-RADS score (i.e., 2, 3, 4A, 4B, or 4X) at the end of case interpretation.

During the second reading (i.e., aided reading) session, the radiologist will be presented with a chest CT with CADe marks displayed on the "left window" and the same chest CT without CADe marks displayed on the "right window". If the radiologist confirms a nodule, the radiologist will mark the location on the chest CT displayed on the right window. This may or may not correspond to the locations of the CADe marks as displayed on the left window. The same as in the baseline study, the radiologist will assign a score (level of suspicion) to each suspicious location and provide a Lung-RADS score for the chest CT case, if at least one location on the CT is marked.

Based on the suspicious locations and their scores (level of suspicion) assessed by study radiologists, LROC curves can be constructed for both the baseline and the aided reads, and the significance of any difference will be calculated. The LOS score will also be used to calculate sensitivity, specificity, PPV and NPV.

Number and types of Chest CT cases:

Retrospective chest CT image series from approximately 350 cases will be included in the study. Approximately two hundred (200) cases will have at least one nodule size greater than 4mm. These cases will have radiology-prove report and/or biopsy-confirmed cancers. Also included as nodule images are those where a nodule was not detected at that time, but was detected and acted on based on a subsequent CT. These are the prior images where the nodule can be identified in the same location as on the "current" image confirmed by the radiologist expert panel (using a majority of three as the decision criterion). In addition, approximately one hundred and fifty (150) cases will not have a nodule size greater than 4mm.

The selected chest CT cases for the 2-arm comparative reader study, randomly selected from a larger pool of CT cases, will be enriched in the following way:

1. Lung nodules (cancers and benign nodules) will be nodule size ranging from 4mm to 30 mm. The proportion of nodules 20 mm or less may take majority of the cases with nodule(s) since this lung lesion size range is where we expect the major impact of this software to be.

2. Non-Solid (ground glass) nodules will be added to the study dataset (based on availability) to determine the performance of the system on non-solid nodules. For this group, to have sufficient cases, we may include more benign (non-malignant) non-solid nodules.

3. In this project, VIRGINIA TECH study team will perform a machine standalone test of the V5med Lung AI algorithm followed by a reader performance evaluation study. V5med Technologies will provide a system configured with the operating point set to be used for the reader studies and a configuration for an "open" system to be used for machine testing and to generate free-response ROC (FROC) curve.

The Design of Two-Arm Comparative Study:

The First Arm Study:

Perform a baseline reader study without any special tool. The reader is asked to mark all potential nodules not less than 4mm. For each mark, the size of region of interest (ROI), reader score (level of suspicion), and time spent will be recorded for the statistical analyses.

The Second Arm Study:

Perform another reader study using the V5med Lung AI as the aided read. The reader is asked to mark all potential nodules not less than 4mm assisted by V5med Lung AI. Some or majority of ROIs' parameters will be pre-filled by V5med Lung AI. The reader will make final determination for each location. The marked location, LOS score, Lung-RADS score, and time spent will be recorded for the statistical analyses and compared to the first arm study.

The primary study hypothesis is that the detection of lung nodules with the use of V5med Lung AI and associated functions is superior to the detection of lung nodules without the use of V5med Lung AI, as measured by the area under the LROC curve.

Conditions

Lung Cancer

Study Design

• Allocation Method: RANDOMIZED
• Intervention Model: PARALLEL
• Primary Study Purpose: DIAGNOSTIC
• Blinding Strategy: DOUBLE

Study Groups

Reader Study without AI (Baseline )

During the baseline reading session the radiologist will mark each location of qualified nodules and assign a score. If at least one suspicious nodule is observed, the radiologist will also indicate a Lung-RADS score (i.e., 2, 3, 4A, 4B, or 4X) at the end of case interpretation.

Group Type: NO_INTERVENTION

No interventions assigned to this group

Reader Study with AI (Aided by V5med Lung AI)

Perform another reader study using the V5med Lung AI as the aided read. The reader is asked to mark all potential nodules not less than 4mm assisted by V5med Lung AI. Some or majority of ROIs' parameters will be pre-filled by V5med Lung AI. The reader will make final determination for each location. The marked location, LOS score, Lung-RADS score, and time spent will be recorded for the statistical analyses and compared to the first arm study.

Group Type: ACTIVE_COMPARATOR

V5med Lung AI

Intervention Type: DEVICE

During the second reading session (concurrent read), the radiologist will be presented with a standard appearing CT with CADe marks placed on the "left window" and the same original without any AI mark with be display on the "right window". Deeming a nodule, the radiologist will mark location. These may or may not correspond to the locations of the CADe markers. As in the baseline study, the radiologist will assign a level of suspicious to each mark and provide a Lung-RADS score and the size of longest dimension.

Interventions

V5med Lung AI

During the second reading session (concurrent read), the radiologist will be presented with a standard appearing CT with CADe marks placed on the "left window" and the same original without any AI mark with be display on the "right window". Deeming a nodule, the radiologist will mark location. These may or may not correspond to the locations of the CADe markers. As in the baseline study, the radiologist will assign a level of suspicious to each mark and provide a Lung-RADS score and the size of longest dimension.

Intervention Type: DEVICE

Eligibility Criteria

Inclusion Criteria

1. Screening chest CT taken of asymptomatic patients age 55 - 77 with history of smoking

2. Non-screening chest CT of an adult (age 18 and older) without comorbidity

3. Primary Lung Cancer, Biopsy-proven with radiology report

4. <= 2.5mm slice spacing, no gaps

5. Standard reconstruction kernels

6. Maximum of 7 lung nodules per CT scan

7. Nodules must be 4-30mm in size (though majority of them will be 20mm or smaller)

Exclusion Criteria

1. Acute pneumothorax

2. Incomplete inclusion of both lungs, within the field of view, including both Apices

3. Excessive motion artifacts or beam-hardening artifacts

4. Symptomatic patients with co-morbidities

• Minimum Eligible Age: 55 Years
• Maximum Eligible Age: 77 Years
• Eligible Sex: ALL
• Accepts Healthy Volunteers: No

Sponsors

V5med Inc.

INDUSTRY

Sponsor Role: lead

Responsible Party

Responsibility Role: SPONSOR

Locations

Arlington Innovation Center: Health Research

Arlington, Virginia, United States

Countries

United States

Other Identifiers

23-806

Identifier Type: -

Identifier Source: org_study_id