Trial Outcomes & Findings for Validation of an Artificial Intelligence-based Algorithm for Skeletal Age Assessment (NCT NCT03530098)
NCT ID: NCT03530098
Last Updated: 2021-06-09
Results Overview
Mean absolute difference between dictated final impressions (baseline measure by Radiologist) and the consensus determination of a panel of radiologists following review.
Recruitment status
COMPLETED
Study phase
NA
Target enrollment
1903 participants
Primary outcome timeframe
Up to 10 minutes to acquire the scan; up to 2 days to complete diagnosis review
Results posted on
2021-06-09
Participant Flow
Participant milestones
| Measure |
Control (Without-AI)
Diagnosis by radiologists made according to current standard of care methods.
|
Experiment (With-AI)
Diagnosis by radiologists informed by "BoneAgeModel" Artificial Intelligence (AI) algorithm incorporated into normal radiologist workflows and considered as a factor in the clinical decision making process. The radiologists' diagnosis will be considered final. BoneAgeModel takes in a hand radiograph and gender, and outputs the skeletal (bone) age.
|
|---|---|---|
|
Overall Study
STARTED
|
939
|
964
|
|
Overall Study
Ground-truth Labeled
|
741
|
794
|
|
Overall Study
Primary Analysis Set
|
739
|
792
|
|
Overall Study
COMPLETED
|
939
|
964
|
|
Overall Study
NOT COMPLETED
|
0
|
0
|
Reasons for withdrawal
Withdrawal data not reported
Baseline Characteristics
Race and Ethnicity were not collected from any participant.
Baseline characteristics by cohort
| Measure |
Control (Without-AI)
n=739 Participants
Diagnosis by radiologists made according to current standard of care methods.
|
Experiment (With-AI)
n=792 Participants
Diagnosis by radiologists informed by "BoneAgeModel" AI algorithm incorporated into normal radiologist workflows and considered as a factor in the clinical decision making process.
|
Total
n=1531 Participants
Total of all reporting groups
|
|---|---|---|---|
|
Age, Continuous
|
11.8 years
STANDARD_DEVIATION 3.6 • n=739 Participants
|
11.5 years
STANDARD_DEVIATION 3.6 • n=792 Participants
|
11.7 years
STANDARD_DEVIATION 3.6 • n=1531 Participants
|
|
Age, Customized
0-4 years
|
12 Participants
n=739 Participants
|
14 Participants
n=792 Participants
|
26 Participants
n=1531 Participants
|
|
Age, Customized
>4-8 years
|
110 Participants
n=739 Participants
|
131 Participants
n=792 Participants
|
241 Participants
n=1531 Participants
|
|
Age, Customized
>8-12 years
|
206 Participants
n=739 Participants
|
249 Participants
n=792 Participants
|
455 Participants
n=1531 Participants
|
|
Age, Customized
>12-16 years
|
331 Participants
n=739 Participants
|
338 Participants
n=792 Participants
|
669 Participants
n=1531 Participants
|
|
Age, Customized
>16-20 years
|
75 Participants
n=739 Participants
|
57 Participants
n=792 Participants
|
132 Participants
n=1531 Participants
|
|
Age, Customized
>20 years
|
5 Participants
n=739 Participants
|
3 Participants
n=792 Participants
|
8 Participants
n=1531 Participants
|
|
Sex: Female, Male
Female
|
338 Participants
n=739 Participants
|
359 Participants
n=792 Participants
|
697 Participants
n=1531 Participants
|
|
Sex: Female, Male
Male
|
401 Participants
n=739 Participants
|
433 Participants
n=792 Participants
|
834 Participants
n=1531 Participants
|
|
Race and Ethnicity Not Collected
|
—
|
—
|
0 Participants
Race and Ethnicity were not collected from any participant.
|
|
Region of Enrollment
United States
|
739 participants
n=739 Participants
|
792 participants
n=792 Participants
|
1531 participants
n=1531 Participants
|
|
Skeletal age final impression (mean)
|
11.6 years
STANDARD_DEVIATION 3.6 • n=739 Participants
|
11.4 years
STANDARD_DEVIATION 3.5 • n=792 Participants
|
11.5 years
STANDARD_DEVIATION 3.5 • n=1531 Participants
|
|
Skeletal age final impression (categorical)
0-4 years
|
21 Participants
n=739 Participants
|
17 Participants
n=792 Participants
|
38 Participants
n=1531 Participants
|
|
Skeletal age final impression (categorical)
>4-8 years
|
110 Participants
n=739 Participants
|
131 Participants
n=792 Participants
|
241 Participants
n=1531 Participants
|
|
Skeletal age final impression (categorical)
>8-12 years
|
191 Participants
n=739 Participants
|
223 Participants
n=792 Participants
|
414 Participants
n=1531 Participants
|
|
Skeletal age final impression (categorical)
>12-16 years
|
344 Participants
n=739 Participants
|
362 Participants
n=792 Participants
|
706 Participants
n=1531 Participants
|
|
Skeletal age final impression (categorical)
>16-20 years
|
73 Participants
n=739 Participants
|
59 Participants
n=792 Participants
|
132 Participants
n=1531 Participants
|
|
Skeletal age final impression (categorical)
>20 years
|
0 Participants
n=739 Participants
|
0 Participants
n=792 Participants
|
0 Participants
n=1531 Participants
|
|
Clinical histories
Endocrine
|
391 Participants
n=739 Participants
|
430 Participants
n=792 Participants
|
821 Participants
n=1531 Participants
|
|
Clinical histories
Orthopedic
|
82 Participants
n=739 Participants
|
63 Participants
n=792 Participants
|
145 Participants
n=1531 Participants
|
|
Clinical histories
Congenital/syndrome
|
9 Participants
n=739 Participants
|
12 Participants
n=792 Participants
|
21 Participants
n=1531 Participants
|
|
Clinical histories
Medical
|
16 Participants
n=739 Participants
|
17 Participants
n=792 Participants
|
33 Participants
n=1531 Participants
|
|
Clinical histories
Other
|
4 Participants
n=739 Participants
|
12 Participants
n=792 Participants
|
16 Participants
n=1531 Participants
|
|
Clinical histories
More than one category
|
20 Participants
n=739 Participants
|
17 Participants
n=792 Participants
|
37 Participants
n=1531 Participants
|
|
Clinical histories
Not available
|
217 Participants
n=739 Participants
|
241 Participants
n=792 Participants
|
458 Participants
n=1531 Participants
|
PRIMARY outcome
Timeframe: Up to 10 minutes to acquire the scan; up to 2 days to complete diagnosis reviewPopulation: Primary analysis set: Participants with ground-truth labeled exam results and no bone deformity. Ground-truth labeled: exam was interpreted by a panel of 4 radiologists and their interpretations were averaged to determine a final label.
Mean absolute difference between dictated final impressions (baseline measure by Radiologist) and the consensus determination of a panel of radiologists following review.
Outcome measures
| Measure |
Control (Without-AI)
n=739 Participants
Diagnosis by radiologists made according to current standard of care methods.
|
Experiment (With-AI)
n=792 Participants
Diagnosis by radiologists informed by "BoneAgeModel" AI algorithm incorporated into normal radiologist workflows and considered as a factor in the clinical decision making process.
|
|---|---|---|
|
Paired Difference of Skeletal Age Estimate
|
5.95 months
Interval 5.53 to 6.37
|
5.36 months
Interval 5.01 to 5.71
|
SECONDARY outcome
Timeframe: Up to approximately 4 minutesPopulation: Primary analysis set: Participants with ground-truth labeled exam results and no bone deformity. Ground-truth labeled: exam was interpreted by a panel of 4 radiologists and their interpretations were averaged to determine a final label.
Amount of time taken by radiologists when using the BoneAgeModel as compared to when they are not.
Outcome measures
| Measure |
Control (Without-AI)
n=739 Participants
Diagnosis by radiologists made according to current standard of care methods.
|
Experiment (With-AI)
n=792 Participants
Diagnosis by radiologists informed by "BoneAgeModel" AI algorithm incorporated into normal radiologist workflows and considered as a factor in the clinical decision making process.
|
|---|---|---|
|
Time for Diagnosis
|
142 seconds
Interval 80.0 to 248.0
|
102 seconds
Interval 59.0 to 196.0
|
Adverse Events
Control (Without-AI)
Serious events: 0 serious events
Other events: 0 other events
Deaths: 0 deaths
Experiment (With-AI)
Serious events: 0 serious events
Other events: 0 other events
Deaths: 0 deaths
Serious adverse events
Adverse event data not reported
Other adverse events
Adverse event data not reported
Additional Information
Results disclosure agreements
- Principal investigator is a sponsor employee
- Publication restrictions are in place