Validation of an Artificial Intelligence-based Algorithm for Skeletal Age Assessment

NCT ID: NCT03530098

Last Updated: 2021-06-09

Study Results

Results available

Outcome measurements, participant flow, baseline characteristics, and adverse events have been published for this study.

View full results

Basic Information

Get a concise snapshot of the trial, including recruitment status, study phase, enrollment targets, and key timeline milestones.

Recruitment Status

COMPLETED

Clinical Phase

NA

Total Enrollment

1903 participants

Study Classification

INTERVENTIONAL

Study Start Date

2018-07-12

Study Completion Date

2019-08-31

Brief Summary

Review the sponsor-provided synopsis that highlights what the study is about and why it is being conducted.

The purpose of this study is to understand the effects of using an Artificial Intelligence algorithm for skeletal age estimation as a computer-aided diagnosis (CADx) system. In this prospective real-time study, the investigators will send de-identified hand radiographs to the Artificial Intelligence algorithm and surface the output of this algorithm to the radiologist, who will incorporate this information with their normal workflows to make an estimation of the bone age. All radiologists involved in the study will be trained to recognize the surfaced prediction to be the output of the Artificial Intelligence algorithm. The radiologists' diagnosis will be final and considered independent to the output of the algorithm.

Detailed Description

Dive into the extended narrative that explains the scientific background, objectives, and procedures in greater depth.

The investigators are targeting to study the effect of their Artificial Intelligence algorithm on the radiologists' estimation of skeletal age. Currently, radiologists make the estimation using only the radiographic images and health records. As part of this study, the radiologists will estimate skeletal age from radiographic images, health records, and the output of the CADx algorithm. The investigators wish to understand how radiologists using the Artificial Intelligence algorithm compare to radiologists who do not for the specific task of estimating skeletal age.

This study is organized as a multi-institutional randomized control trial with two arms - experiment (receiving the Artificial Intelligence algorithm's output) and control (no intervention). Both of these arms will be compared to a clinical reference standard ("gold standard") composed of a panel of radiologists. The metric of comparison will be Mean Absolute Distance (MAD). The investigators plan to use statistical tests such as the t-test to determine any statistically-significant difference in skeletal age estimation between the two groups.

The investigators have recruited and analyzed data from a sample size of 1600 exams. Patients getting these exams will not undergo any research procedures that deviate from the current standard practices.

Conditions

See the medical conditions and disease areas that this research is targeting or investigating.

Bone Age

Study Design

Understand how the trial is structured, including allocation methods, masking strategies, primary purpose, and other design elements.

Allocation Method

RANDOMIZED

Intervention Model

PARALLEL

A hand radiograph will be randomly assigned to one of two groups - control and experiment. In the control group, participating radiologists will diagnose the exam using the current standard of care (no intervention). In the experiment group, the radiologists will factor in the output of the Artificial Intelligence algorithm in their skeletal age estimation. In all cases, the decision of the radiologist will be considered final.
Primary Study Purpose

DIAGNOSTIC

Blinding Strategy

NONE

Study Groups

Review each arm or cohort in the study, along with the interventions and objectives associated with them.

Control (Without-AI)

This is the control arm where no intervention is provided; represents current standard of care.

Group Type NO_INTERVENTION

No interventions assigned to this group

Experiment (With-AI)

This is the experiment arm where the intervention, "BoneAgeModel", is provided. The participating radiologists in this arm will receive the output of the Artificial Intelligence algorithm. They will be asked to incorporate this new information with their normal workflows to make a diagnosis. The radiologists' diagnosis will be considered final.

Group Type EXPERIMENTAL

BoneAgeModel

Intervention Type DEVICE

BoneAgeModel is an Artificial Intelligence tool that takes in a hand radiograph and gender, and outputs the skeletal (bone) age. The intervention involves using this tool as a factor in the clinical decision making process of the participating radiologists. The radiologist's decision will be considered final.

Interventions

Learn about the drugs, procedures, or behavioral strategies being tested and how they are applied within this trial.

BoneAgeModel

BoneAgeModel is an Artificial Intelligence tool that takes in a hand radiograph and gender, and outputs the skeletal (bone) age. The intervention involves using this tool as a factor in the clinical decision making process of the participating radiologists. The radiologist's decision will be considered final.

Intervention Type DEVICE

Eligibility Criteria

Check the participation requirements, including inclusion and exclusion rules, age limits, and whether healthy volunteers are accepted.

Inclusion Criteria

Exams that meet the following inclusion criteria will be included: (1) exams read by radiologists who interpret pediatric skeletal age exams and verbally consent to participate (2) exams that contain a procedure code or study description indicative of a skeletal age exam.

Exams containing more than one radiograph will not be included. Exams for which a trainee provides a preliminary interpretation will be excluded. No further exclusion criteria will be applied on the basis of image quality metrics or manufacturers. No exclusion criteria will be applied on the basis of patient chronological age.
Eligible Sex

ALL

Accepts Healthy Volunteers

No

Sponsors

Meet the organizations funding or collaborating on the study and learn about their roles.

Stanford University

OTHER

Sponsor Role lead

Responsible Party

Identify the individual or organization who holds primary responsibility for the study information submitted to regulators.

Safwan S Halabi, MD

Principal Investigator

Responsibility Role PRINCIPAL_INVESTIGATOR

Principal Investigators

Learn about the lead researchers overseeing the trial and their institutional affiliations.

Curtis Langlotz, M.D. Ph.D.

Role: STUDY_CHAIR

Stanford University

David Eng, B.S.

Role: STUDY_DIRECTOR

Stanford University

Nishith Khandwala, B.S.

Role: STUDY_DIRECTOR

Stanford University

Safwan Halabi, M.D.

Role: PRINCIPAL_INVESTIGATOR

Stanford University

Locations

Explore where the study is taking place and check the recruitment status at each participating site.

Stanford University

Stanford, California, United States

Site Status

Yale New Haven Hospital

New Haven, Connecticut, United States

Site Status

Boston Children's Hospital

Boston, Massachusetts, United States

Site Status

New York University

New York, New York, United States

Site Status

Cincinnati Children's Hospital Medical Center

Cincinnati, Ohio, United States

Site Status

Children's Hospital of Philadelphia

Philadelphia, Pennsylvania, United States

Site Status

Countries

Review the countries where the study has at least one active or historical site.

United States

Provided Documents

Download supplemental materials such as informed consent forms, study protocols, or participant manuals.

Document Type: Study Protocol and Statistical Analysis Plan

View Document

Other Identifiers

Review additional registry numbers or institutional identifiers associated with this trial.

IRB #44764

Identifier Type: -

Identifier Source: org_study_id

More Related Trials

Additional clinical trials that may be relevant based on similarity analysis.

The BoDI (Bone Density Investigation) Study
NCT06117124 NOT_YET_RECRUITING NA