Head-to-Head Evaluation of the Cancer Ontology Supervised Multimodal Orchestration (COSMO) AI System Versus Pathologist-Only Review
NCT ID: NCT07307157
Last Updated: 2025-12-29
Study Results
The study team has not published outcome measurements, participant flow, or safety data for this trial yet. Check back later for updates.
Basic Information
Get a concise snapshot of the trial, including recruitment status, study phase, enrollment targets, and key timeline milestones.
ENROLLING_BY_INVITATION
30 participants
OBSERVATIONAL
2025-06-12
2026-01-31
Brief Summary
Review the sponsor-provided synopsis that highlights what the study is about and why it is being conducted.
The study will characterize the diagnostic accuracy of COSMO and pathologists, inter-observer agreement, and variations in performance across anatomical sites and cancer types with different incidence rates. Results will establish how COSMO compares to pathologists on identical cases and will inform the development of AI-assisted diagnostic systems in clinical practice.
Detailed Description
Dive into the extended narrative that explains the scientific background, objectives, and procedures in greater depth.
Study Aims This head-to-head comparative study aims to: (1) evaluate the diagnostic performance of the COSMO AI system in cancer subtype classification across multiple anatomical sites; (2) characterize the diagnostic accuracy of experienced pathologists on the same cases; (3) directly compare diagnostic performance metrics between COSMO and pathologists; and (4) examine concordance patterns and performance variation by anatomical site, cancer incidence category, pathologist experience, and case complexity.
Study Setting and Participants The study will involve up to 25 board-certified pathologists with 3 to 10+ years of diagnostic experience, recruited from institutions across North America, Europe, and the Asia-Pacific region. Participating pathologists will have domain expertise in neuropathology, pulmonary pathology, urologic pathology, or general anatomical pathology.
Cases and Stratification The study will employ de-identified archival whole-slide images representing up to 300 patients with confirmed reference diagnoses, including 100 brain cancers, 100 lung cancers, and 100 kidney cancers. Cases will be stratified by cancer type and incidence category (common vs. rare or uncommon), consistent with World Health Organization (WHO) guidelines.
Data Collection Pathologists will independently review each case and provide diagnostic classifications along with confidence assessments using a 5-point scale. The digital pathology interface will automatically record time-to-diagnosis metrics. COSMO will process the same cases offline to generate independent diagnostic predictions and confidence scores. Both pathologist and AI predictions will be evaluated against established reference standard diagnoses.
Analysis Framework The primary analysis will characterize diagnostic performance metrics (including accuracy, sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and area under the receiver operating characteristic curve (AUROC)) for both pathologists (at the individual and aggregated levels) and the COSMO system. Secondary analyses will assess performance stratified by anatomical site, cancer incidence category, and pathologist experience level.
Conditions
See the medical conditions and disease areas that this research is targeting or investigating.
Keywords
Explore important study keywords that can help with search, categorization, and topic discovery.
Study Design
Understand how the trial is structured, including allocation methods, masking strategies, primary purpose, and other design elements.
COHORT
PROSPECTIVE
Study Groups
Review each arm or cohort in the study, along with the interventions and objectives associated with them.
AI-Based Evaluation using COSMO
No interventions assigned to this group
Pathologist-Based Evaluation
Digital Pathology Evaluation
Digital Pathology Evaluation
Interventions
Learn about the drugs, procedures, or behavioral strategies being tested and how they are applied within this trial.
Digital Pathology Evaluation
Digital Pathology Evaluation
Eligibility Criteria
Check the participation requirements, including inclusion and exclusion rules, age limits, and whether healthy volunteers are accepted.
Inclusion Criteria
* Minimum of 3 years of clinical diagnostic experience
* Active clinical practice involving diagnostic pathology slide review
* Willingness to independently review and diagnose up to 300 de-identified whole-slide images
* Ability to access the study platform and complete case reviews within the specified study timeline
* Provision of informed consent for study participation
Exclusion Criteria
* Inability to commit sufficient time to complete assigned case reviews
* Presence of significant financial conflicts of interest related to the study outcomes
ALL
No
Sponsors
Meet the organizations funding or collaborating on the study and learn about their roles.
Harvard Medical School (HMS and HSDM)
OTHER
Responsible Party
Identify the individual or organization who holds primary responsibility for the study information submitted to regulators.
Kun-Hsing Yu
Associate Professor
Principal Investigators
Learn about the lead researchers overseeing the trial and their institutional affiliations.
Kun-Hsing Yu, MD, PhD
Role: PRINCIPAL_INVESTIGATOR
Harvard Medical School (HMS and HSDM)
Locations
Explore where the study is taking place and check the recruitment status at each participating site.
Harvard Medical School
Boston, Massachusetts, United States
Countries
Review the countries where the study has at least one active or historical site.
Other Identifiers
Review additional registry numbers or institutional identifiers associated with this trial.
Yu Lab COSMO Study
Identifier Type: -
Identifier Source: org_study_id