DeepRare AI System Outperforms Doctors in Rare Disease Diagnosis

Scientists in Shanghai have unveiled DeepRare, an artificial intelligence system that diagnoses rare diseases more accurately than experienced doctors. In head-to-head testing, the AI correctly identified the disease on its first try 64.4% of the time, while doctors achieved the same accuracy 54.6% of the time.

A paper on the system, developed by a joint team from Shanghai Jiao Tong University's School of Artificial Intelligence and Xinhua Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, was published on the website of the journal Nature on Thursday. The system achieved an overall diagnosis accuracy of approximately 79%, outperforming other diagnosis programs available.

Rare diseases affect less than 1 in 2,000 individuals at any given time but affect hundreds of millions of people worldwide. There are over 7,000 rare diseases, with 80% genetic. Currently, around 300 million people globally are affected by these disorders, and diagnosis often takes five years or more. During that time, many patients endure a diagnostic journey marked by repeated referrals, misdiagnosis, and unnecessary medical interventions.

DeepRare stands for Diagnosis of Rare Diseases with Evidence-traced Autonomous Reasoning Agents. The system is built on an agentic architecture containing over 40 different agents and tools that each help ingest information on symptoms, medical notes, medical literature databases, and genetic sequencing information. The system integrates vast medical literature databases with real-time clinical data, enabling a more sophisticated approach to diagnosis.

Unlike conventional AI, which relies on rapid pattern matching, DeepRare mimics the "slow thinking" of human doctors. It can proactively ask questions to fill in missing information and meticulously refine diagnostic information through a cycle of hypothesis, verification and self-reflection. After processing information, DeepRare references databases of known diseases all over the world, with its agents working together in hypothesis-generation, verification, and refinement loops until they are able to rank possible diagnoses for the patient and provide justification for their rankings.

Every diagnostic conclusion generated by the system is traceable and accompanied by a clear and complete evidence chain, allowing doctors to understand not only what the diagnosis is, but also why it was made. The tool traces the reasoning behind its diagnosis so that doctors can easily see why DeepRare gave them that suggestion.

DeepRare's first test was conducted on 6,401 clinical cases in which the diagnosis was already known. The AI outperformed 15 other existing diagnostic tools. The side-by-side evaluation used a smaller group of 163 difficult cases, with five experienced doctors, each with more than a decade of practice, given the same information as DeepRare.

When provided with only clinical phenotype information without genetic data, DeepRare achieved a top-ranked diagnostic accuracy of 57.18 percent, representing a 23.79 percentage point improvement over the previous best international model. With the support of multimodal data including genetic sequencing, DeepRare's comprehensive top-ranked diagnostic accuracy in complex cases exceeds 70.6 percent, significantly outperforming the widely used international tool, Exomiser, which stands at 53.2 percent.

Even when DeepRare did not get the correct answer the first time around, doctors found that the AI system ranked the correct diagnosis in its top predictions 92% of the time. Ten rare disease specialists were asked to look at the AI's step-by-step reasoning, and they agreed with its logic 95.4% of the time.

DeepRare is currently being used by clinicians online at over 600 different institutions worldwide. Researchers behind DeepRare hope to expand even further and have plans to work with clinical professionals and patients around the world to validate their model on tens of thousands of cases.