Evaluating the Potential of Large Language Models for Respiratory Disease Consultations
NCT ID: NCT06457269
Last Updated: 2024-11-27
Study Results
The study team has not published outcome measurements, participant flow, or safety data for this trial yet. Check back later for updates.
Basic Information
Get a concise snapshot of the trial, including recruitment status, study phase, enrollment targets, and key timeline milestones.
COMPLETED
NA
703 participants
INTERVENTIONAL
2023-10-01
2024-10-12
Brief Summary
Review the sponsor-provided synopsis that highlights what the study is about and why it is being conducted.
The main question aims to answer are:
* How do large language models perform in comparison to human doctors in diagnosing and consulting on respiratory diseases across various clinical scenarios?
In three clinical scenarios including the online query section, the disease diagnosis section and the medical explanation section, research assistants or volunteers will be asked to cross-question all LLMs or real doctors using predefined online questions and their own issues. After each questioning session, a short washout period is implemented to eliminate potential biases.
Detailed Description
Dive into the extended narrative that explains the scientific background, objectives, and procedures in greater depth.
Conditions
See the medical conditions and disease areas that this research is targeting or investigating.
Study Design
Understand how the trial is structured, including allocation methods, masking strategies, primary purpose, and other design elements.
RANDOMIZED
CROSSOVER
DIAGNOSTIC
QUADRUPLE
Study Groups
Review each arm or cohort in the study, along with the interventions and objectives associated with them.
Cross-comparison group(the disease diagnosis section)
Cross-comparison group (including human doctor controls and all LLMs)
Diagnosis by three human doctors
This intervention involves answering patient inquiries by different human doctors. Each patient is randomly assigned by the system to three doctors from different provinces in China selected from the database of doctors. The doctors all come from multiple online consultation platforms in China, and their diagnostic qualifications and medical licenses have undergone strict verification.
Diagnosis by ChatGPT-3.5 (with search capabilities)
This intervention involves answering patient inquiries by ChatGPT-3.5 with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by ChatGPT-3.5 (without search capabilities)
This intervention involves answering patient inquiries by ChatGPT-3.5 without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by ChatGPT-4.0 (with search capabilities)
This intervention involves answering patient inquiries by ChatGPT-4.0 with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by ChatGPT-4.0 (without search capabilities)
This intervention involves answering patient inquiries by ChatGPT-4.0 without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Claude instant (with search capabilities)
This intervention involves answering patient inquiries by Claude instant with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Claude instant (without search capabilities)
This intervention involves answering patient inquiries by Claude instant without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Claude 2 (with search capabilities)
This intervention involves answering patient inquiries by Claude 2 with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Claude 2 (without search capabilities)
This intervention involves answering patient inquiries by Claude 2 without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Gemini Pro (with search capabilities)
This intervention involves answering patient inquiries by Gemini Pro with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Gemini Pro (without search capabilities)
This intervention involves answering patient inquiries by Gemini Pro without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Cross-comparison group(the medical explanation section)
Cross-comparison group (including human doctor controls and all LLMs)
Diagnosis by three human doctors
This intervention involves answering patient inquiries by different human doctors. Each patient is randomly assigned by the system to three doctors from different provinces in China selected from the database of doctors. The doctors all come from multiple online consultation platforms in China, and their diagnostic qualifications and medical licenses have undergone strict verification.
Diagnosis by ChatGPT-3.5 (with search capabilities)
This intervention involves answering patient inquiries by ChatGPT-3.5 with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by ChatGPT-3.5 (without search capabilities)
This intervention involves answering patient inquiries by ChatGPT-3.5 without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by ChatGPT-4.0 (with search capabilities)
This intervention involves answering patient inquiries by ChatGPT-4.0 with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by ChatGPT-4.0 (without search capabilities)
This intervention involves answering patient inquiries by ChatGPT-4.0 without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Claude instant (with search capabilities)
This intervention involves answering patient inquiries by Claude instant with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Claude instant (without search capabilities)
This intervention involves answering patient inquiries by Claude instant without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Claude 2 (with search capabilities)
This intervention involves answering patient inquiries by Claude 2 with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Claude 2 (without search capabilities)
This intervention involves answering patient inquiries by Claude 2 without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Gemini Pro (with search capabilities)
This intervention involves answering patient inquiries by Gemini Pro with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Gemini Pro (without search capabilities)
This intervention involves answering patient inquiries by Gemini Pro without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Interventions
Learn about the drugs, procedures, or behavioral strategies being tested and how they are applied within this trial.
Diagnosis by three human doctors
This intervention involves answering patient inquiries by different human doctors. Each patient is randomly assigned by the system to three doctors from different provinces in China selected from the database of doctors. The doctors all come from multiple online consultation platforms in China, and their diagnostic qualifications and medical licenses have undergone strict verification.
Diagnosis by ChatGPT-3.5 (with search capabilities)
This intervention involves answering patient inquiries by ChatGPT-3.5 with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by ChatGPT-3.5 (without search capabilities)
This intervention involves answering patient inquiries by ChatGPT-3.5 without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by ChatGPT-4.0 (with search capabilities)
This intervention involves answering patient inquiries by ChatGPT-4.0 with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by ChatGPT-4.0 (without search capabilities)
This intervention involves answering patient inquiries by ChatGPT-4.0 without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Claude instant (with search capabilities)
This intervention involves answering patient inquiries by Claude instant with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Claude instant (without search capabilities)
This intervention involves answering patient inquiries by Claude instant without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Claude 2 (with search capabilities)
This intervention involves answering patient inquiries by Claude 2 with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Claude 2 (without search capabilities)
This intervention involves answering patient inquiries by Claude 2 without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Gemini Pro (with search capabilities)
This intervention involves answering patient inquiries by Gemini Pro with search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Diagnosis by Gemini Pro (without search capabilities)
This intervention involves answering patient inquiries by Gemini Pro without search capabilities, before answering any questions, clear the chat history from the previous patient and input the predetermined initialization statement.
Eligibility Criteria
Check the participation requirements, including inclusion and exclusion rules, age limits, and whether healthy volunteers are accepted.
Inclusion Criteria
2. Ability to engage in LLM dialog operations independently or with minimal peer training
3. A health status deemed suitable for study participation by the pulmonology experts
Exclusion Criteria
ALL
No
Sponsors
Meet the organizations funding or collaborating on the study and learn about their roles.
University of Glasgow
OTHER
Nanchong Central Hospital
OTHER_GOV
Affiliated Hospital of North Sichuan Medical College
OTHER
North Sichuan Medical College
OTHER
Responsible Party
Identify the individual or organization who holds primary responsibility for the study information submitted to regulators.
Zining Luo
Principal Investigator
Principal Investigators
Learn about the lead researchers overseeing the trial and their institutional affiliations.
Jiebin Xie, Doctor
Role: PRINCIPAL_INVESTIGATOR
North Sichuan Medical College
Locations
Explore where the study is taking place and check the recruitment status at each participating site.
The Affiliated Hospital of North Sichuan Medical College
Nanchong, Sichuan, China
Countries
Review the countries where the study has at least one active or historical site.
References
Explore related publications, articles, or registry entries linked to this study.
Mercer SW, Maxwell M, Heaney D, Watt GC. The consultation and relational empathy (CARE) measure: development and preliminary validation and reliability of an empathy-based consultation process measure. Fam Pract. 2004 Dec;21(6):699-705. doi: 10.1093/fampra/cmh621. Epub 2004 Nov 4.
Other Identifiers
Review additional registry numbers or institutional identifiers associated with this trial.
22XQT0309
Identifier Type: OTHER_GRANT
Identifier Source: secondary_id
CBY22-QDA15
Identifier Type: OTHER_GRANT
Identifier Source: secondary_id
2022LC005
Identifier Type: OTHER_GRANT
Identifier Source: secondary_id
23JCYJPT0014
Identifier Type: OTHER_GRANT
Identifier Source: secondary_id
1426887-2024-1
Identifier Type: -
Identifier Source: org_study_id