A Study to Evaluate Strategies for Teaching Effective Use of Diagnostic Tests

NCT ID: NCT04130607

Last Updated: 2019-10-17

Basic Information

• Recruitment Status: COMPLETED
• Clinical Phase: NA
• Total Enrollment: 65 participants
• Study Classification: INTERVENTIONAL
• Study Start Date: 2018-05-15
• Study Completion Date: 2019-10-15

Brief Summary

A recent Institute of Medicine monograph brought attention to high rates of diagnostic error and called for better educational efforts to improve diagnostic accuracy.1 Educational methods, however, are rarely tested and some educational efforts may be ineffective and wasteful.2 In this study, we plan to examine whether explicit instruction on diagnostic methods will have an effect on diagnostic accuracy of 2nd-year medical students and internal medicine residents.

Detailed Description

Research has shown that expert diagnosticians use a two-step process to confirm a diagnosis: hypothesis generation to generate diagnostic possibilities, followed by hypothesis verification to confirm the most likely diagnostic possibility.3-5 The first step appears to be non-analytical, related to pattern recognition. The second step could be calculated using analytical reasoning, however, physicians rarely make an overt calculation of conditional probabilities. Instead, experienced clinicians typically use an implicit habit or heuristic called "anchoring and adjusting" to incorporate diagnostic testing information into their thinking.6,7 Cognitive psychologists have postulated that anchoring and adjusting provides a way that probability estimates can be updated based on additional new evidence. Most of the discussion in the literature focuses on how this heuristic can lead to biased thinking because of base-rate neglect or anchoring.6 Very little discussion is on how this heuristic could be improved to yield more accurate probability estimates and whether proper use of the heuristic could be taught.

The degree to which a diagnostic test should lead to an adjustment of a probability estimate depends on the operating characteristics of a test, that is, the sensitivity and specificity. Likelihood ratios, once understood, are easier to incorporate into one's thinking, and thus could be used to calibrate the anchoring and adjusting heuristic.7

In this randomized trial, we tested whether explicit conceptual instruction on Bayesian reasoning and likelihood ratios would improve Bayesian updating, compared with a second intervention where we provided multiple (27) examples of clinical problem solving. The third arm provided minimal teaching about diagnosis, but no explicit teaching or examples.

Conditions

Instructional Methods

Study Design

• Allocation Method: RANDOMIZED
• Intervention Model: PARALLEL
• Primary Study Purpose: HEALTH_SERVICES_RESEARCH
• Blinding Strategy: SINGLE

Study Groups

Analytical

Students will receive brief instruction in probability, sensitivity, specificity, and likelihood ratios, with distributions and calculations. Pretest and posttest probabilities will be computed for two cases for each of the three conditions listed above.

Group Type: EXPERIMENTAL

Conceptual teaching

Intervention Type: OTHER

The present study is designed to contrast two instructional methods - explicit instruction in likelihood ratios and pretest/posttest probabilities versus implicit instruction based on presentation of multiple cases. These will be compared to a "no intervention" control group.

Experiential

Students will receive a brief instruction conceptually discussing sensitivity and specificity (e.g. "a sensitive test will be positive at even low levels of disease. However, this can lead to a number of false positive errors, when the test is positive even when there is no disease. As a result, it is most useful for ruling out a diagnosis"). They will then work through a total of 30 cases, 10 for each condition, in blocked sequence. For each brief written case they will be asked for a probability of diagnosis after the clinical information is presented. The test result will then be given and they will be asked for a post-test probability. Their estimate will be compared to the computed value based on published estimates of sensitivity and specificity and feedback provided.

Group Type: ACTIVE_COMPARATOR

Conceptual teaching

Intervention Type: OTHER

The present study is designed to contrast two instructional methods - explicit instruction in likelihood ratios and pretest/posttest probabilities versus implicit instruction based on presentation of multiple cases. These will be compared to a "no intervention" control group.

No Explicit Instruction or Examples

Students will receive 3 passages from a clinical text related to each of the 3 conditions in the study and asked to study them for 15 min each.

Group Type: PLACEBO_COMPARATOR

Conceptual teaching

Intervention Type: OTHER

The present study is designed to contrast two instructional methods - explicit instruction in likelihood ratios and pretest/posttest probabilities versus implicit instruction based on presentation of multiple cases. These will be compared to a "no intervention" control group.

Interventions

Conceptual teaching

The present study is designed to contrast two instructional methods - explicit instruction in likelihood ratios and pretest/posttest probabilities versus implicit instruction based on presentation of multiple cases. These will be compared to a "no intervention" control group.

Intervention Type: OTHER

Other Intervention Names

Teaching through examples; No active teaching

Eligibility Criteria

Inclusion Criteria

• Medical Student at McMaster University or Eastern Virginia Medical School
• Completed 18 months of coursework

• Minimum Eligible Age: 18 Years
• Eligible Sex: ALL
• Accepts Healthy Volunteers: Yes

Sponsors

McMaster University

OTHER

Sponsor Role: collaborator

Sentara Norfolk General Hospital

OTHER

Sponsor Role: lead

Responsible Party

John Brush

Cardiologist

Responsibility Role: PRINCIPAL_INVESTIGATOR

Locations

Sentara Norfolk General Hospital

Norfolk, Virginia, United States

Countries

United States

References

Brush JE Jr, Lee M, Sherbino J, Taylor-Fishwick JC, Norman G. Effect of Teaching Bayesian Methods Using Learning by Concept vs Learning by Example on Medical Students' Ability to Estimate Probability of a Diagnosis: A Randomized Clinical Trial. JAMA Netw Open. 2019 Dec 2;2(12):e1918023. doi: 10.1001/jamanetworkopen.2019.18023.

Reference Type: DERIVED

PMID: 31860107 (View on PubMed)

Other Identifiers

18-04-EX-0062

Identifier Type: -

Identifier Source: org_study_id