Trial Outcomes & Findings for Trial of Simulation-based Mastery Learning to Communicate Diagnostic Uncertainty (NCT NCT04021771)

Last Updated: 2021-05-25

Results Overview

Investigators will measure the percentage of residents who meet or exceed the Minimum Passing Standard (MPS) on the Uncertainty Communication Checklist (UCC) in the intervention group versus the control group at the second assessment (T2). At T2, resident physicians completed a simulation encounter with a standardized patient (SP) and the SP assessed the physician's performance using a checklist using the UCC. Our team developed the UCC in prior work with extensive patient input. The UCC includes 21 checklist items; items were assessed as either performed correctly (1 point) or incorrectly (0 points) and all items were added into a total score, which can range from 0-21 points. Since 1 = achieved and 0 = did not achieve, 1 is always better for each individual item and the higher the total score the better. Physicians achieved mastery when they met or exceeded the MPS, i.e. when they correctly performed at least 19 of the 21 items.

Recruitment status

COMPLETED

Study phase

Target enrollment

109 participants

Primary outcome timeframe

Although we originally planned for T2 to occur 4-8 weeks after the baseline assessment, T2 ended up taking place 16-19 weeks after baseline assessment due to the residency schedule.

Results posted on

2021-05-25

Participant Flow

Eligible participants included all emergency medicine resident physicians within the emergency medicine residency programs at Thomas Jefferson University and Northwestern University during the 2019-2020 academic year.

Participant milestones

Participant milestones
Measure	Group A (Intervention Group) Participants had at least three study visits. The first visit (T1) consisted of a baseline test, during which participants completed a simulated encounter with a standardized patient (SP) to establish a baseline score. Participants in Group A were given access to the intervention. The second visit (T2) was scheduled on average 16-19 weeks after T1 and consisted of another simulated encounter with the SP. Group A returned for a third visit (T3) to provide information about the decay of learned skills and if performance on the task is maintained by a single intervention. Educational Curriculum Intervention: The intervention included: 1) Online web-based education module created with the Rise 360 platform that has ten lessons, providing the learner with an introduction, background, and a map of how to navigate a discharge conversation for patients with diagnostic uncertainty. Core content are presented with multiple interactive components such as flip-cards, multiple-choice, drag-and-drop, sketch videos, and narrated clips. 2) Mobile application game designed to facilitate practice of the content learned in the online curriculum 3) Deliberate practice sessions allowed trainees to practice communication techniques presented within the online module and to receive feedback on their performance. These sessions were conducted with standardized patients via a video platform, during a scheduled appointment.	Group B (Control Group) Participants had at least three study visits. The first visit (T1) consisted of a baseline test, during which participants completed a simulated encounter with a standardized patient (SP) to establish a baseline score. The second visit (T2) was scheduled on average 16-19 weeks after T1 and consisted of another simulated encounter with the SP; following this session, participants in Group B were introduced to the intervention. The Group B returned for a third visit (T3) to provide information on how the curriculum impacts their score. Educational Curriculum Intervention: The intervention included: 1) Online web-based education module created with the Rise 360 platform that has ten lessons, providing the learner with an introduction, background, and a map of how to navigate a discharge conversation for patients with diagnostic uncertainty. Core content are presented with multiple interactive components such as flip-cards, multiple-choice, drag-and-drop, sketch videos, and narrated clips. 2) Mobile application game designed to facilitate practice of the content learned in the online curriculum 3) Deliberate practice sessions allowed trainees to practice communication techniques presented within the online module and to receive feedback on their performance. These sessions were conducted with standardized patients via a video platform, during a scheduled appointment.
Overall Study STARTED	55	54
Overall Study COMPLETED	55	54
Overall Study NOT COMPLETED	0	0

Reasons for withdrawal

Withdrawal data not reported

Baseline Characteristics

Trial of Simulation-based Mastery Learning to Communicate Diagnostic Uncertainty

Baseline characteristics by cohort

Baseline characteristics by cohort
Measure	Group A (Intervention Group) n=55 Participants Participants had at least three study visits. The first visit (T1) consisted of a baseline test, during which participants completed a simulated encounter with a standardized patient (SP) to establish a baseline score. Participants in Group A were given access to the intervention. The second visit (T2) was scheduled on average 16-19 weeks after T1 and consisted of another simulated encounter with the SP. Group A returned for a third visit (T3) to provide information about the decay of learned skills and if performance on the task is maintained by a single intervention. Educational Curriculum Intervention: The intervention included: 1) Online web-based education module created with the Rise 360 platform that has ten lessons, providing the learner with an introduction, background, and a map of how to navigate a discharge conversation for patients with diagnostic uncertainty. Core content are presented with multiple interactive components such as flip-cards, multiple-choice, drag-and-drop, sketch videos, and narrated clips. 2) Mobile application game designed to facilitate practice of the content learned in the online curriculum 3) Deliberate practice sessions allowed trainees to practice communication techniques presented within the online module and to receive feedback on their performance. These sessions were conducted with standardized patients via a video platform, during a scheduled appointment.	Group B (Control Group) n=54 Participants Participants had at least three study visits. The first visit (T1) consisted of a baseline test, during which participants completed a simulated encounter with a standardized patient (SP) to establish a baseline score. The second visit (T2) was scheduled on average 16-19 weeks after T1 and consisted of another simulated encounter with the SP; following this session, participants in Group B were introduced to the intervention. The Group B returned for a third visit (T3) to provide information on how the curriculum impacts their score. Educational Curriculum Intervention: The intervention included: 1) Online web-based education module created with the Rise 360 platform that has ten lessons, providing the learner with an introduction, background, and a map of how to navigate a discharge conversation for patients with diagnostic uncertainty. Core content are presented with multiple interactive components such as flip-cards, multiple-choice, drag-and-drop, sketch videos, and narrated clips. 2) Mobile application game designed to facilitate practice of the content learned in the online curriculum 3) Deliberate practice sessions allowed trainees to practice communication techniques presented within the online module and to receive feedback on their performance. These sessions were conducted with standardized patients via a video platform, during a scheduled appointment.	Total n=109 Participants Total of all reporting groups
Age, Categorical <=18 years	0 Participants n=5 Participants	0 Participants n=7 Participants	0 Participants n=5 Participants
Age, Categorical Between 18 and 65 years	55 Participants n=5 Participants	54 Participants n=7 Participants	109 Participants n=5 Participants
Age, Categorical >=65 years	0 Participants n=5 Participants	0 Participants n=7 Participants	0 Participants n=5 Participants
Age, Continuous	29.1 years STANDARD_DEVIATION 2.2 • n=5 Participants	29.3 years STANDARD_DEVIATION 3.2 • n=7 Participants	29.2 years STANDARD_DEVIATION 2.7 • n=5 Participants
Sex: Female, Male Female	18 Participants n=5 Participants	22 Participants n=7 Participants	40 Participants n=5 Participants
Sex: Female, Male Male	37 Participants n=5 Participants	32 Participants n=7 Participants	69 Participants n=5 Participants
Ethnicity (NIH/OMB) Hispanic or Latino	4 Participants n=5 Participants	3 Participants n=7 Participants	7 Participants n=5 Participants
Ethnicity (NIH/OMB) Not Hispanic or Latino	48 Participants n=5 Participants	51 Participants n=7 Participants	99 Participants n=5 Participants
Ethnicity (NIH/OMB) Unknown or Not Reported	3 Participants n=5 Participants	0 Participants n=7 Participants	3 Participants n=5 Participants
Race (NIH/OMB) American Indian or Alaska Native	0 Participants n=5 Participants	0 Participants n=7 Participants	0 Participants n=5 Participants
Race (NIH/OMB) Asian	10 Participants n=5 Participants	8 Participants n=7 Participants	18 Participants n=5 Participants
Race (NIH/OMB) Native Hawaiian or Other Pacific Islander	0 Participants n=5 Participants	0 Participants n=7 Participants	0 Participants n=5 Participants
Race (NIH/OMB) Black or African American	4 Participants n=5 Participants	2 Participants n=7 Participants	6 Participants n=5 Participants
Race (NIH/OMB) White	36 Participants n=5 Participants	42 Participants n=7 Participants	78 Participants n=5 Participants
Race (NIH/OMB) More than one race	3 Participants n=5 Participants	0 Participants n=7 Participants	3 Participants n=5 Participants
Race (NIH/OMB) Unknown or Not Reported	2 Participants n=5 Participants	2 Participants n=7 Participants	4 Participants n=5 Participants

PRIMARY outcome

Timeframe: Although we originally planned for T2 to occur 4-8 weeks after the baseline assessment, T2 ended up taking place 16-19 weeks after baseline assessment due to the residency schedule.

Outcome measures

Outcome measures
Measure	Group A (Intervention Group) n=55 Participants Participants had at least three study visits. The first visit (T1) consisted of a baseline test, during which participants completed a simulated encounter with a standardized patient (SP) to establish a baseline score. Participants in Group A were given access to the intervention. The second visit (T2) was scheduled on average 16-19 weeks after T1 and consisted of another simulated encounter with the SP. Group A returned for a third visit (T3) to provide information about the decay of learned skills and if performance on the task is maintained by a single intervention. Educational Curriculum Intervention: The intervention included: 1) Online web-based education module created with the Rise 360 platform that has ten lessons, providing the learner with an introduction, background, and a map of how to navigate a discharge conversation for patients with diagnostic uncertainty. Core content are presented with multiple interactive components such as flip-cards, multiple-choice, drag-and-drop, sketch videos, and narrated clips. 2) Mobile application game designed to facilitate practice of the content learned in the online curriculum 3) Deliberate practice sessions allowed trainees to practice communication techniques presented within the online module and to receive feedback on their performance. These sessions were conducted with standardized patients via a video platform, during a scheduled appointment.	Group B (Control Group) n=54 Participants Participants had at least three study visits. The first visit (T1) consisted of a baseline test, during which participants completed a simulated encounter with a standardized patient (SP) to establish a baseline score. The second visit (T2) was scheduled on average 16-19 weeks after T1 and consisted of another simulated encounter with the SP; following this session, participants in Group B were introduced to the intervention. The Group B returned for a third visit (T3) to provide information on how the curriculum impacts their score. Educational Curriculum Intervention: The intervention included: 1) Online web-based education module created with the Rise 360 platform that has ten lessons, providing the learner with an introduction, background, and a map of how to navigate a discharge conversation for patients with diagnostic uncertainty. Core content are presented with multiple interactive components such as flip-cards, multiple-choice, drag-and-drop, sketch videos, and narrated clips. 2) Mobile application game designed to facilitate practice of the content learned in the online curriculum 3) Deliberate practice sessions allowed trainees to practice communication techniques presented within the online module and to receive feedback on their performance. These sessions were conducted with standardized patients via a video platform, during a scheduled appointment.
% of Residents Who Pass the UCC at the T2 Assessment	29 Participants	2 Participants

SECONDARY outcome

Timeframe: up to eleven months

Investigators will evaluate the association between number of Deliberate Practice (DP) sessions completed and achieving the Minimum Passing Standard (MPS)

Outcome measures

Outcome measures
Measure	Group A (Intervention Group) n=55 Participants Participants had at least three study visits. The first visit (T1) consisted of a baseline test, during which participants completed a simulated encounter with a standardized patient (SP) to establish a baseline score. Participants in Group A were given access to the intervention. The second visit (T2) was scheduled on average 16-19 weeks after T1 and consisted of another simulated encounter with the SP. Group A returned for a third visit (T3) to provide information about the decay of learned skills and if performance on the task is maintained by a single intervention. Educational Curriculum Intervention: The intervention included: 1) Online web-based education module created with the Rise 360 platform that has ten lessons, providing the learner with an introduction, background, and a map of how to navigate a discharge conversation for patients with diagnostic uncertainty. Core content are presented with multiple interactive components such as flip-cards, multiple-choice, drag-and-drop, sketch videos, and narrated clips. 2) Mobile application game designed to facilitate practice of the content learned in the online curriculum 3) Deliberate practice sessions allowed trainees to practice communication techniques presented within the online module and to receive feedback on their performance. These sessions were conducted with standardized patients via a video platform, during a scheduled appointment.	Group B (Control Group) n=54 Participants Participants had at least three study visits. The first visit (T1) consisted of a baseline test, during which participants completed a simulated encounter with a standardized patient (SP) to establish a baseline score. The second visit (T2) was scheduled on average 16-19 weeks after T1 and consisted of another simulated encounter with the SP; following this session, participants in Group B were introduced to the intervention. The Group B returned for a third visit (T3) to provide information on how the curriculum impacts their score. Educational Curriculum Intervention: The intervention included: 1) Online web-based education module created with the Rise 360 platform that has ten lessons, providing the learner with an introduction, background, and a map of how to navigate a discharge conversation for patients with diagnostic uncertainty. Core content are presented with multiple interactive components such as flip-cards, multiple-choice, drag-and-drop, sketch videos, and narrated clips. 2) Mobile application game designed to facilitate practice of the content learned in the online curriculum 3) Deliberate practice sessions allowed trainees to practice communication techniques presented within the online module and to receive feedback on their performance. These sessions were conducted with standardized patients via a video platform, during a scheduled appointment.
Number of DP Sessions number of deliberate practice sessions	1.7 number of deliberate practice sessions Standard Deviation 0.9	1.5 number of deliberate practice sessions Standard Deviation 0.7
Number of DP Sessions 1st attempt: passed	1.0 number of deliberate practice sessions Standard Deviation 0.0	1.0 number of deliberate practice sessions Standard Deviation 0.0
Number of DP Sessions 1st attempt: failed	2.4 number of deliberate practice sessions Standard Deviation 0.8	2.2 number of deliberate practice sessions Standard Deviation 0.4

SECONDARY outcome

Timeframe: Thus this outcome measure was assessed at T2 (16-19 weeks after baseline assessment) and T3; although we originally planned for T3 to occur 4-8 weeks after T2, T3 ended up taking place 11-15 weeks after T2 due to the residency schedule.

This will assess the change within groups on the percentage pass rate on the Uncertainty Communication Checklist (UCC) from the second assessment (T2) to the third assessment (T3), which will assess retention of mastery in the intervention group and will be a supplemental assessment of the interventions' efficacy in the delayed intervention (control) group.

Outcome measures

Outcome measures
Measure	Group A (Intervention Group) n=55 Participants Participants had at least three study visits. The first visit (T1) consisted of a baseline test, during which participants completed a simulated encounter with a standardized patient (SP) to establish a baseline score. Participants in Group A were given access to the intervention. The second visit (T2) was scheduled on average 16-19 weeks after T1 and consisted of another simulated encounter with the SP. Group A returned for a third visit (T3) to provide information about the decay of learned skills and if performance on the task is maintained by a single intervention. Educational Curriculum Intervention: The intervention included: 1) Online web-based education module created with the Rise 360 platform that has ten lessons, providing the learner with an introduction, background, and a map of how to navigate a discharge conversation for patients with diagnostic uncertainty. Core content are presented with multiple interactive components such as flip-cards, multiple-choice, drag-and-drop, sketch videos, and narrated clips. 2) Mobile application game designed to facilitate practice of the content learned in the online curriculum 3) Deliberate practice sessions allowed trainees to practice communication techniques presented within the online module and to receive feedback on their performance. These sessions were conducted with standardized patients via a video platform, during a scheduled appointment.	Group B (Control Group) n=54 Participants Participants had at least three study visits. The first visit (T1) consisted of a baseline test, during which participants completed a simulated encounter with a standardized patient (SP) to establish a baseline score. The second visit (T2) was scheduled on average 16-19 weeks after T1 and consisted of another simulated encounter with the SP; following this session, participants in Group B were introduced to the intervention. The Group B returned for a third visit (T3) to provide information on how the curriculum impacts their score. Educational Curriculum Intervention: The intervention included: 1) Online web-based education module created with the Rise 360 platform that has ten lessons, providing the learner with an introduction, background, and a map of how to navigate a discharge conversation for patients with diagnostic uncertainty. Core content are presented with multiple interactive components such as flip-cards, multiple-choice, drag-and-drop, sketch videos, and narrated clips. 2) Mobile application game designed to facilitate practice of the content learned in the online curriculum 3) Deliberate practice sessions allowed trainees to practice communication techniques presented within the online module and to receive feedback on their performance. These sessions were conducted with standardized patients via a video platform, during a scheduled appointment.
Change in Pass Rates Within Groups From T2 to T3 Achieved Mastery at T2 (UCC >= 19/21)	29 participants	2 participants
Change in Pass Rates Within Groups From T2 to T3 Achieved Mastery at T3 (UCC >= 19/21)	30 participants	31 participants

Adverse Events

Group A (Intervention Group)

Serious events: 0 serious events

Other events: 0 other events

Deaths: 0 deaths

Group B (Control Group)

Serious events: 0 serious events

Other events: 0 other events

Deaths: 0 deaths

Serious adverse events

Adverse event data not reported

Other adverse events

Adverse event data not reported

Additional Information

Dr. Kristin Rising

Thomas Jefferson University

Phone: 215-503-5507

Email: [email protected]

Results disclosure agreements

Principal investigator is a sponsor employee
Publication restrictions are in place