Trial Outcomes & Findings for PROJECT 2 EXAMPLE: Feedback X Prevalence Using Dermatology Stimuli (NCT NCT05244122)

Last Updated: 2022-10-26

Results Overview

D' (d-prime) is defined as z-transform of the true positive rate - z-transform of false positive rate. True positive is when you say that a real melanoma is a melanoma. False positive is when you say that a nevis is a melanoma. A correction of 0.5 error is added to avoid calculation problems when z=0 or z=1. D' of zero indicates no ability to discriminate. D' \> zero indicates some ability to discriminate. The change of interest is the D' for Block 2 when it follows Block 1 compared to the D' for Block 2 averaged across all conditions.

Recruitment status

COMPLETED

Study phase

Target enrollment

1121 participants

Primary outcome timeframe

Participants could be in the study for as little as two blocks in one day up to 24 blocks collected over 6 days.

Results posted on

2022-10-26

Participant Flow

This was an online study. Participation was an option listed on the DiagnosUs site for 6 days in June, 2021.

Participant milestones

Participant milestones
Measure	Feedback X Prevalence Using Dermatology Stimuli In this experiment, observers (Os) completed blocks of 80 trials. On each trial, they saw an image of a spot on the skin. They classified this as a melanoma (cancer) or a nevis (benign). Blocks could be of low prevalence (20% cancer cases, 16 images) or high prevalence (50%, 40 images). Os either did received trial by trial "Feedback" about their performance accuracy, or they did not. Thus, there were four types of block. Low prevalence, No Feedback Low prevalence, Feedback High prevalence, No Feedback High prevalence, Feedback Each of these four types of block was made available to Os on each of 6 days. Os could elect to view each of the four blocks each day. Our particular interest was in the effect of performing one block on performance on an immediately subsequent block.
Overall Study STARTED	1121
Overall Study Completed a Full Block of Trials	630
Overall Study COMPLETED	803
Overall Study NOT COMPLETED	318

Reasons for withdrawal

Reasons for withdrawal
Measure	Feedback X Prevalence Using Dermatology Stimuli In this experiment, observers (Os) completed blocks of 80 trials. On each trial, they saw an image of a spot on the skin. They classified this as a melanoma (cancer) or a nevis (benign). Blocks could be of low prevalence (20% cancer cases, 16 images) or high prevalence (50%, 40 images). Os either did received trial by trial "Feedback" about their performance accuracy, or they did not. Thus, there were four types of block. Low prevalence, No Feedback Low prevalence, Feedback High prevalence, No Feedback High prevalence, Feedback Each of these four types of block was made available to Os on each of 6 days. Os could elect to view each of the four blocks each day. Our particular interest was in the effect of performing one block on performance on an immediately subsequent block.
Overall Study Withdrawal by Subject	318

Baseline Characteristics

Race and Ethnicity were not collected from any participant.

Baseline characteristics by cohort

Baseline characteristics by cohort
Measure	Feedback X Prevalence Using Dermatology Stimuli n=803 Participants In this experiment, observers (Os) completed blocks of 80 trials. On each trial, they saw an image of a spot on the skin. They classified this as a melanoma (cancer) or a nevis (benign). Blocks could be of low prevalence (20% cancer cases, 16 images) or high prevalence (50%, 40 images). Os either did received trial by trial "Feedback" about their performance accuracy, or they did not. Thus, there were four types of block. Low prevalence, No Feedback Low prevalence, Feedback High prevalence, No Feedback High prevalence, Feedback Each of these four types of block was made available to Os on each of 6 days. Os could elect to view each of the four blocks each day. Our particular interest was in the effect of performing one block on performance on an immediately subsequent block.
Age, Customized >=18 yrs	803 Participants n=803 Participants
Expertise Medical Student	337 Participants n=803 Participants
Expertise Pre-Med Student	174 Participants n=803 Participants
Expertise No Medical Experience	145 Participants n=803 Participants
Expertise Medical Doctor	54 Participants n=803 Participants
Expertise Other Healthcare Professional	47 Participants n=803 Participants
Expertise Nurse	25 Participants n=803 Participants
Expertise Medical Technician	11 Participants n=803 Participants
Expertise Physician Assistant	10 Participants n=803 Participants

PRIMARY outcome

Timeframe: Participants could be in the study for as little as two blocks in one day up to 24 blocks collected over 6 days.

Population: Blocks having D' \< 0.5 were excluded.

Outcome measures

Outcome measures
Measure	Low Prevalence No Feedback First, Then Low Prevalence No Feedback n=29 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	Low Prevalence No Feedback First, Then Low Prevalence With Feedback n=92 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	Low Prevalence No Feedback First, Then High Prevalence No Feedback n=261 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	Low Prevalence No Feedback First, Then High Prevalence With Feedback n=147 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	Low Prevalence With Feedback First, Then Low Prevalence No Feedback n=140 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	Low Prevalence With Feedback First, Then Low Prevalence With Feedback n=110 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	Low Prevalence With Feedback First, Then High Prevalence No Feedback n=117 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	Low Prevalence With Feedback First, Then High Prevalence With Feedback n=218 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	High Prevalence No Feedback First, Then Low Prevalence No Feedback n=151 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	High Prevalence No Feedback First, Then Low Prevalence With Feedback n=150 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	High Prevalence No Feedback First, Then High Prevalence No Feedback n=51 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	High Prevalence No Feedback First, Then High Prevalence With Feedback n=118 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	High Prevalence With Feedback First, Then Low Prevalence No Feedback n=134 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	High Prevalence With Feedback First, Then Low Prevalence With Feedback n=218 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	High Prevalence With Feedback First, Then High Prevalence No Feedback n=117 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	High Prevalence With Feedback First, Then High Prevalence With Feedback n=27 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.
Change in D' Between Pairs of Blocks.	-0.138 z-score Interval -0.335 to 0.0585	.0754 z-score Interval -0.126 to 0.277	0.023 z-score Interval -0.0459 to 0.092	-.129 z-score Interval -0.232 to -0.0259	-0.0654 z-score Interval -0.169 to 0.0382	-0.139 z-score Interval -0.259 to -0.0192	0.0603 z-score Interval -0.0432 to 0.164	0.0582 z-score Interval -0.0366 to 0.153	0.0792 z-score Interval -0.0177 to 0.176	0.0452 z-score Interval -0.0759 to 0.166	-0.0123 z-score Interval -0.168 to 0.143	0.035 z-score Interval -0.0883 to 0.158	0.00902 z-score Interval -0.107 to 0.125	0.00723 z-score Interval -0.0947 to 0.109	-0.106 z-score Interval -0.202 to -0.0109	0.078 z-score Interval -0.191 to 0.347

PRIMARY outcome

Timeframe: Participants could be in the study for as little as two blocks in one day up to 24 blocks collected over 6 days.

Population: Blocks having D' \< 0.5 were excluded.

Criterion, c, corresponds to the position of the midpoint between the z-transformed probabilities of hits (correct yes responses) and false alarms (incorrect yes responses). It is calculated as -\[z(p(h))+z(p(FA))\]/2. The criterion, c, z-score quantifies the distance away from being unbiased in units of standard deviations. A Z-score of 0 is said to be unbiased. Negative values for c indicate a more relaxed criterion for saying yes. Positive numbers indicate a more strict criterion for saying yes.

Outcome measures

Outcome measures
Measure	Low Prevalence No Feedback First, Then Low Prevalence No Feedback n=29 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	Low Prevalence No Feedback First, Then Low Prevalence With Feedback n=92 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	Low Prevalence No Feedback First, Then High Prevalence No Feedback n=261 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	Low Prevalence No Feedback First, Then High Prevalence With Feedback n=147 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	Low Prevalence With Feedback First, Then Low Prevalence No Feedback n=140 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	Low Prevalence With Feedback First, Then Low Prevalence With Feedback n=110 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	Low Prevalence With Feedback First, Then High Prevalence No Feedback n=117 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	Low Prevalence With Feedback First, Then High Prevalence With Feedback n=218 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	High Prevalence No Feedback First, Then Low Prevalence No Feedback n=151 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	High Prevalence No Feedback First, Then Low Prevalence With Feedback n=150 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	High Prevalence No Feedback First, Then High Prevalence No Feedback n=51 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	High Prevalence No Feedback First, Then High Prevalence With Feedback n=118 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	High Prevalence With Feedback First, Then Low Prevalence No Feedback n=134 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	High Prevalence With Feedback First, Then Low Prevalence With Feedback n=218 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	High Prevalence With Feedback First, Then High Prevalence No Feedback n=117 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.	High Prevalence With Feedback First, Then High Prevalence With Feedback n=27 Participants Low prevalence means 20% (16 total) Melanoma images High prevalence means 50% (40 total) Melanoma images Feedback means participants were told if the response was correct after each trial. No Feedback means that they were not.
Change in Criterion Between Pairs of Blocks.	0.0488 z-score Interval -0.131 to 0.228	0.00147 z-score Interval -0.0734 to 0.0763	-0.026 z-score Interval -0.0813 to 0.0293	-0.0582 z-score Interval -0.11 to -0.00631	0.182 z-score Interval 0.114 to 0.249	0.131 z-score Interval 0.0674 to 0.195	0.236 z-score Interval 0.166 to 0.305	0.0458 z-score Interval 0.0134 to 0.0783	-0.0124 z-score Interval -0.0752 to 0.0503	0.0136 z-score Interval -0.0407 to 0.0678	-0.00091 z-score Interval -0.125 to 0.124	0.0186 z-score Interval -0.0311 to 0.0683	-0.187 z-score Interval -0.276 to -0.0972	-0.0762 z-score Interval -0.119 to -0.0335	-0.177 z-score Interval -0.252 to -0.103	-0.135 z-score Interval -0.244 to -0.026

Adverse Events

Feedback X Prevalence Using Dermatology Stimuli

Serious events: 0 serious events

Other events: 0 other events

Deaths: 0 deaths

Serious adverse events

Adverse event data not reported

Other adverse events

Adverse event data not reported

Additional Information

Jeremy Wolfe, Principle Investigator

BrighamHospital

Phone: 617-851-1166

Email: [email protected]

Results disclosure agreements

Principal investigator is a sponsor employee
Publication restrictions are in place