Trial Outcomes & Findings for Survey Study - Sensitivity Comparison Between MelaFind and Physician Group (NCT NCT01011153)
NCT ID: NCT01011153
Last Updated: 2012-02-14
Results Overview
Sensitivity is the proportion of positive cases (i.e., histologically confirmed melanoma) identified as positive. Specificity is the proportion of negative cases (i.e., histologically confirmed non-melanoma) identified as negative. Because the number of cases given to each dermatologist varied, both sensitivity and specificity were computed for each dermatologist. The primary outcome as stated was to compare the sensitivity and specificity of all dermatologists to that of MelaFind. These metrics, for both the dermatologists and MelaFind, were calculated based on the same 130 lesions.
COMPLETED
241 participants
April 2010
2012-02-14
Participant Flow
Physicians were selected from the membership list of the AAD and lists provided by SK\&A Healthcare Information Solutions. A letter was then sent inviting them to participate. For each who replied positively,an access code was mailed with which to log on to the study. 241 subjects logged into system,183 signed consents \& completed the Intake Survey.
Participants were grouped according to responses to an Intake Survey, completed after Consent and before reading any study cases. A maximum of 130 cases were reviewed,each one consisting of 3 clinical images and a case history. MelaFind results were not provided as part of each case and were only used for sensitivity and specificity calculations.
Participant milestones
| Measure |
General Dermatologists
Dermatologists were defined as board-certified dermatologists who did not participate in previous EOS Protocols 20061 and 20081
|
Pigmented Skin Lesion Experts
Pigmented Skin Lesion Expert were defined as board-certified dermatologists who spend at least 25% of their practice time evaluating pigmented skin lesions (PSLs)
|
Primary Care Physicians
Primary Care Physicians(PCPs) were defined as physicians who deliver primary care service to adult patients (e.g., internists, general practitioners, family practitioners, and geriatricians).
|
|---|---|---|---|
|
Overall Study
STARTED
|
54
|
75
|
54
|
|
Overall Study
COMPLETED
|
46
|
64
|
45
|
|
Overall Study
NOT COMPLETED
|
8
|
11
|
9
|
Reasons for withdrawal
| Measure |
General Dermatologists
Dermatologists were defined as board-certified dermatologists who did not participate in previous EOS Protocols 20061 and 20081
|
Pigmented Skin Lesion Experts
Pigmented Skin Lesion Expert were defined as board-certified dermatologists who spend at least 25% of their practice time evaluating pigmented skin lesions (PSLs)
|
Primary Care Physicians
Primary Care Physicians(PCPs) were defined as physicians who deliver primary care service to adult patients (e.g., internists, general practitioners, family practitioners, and geriatricians).
|
|---|---|---|---|
|
Overall Study
Less than 78 cases completed
|
8
|
11
|
9
|
Baseline Characteristics
Survey Study - Sensitivity Comparison Between MelaFind and Physician Group
Baseline characteristics by cohort
| Measure |
General Dermatologists
n=54 Participants
Dermatologists were defined as board-certified dermatologists who did not participate in previous EOS Protocols 20061 and 20081
|
Pigmented Skin Lesion Experts
n=75 Participants
Pigmented Skin Lesion Expert were defined as board-certified dermatologists who spend at least 25% of their practice time evaluating pigmented skin lesions (PSLs)
|
Primary Care Physicians
n=54 Participants
Primary Care Physicians(PCPs) were defined as physicians who deliver primary care service to adult patients (e.g., internists, general practitioners, family practitioners, and geriatricians).
|
Total
n=183 Participants
Total of all reporting groups
|
|---|---|---|---|---|
|
Age, Categorical
<=18 years
|
0 Participants
n=5 Participants
|
0 Participants
n=7 Participants
|
0 Participants
n=5 Participants
|
0 Participants
n=4 Participants
|
|
Age, Categorical
Between 18 and 65 years
|
54 Participants
n=5 Participants
|
75 Participants
n=7 Participants
|
54 Participants
n=5 Participants
|
183 Participants
n=4 Participants
|
|
Age, Categorical
>=65 years
|
0 Participants
n=5 Participants
|
0 Participants
n=7 Participants
|
0 Participants
n=5 Participants
|
0 Participants
n=4 Participants
|
|
Sex/Gender, Customized
Female
|
16 participants
n=5 Participants
|
36 participants
n=7 Participants
|
23 participants
n=5 Participants
|
75 participants
n=4 Participants
|
|
Sex/Gender, Customized
Male
|
32 participants
n=5 Participants
|
35 participants
n=7 Participants
|
26 participants
n=5 Participants
|
93 participants
n=4 Participants
|
|
Sex/Gender, Customized
Unknown
|
6 participants
n=5 Participants
|
4 participants
n=7 Participants
|
5 participants
n=5 Participants
|
15 participants
n=4 Participants
|
|
Region of Enrollment
United States
|
54 participants
n=5 Participants
|
75 participants
n=7 Participants
|
54 participants
n=5 Participants
|
183 participants
n=4 Participants
|
PRIMARY outcome
Timeframe: April 2010Population: Participants were invited to enroll in this study via mail. Minimum number of participants was determined by statistician based on power analyses and number of cases completed. The time frame of the study was about 6 months and the comparison between Dermatologists and MelaFind is presented in the statistical analysis section below.
Sensitivity is the proportion of positive cases (i.e., histologically confirmed melanoma) identified as positive. Specificity is the proportion of negative cases (i.e., histologically confirmed non-melanoma) identified as negative. Because the number of cases given to each dermatologist varied, both sensitivity and specificity were computed for each dermatologist. The primary outcome as stated was to compare the sensitivity and specificity of all dermatologists to that of MelaFind. These metrics, for both the dermatologists and MelaFind, were calculated based on the same 130 lesions.
Outcome measures
| Measure |
Dermatologists
n=110 Participants
Dermatologists were defined as board-certified dermatologists who were General Dermatologists and Pigmented Skin Lesion Experts, according to the Intake Survey. Dermatologists in this group did not participate in previous EOS Protocols 20061 and 20081. Each dermatologist was given up to 130 cases (each one consisting of 3 clinical images and a case history) consistinf of 65 positive cases (i.e., histologically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma)
|
MelaFind
n=1 Participants
MelaFind imaged 130 cases which consisted of 65 positive cases (i.e., histolgoically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma).
|
Primary Care Physicians
Physicians who are not Board Certified Dermatologists and Pediatricians. Each PCP was given up to 130 cases (each one consisting of 3 clinical images and a case history) consisting of 65 positive cases (i.e., histologically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma).
|
MelaFind
MelaFind imaged the 130 cases, the 65 positive cases (i.e., histologically confirmed melanoma) and the 65 negative cases (i.e., histologically confirmed non-melanoma). Sensitivity was calculated based on the correct identification of the 65 positive cases and specificity was calculated based on the correct identification of the 65 negative cases.
|
|---|---|---|---|---|
|
Comparison of Biopsy/Referral Sensitivity of MelaFind and Dermatologists (Pigmented Skin Lesion Experts and General Dermatologists)
Proportion of True Negative Cases (Specificity)
|
0.51 Proportion of True Cases
95% Confidence Interval 0 • Interval 0.43 to 0.58
|
0.09 Proportion of True Cases
95% Confidence Interval 0 • Interval 0.04 to 0.19
|
—
|
—
|
|
Comparison of Biopsy/Referral Sensitivity of MelaFind and Dermatologists (Pigmented Skin Lesion Experts and General Dermatologists)
Proportion of True Positive Cases (Sensitivity)
|
0.72 Proportion of True Cases
95% Confidence Interval 8.4 • Interval 0.66 to 0.78
|
0.97 Proportion of True Cases
95% Confidence Interval 0 • Interval 0.9 to 0.99
|
—
|
—
|
SECONDARY outcome
Timeframe: December 2009Population: Each category was diminished after excluded subjects were taken into account. These included subjects who did not complete at least 78 cases, subjects who previously participated in other EOS studies, pediatricians, and a board eligible dermatologist.
Sensitivity is the proportion of positive cases (i.e., histologically confirmed melanoma) identified as positive. Specificity is the proportion of negative cases (i.e., histologically confirmed non-melanoma) identified as negative. Because the number of cases given to each dermatologist varied, both sensitivity and specificity were computed for each dermatologist. The primary outcome as stated was to compare the sensitivity and specificity of each group of physicians to that of Melafind, which is presented in the statistical analysis.
Outcome measures
| Measure |
Dermatologists
n=46 Participants
Dermatologists were defined as board-certified dermatologists who were General Dermatologists and Pigmented Skin Lesion Experts, according to the Intake Survey. Dermatologists in this group did not participate in previous EOS Protocols 20061 and 20081. Each dermatologist was given up to 130 cases (each one consisting of 3 clinical images and a case history) consistinf of 65 positive cases (i.e., histologically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma)
|
MelaFind
n=64 Participants
MelaFind imaged 130 cases which consisted of 65 positive cases (i.e., histolgoically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma).
|
Primary Care Physicians
n=45 Participants
Physicians who are not Board Certified Dermatologists and Pediatricians. Each PCP was given up to 130 cases (each one consisting of 3 clinical images and a case history) consisting of 65 positive cases (i.e., histologically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma).
|
MelaFind
n=1 Participants
MelaFind imaged the 130 cases, the 65 positive cases (i.e., histologically confirmed melanoma) and the 65 negative cases (i.e., histologically confirmed non-melanoma). Sensitivity was calculated based on the correct identification of the 65 positive cases and specificity was calculated based on the correct identification of the 65 negative cases.
|
|---|---|---|---|---|
|
Comparison of Biopsy/Referral Sensitivity and Specificity of MelaFind to the Average of Biopsy/Referral Sensitivity & Specificity in Each of the Three Groups of Physicians: Pigmented Skin Lesion Experts, General Dermatologists, and Primary Care Physicians
Proportion of True Negative Cases (Specificity)
|
0.51 Proportion of True Cases
95% Confidence Interval 0 • Interval 0.43 to 0.6
|
0.50 Proportion of True Cases
95% Confidence Interval 0 • Interval 0.42 to 0.58
|
0.45 Proportion of True Cases
95% Confidence Interval 0 • Interval 0.37 to 0.53
|
0.09 Proportion of True Cases
95% Confidence Interval 0 • Interval 0.04 to 0.19
|
|
Comparison of Biopsy/Referral Sensitivity and Specificity of MelaFind to the Average of Biopsy/Referral Sensitivity & Specificity in Each of the Three Groups of Physicians: Pigmented Skin Lesion Experts, General Dermatologists, and Primary Care Physicians
Proportion of True Positive Cases (Sensitivity)
|
0.73 Proportion of True Cases
95% Confidence Interval 8.7 • Interval 0.67 to 0.8
|
0.71 Proportion of True Cases
95% Confidence Interval 8.1 • Interval 0.65 to 0.77
|
0.71 Proportion of True Cases
95% Confidence Interval 10.9 • Interval 0.64 to 0.78
|
0.97 Proportion of True Cases
95% Confidence Interval 0 • Interval 0.9 to 0.99
|
SECONDARY outcome
Timeframe: December 2009Population: The statistical analysis section contains the Kappa results within Each of the Caregiver Groups
Each physician was given up to 130 cases and asked whether or not they would biopsy the lesion. Interobserver variability was measured via the kappa statistic indicating how well the physicians' answers to that question agreed within each group. Kappa statistics are reported in the statistical analysis. while numbers rep, they dont reflect the sgreement among the subjects
Outcome measures
| Measure |
Dermatologists
n=46 Participants
Dermatologists were defined as board-certified dermatologists who were General Dermatologists and Pigmented Skin Lesion Experts, according to the Intake Survey. Dermatologists in this group did not participate in previous EOS Protocols 20061 and 20081. Each dermatologist was given up to 130 cases (each one consisting of 3 clinical images and a case history) consistinf of 65 positive cases (i.e., histologically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma)
|
MelaFind
n=64 Participants
MelaFind imaged 130 cases which consisted of 65 positive cases (i.e., histolgoically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma).
|
Primary Care Physicians
n=45 Participants
Physicians who are not Board Certified Dermatologists and Pediatricians. Each PCP was given up to 130 cases (each one consisting of 3 clinical images and a case history) consisting of 65 positive cases (i.e., histologically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma).
|
MelaFind
n=155 Participants
MelaFind imaged the 130 cases, the 65 positive cases (i.e., histologically confirmed melanoma) and the 65 negative cases (i.e., histologically confirmed non-melanoma). Sensitivity was calculated based on the correct identification of the 65 positive cases and specificity was calculated based on the correct identification of the 65 negative cases.
|
|---|---|---|---|---|
|
Determine the Interobserver Variability in Each of the Above Metrics Within Each of the Caregiver Groups.
|
5927 Number of Cases
|
8263 Number of Cases
|
5844 Number of Cases
|
20034 Number of Cases
|
SECONDARY outcome
Timeframe: June 2010For each case reviewed, physicians were asked if they thought the lesion was a melanoma (diagnostic sensitivity/specificity) and whether or not they would biopsy or refer the lesion (biopsy/referral sensitivity/specificity. These measurements were compared using areas under the corresponding receiver operating characteristic curves. (see statistical analysis for results) ROC curves (reciver operating curves) are plotted on graphs with an x-axis of sensitivity and a y-axis of 1-specificity.
Outcome measures
| Measure |
Dermatologists
n=46 Participants
Dermatologists were defined as board-certified dermatologists who were General Dermatologists and Pigmented Skin Lesion Experts, according to the Intake Survey. Dermatologists in this group did not participate in previous EOS Protocols 20061 and 20081. Each dermatologist was given up to 130 cases (each one consisting of 3 clinical images and a case history) consistinf of 65 positive cases (i.e., histologically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma)
|
MelaFind
n=64 Participants
MelaFind imaged 130 cases which consisted of 65 positive cases (i.e., histolgoically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma).
|
Primary Care Physicians
n=45 Participants
Physicians who are not Board Certified Dermatologists and Pediatricians. Each PCP was given up to 130 cases (each one consisting of 3 clinical images and a case history) consisting of 65 positive cases (i.e., histologically confirmed melanoma) and 65 negative cases (i.e., histologically confirmed non-melanoma).
|
MelaFind
MelaFind imaged the 130 cases, the 65 positive cases (i.e., histologically confirmed melanoma) and the 65 negative cases (i.e., histologically confirmed non-melanoma). Sensitivity was calculated based on the correct identification of the 65 positive cases and specificity was calculated based on the correct identification of the 65 negative cases.
|
|---|---|---|---|---|
|
To Compare Biopsy/Referral Performance and Diagnostic Performance Using Areas Under the Corresponding Receiver Operating Characteristic (ROC) Curves That Illustrate the Trade-offs Between Sensitivity and Specificity Between Three Groups of Physicians.
Area Under Curve for Biopsy/Referral
|
0.65 Area Under Curve for biopsy/referral
Standard Deviation 0.03
|
0.63 Area Under Curve for biopsy/referral
Standard Deviation 0.03
|
0.59 Area Under Curve for biopsy/referral
Standard Deviation 0.02
|
—
|
|
To Compare Biopsy/Referral Performance and Diagnostic Performance Using Areas Under the Corresponding Receiver Operating Characteristic (ROC) Curves That Illustrate the Trade-offs Between Sensitivity and Specificity Between Three Groups of Physicians.
Area Under Curve for Diagnostic
|
0.68 Area Under Curve for biopsy/referral
Standard Deviation 0.03
|
0.66 Area Under Curve for biopsy/referral
Standard Deviation 0.03
|
0.61 Area Under Curve for biopsy/referral
Standard Deviation 0.03
|
—
|
Adverse Events
General Dermatologists
Pigmented Skin Lesion Experts
Primary Care Physicians
Serious adverse events
Adverse event data not reported
Other adverse events
Adverse event data not reported
Additional Information
Joanna Adrian, Director of Clinical Operatons and Medical Affairs
MELA Sciences, Inc.
Results disclosure agreements
- Principal investigator is a sponsor employee
- Publication restrictions are in place
Restriction type: GT60