Trial Outcomes & Findings for CAMCI: Advancing the Use of Computerized Screening in Healthcare (NCT NCT03512301)

Last Updated: 2025-06-26

Results Overview

Comparison (positive and negative percent agreement, confidence interval estimates, and quadratic weighted kappa) between CAMCI and clinical adjudication classifications for the Test group. Power calculations for agreement were assessed for Cohen Kappa's assuming at least 75% Normals with 80% power and .05 significance level, finding that we were sufficiently powered with at least 120 individuals in the test set, for a Kappa of at least 0.3.

Recruitment status

COMPLETED

Study phase

Target enrollment

773 participants

Primary outcome timeframe

baseline

Results posted on

2025-06-26

Participant Flow

* Recruitment occurred at 4 participating sites representing a variety of regions across the US * Participants were recruited from both clinical and community settings. * Participants were required to be at least 60 years of age, have adequate visual and auditory acuity to allow neuropsychological tests * Exclusionary criteria included significant neurologic disease, history of major depression, psychiatric disorder, consuming 5 or more alcoholic drinks per day, MoCA score less than 10

Participant milestones

Participant milestones
Measure	CAMCI Validation Validation of the Computer Assessment of Memory and Cognitive Impairment (CAMCI) against 1) clinical adjudication of traditional neuropsychological tests administered via paper and pencil and 2) the Montreal Cognitive Assessment (MoCA) tool for early detection of mild cognitive impairment (MCI)
Overall Study STARTED	773
Overall Study COMPLETED	764
Overall Study NOT COMPLETED	9

Reasons for withdrawal

Reasons for withdrawal
Measure	CAMCI Validation Validation of the Computer Assessment of Memory and Cognitive Impairment (CAMCI) against 1) clinical adjudication of traditional neuropsychological tests administered via paper and pencil and 2) the Montreal Cognitive Assessment (MoCA) tool for early detection of mild cognitive impairment (MCI)
Overall Study Protocol Violation	2
Overall Study Did Not Qualify	3
Overall Study Age-ineligible	1
Overall Study Inadequate fluency with the English language	3

Baseline Characteristics

The standardized test is provided in two forms that are selected based on participant age and ability. The age groups for this study are adults ages 60 to 69 years old and older adults ages 65 to 90 years old. There were 355 participants assigned to the adult group and 409 participants assigned to the older adult group.

Baseline characteristics by cohort

Baseline characteristics by cohort
Measure	CAMCI Validation n=764 Participants Validation of the Computer Assessment of Memory and Cognitive Impairment (CAMCI) against 1) clinical adjudication of traditional neuropsychological tests administered via paper and pencil and 2) the Montreal Cognitive Assessment (MoCA) tool for early detection of mild cognitive impairment (MCI)
Age, Continuous	71.0 years n=764 Participants
Sex: Female, Male Female	550 Participants n=764 Participants
Sex: Female, Male Male	214 Participants n=764 Participants
Ethnicity (NIH/OMB) Hispanic or Latino	116 Participants n=764 Participants
Ethnicity (NIH/OMB) Not Hispanic or Latino	619 Participants n=764 Participants
Ethnicity (NIH/OMB) Unknown or Not Reported	29 Participants n=764 Participants
Race (NIH/OMB) American Indian or Alaska Native	0 Participants n=764 Participants
Race (NIH/OMB) Asian	21 Participants n=764 Participants
Race (NIH/OMB) Native Hawaiian or Other Pacific Islander	0 Participants n=764 Participants
Race (NIH/OMB) Black or African American	266 Participants n=764 Participants
Race (NIH/OMB) White	434 Participants n=764 Participants
Race (NIH/OMB) More than one race	43 Participants n=764 Participants
Race (NIH/OMB) Unknown or Not Reported	0 Participants n=764 Participants
Region of Enrollment United States	764 participants n=764 Participants
Neuropsychological classification Normal	521 Participants n=764 Participants
Neuropsychological classification Indeterminate	176 Participants n=764 Participants
Neuropsychological classification Impaired	67 Participants n=764 Participants
Montreal Cognitive Assessment (MoCA) Normal (Score Range 26-30)	337 Participants n=764 Participants
Montreal Cognitive Assessment (MoCA) Indeterminate (Score Range 19-25)	388 Participants n=764 Participants
Montreal Cognitive Assessment (MoCA) Impaired (Score Range 10-18)	39 Participants n=764 Participants
CAMCI Normal (Score Range 21-50)	596 Participants n=764 Participants
CAMCI Indeterminate (Score Range 16-20)	96 Participants n=764 Participants
CAMCI Impaired (Score Range 0-15)	72 Participants n=764 Participants
WRAT5 - Word Reading	59.59 points STANDARD_DEVIATION 8.33 • n=764 Participants
WAIS-IV Digit Span Forward Max Span (0-9) - number of digits recalled on the final correct trial	6.23 points STANDARD_DEVIATION 1.30 • n=764 Participants
WAIS-IV Digit Span Forward Score (0-16) - total of points obtained on each trial/span (0,1,2)	9.42 points STANDARD_DEVIATION 2.23 • n=764 Participants
WAIS-IV Digit Span Backward Max Span (0-8) - number of digits recalled on the final correct trial	4.40 points STANDARD_DEVIATION 1.34 • n=764 Participants
WAIS-IV Digit Span Backward Score (0-16) - total of points obtained on each trial/span (0,1,2)	7.77 points STANDARD_DEVIATION 2.36 • n=764 Participants
WMS-IV Logical Memory I - Immediate Recall Story A 1st Immediate Recall (older adult), Score Range (0-14)	8.59 points STANDARD_DEVIATION 2.72 • n=409 Participants • The standardized test is provided in two forms that are selected based on participant age and ability. The age groups for this study are adults ages 60 to 69 years old and older adults ages 65 to 90 years old. There were 355 participants assigned to the adult group and 409 participants assigned to the older adult group.
WMS-IV Logical Memory I - Immediate Recall Story A 2nd Immediate Recall (older adult), Score Range (0-14)	11.16 points STANDARD_DEVIATION 2.94 • n=409 Participants • The standardized test is provided in two forms that are selected based on participant age and ability. The age groups for this study are adults ages 60 to 69 years old and older adults ages 65 to 90 years old. There were 355 participants assigned to the adult group and 409 participants assigned to the older adult group.
WMS-IV Logical Memory I - Immediate Recall Story B Immediate Recall (older adult), Score Range (0-25)	11.52 points STANDARD_DEVIATION 4.68 • n=409 Participants • The standardized test is provided in two forms that are selected based on participant age and ability. The age groups for this study are adults ages 60 to 69 years old and older adults ages 65 to 90 years old. There were 355 participants assigned to the adult group and 409 participants assigned to the older adult group.
WMS-IV Logical Memory I - Immediate Recall Immediate Recall Total (older adult), Score Range (0-53)	31.27 points STANDARD_DEVIATION 8.69 • n=409 Participants • The standardized test is provided in two forms that are selected based on participant age and ability. The age groups for this study are adults ages 60 to 69 years old and older adults ages 65 to 90 years old. There were 355 participants assigned to the adult group and 409 participants assigned to the older adult group.
WMS-IV Logical Memory I - Immediate Recall Story B Immediate Recall (adult), Score Range (0-25)	13.48 points STANDARD_DEVIATION 4.17 • n=355 Participants • The standardized test is provided in two forms that are selected based on participant age and ability. The age groups for this study are adults ages 60 to 69 years old and older adults ages 65 to 90 years old. There were 355 participants assigned to the adult group and 409 participants assigned to the older adult group.
WMS-IV Logical Memory I - Immediate Recall Story C Immediate Recall (adult), Score Range (0-25)	11.84 points STANDARD_DEVIATION 3.86 • n=355 Participants • The standardized test is provided in two forms that are selected based on participant age and ability. The age groups for this study are adults ages 60 to 69 years old and older adults ages 65 to 90 years old. There were 355 participants assigned to the adult group and 409 participants assigned to the older adult group.
WMS-IV Logical Memory I - Immediate Recall Immediate Recall Total (adult), Score Range (0-50)	25.32 points STANDARD_DEVIATION 7.36 • n=355 Participants • The standardized test is provided in two forms that are selected based on participant age and ability. The age groups for this study are adults ages 60 to 69 years old and older adults ages 65 to 90 years old. There were 355 participants assigned to the adult group and 409 participants assigned to the older adult group.
WMS-IV Logical Memory II - Delayed Recall Story A Delayed Recall (older adult), Score Range (0-14)	7.94 points STANDARD_DEVIATION 3.72 • n=409 Participants • The standardized test is provided in two forms that are selected based on participant age and ability. The age groups for this study are adults ages 60 to 69 years old and older adults ages 65 to 90 years old. There were 355 participants assigned to the adult group and 409 participants assigned to the older adult group.
WMS-IV Logical Memory II - Delayed Recall Story B Delayed Recall (older adult), Score Range (0-25)	9.99 points STANDARD_DEVIATION 5.09 • n=409 Participants • The standardized test is provided in two forms that are selected based on participant age and ability. The age groups for this study are adults ages 60 to 69 years old and older adults ages 65 to 90 years old. There were 355 participants assigned to the adult group and 409 participants assigned to the older adult group.
WMS-IV Logical Memory II - Delayed Recall Delayed Recall Total (older adult), Score Range (0-39)	17.93 points STANDARD_DEVIATION 8.10 • n=409 Participants • The standardized test is provided in two forms that are selected based on participant age and ability. The age groups for this study are adults ages 60 to 69 years old and older adults ages 65 to 90 years old. There were 355 participants assigned to the adult group and 409 participants assigned to the older adult group.
WMS-IV Logical Memory II - Delayed Recall Story B Delayed Recall (adult), Score Range (0-25)	11.57 points STANDARD_DEVIATION 4.69 • n=355 Participants • The standardized test is provided in two forms that are selected based on participant age and ability. The age groups for this study are adults ages 60 to 69 years old and older adults ages 65 to 90 years old. There were 355 participants assigned to the adult group and 409 participants assigned to the older adult group.
WMS-IV Logical Memory II - Delayed Recall Story C Delayed Recall (adult), Score Range (0-25)	10.37 points STANDARD_DEVIATION 4.28 • n=355 Participants • The standardized test is provided in two forms that are selected based on participant age and ability. The age groups for this study are adults ages 60 to 69 years old and older adults ages 65 to 90 years old. There were 355 participants assigned to the adult group and 409 participants assigned to the older adult group.
WMS-IV Logical Memory II - Delayed Recall Delayed Recall Total (adult), Score Range (0-50)	21.93 points STANDARD_DEVIATION 8.27 • n=355 Participants • The standardized test is provided in two forms that are selected based on participant age and ability. The age groups for this study are adults ages 60 to 69 years old and older adults ages 65 to 90 years old. There were 355 participants assigned to the adult group and 409 participants assigned to the older adult group.
Rey-Osterrieth Complex Figure Test (ROCF) Figure Copy	28.31 points STANDARD_DEVIATION 5.61 • n=764 Participants
Rey-Osterrieth Complex Figure Test (ROCF) Immediate Recall - Short Delay (3 minutes)	13.24 points STANDARD_DEVIATION 5.99 • n=764 Participants
Rey-Osterrieth Complex Figure Test (ROCF) Delayed Recall - Long Delay (30 minutes)	12.99 points STANDARD_DEVIATION 5.98 • n=764 Participants
Letter Fluency (F-A-S) Test Fluency - F	12.52 points STANDARD_DEVIATION 4.49 • n=764 Participants
Letter Fluency (F-A-S) Test Fluency - A	11.17 points STANDARD_DEVIATION 4.67 • n=764 Participants
Letter Fluency (F-A-S) Test Fluency - S	13.61 points STANDARD_DEVIATION 4.92 • n=764 Participants
Letter Fluency (F-A-S) Test Fluency Total	37.29 points STANDARD_DEVIATION 12.70 • n=764 Participants
Semantic Fluency Test (Animals)	18.65 points STANDARD_DEVIATION 5.15 • n=764 Participants
Hopkins Verbal Learning Test - Revised (HVLT-R) - Recall Learning Trial 1 Recall	5.46 points STANDARD_DEVIATION 1.85 • n=764 Participants
Hopkins Verbal Learning Test - Revised (HVLT-R) - Recall Learning Trial 2 Recall	7.68 points STANDARD_DEVIATION 2.16 • n=764 Participants
Hopkins Verbal Learning Test - Revised (HVLT-R) - Recall Learning Trial 3 Recall	8.82 points STANDARD_DEVIATION 2.28 • n=764 Participants
Hopkins Verbal Learning Test - Revised (HVLT-R) - Recall Learning Trials Total Recall	21.96 points STANDARD_DEVIATION 5.70 • n=764 Participants
Hopkins Verbal Learning Test - Revised (HVLT-R) - Recall Delayed Recall (20 minutes)	7.25 points STANDARD_DEVIATION 3.21 • n=764 Participants
Hopkins Verbal Learning Test - Revised (HVLT-R) - Retention %	77.77 percentage STANDARD_DEVIATION 28.47 • n=764 Participants
Trail Making Test - Score Part A Score	24.00 points STANDARD_DEVIATION 0.05 • n=764 Participants
Trail Making Test - Score Part B Score	23.54 points STANDARD_DEVIATION 2.45 • n=764 Participants
Hopkins Verbal Learning Test - Revised (HVLT-R) - Recognition Recognition - True Positives	10.94 count STANDARD_DEVIATION 1.45 • n=764 Participants
Hopkins Verbal Learning Test - Revised (HVLT-R) - Recognition Recognition - False Positives	1.42 count STANDARD_DEVIATION 1.61 • n=764 Participants
Hopkins Verbal Learning Test - Revised (HVLT-R) - Recognition Recognition Discrimination Index	9.53 count STANDARD_DEVIATION 2.30 • n=764 Participants
Trail Making Test - Completion Time Part A Time (150 seconds max)	37.28 time in seconds STANDARD_DEVIATION 18.79 • n=764 Participants
Trail Making Test - Completion Time Part B Time (300 seconds max)	106.16 time in seconds STANDARD_DEVIATION 63.65 • n=764 Participants
WAIS-IV Digit Symbol Substitution Test (DSST)	52.36 points STANDARD_DEVIATION 15.45 • n=764 Participants
Boston Naming Test	52.20 points STANDARD_DEVIATION 7.36 • n=764 Participants

PRIMARY outcome

Timeframe: baseline

Population: Test group. Of 773 participants enrolled, 9 were excluded, leaving 764 for analysis. Participants who completed CAMCI version A at baseline (N=680) were assigned through stratified random sampling to Training (75%) or Test (25%) groups, balanced by age, education, race, ethnicity, gender, and cognitive classification. The groups were further adjusted to match the racial and ethnic distribution in the U.S., resulting in a Training group (N=374) and a Test group (N=126).

Outcome measures

Outcome measures
Measure	Clinical Adjudication - Normal n=87 Participants Participants classified as Normal through clinical adjudication.	Clinical Adjudication - Indeterminate n=33 Participants Participants classified as Indeterminate through clinical adjudication.	Clinical Adjudication - Impaired n=6 Participants Participants classified as Impaired through clinical adjudication.
Agreement Analysis (Agreement to Reference Standard) - Test Group CAMCI - Normal (Scores 21-50)	76 Participants	18 Participants	2 Participants
Agreement Analysis (Agreement to Reference Standard) - Test Group CAMCI - Indeterminate (Sores 16-20)	8 Participants	4 Participants	1 Participants
Agreement Analysis (Agreement to Reference Standard) - Test Group CAMCI - Impaired (Scores 0-15)	3 Participants	11 Participants	3 Participants

PRIMARY outcome

Timeframe: baseline

Linear regression agreement analysis (linear regression equation and confidence intervals, and Pearson correlation) between CAMCI and Montreal Cognitive Assessment (MoCA) for the Test group. Comparison (positive and negative percent agreement, confidence interval estimates, and quadratic weighted kappa) between CAMCI and Montreal Cognitive Assessment (MoCA) for the Test group. CAMCI Scores ranged from 0-50, higher scores mean better outcome. MoCA scores ranged from 0-30, higher scores mean better outcome. Power calculations for agreement were assessed for Cohen Kappa's assuming at least 75% Normals with 80% power and .05 significance level, finding that we were sufficiently powered with at least 120 individuals in the test set, for a Kappa of at least 0.3.

Outcome measures

Outcome measures
Measure	Clinical Adjudication - Normal n=57 Participants Participants classified as Normal through clinical adjudication.	Clinical Adjudication - Indeterminate n=63 Participants Participants classified as Indeterminate through clinical adjudication.	Clinical Adjudication - Impaired n=6 Participants Participants classified as Impaired through clinical adjudication.
Agreement Analysis (Agreement to Non-Reference Standard) - Test Group CAMCI - Normal	54 Participants	40 Participants	2 Participants
Agreement Analysis (Agreement to Non-Reference Standard) - Test Group CAMCI - Indeterminate	1 Participants	12 Participants	0 Participants
Agreement Analysis (Agreement to Non-Reference Standard) - Test Group CAMCI - Impaired	2 Participants	11 Participants	4 Participants

OTHER_PRE_SPECIFIED outcome

Timeframe: baseline, 6 months, 12 months, 24 months post baseline

Population: Individuals who received a consistent version of CAMCI across all visits (i.e., CAMCI-A), and were classified as Normal through clinical adjudication at all visits.

To derive a measure of reliable change that addressed unbalanced repeated measures and unequal timing of repeat measures, we built a two-stage hierarchical model for longitudinal data using Bayesian methods. Additionally, the standard deviation (SD) was allowed to change over time, which allowed for computation of an intraclass correlation coefficient (ICC), a measure of reliability. With an SD and ICC that can change over time, reliable change was computed for follow-up months from 3 to 24, in groups of 3. Reliable change was then adjusted for practice effects according to Duff, K. (2012). Computer Assessment of Memory and Cognitive Impairment (CAMCI) scores were calculated as the weighted sum of individual task scores converted to a total score from 0-worst to 50-best. As the intended use of CAMCI is to provide a single score of overall cognitive status, individual task scores were not reported.

Outcome measures

Outcome measures
Measure	Clinical Adjudication - Normal n=137 Participants Participants classified as Normal through clinical adjudication.	Clinical Adjudication - Indeterminate Participants classified as Indeterminate through clinical adjudication.	Clinical Adjudication - Impaired Participants classified as Impaired through clinical adjudication.
Change Metric (Measure of Significant Change) 0-6 Months	11.33 score on a scale Interval 9.81 to 13.1	—	—
Change Metric (Measure of Significant Change) 0-12 Months	11.32 score on a scale Interval 9.79 to 13.1	—	—
Change Metric (Measure of Significant Change) 6-24 Months	11.38 score on a scale Interval 9.95 to 13.12	—	—
Change Metric (Measure of Significant Change) 12-24 Months	11.42 score on a scale Interval 10.0 to 13.18	—	—

Adverse Events

CAMCI Validation

Serious events: 6 serious events

Other events: 1 other events

Deaths: 5 deaths

Serious adverse events

Serious adverse events
Measure	CAMCI Validation n=773 participants at risk Computerized and paper-pencil neuropsychological tests, baseline CAMCI: CAMCI battery of computerized tasks
General disorders Death unrelated to intervention reported by caregiver or test site	0.65% 5/773 • 24 months Participants were seen at baseline only, at baseline-6 months-12 months, or at baseline-12 months-24 months. Adverse events were collected from the time a participant was enrolled until scheduled sessions were completed according to the participant's assigned group (max 24 months). COVID interruptions and project extensions, the study lasted 4 years, 4 months. Adverse events were reviewed by the Data Safety Monitoring Board as appropriate during the entire study period.
General disorders Withdrawal due to health problems unrelated to intervention	0.13% 1/773 • 24 months Participants were seen at baseline only, at baseline-6 months-12 months, or at baseline-12 months-24 months. Adverse events were collected from the time a participant was enrolled until scheduled sessions were completed according to the participant's assigned group (max 24 months). COVID interruptions and project extensions, the study lasted 4 years, 4 months. Adverse events were reviewed by the Data Safety Monitoring Board as appropriate during the entire study period.

Other adverse events

Other adverse events
Measure	CAMCI Validation n=773 participants at risk Computerized and paper-pencil neuropsychological tests, baseline CAMCI: CAMCI battery of computerized tasks
General disorders Strain	0.13% 1/773 • 24 months Participants were seen at baseline only, at baseline-6 months-12 months, or at baseline-12 months-24 months. Adverse events were collected from the time a participant was enrolled until scheduled sessions were completed according to the participant's assigned group (max 24 months). COVID interruptions and project extensions, the study lasted 4 years, 4 months. Adverse events were reviewed by the Data Safety Monitoring Board as appropriate during the entire study period.

Additional Information

Anthony Zuccolotto

Psychology Software Tools, Inc.

Phone: 412-449-0078

Email: [email protected]

Results disclosure agreements

Principal investigator is a sponsor employee
Publication restrictions are in place