Community-Based Care for Minority Adolescents With ADHD: Improving Fidelity With Machine Learning-Assisted Supervision and Fidelity Feedback.

NCT ID: NCT05135065

Last Updated: 2021-11-26

Basic Information

• Recruitment Status: UNKNOWN
• Clinical Phase: NA
• Total Enrollment: 72 participants
• Study Classification: INTERVENTIONAL
• Study Start Date: 2021-11-18
• Study Completion Date: 2022-12-01

Brief Summary

This project proposes to reduce disparities in care among disadvantaged racial/ethnic minority adolescents with ADHD by improving community therapist fidelity to evidence-based behavior therapy through a technology-assisted supervision intervention. In Y01, the research team will work with stakeholders to develop the proposed supervision intervention utilizing two novel technologies: Lyssn + Care4 (LC4S). In Y02, a preliminary clinical trial (N=72) will be conducted in three community mental health agencies in Miami, FL. Adolescent participants will be randomly assigned to receive supervision from a therapist who is trained in LCS4 or provides enhanced supervision as usual(ESAU)using a permuted block randomization strategy that randomizes within site. There will also be double randomization of agency therapists to supervisors. Supervisors will deliver both conditions and investigators will test for contamination to determine the integrity of this design prior to a future R01 that measures patient outcomes. Data from therapists, adolescents and their parents, and supervisors will be collected pre-training, post-training, weekly during service delivery, at EBT completion, and at the end of the trial. The proximal intervention target is therapist fidelity to EBT and the distal targets are service delivery outcomes that include quality, quantity, and speed of delivery. Investigators will also measure indices of consumer fit: cost, acceptability, feasibility, and fidelity to supervision procedures. Sources of data will be audio recorded therapy and supervision sessions, therapist and supervisor report, and project and electronic health records. In longitudinal analyses, time will be modeled as a person-specific variable representing months since baseline. Investigators will nest adolescents within therapists for all analyses.

Detailed Description

Y02 of this study is a small (N=72) phase 1/phase 2 clinical trial of a supervision intervention designed to improve therapist treatment fidelity and subsequent service delivery outcomes. The parallel design includes random assignment of eligible and consenting patients at three community agencies to two active supervision intervention arms (LC4S or ESAU) using a permuted block randomization strategy that accounts for agency. Participants will receive behavioral interventions from community agency staff and their service utilization will be tracked using project and agency electronic health records. Agency therapists and supervisors will also be participants in this trial. Therapists will be randomized to receive either LC4S or ESAU from their supervisor (double randomization) using a block randomization strategy that accounts for site. Supervisors will administer both supervision conditions in this trial; however, investigators will systematically assess for contamination to assess whether this design is appropriate for a future R01.

To minimize bias, adolescent and parent participants will not be informed of the group to which they have been assigned. However, full masking of therapists and supervisors is not feasible in this trial because both supervisors and therapists will know whether they are participating in technology-assisted supervision activities or standard supervision activities due to the nature of these conditions. However, the primary investigator will blind therapists and supervisors to our study hypotheses and the nature of outcome measures to minimize bias in the trial. Many of these measures will be observational and objective (i.e., therapy records, therapy audio recordings), which should reduce bias stemming from self-reports. Investigators will also assess whether there are group effects on therapist accuracy of self-report. All interventions will be delivered by agency staff, who will not be required to follow intervention delivery protocols because an outcome of this study is the extent to which agency therapists follow intervention procedures with guidance provided from their supervisors. Study assessments will be administered electronically via Care4 and data collection will be oversee by study staff.

Each measure of fidelity will be analyzed using a separate mixed /growth model (Duncan et al., 1999). In this design, treatment sessions (level 1) are nested within adolescents (level 2), which are nested within therapists (level 3). Each adolescent attends up to 10 sessions; each therapist treats 3 adolescents. Supervisors serve as an additional higher level, but with so few supervisors (approximately 6), this will be addressed by including dummy predictor variables representing the supervisors. The direct effect of LC4S vs. ESAU on fidelity intercept and slope will be tested for each fidelity outcome (see Table 3). With time centered at the first session, the intercept reflects initial fidelity for the ESAU condition, the group effect reflects the initial fidelity difference between ESAU and LC4S, the time effect reflects the linear change in fidelity over time for the ESAU condition, and the interaction of time and group reflects the difference between ESAU and LC4S in linear change in fidelity over time. As part of the R34, in will estimate the intraclass correlation (ICC) and design effect for the clustering effect of therapist and supervisor on outcome to determine the extent to which additional clustering will be needed in a future R01. ICC ranges from 0 to 1, with larger values reflecting a larger proportion of variance at the higher levels (here, therapist and supervisor rather than adolescent). Investigators will also test both linear and non-linear slopes to ascertain the expected shape of the LGCA in a future R01. Time to therapist MI competence, proportion of EBT delivered by 10th session, and number of sessions and days to completion of the EBT will be modeled using regression (linear, logistic, or Poisson depending on the distribution of the resulting variables) with group as a predictor. Accuracy of therapist self-report will be analyzed using polynomial regression (Laird & LaFleur, 2013).

Using the R package powerlmm, investigators have an estimate of .8 power to detect large effects (d = 0.8) representing group differences at the adolescent (level 2) level; investigators have .4 power to detect medium effects (d = .5) representing group differences at the adolescent level. Our hypotheses, however, are primarily at the level of the therapist (level 3), which has fewer units (i.e., 12 therapists per condition versus 36 adolescents per condition). As such, measures of both adolescent-level and therapist-level effects will be estimated. For power for the ICC calculation, investigators used the ICC.Sample.Size R package based on Zou (2012). With a sample size of 72 participants, 10 observations per participant, alpha equal to .05, and a two tailed test, investigators have greater than .9 power detect an ICC of 0.2. ICC values for cross-sectional data such as children within classrooms are approximately 0.2 (Hedges & Hedberg, 2007). Investigators expect excellent precision for estimating ICC values in this study.

Conditions

Attention Deficit Hyperactivity Disorder

Keywords

ADHD; Adolescents; Community Mental Health

Study Design

• Allocation Method: RANDOMIZED
• Intervention Model: PARALLEL
• Primary Study Purpose: HEALTH_SERVICES_RESEARCH
• Blinding Strategy: SINGLE

Study Groups

Artificial Intelligence-Assisted Supervision Protocol

Measurement-based supervision protocol that incorporates fidelity measurement from a machine learning tool and feedback reports from this tool into a standardized supervision protocol for behavior therapy to task-shift burdensome supervision tasks to a machine, reducing costs and improving precision of fidelity measurement for agencies.

Group Type: EXPERIMENTAL

Artificial Intelligence-Assisted Supervision Protocol

Intervention Type: OTHER

Measurement-based supervision protocol that incorporates fidelity measurement from a machine learning tool and feedback reports from this tool into a standardized supervision protocol for behavior therapy to task-shift burdensome supervision tasks to a machine, reducing costs and improving precision of fidelity measurement for agencies.

Enhanced Supervision as Usual (ESAU) Condition

ESAU therapists will be given standard, paper-based facilitation resources for STAND and will receive 4 hours of training on how to navigate these materials and self-assess fidelity. ESAU therapists will also be trained how to upload recordings into Care4 and complete self-assessments for each session. Supervisors will be given access to these data and recordings once uploaded (but not Lyssn scores or electronic facilitation resources).

Group Type: NO_INTERVENTION

No interventions assigned to this group

Interventions

Artificial Intelligence-Assisted Supervision Protocol

Measurement-based supervision protocol that incorporates fidelity measurement from a machine learning tool and feedback reports from this tool into a standardized supervision protocol for behavior therapy to task-shift burdensome supervision tasks to a machine, reducing costs and improving precision of fidelity measurement for agencies.

Intervention Type: OTHER

Eligibility Criteria

Inclusion Criteria

• DSM-5 ADHD diagnosis, Enrollment in 6th-12th grade, IQ greater than 70, no history of autism spectrum disorder or thought disorder, client of one of two community mental health agencies

Exclusion Criteria

• Autism Spectrum/Thought Disorders, IQ<70

• Minimum Eligible Age: 11 Years
• Maximum Eligible Age: 17 Years
• Eligible Sex: ALL
• Accepts Healthy Volunteers: No

Sponsors

Florida International University

OTHER

Sponsor Role: collaborator

Seattle Children's Hospital

OTHER

Sponsor Role: lead

Responsible Party

Margaret Sibley

Principal Investigator

Responsibility Role: PRINCIPAL_INVESTIGATOR

Principal Investigators

Margaret H Sibley, Ph.D

Role: PRINCIPAL_INVESTIGATOR

Seattle Children's Hospital

Central Contacts

Mercedes Ortiz, B.A

Role: CONTACT

Phone: 206-884-8260

Email: mercedes.ortizrodriguez@seattlechildrens.org

Other Identifiers

R34MH125037

Identifier Type: NIH

Identifier Source: org_study_id

View Link