Using Explainable AI Risk Predictions to Nudge Influenza Vaccine Uptake

NCT ID: NCT05009251

Last Updated: 2025-01-03

Study Results

Results available

Outcome measurements, participant flow, baseline characteristics, and adverse events have been published for this study.

View full results

Basic Information

Get a concise snapshot of the trial, including recruitment status, study phase, enrollment targets, and key timeline milestones.

Recruitment Status

COMPLETED

Clinical Phase

NA

Total Enrollment

45061 participants

Study Classification

INTERVENTIONAL

Study Start Date

2021-09-09

Study Completion Date

2022-07-31

Brief Summary

Review the sponsor-provided synopsis that highlights what the study is about and why it is being conducted.

The study team previously demonstrated that patients are more likely to receive flu vaccine after learning that they are at high risk for flu complications. Building on this past work, the present study will explore whether providing reasons that patients are considered high risk for flu complications (a) further increases the likelihood they will receive flu vaccine and (b) decreases the likelihood that they receive diagnoses of flu and/or flu-like symptoms in the ensuing flu season. It will also examine whether informing patients that their high-risk status was determined by analyzing their medical records or by an artificial intelligence (AI) / machine-learning (ML) algorithm analyzing their medical records will affect the likelihood of receiving the flu vaccine or diagnoses of flu and/or flu-like symptoms.

Detailed Description

Dive into the extended narrative that explains the scientific background, objectives, and procedures in greater depth.

Geisinger has partnered with Medial EarlySign and developed an ML algorithm to identify patients at risk for serious (moderate to severe) flu-associated complications on the basis of their existing electronic health record (EHR) data. Geisinger will apply this algorithm to current patients during the 2021-22 flu season.

This study will evaluate the effect of contacting patients identified as high risk with special messages to encourage vaccination. These communications will inform patients they are at high risk with either (a) no additional explanation, (b) an explanation that this determination comes from an analysis of their medical records, along with a short list of the top factors from their medical record that explain their risk, and (c) the additional explanation that an AI or ML algorithm made this determination, along with a short list of the top factors from their medical record that explain their risk.

Included in the study will be current Geisinger patients 18+ years of age with no contraindications for flu vaccine and who have been assessed by the Medial algorithm and assigned a risk score. The primary study outcomes will be the rates of flu vaccination and flu diagnosis during the 2020-21 season by targeted patients.

Conditions

See the medical conditions and disease areas that this research is targeting or investigating.

Influenza Vaccination Health Promotion Health Behavior Risk Reduction

Study Design

Understand how the trial is structured, including allocation methods, masking strategies, primary purpose, and other design elements.

Allocation Method

RANDOMIZED

Intervention Model

PARALLEL

Patients from the high-risk sample will be randomly assigned to one of five arms:

* No-Contact Control Arm
* Reminder Control Arm
* High Risk Only Arm
* High Risk with Explanation Based on Medical Records Arm
* High Risk with Explanation Based on Algorithm Arm
Primary Study Purpose

PREVENTION

Blinding Strategy

SINGLE

Caregivers
Providers who prescribe vaccination and diagnose conditions will not be randomized to study arms or informed of patient assignment. Although patients will not be explicitly informed which arm they have been randomized to, they will be aware of the messages they receive.

Study Groups

Review each arm or cohort in the study, along with the interventions and objectives associated with them.

No-Contact Control

Subjects in the no-contact control arm will receive no additional pro-vaccination intervention beyond the health system's normal efforts. Although some patients are currently targeted for flu vaccination encouragement due to a conventional non-ML assessment that they are at high risk for complications, these patients are not told that they are at high risk or that they have been targeted.

Group Type NO_INTERVENTION

No interventions assigned to this group

Reminder Control

Subjects in the reminder control arm will receive messages reminding them to get the flu shot without being advised of their risk status.

Group Type EXPERIMENTAL

Reminder

Intervention Type BEHAVIORAL

Mailed letter, short message service (SMS) text, and/or patient portal message

High Risk Only

Subjects in this treatment arm will receive messages telling them they have been identified to be at high risk for flu complications, without specifying how or why the health system believes this to be the case.

Group Type EXPERIMENTAL

Reminder

Intervention Type BEHAVIORAL

Mailed letter, short message service (SMS) text, and/or patient portal message

Risk reduction

Intervention Type BEHAVIORAL

Mailed letter, SMS, and/or patient portal message

High Risk with Explanation Based on Medical Records

Subjects in this treatment arm will receive messages telling them they have been identified to be at high risk for flu complications via review of their medical records and will be provided a human-understandable short list of the top factors from their medical record that explain their risk.

Group Type EXPERIMENTAL

Reminder

Intervention Type BEHAVIORAL

Mailed letter, short message service (SMS) text, and/or patient portal message

Risk reduction

Intervention Type BEHAVIORAL

Mailed letter, SMS, and/or patient portal message

Medical records-based recommendation

Intervention Type BEHAVIORAL

Mailed letter, SMS, and/or patient portal message

High Risk with Explanation Based on Algorithm

Subjects in this treatment arm will receive messages telling them they have been identified to be at high risk for flu complications via analysis of their medical records by a computer algorithm and will be provided a human-understandable short list of the top factors from their medical record that explain their risk.

Group Type EXPERIMENTAL

Reminder

Intervention Type BEHAVIORAL

Mailed letter, short message service (SMS) text, and/or patient portal message

Risk reduction

Intervention Type BEHAVIORAL

Mailed letter, SMS, and/or patient portal message

Medical records-based recommendation

Intervention Type BEHAVIORAL

Mailed letter, SMS, and/or patient portal message

Algorithm-based recommendation

Intervention Type BEHAVIORAL

Mailed letter, SMS, and/or patient portal message

Interventions

Learn about the drugs, procedures, or behavioral strategies being tested and how they are applied within this trial.

Reminder

Mailed letter, short message service (SMS) text, and/or patient portal message

Intervention Type BEHAVIORAL

Risk reduction

Mailed letter, SMS, and/or patient portal message

Intervention Type BEHAVIORAL

Medical records-based recommendation

Mailed letter, SMS, and/or patient portal message

Intervention Type BEHAVIORAL

Algorithm-based recommendation

Mailed letter, SMS, and/or patient portal message

Intervention Type BEHAVIORAL

Other Intervention Names

Discover alternative or legacy names that may be used to describe the listed interventions across different sources.

Credibility Credibility

Eligibility Criteria

Check the participation requirements, including inclusion and exclusion rules, age limits, and whether healthy volunteers are accepted.

Inclusion Criteria

* Aged 18 or older
* Current Geisinger patient at the time of study
* Falls in the top 10% of patients at highest risk, as identified by the flu-complication risk scores of machine learning algorithm (which operates on coded EHR data)

Exclusion Criteria

* Has contraindications for flu vaccination
* Has opted out of receiving communications from Geisinger via all of the modalities being tested
Minimum Eligible Age

18 Years

Eligible Sex

ALL

Accepts Healthy Volunteers

No

Sponsors

Meet the organizations funding or collaborating on the study and learn about their roles.

Geisinger Clinic

OTHER

Sponsor Role collaborator

Massachusetts Institute of Technology

OTHER

Sponsor Role collaborator

National Institute on Aging (NIA)

NIH

Sponsor Role collaborator

National Bureau of Economic Research, Inc.

OTHER

Sponsor Role lead

Responsible Party

Identify the individual or organization who holds primary responsibility for the study information submitted to regulators.

Responsibility Role SPONSOR

Principal Investigators

Learn about the lead researchers overseeing the trial and their institutional affiliations.

Michelle N Meyer, PhD JD

Role: PRINCIPAL_INVESTIGATOR

Geisinger Clinic

Christopher F Chabris, PhD

Role: PRINCIPAL_INVESTIGATOR

Geisinger Clinic

Locations

Explore where the study is taking place and check the recruitment status at each participating site.

Geisinger Clinic

Danville, Pennsylvania, United States

Site Status

Countries

Review the countries where the study has at least one active or historical site.

United States

Provided Documents

Download supplemental materials such as informed consent forms, study protocols, or participant manuals.

Document Type: Study Protocol

View Document

Document Type: Statistical Analysis Plan

View Document

Other Identifiers

Review additional registry numbers or institutional identifiers associated with this trial.

P30AG034532

Identifier Type: NIH

Identifier Source: secondary_id

View Link

2021-0483

Identifier Type: -

Identifier Source: org_study_id

More Related Trials

Additional clinical trials that may be relevant based on similarity analysis.

COVID-19 Messaging for Vaccination
NCT05182554 COMPLETED NA
Promoting Cholesterol Screening
NCT01930149 COMPLETED NA