Trial Outcomes & Findings for Testing Multiple Behavioral Science Strategies to Increase Flu-Shot Rates (NCT NCT04565353)

Last Updated: 2022-02-01

Results Overview

Our field experiment will be conducted with Penn Medicine and Geisinger Health patients via SMS messages sent prior to their first primary care appointment during the study period, hereafter referred to as the "target appointment." The key dependent variable is whether participants receive a flu shot at or before their target appointment (as recorded in their electronic health records). If participants cancel or do not show up for their target appointment after they have been randomized to a treatment and then schedule a new appointment during the study period, their new primary care appointment becomes the target appointment. Participants who have been randomized to a treatment will be counted as not having received a flu shot if they cancel or do not show up for their target appointment and do not schedule a new appointment during the study period. Participants who receive a flu shot before they receive the SMS intervention will be excluded from the analyses.

Recruitment status

COMPLETED

Study phase

Target enrollment

74811 participants

Primary outcome timeframe

The dependent variable will be assessed during a 4-day period after randomization starting three days prior to the participant's target primary care appointment through the date of the target appointment (including the date of the appointment)

Results posted on

2022-02-01

Participant Flow

A small number of patients in each experimental condition did not receive any intervention messages due to technical malfunctions. These patients are excluded from analyses since they were, for all practical purposes, not a part of of our study

Participant milestones

Participant milestones
Measure	Holdout Control Participants will only receive the standard appointment reminders from their providers.	Default Reservations Opt-Out Condition The day before their scheduled appointment, participants receive a text reading: "A flu shot has been reserved for you to receive at your appointment tomorrow. Reply Y if you want this shot held for you, N if you don't." The text will include a picture of a vial that says "Your Flu Shot" on it. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Default Reservations Opt-In Condition The day before their scheduled appointment, participants receive a text reading: "Reply Y if you would like to receive a flu shot at your appointment tomorrow, N if not." The text will include a picture of vial with no text on it. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Intergroup Competition Treatment Condition Three days before their scheduled appointment, participants will receive a text message with information about how much "your region" lagged behind "another region" in flu shot rate last year ("your region" and "another region" will depend on the study site and include realistic flu shot rates based on historical CDC data). Participants will receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Intergroup Competition Control Condition Three days before their scheduled appointment, participants will receive a text message with information about how much "your region" lagged behind the target flu shot rate of 70% last year ("your region" will depend on the study site and include realistic flu shot rates based on historical CDC data). Participants will receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Flu Shot for You Symbolic Condition Three days before their scheduled appointment, participants will receive a text message encouraging them to get a flu shot. They will also be told that they have the opportunity to dedicate getting the flu shot to someone by texting back the initials of an individual. Participants will receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Flu Shot for You Herd Immunity Condition Three days before their scheduled appointment, participants will receive a text message encouraging them to get a flu shot. They will also be told that they have the opportunity of getting the flu shot to protect a vulnerable loved one by texting back the initials of an individual. Participants will receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Flu Shot for You Control Condition Three days before their scheduled appointment, participants will receive a text message encouraging them to get a flu shot. They will also receive a reminder message on the day of the appointment. They will not receive any further information, nor will they be asked to respond with initials of an individual. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Prosocial Condition The day before their scheduled appointment, participants will receive a message describing the pro-social benefits of getting a flu shot, and a reminder to ask for their flu shot. The described pro-social benefits will vary whether they emphasize the COVID-19 pandemic (e.g., protecting loved ones from the risk of hospital-acquired COVID-19 infection) or not (e.g., protecting loved ones from serious complications from the flu). Participants will also receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Self-Oriented Condition The day before their scheduled appointment, participants will receive a message describing the self-oriented benefits of getting a flu shot, and a reminder to ask for their flu shot. The described self-oriented benefits will vary whether they emphasize the COVID-19 pandemic (e.g., protecting oneself from the risk of hospital-acquired COVID-19 infection) or not (e.g., protecting oneself from serious complications from the flu). Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Information Vivid Condition Three days before their scheduled appointment, participants will receive a message inviting them to watch a 2-minute wellness video and answer some related questions, as well as encouragement to get the flu shot. In this condition, the video will contain vivid information about getting the flu. Participants will also receive a reminder message the day before the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Information Basic Condition Three days before their scheduled appointment, participants will receive a message inviting them to watch a 2-minute wellness video and answer some related questions, as well as encouragement to get the flu shot. In this condition, the video will contain basic information about getting the flu. Participants will also receive a reminder message the day before the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Information Control Condition Three days before their scheduled appointment, participants will receive a message inviting them to watch a 2-minute wellness video and answer some related questions, as well as encouragement to get the flu shot. In this condition, the video will contain information about exercising. Participants will also receive a reminder message the day before the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Sharing Humor Condition The day before their scheduled appointment, participants will receive a text message encouraging them to get the flu shot. The message will include a joke about the flu and will encourage participants to share the joke with nurses, doctors, or pharmacists. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	No Humor Condition The day before their scheduled appointment, participants will receive a text message encouraging them to get the flu shot. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Healthy Habits Easy Health Behavior Condition The day before their scheduled appointment, participants will be asked if they have completed a series of easy health behaviors (e.g., whether they walked 500 feet yesterday, at least two serving of fruits and vegetables in the last week, and slept at least 6 hours the previous night). They will then be encouraged to get a flu shot at their appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Healthy Habits Difficult Health Behavior Condition The day before their scheduled appointment, participants will be asked if they have completed a series of difficult health behaviors (e.g., whether they walked 3 miles yesterday, ate 4-6 servings of fruits and vegetables yesterday, and slept at least 9 hours the previous night). They will then be encouraged to get a flu shot at their appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Healthy Habits Control Condition The day before their scheduled appointment, participants will receive a text message encouraging them to get a flu shot. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Just-In-Time Reminders 24-Hour Condition Seventy-two hours before their appointment, participants will receive a text message alerting them to the availability of the flu vaccine at their upcoming appointment. Participants will be told they'll receive a reminder, which they can opt-out from receiving. The text message reminder will be sent 24 hours before their appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Just-In-Time Reminders 15-Minute Condition Just-In-Time Reminders 15-Minute Condition: Seventy-two hours before their appointment, participants will receive a text message alerting them to the availability of the flu vaccine at their upcoming appointment. Participants will be told they'll receive a reminder, which they can opt-out from receiving. The text message reminder will be sent 15 minutes before their appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.
Overall Study STARTED	3742	3737	3709	3735	3758	3769	3719	3732	3728	3777	3672	3751	3737	3714	3773	3739	3776	3744	3741	3758
Overall Study COMPLETED	3742	3688	3670	3693	3717	3716	3665	3693	3662	3734	3623	3710	3685	3664	3736	3690	3734	3696	3698	3712
Overall Study NOT COMPLETED	0	49	39	42	41	53	54	39	66	43	49	41	52	50	37	49	42	48	43	46

Reasons for withdrawal

Withdrawal data not reported

Baseline Characteristics

Testing Multiple Behavioral Science Strategies to Increase Flu-Shot Rates

Baseline characteristics by cohort

Baseline characteristics by cohort
Measure	Holdout Control n=3742 Participants Participants will only receive the standard appointment reminders from their providers.	Default Reservations Opt-Out Condition n=3737 Participants The day before their scheduled appointment, participants receive a text reading: "A flu shot has been reserved for you to receive at your appointment tomorrow. Reply Y if you want this shot held for you, N if you don't." The text will include a picture of a vial that says "Your Flu Shot" on it. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Default Reservations Opt-In Condition n=3709 Participants The day before their scheduled appointment, participants receive a text reading: "Reply Y if you would like to receive a flu shot at your appointment tomorrow, N if not." The text will include a picture of vial with no text on it. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Intergroup Competition Treatment Condition n=3735 Participants Three days before their scheduled appointment, participants will receive a text message with information about how much "your region" lagged behind "another region" in flu shot rate last year ("your region" and "another region" will depend on the study site and include realistic flu shot rates based on historical CDC data). Participants will receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Intergroup Competition Control Condition n=3758 Participants Three days before their scheduled appointment, participants will receive a text message with information about how much "your region" lagged behind the target flu shot rate of 70% last year ("your region" will depend on the study site and include realistic flu shot rates based on historical CDC data). Participants will receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Flu Shot for You Symbolic Condition n=3769 Participants Three days before their scheduled appointment, participants will receive a text message encouraging them to get a flu shot. They will also be told that they have the opportunity to dedicate getting the flu shot to someone by texting back the initials of an individual. Participants will receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Flu Shot for You Herd Immunity Condition n=3719 Participants Three days before their scheduled appointment, participants will receive a text message encouraging them to get a flu shot. They will also be told that they have the opportunity of getting the flu shot to protect a vulnerable loved one by texting back the initials of an individual. Participants will receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Flu Shot for You Control Condition n=3732 Participants Three days before their scheduled appointment, participants will receive a text message encouraging them to get a flu shot. They will also receive a reminder message on the day of the appointment. They will not receive any further information, nor will they be asked to respond with initials of an individual. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Prosocial Condition n=3728 Participants The day before their scheduled appointment, participants will receive a message describing the pro-social benefits of getting a flu shot, and a reminder to ask for their flu shot. The described pro-social benefits will vary whether they emphasize the COVID-19 pandemic (e.g., protecting loved ones from the risk of hospital-acquired COVID-19 infection) or not (e.g., protecting loved ones from serious complications from the flu). Participants will also receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Self-Oriented Condition n=3777 Participants The day before their scheduled appointment, participants will receive a message describing the self-oriented benefits of getting a flu shot, and a reminder to ask for their flu shot. The described self-oriented benefits will vary whether they emphasize the COVID-19 pandemic (e.g., protecting oneself from the risk of hospital-acquired COVID-19 infection) or not (e.g., protecting oneself from serious complications from the flu). Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Information Vivid Condition n=3672 Participants Three days before their scheduled appointment, participants will receive a message inviting them to watch a 2-minute wellness video and answer some related questions, as well as encouragement to get the flu shot. In this condition, the video will contain vivid information about getting the flu. Participants will also receive a reminder message the day before the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Information Basic Condition n=3751 Participants Three days before their scheduled appointment, participants will receive a message inviting them to watch a 2-minute wellness video and answer some related questions, as well as encouragement to get the flu shot. In this condition, the video will contain basic information about getting the flu. Participants will also receive a reminder message the day before the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Information Control Condition n=3737 Participants Three days before their scheduled appointment, participants will receive a message inviting them to watch a 2-minute wellness video and answer some related questions, as well as encouragement to get the flu shot. In this condition, the video will contain information about exercising. Participants will also receive a reminder message the day before the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Sharing Humor Condition n=3714 Participants The day before their scheduled appointment, participants will receive a text message encouraging them to get the flu shot. The message will include a joke about the flu and will encourage participants to share the joke with nurses, doctors, or pharmacists. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	No Humor Condition n=3773 Participants The day before their scheduled appointment, participants will receive a text message encouraging them to get the flu shot. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Healthy Habits Easy Health Behavior Condition n=3739 Participants The day before their scheduled appointment, participants will be asked if they have completed a series of easy health behaviors (e.g., whether they walked 500 feet yesterday, at least two serving of fruits and vegetables in the last week, and slept at least 6 hours the previous night). They will then be encouraged to get a flu shot at their appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Healthy Habits Difficult Health Behavior Condition n=3776 Participants The day before their scheduled appointment, participants will be asked if they have completed a series of difficult health behaviors (e.g., whether they walked 3 miles yesterday, ate 4-6 servings of fruits and vegetables yesterday, and slept at least 9 hours the previous night). They will then be encouraged to get a flu shot at their appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Healthy Habits Control Condition n=3744 Participants The day before their scheduled appointment, participants will receive a text message encouraging them to get a flu shot. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Just-In-Time Reminders 24-Hour Condition n=3741 Participants Seventy-two hours before their appointment, participants will receive a text message alerting them to the availability of the flu vaccine at their upcoming appointment. Participants will be told they'll receive a reminder, which they can opt-out from receiving. The text message reminder will be sent 24 hours before their appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Just-In-Time Reminders 15-Minute Condition n=3758 Participants Just-In-Time Reminders 15-Minute Condition: Seventy-two hours before their appointment, participants will receive a text message alerting them to the availability of the flu vaccine at their upcoming appointment. Participants will be told they'll receive a reminder, which they can opt-out from receiving. The text message reminder will be sent 15 minutes before their appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Total n=74811 Participants Total of all reporting groups
Age, Categorical <=18 years	22 Participants n=5 Participants	24 Participants n=7 Participants	21 Participants n=5 Participants	20 Participants n=4 Participants	31 Participants n=21 Participants	21 Participants n=10 Participants	21 Participants n=115 Participants	15 Participants n=24 Participants	24 Participants n=42 Participants	24 Participants n=42 Participants	22 Participants n=42 Participants	31 Participants n=42 Participants	24 Participants n=36 Participants	14 Participants n=36 Participants	24 Participants n=24 Participants	19 Participants n=135 Participants	16 Participants n=136 Participants	19 Participants n=44 Participants	20 Participants n=667 Participants	28 Participants n=7 Participants	440 Participants n=6 Participants
Age, Categorical Between 18 and 65 years	2950 Participants n=5 Participants	2920 Participants n=7 Participants	2921 Participants n=5 Participants	2940 Participants n=4 Participants	2942 Participants n=21 Participants	2941 Participants n=10 Participants	2916 Participants n=115 Participants	2923 Participants n=24 Participants	2932 Participants n=42 Participants	2955 Participants n=42 Participants	2902 Participants n=42 Participants	2948 Participants n=42 Participants	2961 Participants n=36 Participants	2950 Participants n=36 Participants	2942 Participants n=24 Participants	2927 Participants n=135 Participants	2956 Participants n=136 Participants	2932 Participants n=44 Participants	2947 Participants n=667 Participants	2959 Participants n=7 Participants	58764 Participants n=6 Participants
Age, Categorical >=65 years	770 Participants n=5 Participants	793 Participants n=7 Participants	767 Participants n=5 Participants	775 Participants n=4 Participants	785 Participants n=21 Participants	807 Participants n=10 Participants	782 Participants n=115 Participants	794 Participants n=24 Participants	772 Participants n=42 Participants	798 Participants n=42 Participants	748 Participants n=42 Participants	772 Participants n=42 Participants	752 Participants n=36 Participants	750 Participants n=36 Participants	807 Participants n=24 Participants	793 Participants n=135 Participants	804 Participants n=136 Participants	793 Participants n=44 Participants	774 Participants n=667 Participants	771 Participants n=7 Participants	15607 Participants n=6 Participants
Age, Continuous	50.3 years STANDARD_DEVIATION 16.2 • n=5 Participants	50.7 years STANDARD_DEVIATION 16.4 • n=7 Participants	50.5 years STANDARD_DEVIATION 16.3 • n=5 Participants	50.7 years STANDARD_DEVIATION 16.2 • n=4 Participants	50.5 years STANDARD_DEVIATION 16.2 • n=21 Participants	50.9 years STANDARD_DEVIATION 16.3 • n=10 Participants	50.5 years STANDARD_DEVIATION 16.4 • n=115 Participants	50.8 years STANDARD_DEVIATION 16.6 • n=24 Participants	50.9 years STANDARD_DEVIATION 16.2 • n=42 Participants	50.6 years STANDARD_DEVIATION 16.2 • n=42 Participants	50.8 years STANDARD_DEVIATION 16.1 • n=42 Participants	50.8 years STANDARD_DEVIATION 16.1 • n=42 Participants	50.7 years STANDARD_DEVIATION 16.1 • n=36 Participants	50.5 years STANDARD_DEVIATION 16.2 • n=36 Participants	50.9 years STANDARD_DEVIATION 16.3 • n=24 Participants	51.0 years STANDARD_DEVIATION 16.3 • n=135 Participants	51.0 years STANDARD_DEVIATION 16.0 • n=136 Participants	50.7 years STANDARD_DEVIATION 16.3 • n=44 Participants	50.7 years STANDARD_DEVIATION 16.2 • n=667 Participants	50.5 years STANDARD_DEVIATION 16.0 • n=7 Participants	50.7 years STANDARD_DEVIATION 16.2 • n=6 Participants
Sex: Female, Male Female	2056 Participants n=5 Participants	2109 Participants n=7 Participants	2079 Participants n=5 Participants	2067 Participants n=4 Participants	2096 Participants n=21 Participants	2097 Participants n=10 Participants	2018 Participants n=115 Participants	2091 Participants n=24 Participants	2083 Participants n=42 Participants	2120 Participants n=42 Participants	2030 Participants n=42 Participants	2107 Participants n=42 Participants	2060 Participants n=36 Participants	2104 Participants n=36 Participants	2098 Participants n=24 Participants	2085 Participants n=135 Participants	2159 Participants n=136 Participants	2120 Participants n=44 Participants	2101 Participants n=667 Participants	2091 Participants n=7 Participants	41771 Participants n=6 Participants
Sex: Female, Male Male	1686 Participants n=5 Participants	1628 Participants n=7 Participants	1630 Participants n=5 Participants	1668 Participants n=4 Participants	1662 Participants n=21 Participants	1672 Participants n=10 Participants	1701 Participants n=115 Participants	1641 Participants n=24 Participants	1645 Participants n=42 Participants	1657 Participants n=42 Participants	1642 Participants n=42 Participants	1644 Participants n=42 Participants	1677 Participants n=36 Participants	1610 Participants n=36 Participants	1675 Participants n=24 Participants	1654 Participants n=135 Participants	1617 Participants n=136 Participants	1624 Participants n=44 Participants	1640 Participants n=667 Participants	1667 Participants n=7 Participants	33040 Participants n=6 Participants
Race/Ethnicity, Customized Race category · Asian	86 Participants n=5 Participants	82 Participants n=7 Participants	91 Participants n=5 Participants	99 Participants n=4 Participants	86 Participants n=21 Participants	75 Participants n=10 Participants	87 Participants n=115 Participants	88 Participants n=24 Participants	93 Participants n=42 Participants	83 Participants n=42 Participants	90 Participants n=42 Participants	91 Participants n=42 Participants	87 Participants n=36 Participants	88 Participants n=36 Participants	93 Participants n=24 Participants	84 Participants n=135 Participants	80 Participants n=136 Participants	89 Participants n=44 Participants	85 Participants n=667 Participants	77 Participants n=7 Participants	1734 Participants n=6 Participants
Race/Ethnicity, Customized Race category · Black	751 Participants n=5 Participants	729 Participants n=7 Participants	736 Participants n=5 Participants	688 Participants n=4 Participants	730 Participants n=21 Participants	717 Participants n=10 Participants	711 Participants n=115 Participants	705 Participants n=24 Participants	719 Participants n=42 Participants	755 Participants n=42 Participants	676 Participants n=42 Participants	667 Participants n=42 Participants	713 Participants n=36 Participants	673 Participants n=36 Participants	741 Participants n=24 Participants	676 Participants n=135 Participants	722 Participants n=136 Participants	714 Participants n=44 Participants	698 Participants n=667 Participants	701 Participants n=7 Participants	14222 Participants n=6 Participants
Race/Ethnicity, Customized Race category · Hispanic	187 Participants n=5 Participants	177 Participants n=7 Participants	173 Participants n=5 Participants	168 Participants n=4 Participants	157 Participants n=21 Participants	180 Participants n=10 Participants	156 Participants n=115 Participants	174 Participants n=24 Participants	174 Participants n=42 Participants	177 Participants n=42 Participants	181 Participants n=42 Participants	143 Participants n=42 Participants	177 Participants n=36 Participants	183 Participants n=36 Participants	154 Participants n=24 Participants	160 Participants n=135 Participants	162 Participants n=136 Participants	175 Participants n=44 Participants	145 Participants n=667 Participants	188 Participants n=7 Participants	3391 Participants n=6 Participants
Race/Ethnicity, Customized Race category · Other	140 Participants n=5 Participants	114 Participants n=7 Participants	149 Participants n=5 Participants	126 Participants n=4 Participants	134 Participants n=21 Participants	147 Participants n=10 Participants	130 Participants n=115 Participants	116 Participants n=24 Participants	140 Participants n=42 Participants	108 Participants n=42 Participants	139 Participants n=42 Participants	128 Participants n=42 Participants	144 Participants n=36 Participants	131 Participants n=36 Participants	128 Participants n=24 Participants	135 Participants n=135 Participants	113 Participants n=136 Participants	144 Participants n=44 Participants	134 Participants n=667 Participants	138 Participants n=7 Participants	2638 Participants n=6 Participants
Race/Ethnicity, Customized Race category · White	2578 Participants n=5 Participants	2635 Participants n=7 Participants	2560 Participants n=5 Participants	2654 Participants n=4 Participants	2651 Participants n=21 Participants	2650 Participants n=10 Participants	2635 Participants n=115 Participants	2649 Participants n=24 Participants	2602 Participants n=42 Participants	2654 Participants n=42 Participants	2586 Participants n=42 Participants	2722 Participants n=42 Participants	2616 Participants n=36 Participants	2639 Participants n=36 Participants	2657 Participants n=24 Participants	2684 Participants n=135 Participants	2699 Participants n=136 Participants	2622 Participants n=44 Participants	2679 Participants n=667 Participants	2654 Participants n=7 Participants	52826 Participants n=6 Participants

PRIMARY outcome

Timeframe: The dependent variable will be assessed during a 4-day period after randomization starting three days prior to the participant's target primary care appointment through the date of the target appointment (including the date of the appointment)

Outcome measures

Outcome measures
Measure	Information Control Condition n=3685 Participants Three days before their scheduled appointment, participants will receive a message inviting them to watch a 2-minute wellness video and answer some related questions, as well as encouragement to get the flu shot. In this condition, the video will contain information about exercising. Participants will also receive a reminder message the day before the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Sharing Humor Condition n=3664 Participants The day before their scheduled appointment, participants will receive a text message encouraging them to get the flu shot. The message will include a joke about the flu and will encourage participants to share the joke with nurses, doctors, or pharmacists. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	No Humor Condition n=3736 Participants The day before their scheduled appointment, participants will receive a text message encouraging them to get the flu shot. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Healthy Habits Easy Health Behavior Condition n=3690 Participants The day before their scheduled appointment, participants will be asked if they have completed a series of easy health behaviors (e.g., whether they walked 500 feet yesterday, at least two serving of fruits and vegetables in the last week, and slept at least 6 hours the previous night). They will then be encouraged to get a flu shot at their appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Healthy Habits Difficult Health Behavior Condition n=3734 Participants The day before their scheduled appointment, participants will be asked if they have completed a series of difficult health behaviors (e.g., whether they walked 3 miles yesterday, ate 4-6 servings of fruits and vegetables yesterday, and slept at least 9 hours the previous night). They will then be encouraged to get a flu shot at their appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Healthy Habits Control Condition n=3696 Participants The day before their scheduled appointment, participants will receive a text message encouraging them to get a flu shot. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Just-In-Time Reminders 24-Hour Condition n=3698 Participants Seventy-two hours before their appointment, participants will receive a text message alerting them to the availability of the flu vaccine at their upcoming appointment. Participants will be told they'll receive a reminder, which they can opt-out from receiving. The text message reminder will be sent 24 hours before their appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Just-In-Time Reminders 15-Minute Condition n=3712 Participants Just-In-Time Reminders 15-Minute Condition: Seventy-two hours before their appointment, participants will receive a text message alerting them to the availability of the flu vaccine at their upcoming appointment. Participants will be told they'll receive a reminder, which they can opt-out from receiving. The text message reminder will be sent 15 minutes before their appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Holdout Control n=3742 Participants Participants will only receive the standard appointment reminders from their providers.	Default Reservations Opt-Out Condition n=3688 Participants The day before their scheduled appointment, participants receive a text reading: "A flu shot has been reserved for you to receive at your appointment tomorrow. Reply Y if you want this shot held for you, N if you don't." The text will include a picture of a vial that says "Your Flu Shot" on it. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Default Reservations Opt-In Condition n=3670 Participants The day before their scheduled appointment, participants receive a text reading: "Reply Y if you would like to receive a flu shot at your appointment tomorrow, N if not." The text will include a picture of vial with no text on it. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Intergroup Competition Treatment Condition n=3693 Participants Three days before their scheduled appointment, participants will receive a text message with information about how much "your region" lagged behind "another region" in flu shot rate last year ("your region" and "another region" will depend on the study site and include realistic flu shot rates based on historical CDC data). Participants will receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Intergroup Competition Control Condition n=3717 Participants Three days before their scheduled appointment, participants will receive a text message with information about how much "your region" lagged behind the target flu shot rate of 70% last year ("your region" will depend on the study site and include realistic flu shot rates based on historical CDC data). Participants will receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Flu Shot for You Symbolic Condition n=3716 Participants Three days before their scheduled appointment, participants will receive a text message encouraging them to get a flu shot. They will also be told that they have the opportunity to dedicate getting the flu shot to someone by texting back the initials of an individual. Participants will receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Flu Shot for You Herd Immunity Condition n=3665 Participants Three days before their scheduled appointment, participants will receive a text message encouraging them to get a flu shot. They will also be told that they have the opportunity of getting the flu shot to protect a vulnerable loved one by texting back the initials of an individual. Participants will receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Flu Shot for You Control Condition n=3693 Participants Three days before their scheduled appointment, participants will receive a text message encouraging them to get a flu shot. They will also receive a reminder message on the day of the appointment. They will not receive any further information, nor will they be asked to respond with initials of an individual. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Prosocial Condition n=3662 Participants The day before their scheduled appointment, participants will receive a message describing the pro-social benefits of getting a flu shot, and a reminder to ask for their flu shot. The described pro-social benefits will vary whether they emphasize the COVID-19 pandemic (e.g., protecting loved ones from the risk of hospital-acquired COVID-19 infection) or not (e.g., protecting loved ones from serious complications from the flu). Participants will also receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Self-Oriented Condition n=3734 Participants The day before their scheduled appointment, participants will receive a message describing the self-oriented benefits of getting a flu shot, and a reminder to ask for their flu shot. The described self-oriented benefits will vary whether they emphasize the COVID-19 pandemic (e.g., protecting oneself from the risk of hospital-acquired COVID-19 infection) or not (e.g., protecting oneself from serious complications from the flu). Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Information Vivid Condition n=3623 Participants Three days before their scheduled appointment, participants will receive a message inviting them to watch a 2-minute wellness video and answer some related questions, as well as encouragement to get the flu shot. In this condition, the video will contain vivid information about getting the flu. Participants will also receive a reminder message the day before the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Information Basic Condition n=3710 Participants Three days before their scheduled appointment, participants will receive a message inviting them to watch a 2-minute wellness video and answer some related questions, as well as encouragement to get the flu shot. In this condition, the video will contain basic information about getting the flu. Participants will also receive a reminder message the day before the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.
Number of Participants With Flu Shot Receipt at or Before Appointment	1171 Participants	1135 Participants	1199 Participants	1181 Participants	1193 Participants	1130 Participants	1240 Participants	1227 Participants	1113 Participants	1210 Participants	1150 Participants	1163 Participants	1183 Participants	1200 Participants	1192 Participants	1191 Participants	1157 Participants	1224 Participants	1144 Participants	1231 Participants

SECONDARY outcome

Timeframe: Up to 6 months after randomization.

The secondary dependent variable is whether participants receive the flu shot at or before their target appointment or at any time after this appointment (by March 31, 2021), as recorded in their electronic health records. Participants who receive a flu shot before they receive the SMS intervention will be excluded from the analyses.

Outcome measures

Outcome measures
Measure	Information Control Condition n=3685 Participants Three days before their scheduled appointment, participants will receive a message inviting them to watch a 2-minute wellness video and answer some related questions, as well as encouragement to get the flu shot. In this condition, the video will contain information about exercising. Participants will also receive a reminder message the day before the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Sharing Humor Condition n=3664 Participants The day before their scheduled appointment, participants will receive a text message encouraging them to get the flu shot. The message will include a joke about the flu and will encourage participants to share the joke with nurses, doctors, or pharmacists. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	No Humor Condition n=3736 Participants The day before their scheduled appointment, participants will receive a text message encouraging them to get the flu shot. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Healthy Habits Easy Health Behavior Condition n=3690 Participants The day before their scheduled appointment, participants will be asked if they have completed a series of easy health behaviors (e.g., whether they walked 500 feet yesterday, at least two serving of fruits and vegetables in the last week, and slept at least 6 hours the previous night). They will then be encouraged to get a flu shot at their appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Healthy Habits Difficult Health Behavior Condition n=3734 Participants The day before their scheduled appointment, participants will be asked if they have completed a series of difficult health behaviors (e.g., whether they walked 3 miles yesterday, ate 4-6 servings of fruits and vegetables yesterday, and slept at least 9 hours the previous night). They will then be encouraged to get a flu shot at their appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Healthy Habits Control Condition n=3696 Participants The day before their scheduled appointment, participants will receive a text message encouraging them to get a flu shot. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Just-In-Time Reminders 24-Hour Condition n=3698 Participants Seventy-two hours before their appointment, participants will receive a text message alerting them to the availability of the flu vaccine at their upcoming appointment. Participants will be told they'll receive a reminder, which they can opt-out from receiving. The text message reminder will be sent 24 hours before their appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Just-In-Time Reminders 15-Minute Condition n=3712 Participants Just-In-Time Reminders 15-Minute Condition: Seventy-two hours before their appointment, participants will receive a text message alerting them to the availability of the flu vaccine at their upcoming appointment. Participants will be told they'll receive a reminder, which they can opt-out from receiving. The text message reminder will be sent 15 minutes before their appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Holdout Control n=3742 Participants Participants will only receive the standard appointment reminders from their providers.	Default Reservations Opt-Out Condition n=3688 Participants The day before their scheduled appointment, participants receive a text reading: "A flu shot has been reserved for you to receive at your appointment tomorrow. Reply Y if you want this shot held for you, N if you don't." The text will include a picture of a vial that says "Your Flu Shot" on it. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Default Reservations Opt-In Condition n=3670 Participants The day before their scheduled appointment, participants receive a text reading: "Reply Y if you would like to receive a flu shot at your appointment tomorrow, N if not." The text will include a picture of vial with no text on it. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Intergroup Competition Treatment Condition n=3693 Participants Three days before their scheduled appointment, participants will receive a text message with information about how much "your region" lagged behind "another region" in flu shot rate last year ("your region" and "another region" will depend on the study site and include realistic flu shot rates based on historical CDC data). Participants will receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Intergroup Competition Control Condition n=3717 Participants Three days before their scheduled appointment, participants will receive a text message with information about how much "your region" lagged behind the target flu shot rate of 70% last year ("your region" will depend on the study site and include realistic flu shot rates based on historical CDC data). Participants will receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Flu Shot for You Symbolic Condition n=3716 Participants Three days before their scheduled appointment, participants will receive a text message encouraging them to get a flu shot. They will also be told that they have the opportunity to dedicate getting the flu shot to someone by texting back the initials of an individual. Participants will receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Flu Shot for You Herd Immunity Condition n=3665 Participants Three days before their scheduled appointment, participants will receive a text message encouraging them to get a flu shot. They will also be told that they have the opportunity of getting the flu shot to protect a vulnerable loved one by texting back the initials of an individual. Participants will receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Flu Shot for You Control Condition n=3693 Participants Three days before their scheduled appointment, participants will receive a text message encouraging them to get a flu shot. They will also receive a reminder message on the day of the appointment. They will not receive any further information, nor will they be asked to respond with initials of an individual. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Prosocial Condition n=3662 Participants The day before their scheduled appointment, participants will receive a message describing the pro-social benefits of getting a flu shot, and a reminder to ask for their flu shot. The described pro-social benefits will vary whether they emphasize the COVID-19 pandemic (e.g., protecting loved ones from the risk of hospital-acquired COVID-19 infection) or not (e.g., protecting loved ones from serious complications from the flu). Participants will also receive a reminder message on the day of the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Self-Oriented Condition n=3734 Participants The day before their scheduled appointment, participants will receive a message describing the self-oriented benefits of getting a flu shot, and a reminder to ask for their flu shot. The described self-oriented benefits will vary whether they emphasize the COVID-19 pandemic (e.g., protecting oneself from the risk of hospital-acquired COVID-19 infection) or not (e.g., protecting oneself from serious complications from the flu). Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Information Vivid Condition n=3623 Participants Three days before their scheduled appointment, participants will receive a message inviting them to watch a 2-minute wellness video and answer some related questions, as well as encouragement to get the flu shot. In this condition, the video will contain vivid information about getting the flu. Participants will also receive a reminder message the day before the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.	Information Basic Condition n=3710 Participants Three days before their scheduled appointment, participants will receive a message inviting them to watch a 2-minute wellness video and answer some related questions, as well as encouragement to get the flu shot. In this condition, the video will contain basic information about getting the flu. Participants will also receive a reminder message the day before the appointment. Flu shot text messages: Participants will receive text messages per descriptions listed in the arms.
Number of Participants With Flu Shot Receipt Before March 31, 2021	1342 Participants	1310 Participants	1351 Participants	1370 Participants	1360 Participants	1297 Participants	1388 Participants	1388 Participants	1296 Participants	1391 Participants	1330 Participants	1343 Participants	1338 Participants	1371 Participants	1369 Participants	1370 Participants	1360 Participants	1418 Participants	1316 Participants	1385 Participants

Deaths: 0 deaths

Serious adverse events

Adverse event data not reported

Other adverse events

Adverse event data not reported

Additional Information

Dr. Joseph Kay

Behavior Change for Good initiative

Phone: 2157466609

Email: [email protected]

Results disclosure agreements

Principal investigator is a sponsor employee
Publication restrictions are in place