Trial Outcomes & Findings for eLearning for Suicide Prevention (NCT NCT05178121)

Results Overview

The System Usability Scale, a self-report measure using a Likert-type rating scale, will assess acceptability of the two technologic training components: Client Bot Emily and Lyssn Advisor. Client Bot Emily is a virtual patient/chat bot with the persona of a suicidal patient. Nurses chat with Emily and practice skills of asking open-ended questions and making reflections about statements Emily makes. Lyssn Advisor is an online platform that automatically codes transcripts of audio or video recorded role-plays for counseling skills; nurses then log in to the platform to see their scores, which provides feedback on their counseling performance. Scores on the SUS range from 0-100, which higher scores indicating greater usability or acceptability.

Recruitment status

COMPLETED

Study phase

NA

Target enrollment

19 participants

Primary outcome timeframe

Once, immediately after engaging with each of the training technologies

Results posted on

2023-12-18

Participant Flow

Nurses recruited from medical center via email, flyer, and unit manager notifications of the study. Recruitment period: November 1, 2021 - May 31, 2022

Participant milestones

Participant milestones
Measure	Project WISE eLearning Training in Suicide Safety Planning eLearning Training in Suicide Safety Planning: The training includes a web-based didactic in suicide safety planning, interactive practice in counseling microskills with a chat bot, role-playing safety planning with a standardized patient, and reviewing automated feedback on safety planning counseling skills generated from a computer coding system.
Overall Study STARTED	18
Overall Study Completed Training	15
Overall Study Completed Post-Training Survey	14
Overall Study Completed Post-Training Role-Play	14
Overall Study Completed 6-Month Role-Play	10
Overall Study COMPLETED	13
Overall Study NOT COMPLETED	5

Reasons for withdrawal

Reasons for withdrawal
Measure	Project WISE eLearning Training in Suicide Safety Planning eLearning Training in Suicide Safety Planning: The training includes a web-based didactic in suicide safety planning, interactive practice in counseling microskills with a chat bot, role-playing safety planning with a standardized patient, and reviewing automated feedback on safety planning counseling skills generated from a computer coding system.
Overall Study Other	4
Overall Study Data collection ended before participant reached 6 months	1

Baseline Characteristics

eLearning for Suicide Prevention

Baseline characteristics by cohort

Baseline characteristics by cohort
Measure	Project WISE eLearning Training in Suicide Safety Planning n=18 Participants eLearning Training in Suicide Safety Planning: The training includes a web-based didactic in suicide safety planning, interactive practice in counseling microskills with a chat bot, role-playing safety planning with a standardized patient, and reviewing automated feedback on safety planning counseling skills generated from a computer coding system.
Age, Continuous	31.4 years STANDARD_DEVIATION 6.0 • n=5 Participants
Sex: Female, Male Female	17 Participants n=5 Participants
Sex: Female, Male Male	1 Participants n=5 Participants
Ethnicity (NIH/OMB) Hispanic or Latino	1 Participants n=5 Participants
Ethnicity (NIH/OMB) Not Hispanic or Latino	16 Participants n=5 Participants
Ethnicity (NIH/OMB) Unknown or Not Reported	1 Participants n=5 Participants
Race (NIH/OMB) American Indian or Alaska Native	0 Participants n=5 Participants
Race (NIH/OMB) Asian	3 Participants n=5 Participants
Race (NIH/OMB) Native Hawaiian or Other Pacific Islander	0 Participants n=5 Participants
Race (NIH/OMB) Black or African American	0 Participants n=5 Participants
Race (NIH/OMB) White	14 Participants n=5 Participants
Race (NIH/OMB) More than one race	0 Participants n=5 Participants
Race (NIH/OMB) Unknown or Not Reported	1 Participants n=5 Participants
Region of Enrollment United States	18 participants n=5 Participants

PRIMARY outcome

Timeframe: Once, immediately after engaging with each of the training technologies

The System Usability Scale, a self-report measure using a Likert-type rating scale, will assess acceptability of the two technologic training components: Client Bot Emily and Lyssn Advisor. Client Bot Emily is a virtual patient/chat bot with the persona of a suicidal patient. Nurses chat with Emily and practice skills of asking open-ended questions and making reflections about statements Emily makes. Lyssn Advisor is an online platform that automatically codes transcripts of audio or video recorded role-plays for counseling skills; nurses then log in to the platform to see their scores, which provides feedback on their counseling performance. Scores on the SUS range from 0-100, which higher scores indicating greater usability or acceptability.

Outcome measures

Outcome measures
Measure	Project WISE eLearning Training in Suicide Safety Planning n=16 Participants eLearning Training in Suicide Safety Planning: The training includes a web-based didactic in suicide safety planning, interactive practice in counseling microskills with a chat bot "Client Bot Emily", role-playing safety planning with a standardized patient, and reviewing automated feedback on safety planning counseling skills generated from a computer coding system (Lyssn Advisor).
Training Acceptability - System Usability Scale for Client Bot Emily	70.3 score on a scale Standard Deviation 19.7

PRIMARY outcome

Timeframe: Immediately after completing the training

A 4-item self-report measure created for this study will assess satisfaction with all of the training components using a Likert-type scale of 0 = Not at all satisfied to 4 = Completely satisfied.

Outcome measures

Outcome measures
Measure	Project WISE eLearning Training in Suicide Safety Planning n=14 Participants eLearning Training in Suicide Safety Planning: The training includes a web-based didactic in suicide safety planning, interactive practice in counseling microskills with a chat bot "Client Bot Emily", role-playing safety planning with a standardized patient, and reviewing automated feedback on safety planning counseling skills generated from a computer coding system (Lyssn Advisor).
Training Acceptability Self-Report - Post-Training Satisfaction with the didactic and demonstration	3.1 score on a scale Standard Deviation 0.8
Training Acceptability Self-Report - Post-Training Satisfaction with Client Bot Emily	1.1 score on a scale Standard Deviation 1.1
Training Acceptability Self-Report - Post-Training Satisfaction with the role-play practice	3.5 score on a scale Standard Deviation 0.5
Training Acceptability Self-Report - Post-Training Satisfaction with the Lyssn Advisor feedback	2.5 score on a scale Standard Deviation 0.6

PRIMARY outcome

Timeframe: 6-months after completing the training

A 4-item self-report measure created for this study will assess satisfaction with all of the training components using a Likert-type scale of 0 = Not at all satisfied to 4 = Completely satisfied.

Outcome measures

Outcome measures
Measure	Project WISE eLearning Training in Suicide Safety Planning n=13 Participants eLearning Training in Suicide Safety Planning: The training includes a web-based didactic in suicide safety planning, interactive practice in counseling microskills with a chat bot "Client Bot Emily", role-playing safety planning with a standardized patient, and reviewing automated feedback on safety planning counseling skills generated from a computer coding system (Lyssn Advisor).
Training Acceptability Self-Report - 6-Month Follow-up Satisfaction with the didactic and demonstration	3.2 score on a scale Standard Deviation 0.8
Training Acceptability Self-Report - 6-Month Follow-up Satisfaction with Client Bot Emily	1.5 score on a scale Standard Deviation 1.2
Training Acceptability Self-Report - 6-Month Follow-up Satisfaction with the role-play practice	3.5 score on a scale Standard Deviation 0.5
Training Acceptability Self-Report - 6-Month Follow-up Satisfaction with the Lyssn Advisor feedback	2.6 score on a scale Standard Deviation 1.0

PRIMARY outcome

Timeframe: Immediately after participants indicate they have completing the training activities

Data is collected from the technologic platform on whether and how often participants accessed the training technologies. Engagement is indicated by participants accessing the platform using their unique login and interacting with the webpage associated with the Client Bot. The unit of measurement will be the number of participants who accessed the Client Bot during the training period.

Outcome measures

Outcome measures
Measure	Project WISE eLearning Training in Suicide Safety Planning n=18 Participants eLearning Training in Suicide Safety Planning: The training includes a web-based didactic in suicide safety planning, interactive practice in counseling microskills with a chat bot "Client Bot Emily", role-playing safety planning with a standardized patient, and reviewing automated feedback on safety planning counseling skills generated from a computer coding system (Lyssn Advisor).
Training Engagement With Client Bot User Data - Post-Training	16 Participants

PRIMARY outcome

Timeframe: Immediately after participants indicate they have completing the training activities

Population: We were unable to collect this data

Data is collected from the technologic platform on whether and how often participants accessed the training technologies. Engagement is indicated by participants accessing the platform using their unique login and interacting with the webpage associated with the Lyssn Advisor. The unit of measurement will be the frequency of times participants visits the Lyssn Advisor webpage.

Outcome measures

Outcome data not reported

PRIMARY outcome

Timeframe: 6-months after participants indicate they have completed the training activities

Population: 16 participants interacted with Client Bot Emily during the training period.

Data is collected from the technologic platform on whether and how often participants accessed the training technologies. Engagement is indicated by participants accessing the platform using their unique login and interacting with the webpage associated with the Client Bot. The unit of measurement will be the number of participants who logged on and interacted with the Client Bot after the training period and before completing the 6-month survey.

Outcome measures

Outcome measures
Measure	Project WISE eLearning Training in Suicide Safety Planning n=16 Participants eLearning Training in Suicide Safety Planning: The training includes a web-based didactic in suicide safety planning, interactive practice in counseling microskills with a chat bot "Client Bot Emily", role-playing safety planning with a standardized patient, and reviewing automated feedback on safety planning counseling skills generated from a computer coding system (Lyssn Advisor).
Training Engagement With Client Bot User Data - 6-Month Follow-up	0 Participants

PRIMARY outcome

Timeframe: 6-months after participants indicate they have completed the training activities

Population: Participant login to the Lyssn system after initial training was not able to be captured, therefore, the outcome data for this variable were not collected.

Data is collected from the technologic platform on whether and how often participants accessed the training technologies. Engagement is indicated by participants accessing the platform using their unique login and interacting with the webpage associated with the Lyssn Advisor. The unit of measurement will be the frequency of times participants visits the Lyssn Advisor webpage during the 6-month follow-up period.

Outcome measures

Outcome data not reported

PRIMARY outcome

Timeframe: Immediately after participants indicate they have completed the training activities

Investigators will collect self-reported completion of the training activities using a 4-item measure. Participants will report on a sliding scale from 0-100, what percent of each training activity they believe they completed.

Outcome measures

Outcome measures
Measure	Project WISE eLearning Training in Suicide Safety Planning n=14 Participants eLearning Training in Suicide Safety Planning: The training includes a web-based didactic in suicide safety planning, interactive practice in counseling microskills with a chat bot "Client Bot Emily", role-playing safety planning with a standardized patient, and reviewing automated feedback on safety planning counseling skills generated from a computer coding system (Lyssn Advisor).
Training Engagement Self-Report - Post-Training Percent completion of didactic and demonstration	94.3 percentage of completion Standard Deviation 14.4
Training Engagement Self-Report - Post-Training Percent completion of Client Bot Emily activity	81.8 percentage of completion Standard Deviation 18.1
Training Engagement Self-Report - Post-Training Percent completion of role-play practice	95.3 percentage of completion Standard Deviation 11.0
Training Engagement Self-Report - Post-Training Percent completion of Lyssn Advisor activity	90.5 percentage of completion Standard Deviation 12.2

PRIMARY outcome

Timeframe: 6-months after participants indicate they have completed the training activities

Participants will self-report whether or not they accessed any of the training technologies during the follow-up period.

Outcome measures

Outcome measures
Measure	Project WISE eLearning Training in Suicide Safety Planning n=13 Participants eLearning Training in Suicide Safety Planning: The training includes a web-based didactic in suicide safety planning, interactive practice in counseling microskills with a chat bot "Client Bot Emily", role-playing safety planning with a standardized patient, and reviewing automated feedback on safety planning counseling skills generated from a computer coding system (Lyssn Advisor).
Client Bot Training Engagement Self-Report - 6 Month Follow-up	3 Participants

PRIMARY outcome

Timeframe: 6 Months after completing the training

Population: 19 participants enrolled in the study; however, of these, 18 participants participated in some aspect of the study, including completed the baseline survey

Percent of participants retained throughout study follow-up

Outcome measures

Outcome measures
Measure	Project WISE eLearning Training in Suicide Safety Planning n=18 Participants eLearning Training in Suicide Safety Planning: The training includes a web-based didactic in suicide safety planning, interactive practice in counseling microskills with a chat bot "Client Bot Emily", role-playing safety planning with a standardized patient, and reviewing automated feedback on safety planning counseling skills generated from a computer coding system (Lyssn Advisor).
Participant Retention Retained through the 6-month survey	13 Participants
Participant Retention Retained through the 6-month role-play	10 Participants

PRIMARY outcome

Timeframe: Once, immediately after engaging with each of the training technologies

The System Usability Scale, a self-report measure using a Likert-type rating scale, will assess acceptability of the two technologic training components: Client Bot Emily and Lyssn Advisor. Client Bot Emily is a virtual patient/chat bot with the persona of a suicidal patient. Nurses chat with Emily and practice skills of asking open-ended questions and making reflections about statements Emily makes. Lyssn Advisor is an online platform that automatically codes transcripts of audio or video recorded role-plays for counseling skills; nurses then log in to the platform to see their scores, which provides feedback on their counseling performance. Scores on the SUS range from 0-100, which higher scores indicating greater usability or acceptability.

Outcome measures

Outcome measures
Measure	Project WISE eLearning Training in Suicide Safety Planning n=14 Participants eLearning Training in Suicide Safety Planning: The training includes a web-based didactic in suicide safety planning, interactive practice in counseling microskills with a chat bot "Client Bot Emily", role-playing safety planning with a standardized patient, and reviewing automated feedback on safety planning counseling skills generated from a computer coding system (Lyssn Advisor).
Training Acceptability - System Usability Scale for Lyssn Advisor	65.4 score on a scale Standard Deviation 16.3

PRIMARY outcome

Timeframe: 6-months after participants indicate they have completed the training activities

Population: 13 nurses completed the 6-month follow-up survey and therefore reported on technology use during that period of time.

Participants will self-report whether or not they accessed any of the training technologies during the follow-up period.

Outcome measures

Outcome measures
Measure	Project WISE eLearning Training in Suicide Safety Planning n=13 Participants eLearning Training in Suicide Safety Planning: The training includes a web-based didactic in suicide safety planning, interactive practice in counseling microskills with a chat bot "Client Bot Emily", role-playing safety planning with a standardized patient, and reviewing automated feedback on safety planning counseling skills generated from a computer coding system (Lyssn Advisor).
Lyssn Advisor Training Engagement Self-Report - 6 Month Follow-up	6 Participants

Adverse Events

Project WISE eLearning Training in Suicide Safety Planning

Serious events: 0 serious events

Other events: 0 other events

Deaths: 0 deaths

Serious adverse events

Adverse event data not reported

Other adverse events

Adverse event data not reported

Additional Information

Doyanne Darnell

University of Washington

Phone: 206-744-9108

Email: [email protected]

Results disclosure agreements

Principal investigator is a sponsor employee
Publication restrictions are in place