Machine Learning Sepsis Alert Notification Using Clinical Data

NCT ID: NCT04005001

Last Updated: 2022-05-03

Basic Information

• Recruitment Status: UNKNOWN
• Clinical Phase: PHASE2
• Total Enrollment: 37986 participants
• Study Classification: INTERVENTIONAL
• Study Start Date: 2021-09-25
• Study Completion Date: 2022-08-31

Brief Summary

Machine learning is a powerful method to create clinical decision support (CDS) tools, when training labels reflect the desired alert behavior. In our Phase I work for this project, we developed HindSight, an encoding software that was designed to examine discharged patients' electronic health records (EHRs), identify clinicians' sepsis treatment decisions and patient outcomes, and pass those labeled outcomes and treatment decisions to an online algorithm for retraining of our machine-learning-based CDS tool for real-time sepsis alert notification, InSight. HindSight improved the performance of InSight sepsis alerts in retrospective work. In this study, we propose to assess the clinical utility of HindSight by conducting a multicenter prospective randomized controlled trial (RCT) for more accurate sepsis alerts.

Detailed Description

We will evaluate the performance of HindSight in a randomized controlled trial (RCT). HindSight is a novel encoding software designed to optimize alerts for sepsis alert notification. HindSight identifies clinicians' sepsis-related decisions in the electronic health records of former patients and passes those events to InSight, thus supplying InSight with labeled examples of true positive sepsis cases for retraining. In our retrospective work, we have shown that HindSight enables InSight to adapt to site-specific deviations of real-world clinical deployment by successfully reducing false and irrelevant alarms, without human supervision. The goal of this project is to demonstrate that the retrospective success of HindSight can be successfully translated to live clinical environments. In our Phase I work, HindSight achieved an area under the receiver-operating characteristic (AUROC) of 0.899, 0.831 and 0.877 for clinician sepsis evaluation, treatment, and onset, respectively. By using an online learning algorithm to incorporate HindSight-labeled data into the InSight predictor, we showed that the online-trained InSight can adapt to the HindSight-labeled data and outperform both baseline and periodically re-trained versions of InSight (p < 0.05). In Aim 1, we will prospectively validate HindSight's performance on real-time patient data streams in three diverse hospitals non-interventionally. In Aim 2, we will evaluate the effect of the tool in a prospective, interventional RCT. HindSight will first be evaluated by live deployment at four academic and community hospitals, during which time it will not provide alerts of future sepsis onset. Following any necessary algorithm optimization arising from live hospital validation, we will perform an RCT to evaluate reductions in false alerts from InSight trained on HindSight sepsis labels (experimental arm), compared to InSight trained on gold standard Sepsis-3 labels (control arm).

Conditions

Sepsis; Severe Sepsis; Septic Shock

Study Design

• Allocation Method: RANDOMIZED
• Intervention Model: PARALLEL
• Primary Study Purpose: OTHER
• Blinding Strategy: TRIPLE

Study Groups

Experimental

The experimental arm will involve patients monitored by HindSight.

Group Type: EXPERIMENTAL

HindSight

Intervention Type: OTHER

HindSight will examine the dynamic trends of clinical measurements taken from a patient's EHR and analyzes correlations between vital signs to alert for the onset of sepsis.This machine learning based tool is optimized by encoder and utilizes periodic retraining to improve its performance over time.

Control

The control arm will involve patients monitored by InSight.

Group Type: ACTIVE_COMPARATOR

InSight

Intervention Type: OTHER

Compared to the ability of the InSight software's recognition of sepsis onset to HindSight's performance. The study determines if the HindSight software has equivalent or better performance than the InSight software.

Interventions

HindSight

HindSight will examine the dynamic trends of clinical measurements taken from a patient's EHR and analyzes correlations between vital signs to alert for the onset of sepsis.This machine learning based tool is optimized by encoder and utilizes periodic retraining to improve its performance over time.

Intervention Type: OTHER

InSight

Compared to the ability of the InSight software's recognition of sepsis onset to HindSight's performance. The study determines if the HindSight software has equivalent or better performance than the InSight software.

Intervention Type: OTHER

Other Intervention Names

HindSight-Clinical decision support (CDS) system for sepsis alert notification

Eligibility Criteria

Inclusion Criteria

• During the study period, all patients over the age of 18 presenting to the emergency department or admitted to an inpatient unit at the participating facilities will automatically be enrolled in the study, until the enrollment target for the study is met

Exclusion Criteria

• Patients under the age of 18
• Prisoners

• Minimum Eligible Age: 18 Years
• Eligible Sex: ALL
• Accepts Healthy Volunteers: Yes

Sponsors

Cape Regional Medical Center

UNKNOWN

Sponsor Role: collaborator

Cooper University Medical Center

UNKNOWN

Sponsor Role: collaborator

Baystate Health

OTHER

Sponsor Role: collaborator

National Institute on Alcohol Abuse and Alcoholism (NIAAA)

NIH

Sponsor Role: collaborator

Dascena

INDUSTRY

Sponsor Role: lead

Responsible Party

Responsibility Role: SPONSOR

Principal Investigators

Jana Hoffman, PhD

Role: PRINCIPAL_INVESTIGATOR

Dascena

Locations

Baystate Health

Springfield, Massachusetts, United States

Site Status: RECRUITING

Cooper University Health Care

Camden, New Jersey, United States

Site Status: RECRUITING

Cape Regional Medical Center

Cape May, New Jersey, United States

Site Status: RECRUITING

Countries

United States

Central Contacts

Jana Hoffman, PhD

Role: CONTACT

jana@dascena.com

2158806619

Gina Barnes, MPH

Role: CONTACT

gbarnes@dascena.com

2158806619

Facility Contacts

Gregory Braden

Role: primary

Sharad Patel

Role: primary

Snehal Gandhi

Role: backup

Andrea McCoy

Role: primary

References

Desautels T, Calvert J, Hoffman J, Mao Q, Jay M, Fletcher G, Barton C, Chettipally U, Kerem Y, Das R. Using Transfer Learning for Improved Mortality Prediction in a Data-Scarce Hospital Setting. Biomed Inform Insights. 2017 Jun 12;9:1178222617712994. doi: 10.1177/1178222617712994. eCollection 2017.

Reference Type: BACKGROUND

PMID: 28638239 (View on PubMed)

Calvert J, Mao Q, Rogers AJ, Barton C, Jay M, Desautels T, Mohamadlou H, Jan J, Das R. A computational approach to mortality prediction of alcohol use disorder inpatients. Comput Biol Med. 2016 Aug 1;75:74-9. doi: 10.1016/j.compbiomed.2016.05.015. Epub 2016 May 24.

Reference Type: BACKGROUND

PMID: 27253619 (View on PubMed)

Calvert JS, Price DA, Barton CW, Chettipally UK, Das R. Discharge recommendation based on a novel technique of homeostatic analysis. J Am Med Inform Assoc. 2017 Jan;24(1):24-29. doi: 10.1093/jamia/ocw014. Epub 2016 Mar 28.

Reference Type: BACKGROUND

PMID: 27026611 (View on PubMed)

Calvert J, Mao Q, Hoffman JL, Jay M, Desautels T, Mohamadlou H, Chettipally U, Das R. Using electronic health record collected clinical variables to predict medical intensive care unit mortality. Ann Med Surg (Lond). 2016 Sep 6;11:52-57. doi: 10.1016/j.amsu.2016.09.002. eCollection 2016 Nov.

Reference Type: BACKGROUND

PMID: 27699003 (View on PubMed)

Shimabukuro DW, Barton CW, Feldman MD, Mataraso SJ, Das R. Effect of a machine learning-based severe sepsis prediction algorithm on patient survival and hospital length of stay: a randomised clinical trial. BMJ Open Respir Res. 2017 Nov 9;4(1):e000234. doi: 10.1136/bmjresp-2017-000234. eCollection 2017.

Reference Type: BACKGROUND

PMID: 29435343 (View on PubMed)

Other Identifiers

2R44AA030000-02

Identifier Type: NIH

Identifier Source: secondary_id

View Link

19-569185

Identifier Type: -

Identifier Source: org_study_id