Free Text Prediction Algorithm for Appendicitis

NCT ID: NCT03414853

Last Updated: 2021-03-03

Basic Information

• Recruitment Status: COMPLETED
• Total Enrollment: 689 participants
• Study Classification: OBSERVATIONAL
• Study Start Date: 2017-12-04
• Study Completion Date: 2020-07-01

Brief Summary

Computer-aided diagnostic software has been used to assist physicians in various ways. Text-based prediction algorithms have been trained on past medical records through data mining and feature analysis. Currently, all text-based machine learning prediction problem models have been built on extracted data with no research completed on free text based prediction algorithms. This study aims to determine the accuracy of a free text prediction algorithm in predicting the probability of appendicitis in patients presenting to the Emergency Department with abdominal pain and gastrointestinal symptoms.

Detailed Description

Developing machine learning models that have a strong prediction power for diagnosis of appendicitis from physician entered free text input can improve diagnostic accuracy of doctors. It also offers the possibility of using prediction algorithms to improve routine clinical care. In the future, multiple machine learning models can be combined to increase prediction accuracy and prediction algorithms can be extended to other diagnoses.

18,000 cases of emergency department presentations over 10 years were used as a training and validation dataset. To develop the appendicitis prediction model, deep learning neural networks with a customized medical ontology were used. The diagnostic accuracy of the model is expressed as sensitivity (recall), specificity and F1 score (harmonic mean). The developed diagnosis predictive model shows high sensitivity (86.3%), specificity (91.9%) and F1 score (88.8) in diagnosing appendicitis from patients presenting with abdominal pain.

The predictive model algorithm will also highlight words in the free text (entered by the attending physician) that it assigns higher probability for predicting an outcome. The doctors will be instructed to provide a percentage likelihood of appendicitis based on the clinical presentation and any available laboratory investigations. The doctor is then shown the prediction of the algorithm as well as the highlighted words for the patient entered. He/she must then provide another prediction of the likelihood of appendicitis after seeing the algorithm generated prediction.

The aim is to evaluate the performance of the algorithm and to assess if usage of the algorithm is able to help emergency doctors improve their diagnosis of appendicitis. The prediction results will be tabulated to assess accuracy of the algorithm, doctors before algorithm input and doctors after receiving algorithm input. The accuracy will be expressed as sensitivity, specificity, accuracy, positive prediction value, F1 score and F0.5 score.

Approximately 100 emergency doctors will be recruited over the course of 1 year as participants in the study. The doctors will be split randomly assigned to two groups - the algorithm arm and the no algorithm arm. The randomization will be by time (weekly) using variable block randomization of 4 and 6. The patients will be followed up for the final discharge diagnoses.

Conditions

Acute Appendicitis; Abdominal Pain

Study Design

• Observational Model Type: COHORT
• Study Time Perspective: PROSPECTIVE

Study Groups

With algorithm use

Free text prediction algorithm for appendicitis

Intervention Type: DIAGNOSTIC_TEST

A free-text prediction software that predicts the probability of acute appendicitis

No algorithm use

No interventions assigned to this group

Interventions

Free text prediction algorithm for appendicitis

A free-text prediction software that predicts the probability of acute appendicitis

Intervention Type: DIAGNOSTIC_TEST

Eligibility Criteria

Inclusion Criteria

Eligibility criteria of patients-

• Presence of abdominal pain, OR
• Presence of gastrointestinal symptoms such as nausea, vomiting or diarrhea, OR
• Fever with anorexia

Exclusion Criteria

• Previous history of appendicectomy
• Refusal of consent

• Minimum Eligible Age: 21 Years
• Maximum Eligible Age: 99 Years
• Eligible Sex: ALL
• Accepts Healthy Volunteers: No

Sponsors

National University Hospital, Singapore

OTHER

Sponsor Role: lead

Responsible Party

Responsibility Role: SPONSOR

Principal Investigators

Kee Yuan Ngiam, Dr

Role: PRINCIPAL_INVESTIGATOR

National University Hospital, Singapore

Locations

National University Hospital

Singapore, , Singapore

Countries

Singapore

Other Identifiers

N-171-000-456-001

Identifier Type: -

Identifier Source: org_study_id