Prospective Validation of an Artificial Intelligence Tool for Pre-Anesthetic Assessment
NCT ID: NCT07290647
Last Updated: 2025-12-18
Study Results
The study team has not published outcome measurements, participant flow, or safety data for this trial yet. Check back later for updates.
Basic Information
Get a concise snapshot of the trial, including recruitment status, study phase, enrollment targets, and key timeline milestones.
NOT_YET_RECRUITING
270 participants
OBSERVATIONAL
2026-03-01
2028-06-30
Brief Summary
Review the sponsor-provided synopsis that highlights what the study is about and why it is being conducted.
Detailed Description
Dive into the extended narrative that explains the scientific background, objectives, and procedures in greater depth.
The pre-anesthetic assessment is a critical process preceding surgical procedures, playing a fundamental role in ensuring perioperative safety and quality of care. Over recent decades, anesthesiologists have contributed significantly to improving perioperative safety and quality, as highlighted in the Institute of Medicine's report on quality of health care. This evaluation has been continually refined to enhance clinical outcomes and reduce costs associated with unnecessary preoperative laboratory tests and examinations.
The benefits of pre-anesthetic assessment are well-documented, including increased operational efficiency through early identification of potential complications, optimization of preoperative management for patients with underlying conditions such as diabetes, cardiopulmonary diseases, and renal insufficiency, and minimization of postoperative complications. In Brazil, the Federal Council of Medicine's Resolution No. 1.802/2006 mandates pre-anesthetic evaluation as an essential component for patient safety, recommending it be performed before hospital admission for elective procedures.
Recently, Brazil has experienced a surge in demand for surgical procedures, exacerbated by the COVID-19 pandemic. A 2022 survey by the Oswaldo Cruz Foundation (Fiocruz) revealed a backlog of 910,621 surgeries in the Unified Health System (SUS), with significant deficits in digestive system surgeries (374,475), genitourinary procedures (241,752), circulatory system interventions (104,925), and upper airway, face, head, and neck surgeries (102,352). This pressure on the public health system led to the launch of the National Program for Reducing Waiting Lists in 2024 by the Brazilian federal government. Consequently, there is a need to rethink the flow and process of pre-anesthetic assessments to ensure safe and adequate care for this pent-up demand.
With technological advancements and the increasing volume of available medical data, clinical evaluations have become more complex, requiring faster and more precise decisions. In this context, artificial intelligence (AI) emerges as a promising tool to transform anesthesiology, particularly in pre-anesthetic assessment. AI tools, including Natural Language Processing (NLP) and Large Language Models (LLMs), have shown potential to improve accuracy, efficiency, and personalization of medical care.
Recent studies have demonstrated the positive impact of AI in this area. For instance, NLP for reviewing medical records and identifying relevant preoperative clinical information has shown high concordance (81.24%) between machine and anesthesiologist assessments regarding the presence or absence of conditions, and it identified medical conditions in 16.6% of cases overlooked by the anesthesiologist. Personalized decision support systems using OWL ontologies and semantic web technologies have proven effective in generating risk assessment reports and customized clinical recommendations. Machine learning models for perioperative risk prediction have enabled the creation of individual risk profiles and personalized assessments.
Large Language Models (LLMs), such as GPT, have shown potential to enhance patient communication and education during the surgical journey. A 2024 comparative study of GPT versions indicated its ability to provide accurate and readable responses to patients on anesthesia-related questions. However, studies on the clinical application of AI in anesthesiology involving LLMs remain scarce; a 2023 systematic review identified no studies using this technology. Moreover, most studies and developments have been conducted in international contexts, not addressing the Portuguese language or the specificities of the Brazilian healthcare system. To date, no published studies exist on the development and validation of an LLM tool in Portuguese for pre-anesthetic assessment, particularly considering Brazil's epidemiological profile and healthcare challenges.
Additionally, concerns about the reliability of AI tools in medicine are notable. A recent study published in Nature Medicine revealed that many AI devices approved by the FDA in the United States lack adequate clinical validation. Of 521 FDA-authorized AI devices between 1995 and 2022, only 56% reported some form of clinical validation, with just 28.4% validated prospectively and 4.2% through randomized clinical trials. Notably, 43.4% had no publicly available clinical validation data. These findings underscore the importance of conducting rigorous clinical validation studies, especially in specific contexts, before implementing AI tools in medical practice.
Therefore, this project proposes to develop and prospectively validate an AI tool based on an LLM in Portuguese for pre-anesthetic assessment in Brazil, in a clinical setting. This addresses a gap in the literature and meets the specific needs of the national context. The goal is to enhance the accuracy, personalization, reliability, and efficiency of preoperative evaluations using advanced AI, considering the peculiarities of the Brazilian population and healthcare system. Implementing this tool could potentially transform anesthesiology practice in the country, providing improved decision support and promoting greater surgical safety.
Study Objectives:
Primary Objective:
\- To prospectively evaluate the accuracy and consistency of the pre-anesthetic assessment performed by the AI tool compared to assessments conducted by anesthesiologists in a national tertiary hospital.
Secondary Objectives:
* To determine the level of concordance between the preoperative risk assessment performed by the AI tool and human anesthesiologists, considering aspects such as the American Society of Anesthesiologists (ASA) classification and validated surgical risk models in a national tertiary hospital.
* To investigate anesthesiologists' perceptions of the AI tool's utility, including their confidence in the results generated by the tool and their willingness to integrate it into clinical practice.
* To assess whether the use of the AI tool influences the overall quality of the pre-anesthetic assessment, including the detection of risk conditions that may be underestimated or overlooked in conventional human evaluations.
* To evaluate any difficulties patients face in using the AI tool in a national tertiary hospital.
Study Design:
This is a prospective observational longitudinal cohort study conducted at two centers, the Anesthesia Service of Hospital Nossa Senhora da Conceição in Porto Alegre, Brazil and Santa Casa de Ribeirão Preto, Brazil.
Participants:
Inclusion Criteria:
* Patients aged 18 years or older.
* Scheduled for elective non-cardiac surgeries at both institutions.
Exclusion Criteria:
* Patients undergoing diagnostic procedures with isolated sedation or local anesthesia.
* If a patient undergoes more than one surgical intervention during the same hospitalization, only the major procedure will be considered.
Sample Size:
The sample size was calculated based on the primary outcome of concordance between AI and human preoperative risk assessments using Cohen's Kappa coefficient. Assuming an expected concordance of 80% (based on prior studies showing around 0.80 for perioperative risk assessments between AI and clinicians), with a Kappa value ≥ 0.70 considered substantial concordance, a margin of error of 5%, and a 95% confidence level, the minimum sample size is approximately 246 patients. Accounting for potential losses to follow-up and exclusions (10% margin), the total sample will be 270 patients.
Procedures:
1. Surgical Indication:
* The surgeon determines the patient's surgical indication and records the surgery name according to the procedure code in the Ex-Care risk model.
2. Use of the AI Tool:
* The surgeon instructs the patient to use the AI tool for pre-anesthetic assessment. The patient accesses the tool and completes the requested information.
3. Processing of the Assessment:
* The tool processes the patient's assessment, generating two types of results:
* Generic Orientations: A set of general instructions sent directly to the patient to aid in surgical preparation.
* Specific Assessment: A detailed evaluation, including recommendations and warning signs, generated for medical use but not made available to the anesthesiologist performing the preoperative evaluation.
4. Pre-Anesthetic Evaluation:
* The patient undergoes a traditional pre-anesthetic evaluation by an anesthesiologist, who will not have access to the AI-generated assessment.
5. Comparison of Assessments:
* A third anesthesiologist, blinded to the prior process, will compare the two assessments (AI tool versus human evaluation). This comparison will consider the quality of collected information, precision of clinical judgment, and identification of potential risks or complications.
Data Collection:
Data will be collected prospectively, including variables such as age, sex, surgery code, and results from both assessments (AI tool and anesthesiologist). A data collection form (attached in the protocol after references) outlines the variables included in this research protocol.
Outcome Measures:
Primary Outcome:
\- Concordance between preoperative risk assessments by the AI tool and the human anesthesiologist, in terms of quality of collected information and clinical judgment.
Secondary Outcomes:
* Level of agreement on ASA classification and validated surgical risk models.
* Anesthesiologists' perceptions of the tool's utility, confidence, and integration potential (assessed via surveys).
* Impact on assessment quality, including detection of overlooked risks.
* Patient-reported difficulties in using the AI tool (assessed via feedback).
Statistical Analysis:
Concordance tests will be used to compare preoperative risk assessment results between the AI tool and anesthesiologist. Cohen's Kappa coefficient will measure agreement for categorical variables (information quality and clinical judgment). For continuous variables (e.g., age), Student's t-test or Mann-Whitney test will be applied, depending on data distribution (normality checked via Shapiro-Wilk test). Statistical significance will be set at P \< 0.05.
Timeline:
The study will span 36 months: 24 months for data collection, 6 months for analysis, and 6 months for result formulation and publication.
Funding:
Costs for data collection materials, computers, statistical software, and documentation will be covered by the involved authors. Budget details include: folders for storage (R$250.00), data collection sheets (R$500.00), and analysis software (R$1,000.00).
Dissemination:
Results will be stored in a secure, confidential database and submitted for publication in an indexed scientific journal upon completion.
Conditions
See the medical conditions and disease areas that this research is targeting or investigating.
Keywords
Explore important study keywords that can help with search, categorization, and topic discovery.
Study Design
Understand how the trial is structured, including allocation methods, masking strategies, primary purpose, and other design elements.
COHORT
PROSPECTIVE
Interventions
Learn about the drugs, procedures, or behavioral strategies being tested and how they are applied within this trial.
AI-based pre-anesthetic assessment
Patients use an artificial intelligence (AI) tool based on a Large Language Model (LLM) in Portuguese to complete a pre-anesthetic self-assessment. The tool collects patient information and generates two outputs: Generic Orientations: General instructions sent directly to the patient to aid in surgical preparation.
Specific Assessment: A detailed evaluation, including recommendations and warning signs for medical use, which is not shared with the anesthesiologist performing the traditional evaluation.
Anesthesiologist-led pre-anesthetic evaluation
Each patient undergoes a standard pre-anesthetic evaluation conducted by an anesthesiologist, following routine clinical practice at both institutions. This evaluation is performed without access to the AI tool's results to ensure blinding.
Purpose: To serve as the comparator for the AI-based assessment, allowing evaluation of concordance in risk assessment, quality of information collected, and clinical judgment.
Details: The anesthesiologist conducts a clinical interview and review of medical records, assessing factors such as the American Society of Anesthesiologists (ASA) classification, perioperative risk models (for instance, Ex-Care model), and potential complications.
Eligibility Criteria
Check the participation requirements, including inclusion and exclusion rules, age limits, and whether healthy volunteers are accepted.
Inclusion Criteria
* Patients scheduled for elective non-cardiac surgeries
Exclusion Criteria
* If a patient undergoes more than one surgical intervention during the same hospitalization, only the major procedure will be considered (i.e., additional procedures during the same admission are not eligible for separate inclusion).
18 Years
ALL
No
Sponsors
Meet the organizations funding or collaborating on the study and learn about their roles.
Hospital Nossa Senhora da Conceicao
OTHER
Responsible Party
Identify the individual or organization who holds primary responsibility for the study information submitted to regulators.
Andre Prato Schmidt
MD, PhD (Anesthesiologist - Department Chair).
Locations
Explore where the study is taking place and check the recruitment status at each participating site.
Hospital Nossa Senhora da Conceição (Grupo Hospitalar Conceição)
Porto Alegre, Rio Grande do Sul, Brazil
Countries
Review the countries where the study has at least one active or historical site.
Central Contacts
Reach out to these primary contacts for questions about participation or study logistics.
Facility Contacts
Find local site contact details for specific facilities participating in the trial.
Andre P. Schmidt, MD, PhD
Role: primary
References
Explore related publications, articles, or registry entries linked to this study.
Wongtangman K, Aasman B, Garg S, Witt AS, Harandi AA, Azimaraghi O, Mirhaji P, Soby S, Anand P, Himes CP, Smith RV, Santer P, Freda J, Eikermann M, Ramaswamy P. Development and validation of a machine learning ASA-score to identify candidates for comprehensive preoperative screening and risk stratification. J Clin Anesth. 2023 Aug;87:111103. doi: 10.1016/j.jclinane.2023.111103. Epub 2023 Mar 8.
Abdel Malek M, van Velzen M, Dahan A, Martini C, Sitsen E, Sarton E, Boon M. Generation of preoperative anaesthetic plans by ChatGPT-4.0: a mixed-method study. Br J Anaesth. 2025 May;134(5):1333-1340. doi: 10.1016/j.bja.2024.08.038. Epub 2024 Nov 14.
Yoon HK, Yang HL, Jung CW, Lee HC. Artificial intelligence in perioperative medicine: a narrative review. Korean J Anesthesiol. 2022 Jun;75(3):202-215. doi: 10.4097/kja.22157. Epub 2022 Mar 29.
Other Identifiers
Review additional registry numbers or institutional identifiers associated with this trial.
85674025.4.0000.5530
Identifier Type: -
Identifier Source: org_study_id