Emergency department triage prediction of clinical outcomes using machine learning models

A pre-trained language model for emergency department intervention prediction using routine physiological data and clinical narratives

INTRODUCTION

The urgency and complexity of emergency room (ER) settings require precise and swift decision-making processes for patient care. Ensuring the timely execution of critical examinations and interventions is vital for reducing diagnostic errors, but the literature highlights a need for innovative approaches to optimize diagnostic accuracy and patient outcomes. In response, our study endeavors to create predictive models for timely examinations and interventions by leveraging the patient's symptoms and vital signs recorded during triage, and in so doing, augment traditional diagnostic methodologies.

METHODS

Focusing on four key areas-medication dispensing, vital interventions, laboratory testing, and emergency radiology exams, the study employed Natural Language Processing (NLP) and seven advanced machine learning techniques. The research was centered around the innovative use of BioClinicalBERT, a state-of-the-art NLP framework.

RESULTS

BioClinicalBERT emerged as the superior model, outperforming others in predictive accuracy. The integration of physiological data with patient narrative symptoms demonstrated greater effectiveness compared to models based solely on textual data. The robustness of our approach was confirmed by an Area Under the Receiver Operating Characteristic curve (AUROC) score of 0.9.

CONCLUSION

The findings of our study underscore the feasibility of establishing a decision support system for emergency patients, targeting timely interventions and examinations based on a nuanced analysis of symptoms. By using an advanced natural language processing technique, our approach shows promise for enhancing diagnostic accuracy. However, the current model is not yet fully mature for direct implementation into daily clinical practice. Recognizing the imperative nature of precision in the ER environment, future research endeavors must focus on refining and expanding predictive models to include detailed timely examinations and interventions. Although the progress achieved in this study represents an encouraging step towards a more innovative and technology-driven paradigm in emergency care, full clinical integration warrants further exploration and validation.

Exploring the potential of artificial intelligence models for triage in the emergency department

OBJECTIVE

To perform a comparative analysis of the three-level triage protocol conducted by triage nurses and emergency medicine doctors with the use of ChatGPT, Gemini, and Pi, which are recognized artificial intelligence (AI) models widely used in the daily life.

MATERIALS AND METHODS

The study was prospectively conducted with patients presenting to the emergency department of a tertiary care hospital from 1 April 2024, to 7 April 2024. Among the patients who presented to the emergency department over this period, data pertaining to their primary complaints, arterial blood pressure values, heart rates, peripheral oxygen saturation values measured by pulse oximetry, body temperature values, age, and gender characteristics were analyzed. The triage categories determined by triage nurses, the abovementioned AI chatbots, and emergency medicine doctors were compared.

RESULTS

The study included 500 patients, of whom 23.8% were categorized identically by all triage evaluators. Compared to the triage conducted by emergency medicine doctors, triage nurses overtriaged 6.4% of the patients and undertriaged 3.1% of the yellow-coded patients and 3.4% of the red-coded patients. Of the AI chatbots, ChatGPT exhibited the closest triage approximation to that of emergency medicine doctors; however, its undertriage rates were 26.5% for yellow-coded patients and 42.6% for red-coded patients.

CONCLUSION

The undertriage rates observed in AI models were considerably high. Hence, it does not yet seem appropriate to solely rely on the specified AI models for triage purposes in the emergency department.

Predictability of adult patient medical emergency condition from triage vital signs and comorbidities: a single-center, observational study

BACKGROUND

Vital signs and comorbid diseases are the first information evaluated in patients admitted to the emergency department (ED). In most EDs, triage of patients takes place with vital signs and admission complaints only. Comorbidities are generally underestimated when determining the patient's status at the triage area. This study aims to assess the relationship between initial vital signs, comorbid diseases, and medical emergency conditions (MEC) in patients admitted to the ED.

METHODS

This prospective study was designed as a single-center observational study, including patients admitted to a tertiary ED between 16.06.2022 and 09.09.2022. Patients younger than 18, readmitted to the ED within 24 h, or absence of vital signs due to cardiac arrest were excluded from the study. Vital signs and comorbid diseases of all patients were recorded. The mortality within 24 h, the need for intensive care unit admission, emergency surgery, and life-saving procedures were considered "medical emergency conditions". The role of vital signs and comorbid diseases in predicting emergencies was analyzed by binary logistic regression.

RESULTS

A total of 10,022 patients were included in the study; 5056 (50.4%) were female, and 4966 (49.6%) were male. Six hundred four patients presented with an MEC. All vital signs -except diastolic hypertension and tachycardia- and comorbidities were found statistically significant. Hypoxia (Odd's Ratio [OR]: 1.73), diastolic hypotension (OR: 3.71), tachypnea (OR: 8.09), and tachycardia (OR: 1.61) were associated with MECs. Hemiplegia (OR: 5.7), leukemia (OR: 4.23), and moderate-severe liver disease (OR: 2.99) were the most associated comorbidities with MECs. In our study, an MEC was detected in 3.6% (186 patients) of the patients with no abnormal vital signs and without any comorbidities.

CONCLUSION

Among the vital signs, hypoxia, diastolic hypotension, tachypnea, and tachycardia should be considered indicators of an MEC. Hemiplegia, leukemia, and moderate-severe liver disease are the most relevant comorbidities that may accompany the MECs.

Orthopedic disease classification based on breadth-first search algorithm

Orthopedic diseases are widespread worldwide, impacting the body's musculoskeletal system, particularly those involving bones or hips. They have the potential to cause discomfort and impair functionality. This paper aims to address the lack of supplementary diagnostics in orthopedics and improve the method of diagnosing orthopedic diseases. The study uses binary breadth-first search (BBFS), binary particle swarm optimization (BPSO), binary grey wolf optimizer (BGWO), and binary whale optimization algorithm (BWAO) for feature selections, and the BBFS makes an average error of 47.29% less than others. Then we apply six machine learning models, i.e., RF, SGD, NBC, DC, QDA, and ET. The dataset used contains 310 instances and six distinct features. Through experimentation, the RF model led to optimal outcomes during comparison to the remaining models, with an accuracy of 91.4%. The parameters of the RF model were optimized using four optimization algorithms: BFS, PSO, WAO, and GWO. To check how well the optimized RF works on the dataset, this paper uses prediction evaluation metrics such as accuracy, sensitivity, specificity, F-score, and the AUC curve. The results showed that the BFS-RF can improve the performance of the original classifier compared with others with 99.41% accuracy.

Development and validation of a machine learning framework for improved resource allocation in the emergency department

OBJECTIVE

The Emergency Severity Index (ESI) is the most commonly used system in over 70% of all U.S. emergency departments (ED) that uses predicted resource utilization as a means to triage [1], Mistriage, which includes both undertriage and overtriage has been a persistent issue, affecting 32.2% of total ED visits [2]. Our goal is to develop a machine learning framework that predicts patients' resource needs, thereby improving resource allocation during triage.

METHODS

This retrospective study analyzed ED visits from the Medical Information Mart for Intensive Care IV, dividing the data into training (80%) and testing (20%) cohorts. We utilized data available during triage, including patient vital signs, age, gender, mode of arrival, medication history, and chief complaint. Azure AutoML was used to create different machine learning models trained to predict the 144 target columns including laboratory panels and imaging modalities as well as medications required during patients' ED visits. The 144 models' performance was evaluated using the area under the receiver operating characteristic curve (AUROC), F1 score, accuracy, precision and recall.

RESULTS

A total of 391,472 ED visits were analyzed. 144 Voting ensemble models were created for each target. All frameworks achieved on average an AUC score of 0.82 and accuracy of 0.76. We gathered the feature importance for each target and observed that 'chief complaint', among others, had a high aggregate feature importance across different targets.

CONCLUSION

This study shows the high accuracy in predicting resource needs for patients in the ED using a machine learning model. This can greatly improve patient flow and resource allocation in already resource limited emergency departments.

Emergency Severity Index Version 4 and Triage of Pediatric Emergency Department Patients

Importance

Most emergency departments (EDs) across the US use the Emergency Severity Index (ESI) to predict acuity and resource needs. A comprehensive assessment of ESI accuracy among pediatric patients is lacking.

Objective

To assess the frequency of mistriage using ESI (version 4) among pediatric ED visits using automated measures of mistriage and identify characteristics associated with mistriage.

Design, Setting, and Participants

This cohort study used operational measures for each ESI level to classify encounters as undertriaged, overtriaged, or correctly triaged to assess the accuracy of the ESI and identify characteristics of mistriage. Participants were pediatric patients at 21 EDs within Kaiser Permanente Northern California from January 1, 2016, to December 31, 2020. During that time, version 4 of the ESI was in use by these EDs. Visits with missing ESI, incomplete ED time variables, patients transferred from another ED, and those who left against medical advice or without being seen were excluded. Data were analyzed between January 2022 and June 2023.

Exposures

Assigned ESI level.

Main Outcomes and Measures

Rates of undertriage and overtriage by assigned ESI level based on mistriage algorithm, patient and visit characteristics associated with undertriage and overtriage.

Results

This study included 1 016 816 pediatric ED visits; the mean (SD) age of patients was 7.3 (5.6) years, 479 610 (47.2%) were female, and 537 206 (52.8%) were male. Correct triage occurred in 346 918 visits (34.1%; 95% CI, 34.0%-34.2%), while overtriage and undertriage occurred in 594 485 visits (58.5%; 95% CI, 58.4%-58.6%) and 75 413 visits (7.4%; 95% CI, 7.4%-7.5%), respectively. In adjusted analyses, undertriage was more common among children at least 6 years old compared with those younger 6 years; male patients compared with female patients; patients with Asian, Black, or Hispanic or other races or ethnicities compared with White patients; patients with comorbid illnesses compared with those without; and patients who arrived by ambulance compared with nonambulance patients.

Conclusions and Relevance

This multicenter retrospective study found that mistriage with ESI version 4 was common in pediatric ED visits. There is an opportunity to improve pediatric ED triage, both in early identification of critically ill patients (limit undertriage) and in more accurate identification of low-acuity patients with low resource needs (limit overtriage). Future research should include assessments based on version 5 of the ESI, which was released after this study was completed.

Acute cholecystitis diagnosis in the emergency department: an artificial intelligence-based approach

OBJECTIVES

This study aimed to assess the diagnostic performance of a support vector machine (SVM) algorithm for acute cholecystitis and evaluate its effectiveness in accurately diagnosing this condition.

METHODS

Using a retrospective analysis of patient data from a single center, individuals with abdominal pain lasting one week or less were included. The SVM model was trained and optimized using standard procedures. Model performance was assessed through sensitivity, specificity, accuracy, and AUC-ROC, with probability calibration evaluated using the Brier score.

RESULTS

Among 534 patients, 198 (37.07%) were diagnosed with acute cholecystitis. The SVM model showed balanced performance, with a sensitivity of 83.08% (95% CI: 71.73-91.24%), a specificity of 80.21% (95% CI: 70.83-87.64%), and an accuracy of 81.37% (95% CI: 74.48-87.06%). The positive predictive value (PPV) was 73.97% (95% CI: 65.18-81.18%), the negative predictive value (NPV) was 87.50% (95% CI: 80.19-92.37%), and the AUC-ROC was 0.89 (95% CI: 0.85 to 0.93). The Brier score indicated well-calibrated probability estimates.

CONCLUSION

The SVM algorithm demonstrated promising potential for accurately diagnosing acute cholecystitis. Further refinement and validation are needed to enhance its reliability in clinical practice.

Using Machine Learning to Select Breast Implant Volume

BACKGROUND

In breast augmentation surgery, selection of the appropriate breast implant size is a crucial step that can greatly affect patient satisfaction and the outcome of the procedure. However, this decision is often based on the subjective judgment of the surgeon and the patient, which can lead to suboptimal results. The authors aimed to develop a machine-learning approach that can accurately predict the size of breast implants selected for breast augmentation surgery.

METHODS

The authors collected data on patient demographic characteristics, medical history, and surgeon preferences from a sample of 1000 consecutive patients who underwent breast augmentation. This information was used to train and test a supervised machine-learning model to predict the size of breast implant needed.

RESULTS

The study demonstrated the effectiveness of the algorithm in predicting breast implant size, achieving a Pearson correlation coefficient of 0.9335 ( P < 0.001). The model generated accurate predictions in 86% of instances, with a mean absolute error of 27.10 mL. Its effectiveness was confirmed in the reoperation group, in which 36 of 57 patients (63%) would have received a more suitable implant size if the model's suggestion had been followed, potentially avoiding reoperation.

CONCLUSIONS

The findings show that machine learning can accurately predict the needed size of breast implants in augmentation surgery. By integrating the artificial intelligence model into a decision support system for breast augmentation surgery, essential guidance can be provided to surgeons and patients. This approach not only streamlines the implant selection process but also facilitates enhanced communication and decision-making, ultimately leading to more reliable outcomes and improved patient satisfaction.

Machine learning models predict triage levels, massive transfusion protocol activation, and mortality in trauma utilizing patients hemodynamics on admission

BACKGROUND

The effective management of trauma patients necessitates efficient triaging, timely activation of Massive Blood Transfusion Protocols (MTP), and accurate prediction of in-hospital outcomes. Machine learning (ML) algorithms have emerged as up-and-coming tools in the domains of optimizing triage decisions, improving intervention strategies, and predicting clinical outcomes, consistently outperforming traditional methodologies. This study aimed to develop, assess, and compare several ML models for the triaging processes, activation of MTP, and mortality prediction.

METHODS

In a 10-year retrospective study, the predictive capabilities of seven ML models for trauma patients were systematically assessed using on-admission patients' hemodynamic data. All patient's data were randomly divided into training (80 %) and test (20 %) sets. Employing Python for data preprocessing, feature scaling, and model development, we evaluated K-Nearest Neighbors (KNN), Logistic Regression (LR), Decision Tree (DT), Support Vector Machines (SVM) with RBF kernels, Random Forest (RF), Extreme Gradient Boosting (XGBoost), and Artificial Neural Network (ANN). We employed various imputation techniques and addressed data imbalance through down-sampling, up-sampling, and synthetic minority for the over-sampling technique (SMOTE). Hyperparameter tuning, coupled with 5-fold cross-validation, was performed. The evaluation included essential metrics like sensitivity, specificity, F1 score, accuracy, Area Under the Receiver Operating Curve (AUC ROC), and Area Under the Precision recall Curve (AUC PR), ensuring robust predictive capability.

RESULT

This study included 17,390 adult trauma patients; of them, 19.5 % (3385) were triaged at a critical level, 3.8 % (664) required MTP, and 7.7 % (1335) died in the hospital. The model's performance improved using imputation and balancing techniques. The overall models demonstrated notable performance metrics for predicting triage, MTP activation, and mortality with F1 scores of 0.75, 0.42, and 0.79, sensitivities of 0.73, 0.82, and 0.9, and AUC ROC values of 0.89, 0.95 and 0.99 respectively.

CONCLUSION

Machine learning, especially RF models, effectively predicted trauma triage, MTP activation, and mortality. Featured critical hemodynamic variables include shock indices, systolic blood pressure, and mean arterial pressure. Therefore, models can do better than individual parameters for the early management and disposition of patients in the ED. Future research should focus on creating sensitive and interpretable models to enhance trauma care.

Aspiring to clinical significance: Insights from developing and evaluating a machine learning model to predict emergency department return visit admissions

Return visit admissions (RVA), which are instances where patients discharged from the emergency department (ED) rapidly return and require hospital admission, have been associated with quality issues and adverse outcomes. We developed and validated a machine learning model to predict 72-hour RVA using electronic health records (EHR) data. Study data were extracted from EHR data in 2019 from three urban EDs. The development and independent validation datasets included 62,154 patients from two EDs and 73,453 patients from one ED, respectively. Multiple machine learning algorithms were evaluated, including deep significance clustering (DICE), regularized logistic regression (LR), Gradient Boosting Decision Tree, and XGBoost. These machine learning models were also compared against an existing clinical risk score. To support clinical actionability, clinician investigators conducted manual chart reviews of the cases identified by the model. Chart reviews categorized predicted cases across index ED discharge diagnosis and RVA root cause classifications. The best-performing model achieved an AUC of 0.87 in the development site (test set) and 0.75 in the independent validation set. The model, which combined DICE and LR, boosted predictive performance while providing well-defined features. The model was relatively robust to sensitivity analyses regarding performance across age, race, and by varying predictor availability but less robust across diagnostic groups. Clinician examination demonstrated discrete model performance characteristics within clinical subtypes of RVA. This machine learning model demonstrated a strong predictive performance for 72- RVA. Despite the limited clinical actionability potentially due to model complexity, the rarity of the outcome, and variable relevance, the clinical examination offered guidance on further variable inclusion for enhanced predictive accuracy and actionability.

Traditional Machine Learning, Deep Learning, and BERT (Large Language Model) Approaches for Predicting Hospitalizations From Nurse Triage Notes: Comparative Evaluation of Resource Management

BACKGROUND

Predicting hospitalization from nurse triage notes has the potential to augment care. However, there needs to be careful considerations for which models to choose for this goal. Specifically, health systems will have varying degrees of computational infrastructure available and budget constraints.

OBJECTIVE

To this end, we compared the performance of the deep learning, Bidirectional Encoder Representations from Transformers (BERT)-based model, Bio-Clinical-BERT, with a bag-of-words (BOW) logistic regression (LR) model incorporating term frequency-inverse document frequency (TF-IDF). These choices represent different levels of computational requirements.

METHODS

A retrospective analysis was conducted using data from 1,391,988 patients who visited emergency departments in the Mount Sinai Health System spanning from 2017 to 2022. The models were trained on 4 hospitals' data and externally validated on a fifth hospital's data.

RESULTS

The Bio-Clinical-BERT model achieved higher areas under the receiver operating characteristic curve (0.82, 0.84, and 0.85) compared to the BOW-LR-TF-IDF model (0.81, 0.83, and 0.84) across training sets of 10,000; 100,000; and ~1,000,000 patients, respectively. Notably, both models proved effective at using triage notes for prediction, despite the modest performance gap.

CONCLUSIONS

Our findings suggest that simpler machine learning models such as BOW-LR-TF-IDF could serve adequately in resource-limited settings. Given the potential implications for patient care and hospital resource management, further exploration of alternative models and techniques is warranted to enhance predictive performance in this critical domain.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1101/2023.08.07.23293699.

Using machine learning methods to predict all-cause somatic hospitalizations in adults: A systematic review

AIM

In this review, we investigated how Machine Learning (ML) was utilized to predict all-cause somatic hospital admissions and readmissions in adults.

METHODS

We searched eight databases (PubMed, Embase, Web of Science, CINAHL, ProQuest, OpenGrey, WorldCat, and MedNar) from their inception date to October 2023, and included records that predicted all-cause somatic hospital admissions and readmissions of adults using ML methodology. We used the CHARMS checklist for data extraction, PROBAST for bias and applicability assessment, and TRIPOD for reporting quality.

RESULTS

We screened 7,543 studies of which 163 full-text records were read and 116 met the review inclusion criteria. Among these, 45 predicted admission, 70 predicted readmission, and one study predicted both. There was a substantial variety in the types of datasets, algorithms, features, data preprocessing steps, evaluation, and validation methods. The most used types of features were demographics, diagnoses, vital signs, and laboratory tests. Area Under the ROC curve (AUC) was the most used evaluation metric. Models trained using boosting tree-based algorithms often performed better compared to others. ML algorithms commonly outperformed traditional regression techniques. Sixteen studies used Natural language processing (NLP) of clinical notes for prediction, all studies yielded good results. The overall adherence to reporting quality was poor in the review studies. Only five percent of models were implemented in clinical practice. The most frequently inadequately addressed methodological aspects were: providing model interpretations on the individual patient level, full code availability, performing external validation, calibrating models, and handling class imbalance.

CONCLUSION

This review has identified considerable concerns regarding methodological issues and reporting quality in studies investigating ML to predict hospitalizations. To ensure the acceptability of these models in clinical settings, it is crucial to improve the quality of future studies.

Identification of avoidable patients at triage in a Paediatric Emergency Department: a decision support system using predictive analytics

BACKGROUND

Crowding has been a longstanding issue in emergency departments. To address this, a fast-track system for avoidable patients is being implemented in the Paediatric Emergency Department where our study is conducted. Our goal is to develop an optimized Decision Support System that helps in directing patients to this fast track. We evaluated various Machine Learning models, focusing on a balance between complexity, predictive performance, and interpretability.

METHODS

This is a retrospective study considering all visits to a university-affiliated metropolitan hospital's PED between 2014 and 2019. Using information available at the time of triage, we trained several models to predict whether a visit is avoidable and should be directed to a fast-track area.

RESULTS

A total of 507,708 visits to the PED were used in the training and testing of the models. Regarding the outcome, 41.6% of the visits were considered avoidable. Except for the classification made by triage rules, i.e. considering levels 1,2, and 3 as non-avoidable and 4 and 5 as avoidable, all models had similar results in model's evaluation metrics, e.g. Area Under the Curve ranging from 74% to 80%.

CONCLUSIONS

Regarding predictive performance, the pruned decision tree had evaluation metrics results that were comparable to the other ML models. Furthermore, it offers a low complexity and easy to implement solution. When considering interpretability, a paramount requisite in healthcare since it relates to the trustworthiness and transparency of the system, the pruned decision tree excels. Overall, this paper contributes to the growing body of research on the use of machine learning in healthcare. It highlights practical benefits for patients and healthcare systems of the use ML-based DSS in emergency medicine. Moreover, the obtained results can potentially help to design patients' flow management strategies in PED settings, which has been sought as a solution for addressing the long-standing problem of overcrowding.

Assessing triage efficiency in Italy: a comparative study using simulated cases among nurses

BACKGROUND

Emergency Departments (EDs) across Italy use different triage systems, which vary from region to region. This study aimed to assess whether nurses working in different EDs assign triage codes in a similar and standardized manner.

METHODS

A multicenter observational simulation study involved the EDs of Bolzano Hospital, Merano Hospital, Pisa University Hospital, and Rovereto Hospital. All participating nurses were given 30 simulated clinical cases (vignettes) and asked to assign triage codes according to the triage systems used in their EDs. Subsequently, we assessed inter-rater agreement and evaluated if code assignment had different performance among hospitals in relation to different clinical outcomes.

RESULTS

Eighty-seven nurses participated in this study. There was marked variation in assigned triage codes both across hospitals and among individual operators. The kappa values for inter-rater agreement were 0.632 for Bolzano Hospital, 0.589 for Merano Hospital, 0.464 for Pisa University Hospital, and 0.574 for Rovereto Hospital. Sensitivity and specificity levels varied considerably for the same outcomes when comparing different hospitals.

CONCLUSION

There is a high degree of subjectivity in triage code assignment by ED nurses. In the interest of equitable care for patients, this variability within the same country is hardly acceptable.

The AI Future of Emergency Medicine

In the coming years, artificial intelligence (AI) and machine learning will likely give rise to profound changes in the field of emergency medicine, and medicine more broadly. This article discusses these anticipated changes in terms of 3 overlapping yet distinct stages of AI development. It reviews some fundamental concepts in AI and explores their relation to clinical practice, with a focus on emergency medicine. In addition, it describes some of the applications of AI in disease diagnosis, prognosis, and treatment, as well as some of the practical issues that they raise, the barriers to their implementation, and some of the legal and regulatory challenges they create.

Patient stratification based on the risk of severe illness in emergency departments through collaborative machine learning models

OBJECTIVES

Emergency department (ED) overcrowding presents a global challenge that inhibits prompt care for critically ill patients. Traditional 5-level triage system that heavily rely on the judgment of the triage staff could fail to detect subtle symptoms in critical patients, thus leading to delayed treatment. Unlike previous rivalry-focused approaches, our study aimed to establish a collaborative machine learning (ML) model that renders risk scores for severe illness, which may assist the triage staff to provide a better patient stratification for timely critical cares.

METHODS

This retrospective study was conducted at a tertiary teaching hospital. Data were collected from January 2015 to October 2022. Demographic and clinical information were collected at triage. The study focused on severe illness as the outcome. We developed artificial neural network (ANN) models, with or without utilizing the Taiwan Triage and Acuity Scale (TTAS) score as one of the predictors. The model using the TTAS score is termed a machine-human collaborative model (ANN-MH), while the model without it is referred to as a machine-only model (ANN-MO). The predictive power of these models was assessed using the area under the receiver-operating-characteristic (AUROC) and the precision-recall curves (AUPRC); their sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and F1 score were compared.

RESULTS

The study analyzed 668,602 ED visits from 2015 to 2022. Among them, 278,724 visits from 2015 to 2018 were used for model training and validation, while 320,201 visits from 2019 to 2022 were for testing model performance. Approximately 2.6% of visits were by severely ill patients, whose TTAS scores ranged from 1 to 5. The ANN-MH model achieved a testing AUROC of 0.918 and AUPRC of 0.369, while for the ANN-MO model the AUROC and AUPRC were 0.909 and 0.339, respectively. Based on these metrics, the ANN-MH model outperformed the ANN-MO model, and both surpassed human triage classification. Subgroup analyses further highlighted the models' capability to identify higher-risk patients within the same triage level.

CONCLUSIONS

The traditional 5-level triage system often falls short, leading to under-triage of critical patients. Our models include a score-based differentiation within a triage level to offer advanced risk stratification, thereby promoting patient safety.

Interpretable machine learning for the prediction of death risk in patients with acute diquat poisoning

The aim of this study was to develop and validate predictive models for assessing the risk of death in patients with acute diquat (DQ) poisoning using innovative machine learning techniques. Additionally, predictive models were evaluated through the application of SHapley Additive ExPlanations (SHAP). A total of 201 consecutive patients from the emergency departments of the First Hospital and Shengjing Hospital of China Medical University admitted for deliberate oral intake of DQ from February 2018 to August 2023 were analysed. The initial clinical data of the patients with acute DQ poisoning were collected. Machine learning methods such as logistic regression, random forest, support vector machine (SVM), and gradient boosting were applied to build the prediction models. The whole sample was split into a training set and a test set at a ratio of 8:2. The performances of these models were assessed in terms of discrimination, calibration, and clinical decision curve analysis (DCA). We also used the SHAP interpretation tool to provide an intuitive explanation of the risk of death in patients with DQ poisoning. Logistic regression, random forest, SVM, and gradient boosting models were established, and the areas under the receiver operating characteristic curves (AUCs) were 0.91, 0.98, 0.96 and 0.94, respectively. The net benefits were similar across all four models. The four machine learning models can be reliable tools for predicting death risk in patients with acute DQ poisoning. Their combination with SHAP provides explanations for individualized risk prediction, increasing the model transparency.

Emergency Patient Triage Improvement through a Retrieval-Augmented Generation Enhanced Large-Scale Language Model

OBJECTIVES

Emergency medical triage is crucial for prioritizing patient care in emergency situations, yet its effectiveness can vary significantly based on the experience and training of the personnel involved. This study aims to evaluate the efficacy of integrating Retrieval Augmented Generation (RAG) with Large Language Models (LLMs), specifically OpenAI's GPT models, to standardize triage procedures and reduce variability in emergency care.

METHODS

We created 100 simulated triage scenarios based on modified cases from the Japanese National Examination for Emergency Medical Technicians. These scenarios were processed by the RAG-enhanced LLMs, and the models were given patient vital signs, symptoms, and observations from emergency medical services (EMS) teams as inputs. The primary outcome was the accuracy of triage classifications, which was used to compare the performance of the RAG-enhanced LLMs with that of emergency medical technicians and emergency physicians. Secondary outcomes included the rates of under-triage and over-triage.

RESULTS

The Generative Pre-trained Transformer 3.5 (GPT-3.5) with RAG model achieved a correct triage rate of 70%, significantly outperforming Emergency Medical Technicians (EMTs) with 35% and 38% correct rates, and emergency physicians with 50% and 47% correct rates (p < 0.05). Additionally, this model demonstrated a substantial reduction in under-triage rates to 8%, compared with 33% for GPT-3.5 without RAG, and 39% for GPT-4 without RAG.

CONCLUSIONS

The integration of RAG with LLMs shows promise in improving the accuracy and consistency of medical assessments in emergency settings. Further validation in diverse medical settings with broader datasets is necessary to confirm the effectiveness and adaptability of these technologies in live environments.

Admission prioritization of heart failure patients with multiple comorbidities

The primary objective of this study was to enhance the operational efficiency of the current healthcare system by proposing a quicker and more effective approach for healthcare providers to deliver services to individuals facing acute heart failure (HF) and concurrent medical conditions. The aim was to support healthcare staff in providing urgent services more efficiently by developing an automated decision-support Patient Prioritization (PP) Tool that utilizes a tailored machine learning (ML) model to prioritize HF patients with chronic heart conditions and concurrent comorbidities during Urgent Care Unit admission. The study applies key ML models to the PhysioNet dataset, encompassing hospital admissions and mortality records of heart failure patients at Zigong Fourth People's Hospital in Sichuan, China, between 2016 and 2019. In addition, the model outcomes for the PhysioNet dataset are compared with the Healthcare Cost and Utilization Project (HCUP) Maryland (MD) State Inpatient Data (SID) for 2014, a secondary dataset containing heart failure patients, to assess the generalizability of results across diverse healthcare settings and patient demographics. The ML models in this project demonstrate efficiencies surpassing 97.8% and specificities exceeding 95% in identifying HF patients at a higher risk and ranking them based on their mortality risk level. Utilizing this machine learning for the PP approach underscores risk assessment, supporting healthcare professionals in managing HF patients more effectively and allocating resources to those in immediate need, whether in hospital or telehealth settings.

Strategies to identify medical patients suitable for management through same-day emergency care services: A systematic review

Same-day emergency care (SDEC) in unplanned and emergency care is an NHS England (NHSE) priority. Optimal use of these services requires rapid identification of suitable patients. NHSE suggests the use of one tool for this purpose. This systematic review compares studies that evaluate the performance of selection tools for SDEC pathways. Nine studies met the inclusion criteria. Three scores were evaluated: the Amb score (seven studies), Glasgow Admission Prediction Score (GAPS) (six studies) and Sydney Triage to Admission Risk Tool (START) (two studies). There was heterogeneity in the populations assessed, exclusion criteria used and definitions used for SDEC suitability, with proportions of patients deemed 'suitable' for SDEC ranging from 20 to 80%. Reported score sensitivity and specificity ranged between 18-99% and 10-89%. Score performance could not be compared due to heterogeneity between studies. No studies assessed clinical implementation. The current evidence to support the use of a specific tool for SDEC is limited and requires further evaluation.

Machine learning for the prediction of in-hospital mortality in patients with spontaneous intracerebral hemorrhage in intensive care unit

This study aimed to develop a machine learning (ML)-based tool for early and accurate prediction of in-hospital mortality risk in patients with spontaneous intracerebral hemorrhage (sICH) in the intensive care unit (ICU). We did a retrospective study in our study and identified cases of sICH from the MIMIC IV (n = 1486) and Zhejiang Hospital databases (n = 110). The model was constructed using features selected through LASSO regression. Among five well-known models, the selection of the best model was based on the area under the curve (AUC) in the validation cohort. We further analyzed calibration and decision curves to assess prediction results and visualized the impact of each variable on the model through SHapley Additive exPlanations. To facilitate accessibility, we also created a visual online calculation page for the model. The XGBoost exhibited high accuracy in both internal validation (AUC = 0.907) and external validation (AUC = 0.787) sets. Calibration curve and decision curve analyses showed that the model had no significant bias as well as being useful for supporting clinical decisions. XGBoost is an effective algorithm for predicting in-hospital mortality in patients with sICH, indicating its potential significance in the development of early warning systems.

Triage Performance Across Large Language Models, ChatGPT, and Untrained Doctors in Emergency Medicine: Comparative Study

BACKGROUND

Large language models (LLMs) have demonstrated impressive performances in various medical domains, prompting an exploration of their potential utility within the high-demand setting of emergency department (ED) triage. This study evaluated the triage proficiency of different LLMs and ChatGPT, an LLM-based chatbot, compared to professionally trained ED staff and untrained personnel. We further explored whether LLM responses could guide untrained staff in effective triage.

OBJECTIVE

This study aimed to assess the efficacy of LLMs and the associated product ChatGPT in ED triage compared to personnel of varying training status and to investigate if the models' responses can enhance the triage proficiency of untrained personnel.

METHODS

A total of 124 anonymized case vignettes were triaged by untrained doctors; different versions of currently available LLMs; ChatGPT; and professionally trained raters, who subsequently agreed on a consensus set according to the Manchester Triage System (MTS). The prototypical vignettes were adapted from cases at a tertiary ED in Germany. The main outcome was the level of agreement between raters' MTS level assignments, measured via quadratic-weighted Cohen κ. The extent of over- and undertriage was also determined. Notably, instances of ChatGPT were prompted using zero-shot approaches without extensive background information on the MTS. The tested LLMs included raw GPT-4, Llama 3 70B, Gemini 1.5, and Mixtral 8x7b.

RESULTS

GPT-4-based ChatGPT and untrained doctors showed substantial agreement with the consensus triage of professional raters (κ=mean 0.67, SD 0.037 and κ=mean 0.68, SD 0.056, respectively), significantly exceeding the performance of GPT-3.5-based ChatGPT (κ=mean 0.54, SD 0.024; P<.001). When untrained doctors used this LLM for second-opinion triage, there was a slight but statistically insignificant performance increase (κ=mean 0.70, SD 0.047; P=.97). Other tested LLMs performed similar to or worse than GPT-4-based ChatGPT or showed odd triaging behavior with the used parameters. LLMs and ChatGPT models tended toward overtriage, whereas untrained doctors undertriaged.

CONCLUSIONS

While LLMs and the LLM-based product ChatGPT do not yet match professionally trained raters, their best models' triage proficiency equals that of untrained ED doctors. In its current form, LLMs or ChatGPT thus did not demonstrate gold-standard performance in ED triage and, in the setting of this study, failed to significantly improve untrained doctors' triage when used as decision support. Notable performance enhancements in newer LLM versions over older ones hint at future improvements with further technological development and specific training.

Can artificial intelligence help ED nurses more accurately triage patients?

ABSTRACT

The Emergency Severity Index (ESI) is the most popular tool used to triage patients in the US and abroad. Evidence has shown that ESI has its limitations in correctly assigning acuity. To address this, AI can be incorporated into the triage process, decreasing the likelihood of assigning an incorrect ESI level.

Drinking from the Holy Grail-Does a Perfect Triage System Exist? And Where to Look for It?

The Emergency Department (ED) is a facility meant to treat patients in need of medical assistance. The choice of triage system hugely impactsed the organization of any given ED and it is important to analyze them for their effectiveness. The goal of this review is to briefly describe selected triage systems in an attempt to find the perfect one. Papers published in PubMed from 1990 to 2022 were reviewed. The following terms were used for comparison: "ED" and "triage system". The papers contained data on the design and function of the triage system, its validation, and its performance. After studies comparing the distinct means of patient selection were reviewed, they were meant to be classified as either flawed or non-ideal. The validity of all the comparable segregation systems was similar. A possible solution would be to search for a new, measurable parameter for a more accurate risk estimation, which could be a game changer in terms of triage assessment. The dynamic development of artificial intelligence (AI) technologies has recently been observed. The authors of this study believe that the future segregation system should be a combination of the experience and intuition of trained healthcare professionals and modern technology (artificial intelligence).

Prediction of tuberculosis clusters in the riverine municipalities of the Brazilian Amazon with machine learning

OBJECTIVE

Tuberculosis (TB) is the second most deadly infectious disease globally, posing a significant burden in Brazil and its Amazonian region. This study focused on the "riverine municipalities" and hypothesizes the presence of TB clusters in the area. We also aimed to train a machine learning model to differentiate municipalities classified as hot spots vs. non-hot spots using disease surveillance variables as predictors.

METHODS

Data regarding the incidence of TB from 2019 to 2022 in the riverine town was collected from the Brazilian Health Ministry Informatics Department. Moran's I was used to assess global spatial autocorrelation, while the Getis-Ord GI* method was employed to detect high and low-incidence clusters. A Random Forest machine-learning model was trained using surveillance variables related to TB cases to predict hot spots among non-hot spot municipalities.

RESULTS

Our analysis revealed distinct geographical clusters with high and low TB incidence following a west-to-east distribution pattern. The Random Forest Classification model utilizes six surveillance variables to predict hot vs. non-hot spots. The machine learning model achieved an Area Under the Receiver Operator Curve (AUC-ROC) of 0.81.

CONCLUSION

Municipalities with higher percentages of recurrent cases, deaths due to TB, antibiotic regimen changes, percentage of new cases, and cases with smoking history were the best predictors of hot spots. This prediction method can be leveraged to identify the municipalities at the highest risk of being hot spots for the disease, aiding policymakers with an evidenced-based tool to direct resource allocation for disease control in the riverine municipalities.

Human intelligence versus Chat-GPT: who performs better in correctly classifying patients in triage?

INTRODUCTION

Chat-GPT is rapidly emerging as a promising and potentially revolutionary tool in medicine. One of its possible applications is the stratification of patients according to the severity of clinical conditions and prognosis during the triage evaluation in the emergency department (ED).

METHODS

Using a randomly selected sample of 30 vignettes recreated from real clinical cases, we compared the concordance in risk stratification of ED patients between healthcare personnel and Chat-GPT. The concordance was assessed with Cohen's kappa, and the performance was evaluated with the area under the receiver operating characteristic curve (AUROC) curves. Among the outcomes, we considered mortality within 72 h, the need for hospitalization, and the presence of a severe or time-dependent condition.

RESULTS

The concordance in triage code assignment between triage nurses and Chat-GPT was 0.278 (unweighted Cohen's kappa; 95% confidence intervals: 0.231-0.388). For all outcomes, the ROC values were higher for the triage nurses. The most relevant difference was found in 72-h mortality, where triage nurses showed an AUROC of 0.910 (0.757-1.000) compared to only 0.669 (0.153-1.000) for Chat-GPT.

CONCLUSIONS

The current level of Chat-GPT reliability is insufficient to make it a valid substitute for the expertise of triage nurses in prioritizing ED patients. Further developments are required to enhance the safety and effectiveness of AI for risk stratification of ED patients.

Use of Artificial Intelligence in Triage in Hospital Emergency Departments: A Scoping Review

The integration of artificial intelligence (AI) and machine learning (ML) in healthcare has become a major point of interest and raises the question of its impact on the emergency department (ED) triaging process. AI's capacity to emulate human cognitive processes coupled with advancements in computing has shown positive outcomes in various aspects of healthcare but little is known about the use of AI in triaging patients in ED. AI algorithms may allow for earlier diagnosis and intervention; however, overconfident answers may present dangers to patients. The purpose of this review was to explore comprehensively recently published literature regarding the effect of AI and ML in ED triage and identify research gaps. A systemized search was conducted in September 2023 using the electronic databases EMBASE, Ovid MEDLINE, and Web of Science. To meet inclusion criteria, articles had to be peer-reviewed, written in English, and based on primary data research studies published in US journals 2013-2023. Other criteria included 1) studies with patients needing to be admitted to hospital EDs, 2) AI must have been used when triaging a patient, and 3) patient outcomes must be represented. The search was conducted using controlled descriptors from the Medical Subject Headings (MeSH) that included the terms "artificial intelligence" OR "machine learning" AND "emergency ward" OR "emergency care" OR "emergency department" OR "emergency room" AND "patient triage" OR "triage" OR "triaging." The search initially identified 1,142 citations. After a rigorous, systemized screening process and critical appraisal of the evidence, 29 studies were selected for the final review. The findings indicated that 1) ML models consistently demonstrated superior discrimination abilities compared to conventional triage systems, 2) the integration of AI into the triage process yielded significant enhancements in predictive accuracy, disease identification, and risk assessment, 3) ML accurately determined the necessity of hospitalization for patients requiring urgent attention, and 4) ML improved resource allocation and quality of patient care, including predicting length of stay. The suggested superiority of ML models in prioritizing patients in the ED holds the potential to redefine triage precision.

Establishment and assessment of mortality risk prediction model in patients with sepsis based on early-stage peripheral lymphocyte subsets

This study is aimed to explore the value of lymphocyte subsets in evaluating the severity and prognosis of sepsis. The counts of lymphocytes, CD3+ T cells, CD4+ T cells, CD8+ T cells, CD19+ B cells, and NK cells significantly decreased between day 1 and day 3 in both the survivor and the non-survivor groups. The peripheral lymphocyte subsets (PLS) at day 1 were not significantly different between the survivor and the non-survivor groups. However, at day 3, the counts of lymphocytes, CD3+ T cells, CD4+ T cells, and NK cells were remarkably lower in the non-survivor group. No significant differences in CD8+ T cells, or CD19+ B cells were observed. The PLS index was independently and significantly associated with the 28-day mortality risk in septic patients (OR: 3.08, 95% CI: 1.18-9.67). Based on these clinical parameters and the PLS index, we developed a nomograph for evaluating the individual mortality risk in sepsis. The area under the curve of prediction with the PLS index was significantly higher than that from the model with only clinical parameters (0.912 vs. 0.817). Our study suggests that the decline of PLS occurred in the early stage of sepsis. The new novel PLS index can be an independent predictor of 28-day mortality in septic patients. The prediction model based on clinical parameters and the PLS index has relatively high predicting ability.

Randomized controlled open-label trial to evaluate prioritization software for the secondary triage of patients in the pediatric emergency department

BACKGROUND

The continual increase in patient attendance at the emergency department (ED) is a worldwide health issue. The aim of this study was to determine whether the use of a secondary prioritization software reduces the patients' median length of stay (LOS) in the pediatric ED.

METHODS

A randomized, controlled, open-label trial was conducted over a 30-day period between March 15th and April 23rd 2021 at Lille University Hospital. Work days were randomized to use the patient prioritization software or the pediatric ED's standard dashboard. All time intervals between admission and discharge were recorded prospectively by a physician not involved in patient care during the study period. The study's primary endpoint was the LOS in the pediatric ED, which was expected to be 15 min shorter in the intervention group than in the control group. The secondary endpoints were specific time intervals during the stay in the pediatric ED and levels of staff satisfaction.

RESULTS

1599 patients were included: 798 in the intervention group and 801 in the control group. The median [interquartile range] LOS was 172 min [113-255] in the intervention group and 167 min [108-254) in the control group (p = 0.46). In the intervention group, the time interval between admission to the first medical evaluation for high-priority patients and the time interval between the senior physician's final evaluation and patient discharge were shorter (p < 0.01). The median satisfaction score was 68 [55-80] (average).

CONCLUSION

The patients' total LOS was not significantly shorter on days of intervention. However, use of the electronic patient prioritization tool was associated with significant decreases in some important time intervals during care in the pediatric ED.

CLINICALTRIALS

gov: NCT05994196 Trial registration number: NCT05994196. Date of registration: August 16th, 2023.

Machine-learning models are superior to severity scoring systems for the prediction of the mortality of critically ill patients in a tertiary medical center

BACKGROUND

Intensive care unit (ICU) mortality prediction helps to guide therapeutic decision making for critically ill patients. Several scoring systems based on statistical techniques have been developed for this purpose. In this study, we developed a machine-learning model to predict patient mortality in the very early stage of ICU admission.

METHODS

This study was performed with data from all patients admitted to the intensive care units of a tertiary medical center in Taiwan from 2009 to 2018. The patients' comorbidities, co-medications, vital signs, and laboratory data on the day of ICU admission were obtained from electronic medical records. We constructed random forest and extreme gradient boosting (XGBoost) models to predict ICU mortality, and compared their performance with that of traditional scoring systems.

RESULTS

Data from 12,377 patients was allocated to training (n = 9901) and testing (n = 2476) datasets. The median patient age was 70.0 years; 9210 (74.41%) patients were under mechanical ventilation in the ICU. The areas under receiver operating characteristic curves for the random forest and XGBoost models (0.876 and 0.880, respectively) were larger than those for the Acute Physiology and Chronic Health Evaluation II score (0.738), Sequential Organ Failure Assessment score (0.747), and Simplified Acute Physiology Score II (0.743). The fraction of inspired oxygen on ICU admission was the most important predictive feature across all models.

CONCLUSION

The XGBoost model most accurately predicted ICU mortality and was superior to traditional scoring systems. Our results highlight the utility of machine learning for ICU mortality prediction in the Asian population.

Interpretable Deep Learning System for Identifying Critical Patients Through the Prediction of Triage Level, Hospitalization, and Length of Stay: Prospective Study

BACKGROUND

Triage is the process of accurately assessing patients' symptoms and providing them with proper clinical treatment in the emergency department (ED). While many countries have developed their triage process to stratify patients' clinical severity and thus distribute medical resources, there are still some limitations of the current triage process. Since the triage level is mainly identified by experienced nurses based on a mix of subjective and objective criteria, mis-triage often occurs in the ED. It can not only cause adverse effects on patients, but also impose an undue burden on the health care delivery system.

OBJECTIVE

Our study aimed to design a prediction system based on triage information, including demographics, vital signs, and chief complaints. The proposed system can not only handle heterogeneous data, including tabular data and free-text data, but also provide interpretability for better acceptance by the ED staff in the hospital.

METHODS

In this study, we proposed a system comprising 3 subsystems, with each of them handling a single task, including triage level prediction, hospitalization prediction, and length of stay prediction. We used a large amount of retrospective data to pretrain the model, and then, we fine-tuned the model on a prospective data set with a golden label. The proposed deep learning framework was built with TabNet and MacBERT (Chinese version of bidirectional encoder representations from transformers [BERT]).

RESULTS

The performance of our proposed model was evaluated on data collected from the National Taiwan University Hospital (901 patients were included). The model achieved promising results on the collected data set, with accuracy values of 63%, 82%, and 71% for triage level prediction, hospitalization prediction, and length of stay prediction, respectively.

CONCLUSIONS

Our system improved the prediction of 3 different medical outcomes when compared with other machine learning methods. With the pretrained vital sign encoder and repretrained mask language modeling MacBERT encoder, our multimodality model can provide a deeper insight into the characteristics of electronic health records. Additionally, by providing interpretability, we believe that the proposed system can assist nursing staff and physicians in taking appropriate medical decisions.

Assessing SOFA score trajectories in sepsis using machine learning: A pragmatic approach to improve the accuracy of mortality prediction

INTRODUCTION

An increasing amount of longitudinal health data is available on critically ill septic patients in the age of digital medicine, including daily sequential organ failure assessment (SOFA) score measurements. Thus, the assessment in sepsis focuses increasingly on the evaluation of the individual disease's trajectory. Machine learning (ML) algorithms may provide a promising approach here to improve the evaluation of daily SOFA score dynamics. We tested whether ML algorithms can outperform the conventional ΔSOFA score regarding the accuracy of 30-day mortality prediction.

METHODS

We used the multicentric SepsisDataNet.NRW study cohort that prospectively enrolled 252 sepsis patients between 03/2018 and 09/2019 for training ML algorithms, i.e. support vector machine (SVM) with polynomial kernel and artificial neural network (aNN). We used the Amsterdam UMC database covering 1,790 sepsis patients for external and independent validation.

RESULTS

Both SVM (AUC 0.84; 95% CI: 0.71-0.96) and aNN (AUC 0.82; 95% CI: 0.69-0.95) assessing the SOFA scores of the first seven days led to a more accurate prognosis of 30-day mortality compared to the ΔSOFA score between day 1 and 7 (AUC 0.73; 95% CI: 0.65-0.80; p = 0.02 and p = 0.05, respectively). These differences were even more prominent the shorter the time interval considered. Using the SOFA scores of day 1 to 3 SVM (AUC 0.82; 95% CI: 0.68 0.95) and aNN (AUC 0.80; 95% CI: 0.660.93) led to a more accurate prognosis of 30-day mortality compared to the ΔSOFA score (AUC 0.66; 95% CI: 0.58-0.74; p < 0.01 and p < 0.01, respectively). Strikingly, all these findings could be confirmed in the independent external validation cohort.

CONCLUSIONS

The ML-based algorithms using daily SOFA scores markedly improved the accuracy of mortality compared to the conventional ΔSOFA score. Therefore, this approach could provide a promising and automated approach to assess the individual disease trajectory in sepsis. These findings reflect the potential of incorporating ML algorithms as robust and generalizable support tools on intensive care units.

Explainable machine learning for early predicting treatment failure risk among patients with TB-diabetes comorbidity

The present study aims to assess the treatment outcome of patients with diabetes and tuberculosis (TB-DM) at an early stage using machine learning (ML) based on electronic medical records (EMRs). A total of 429 patients were included at Chongqing Public Health Medical Center. The random-forest-based Boruta algorithm was employed to select the essential variables, and four models with a fivefold cross-validation scheme were used for modeling and model evaluation. Furthermore, we adopted SHapley additive explanations to interpret results from the tree-based model. 9 features out of 69 candidate features were chosen as predictors. Among these predictors, the type of resistance was the most important feature, followed by activated partial throm-boplastic time (APTT), thrombin time (TT), platelet distribution width (PDW), and prothrombin time (PT). All the models we established performed above an AUC 0.7 with good predictive performance. XGBoost, the optimal performing model, predicts the risk of treatment failure in the test set with an AUC 0.9281. This study suggests that machine learning approach (XGBoost) presented in this study identifies patients with TB-DM at higher risk of treatment failure at an early stage based on EMRs. The application of a convenient and economy EMRs based on machine learning provides new insight into TB-DM treatment strategies in low and middle-income countries.

Integrating Phenotypic Information of Obstructive Sleep Apnea and Deep Representation of Sleep-Event Sequences for Cardiovascular Risk Prediction

Background

Advances in mobile, wearable and machine learning (ML) technologies for gathering and analyzing long-term health data have opened up new possibilities for predicting and preventing cardiovascular diseases (CVDs). Meanwhile, the association between obstructive sleep apnea (OSA) and CV risk has been well-recognized. This study seeks to explore effective strategies of incorporating OSA phenotypic information and overnight physiological information for precise CV risk prediction in the general population.

Methods

1,874 participants without a history of CVDs from the MESA dataset were included for the 5-year CV risk prediction. Four OSA phenotypes were first identified by the K-mean clustering based on static polysomnographic (PSG) features. Then several phenotype-agnostic and phenotype-specific ML models, along with deep learning (DL) models that integrate deep representations of overnight sleep-event feature sequences, were built for CV risk prediction. Finally, feature importance analysis was conducted by calculating SHapley Additive exPlanations (SHAP) values for all features across the four phenotypes to provide model interpretability.

Results

All ML models showed improved performance after incorporating the OSA phenotypic information. The DL model trained with the proposed phenotype-contrastive training strategy performed the best, achieving an area under the Receiver Operating Characteristic (ROC) curve of 0.877. Moreover, PSG and FOOD FREQUENCY features were recognized as significant CV risk factors across all phenotypes, with each phenotype emphasizing unique features.

Conclusion

Models that are aware of OSA phenotypes are preferred, and lifestyle factors should be a greater focus for precise CV prevention and risk management in the general population.

Triage in major incidents: development and external validation of novel machine learning-derived primary and secondary triage tools

BACKGROUND

Major incidents (MIs) are an important cause of death and disability. Triage tools are crucial to identifying priority 1 (P1) patients-those needing time-critical, life-saving interventions. Existing expert opinion-derived tools have limited evidence supporting their use. This study employs machine learning (ML) to develop and validate models for novel primary and secondary triage tools.

METHODS

Adults (16+ years) from the UK Trauma Audit and Research Network (TARN) registry (January 2008-December 2017) served as surrogates for MI victims, with P1 patients identified using predefined criteria. The TARN database was split chronologically into model training and testing (70:30) datasets. Input variables included physiological parameters, age, mechanism and anatomical location of injury. Random forest, extreme gradient boosted tree, logistic regression and decision tree models were trained to predict P1 status, and compared with existing tools (Battlefield Casualty Drills (BCD) Triage Sieve, CareFlight, Modified Physiological Triage Tool, MPTT-24, MSTART, National Ambulance Resilience Unit Triage Sieve and RAMP). Primary and secondary candidate models were selected; the latter was externally validated on patients from the UK military's Joint Theatre Trauma Registry (JTTR).

RESULTS

Models were internally tested in 57 979 TARN patients. The best existing tool was the BCD Triage Sieve (sensitivity 68.2%, area under the receiver operating curve (AUC) 0.688). Inability to breathe spontaneously, presence of chest injury and mental status were most predictive of P1 status. A decision tree model including these three variables exhibited the best test characteristics (sensitivity 73.0%, AUC 0.782), forming the candidate primary tool. The proposed secondary tool (sensitivity 77.9%, AUC 0.817), applicable via a portable device, includes a fourth variable (injury mechanism). This performed favourably on external validation (sensitivity of 97.6%, AUC 0.778) in 5956 JTTR patients.

CONCLUSION

Novel triage tools developed using ML outperform existing tools in a nationally representative trauma population. The proposed primary tool requires external validation prior to consideration for practical use. The secondary tool demonstrates good external validity and may be used to support decision-making by healthcare workers responding to MIs.

A Machine Learning Predictive Model of Bloodstream Infection in Hospitalized Patients

The aim of the study was to build a machine learning-based predictive model to discriminate between hospitalized patients at low risk and high risk of bloodstream infection (BSI). A Data Mart including all patients hospitalized between January 2016 and December 2019 with suspected BSI was built. Multivariate logistic regression was applied to develop a clinically interpretable machine learning predictive model. The model was trained on 2016-2018 data and tested on 2019 data. A feature selection based on a univariate logistic regression first selected candidate predictors of BSI. A multivariate logistic regression with stepwise feature selection in five-fold cross-validation was applied to express the risk of BSI. A total of 5660 hospitalizations (4026 and 1634 in the training and the validation subsets, respectively) were included. Eleven predictors of BSI were identified. The performance of the model in terms of AUROC was 0.74. Based on the interquartile predicted risk score, 508 (31.1%) patients were defined as being at low risk, 776 (47.5%) at medium risk, and 350 (21.4%) at high risk of BSI. Of them, 14.2% (72/508), 30.8% (239/776), and 64% (224/350) had a BSI, respectively. The performance of the predictive model of BSI is promising. Computational infrastructure and machine learning models can help clinicians identify people at low risk for BSI, ultimately supporting an antibiotic stewardship approach.

A comparative study of explainable ensemble learning and logistic regression for predicting in-hospital mortality in the emergency department

This study addresses the challenges associated with emergency department (ED) overcrowding and emphasizes the need for efficient risk stratification tools to identify high-risk patients for early intervention. While several scoring systems, often based on logistic regression (LR) models, have been proposed to indicate patient illness severity, this study aims to compare the predictive performance of ensemble learning (EL) models with LR for in-hospital mortality in the ED. A cross-sectional single-center study was conducted at the ED of Imam Reza Hospital in northeast Iran from March 2016 to March 2017. The study included adult patients with one to three levels of emergency severity index. EL models using Bagging, AdaBoost, random forests (RF), Stacking and extreme gradient boosting (XGB) algorithms, along with an LR model, were constructed. The training and validation visits from the ED were randomly divided into 80% and 20%, respectively. After training the proposed models using tenfold cross-validation, their predictive performance was evaluated. Model performance was compared using the Brier score (BS), The area under the receiver operating characteristics curve (AUROC), The area and precision-recall curve (AUCPR), Hosmer-Lemeshow (H-L) goodness-of-fit test, precision, sensitivity, accuracy, F1-score, and Matthews correlation coefficient (MCC). The study included 2025 unique patients admitted to the hospital's ED, with a total percentage of hospital deaths at approximately 19%. In the training group and the validation group, 274 of 1476 (18.6%) and 152 of 728 (20.8%) patients died during hospitalization, respectively. According to the evaluation of the presented framework, EL models, particularly Bagging, predicted in-hospital mortality with the highest AUROC (0.839, CI (0.802-0.875)) and AUCPR = 0.64 comparable in terms of discrimination power with LR (AUROC (0.826, CI (0.787-0.864)) and AUCPR = 0.61). XGB achieved the highest precision (0.83), sensitivity (0.831), accuracy (0.842), F1-score (0.833), and the highest MCC (0.48). Additionally, the most accurate models in the unbalanced dataset belonged to RF with the lowest BS (0.128). Although all studied models overestimate mortality risk and have insufficient calibration (P > 0.05), stacking demonstrated relatively good agreement between predicted and actual mortality. EL models are not superior to LR in predicting in-hospital mortality in the ED. Both EL and LR models can be considered as screening tools to identify patients at risk of mortality.

Machine learning in clinical practice: Evaluation of an artificial intelligence tool after implementation

OBJECTIVE

Artificial intelligence (AI) has gradually found its way into healthcare, and its future integration into clinical practice is inevitable. In the present study, we evaluate the accuracy of a novel AI algorithm designed to predict admission based on a triage note after clinical implementation. This is the first of such studies to investigate real-time AI performance in the emergency setting.

METHODS

The novel AI algorithm that predicts admission using a triage note was translated into clinical practice and integrated within St Vincent's Hospital Melbourne's electronic emergency patient management system. The data were collected from 1 January 2021 to 17 August 2022 to evaluate the diagnostic accuracy of the AI system after implementation.

RESULTS

A total of 77 125 ED presentations were included. The live AI algorithm has a sensitivity of 73.1% (95% confidence interval 72.5-73.8), specificity of 74.3% (73.9-74.7), positive predictive value of 50% (49.6-50.4) and negative predictive value of 88.7% (88.5-89) with a total accuracy of 74% (73.7-74.3). The accuracy of the system was at the lowest for admission to psychiatric units (34%) and at the highest for gastroenterology and medical admission (84% and 80%, respectively).

CONCLUSION

Our study showed the diagnostic evaluation of a real-time AI clinical decision-support tool became less accurate than the original. Although real-time sensitivity and specificity of the AI tool was still acceptable as a decision-support tool in the ED, we propose that continuous training and evaluation of AI-enabled clinical support tools in healthcare are conducted to ensure consistent accuracy and performance to prevent inadvertent consequences.

Predicting postoperative outcomes in lumbar spinal fusion: development of a machine learning model

BACKGROUND CONTEXT

Degenerative lumbar spondylolisthesis (DLS) is a prevalent spinal disorder, often requiring surgical intervention. Accurately predicting surgical outcomes is crucial to guide clinical decision-making, but this is challenging due to the multifactorial nature of postoperative results. Traditional risk assessment tools have limitations, and with the advent of machine learning, there is potential to enhance the precision and comprehensiveness of preoperative evaluations.

PURPOSE

We aimed to develop a machine-learning algorithm to predict surgical outcomes in patients with degenerative lumbar spondylolisthesis (DLS) undergoing spinal fusion surgery, only using preoperative data.

STUDY DESIGN

Retrospective cross-sectional study.

PATIENT SAMPLE

Patients with DLS undergoing lumbar spinal fusion surgery.

OUTCOME MEASURES

This study aimed to predict the occurrence of lower back pain (LBP) ≥4 on the numeric analogue scale (NAS) 2 years after surgery. LBP was evaluated as the average pain patients experienced at rest in the week before questioning. NAS ranges from 0 to 10, 0 representing no pain and 10 representing the worst pain imaginable.

METHODS

We conducted a retrospective analysis of prospectively enrolled patients who underwent spinal fusion surgery for degenerative lumbar spondylolistheses at our institution in the United States between January 2016 and December 2018. The initial patient characteristics to be included in the training of the model were chosen by clinical expertise and through a literature review and included demographic characteristics, comorbidities, and radiologic features. The data was split into a training and validation datasets using a 60/40 split. Four different machine learning models were trained, including the modern XGBoost model, logistic regression, random-forest, and support vector machine (SVM). The models were evaluated according to the area under the curve (AUC) of the receiver operating characteristics (ROC) curve. An AUC of 0.7 to 0.8 was considered fair, 0.8 to 0.9 good, and ≥ 0.9 excellent. Additionally, a calibration plot and the Brier score were calculated for each model.

RESULTS

A total of 135 patients (66% female) were included. A total of 38 (28%) patients reported LBP ≥ 4 after 2 years, representing the positive class. The XGBoost model demonstrated the best performance in the validation set with an AUC of 0.81 (95% CI 0.67-0.95). The other machine learning models performed significantly worse: with an AUC of 0.52 (95% CI 0.37-0.68) for the SVM, 0.56 (95% CI 0.37-0.76) for the logistic regression and an AUC of 0.56 (95% CI 0.37-0.78) for the random forest. In the XGBoost model age, composition of the erector spinae, and severity of lumbar spinal stenosis as were identified as the most important features.

CONCLUSIONS

This study represents a novel approach to predicting surgical outcomes in spinal fusion patients. The XGBoost demonstrated a better performance compared with classical models and highlighted the potential contributions of age and paraspinal musculature atrophy as significant factors. These findings have important implications for enhancing patient care through the identification of high-risk individuals and modifiable risk factors. As the incorporation of machine learning algorithms into clinical decision-making continues to gain traction in research and clinical practice, our insights reinforce this trajectory by showcasing the potential of these techniques in forecasting surgical results.

AI algorithm for personalized resource allocation and treatment of hemorrhage casualties

A deep neural network-based artificial intelligence (AI) model was assessed for its utility in predicting vital signs of hemorrhage patients and optimizing the management of fluid resuscitation in mass casualties. With the use of a cardio-respiratory computational model to generate synthetic data of hemorrhage casualties, an application was created where a limited data stream (the initial 10 min of vital-sign monitoring) could be used to predict the outcomes of different fluid resuscitation allocations 60 min into the future. The predicted outcomes were then used to select the optimal resuscitation allocation for various simulated mass-casualty scenarios. This allowed the assessment of the potential benefits of using an allocation method based on personalized predictions of future vital signs versus a static population-based method that only uses currently available vital-sign information. The theoretical benefits of this approach included up to 46% additional casualties restored to healthy vital signs and a 119% increase in fluid-utilization efficiency. Although the study is not immune from limitations associated with synthetic data under specific assumptions, the work demonstrated the potential for incorporating neural network-based AI technologies in hemorrhage detection and treatment. The simulated injury and treatment scenarios used delineated possible benefits and opportunities available for using AI in pre-hospital trauma care. The greatest benefit of this technology lies in its ability to provide personalized interventions that optimize clinical outcomes under resource-limited conditions, such as in civilian or military mass-casualty events, involving moderate and severe hemorrhage.

Prediction of major adverse cardiac events in the emergency department using an artificial neural network with a systematic grid search

BACKGROUND

The aim of our research was to design and evaluate an Artificial Neural Network (ANN) model using a systemic grid search for the early prediction of major adverse cardiac events (MACE) among patients presenting to the triage of an emergency department.

METHODS

This is a single-center, cross-sectional study using electronic health records from January 2017 to December 2020. The research population consists of adults coming to our emergency department triage at Aga Khan University Hospital. The MACE during hospitalization was the main outcome. To enhance the architecture of an ANN using triage data, we used a systematic grid search strategy. Four hidden ANN layers were used, followed by an output layer. Following each hidden layer was back normalization and a dropout layer. MACE was predicted using three binary classifiers: ANN, Random Forests (RF), and logistic regression (LR). The overall accuracy, sensitivity, specificity, precision, and recall of these models were examined. Each model was evaluated using the receiver operating characteristic curve (ROC) and an F1-score with a 95% confidence interval.

RESULTS

A total of 97,333 emergency department visits were recorded during the study period, with 33% of patients having cardiovascular symptoms. The mean age was 54.08 (19.18) years old. The MACE was observed in 23,052 (23.7%) of the patients, in-hospital (up to 30 days) mortality in 10,888 (11.2%) patients, and cardiac arrest in 5483 (5.6%) patients. The data used for training and validation were 77,866 and 19,467 in an 80:20 ratio, respectively. The AUC score for MACE with ANN was 0.97, which was greater than RF (0.96) and LR (0.96). Similarly, the precision-recall curve for MACE utilizing ANN was greater (0.94 vs. 0.93 for RF and 0.93 for LR). The sensitivity for MACE prediction using ANN, RF, and LR classifiers was 99.3%, 99.4%, and 99.2%, respectively, with the specificities being 94.5%, 94.2%, and 94.2%, respectively.

CONCLUSION

When triage data is used to predict MACE, death, and cardiac arrest, ANN with systemic grid search gives precise and valid outcomes and will benefit in predicting MACE in emergency rooms with limited resources that have to deal with a substantial number of patients.

Improved interpretable machine learning emergency department triage tool addressing class imbalance

Objective

The Score for Emergency Risk Prediction (SERP) is a novel mortality risk prediction score which leverages machine learning in supporting triage decisions. In its derivation study, SERP-2d, SERP-7d and SERP-30d demonstrated good predictive performance for 2-day, 7-day and 30-day mortality. However, the dataset used had significant class imbalance. This study aimed to determine if addressing class imbalance can improve SERP's performance, ultimately improving triage accuracy.

Methods

The Singapore General Hospital (SGH) emergency department (ED) dataset was used, which contains 1,833,908 ED records between 2008 and 2020. Records between 2008 and 2017 were randomly split into a training set (80%) and validation set (20%). The 2019 and 2020 records were used as test sets. To address class imbalance, we used random oversampling and random undersampling in the AutoScore-Imbalance framework to develop SERP+-2d, SERP+-7d, and SERP+-30d scores. The performance of SERP+, SERP, and the commonly used triage risk scores was compared.

Results

The developed SERP+ scores had five to six variables. The AUC of SERP+ scores (0.874 to 0.905) was higher than that of the corresponding SERP scores (0.859 to 0.894) on both test sets. This superior performance was statistically significant for SERP+-7d (2019: Z = -5.843, p < 0.001, 2020: Z = -4.548, p < 0.001) and SERP+-30d (2019: Z = -3.063, p = 0.002, 2020: Z = -3.256, p = 0.001). SERP+ outperformed SERP marginally on sensitivity, specificity, balanced accuracy, and positive predictive value measures. Negative predictive value was the same for SERP+ and SERP. Additionally, SERP+ showed better performance compared to the commonly used triage risk scores.

Conclusions

Accounting for class imbalance during training improved score performance for SERP+. Better stratification of even a small number of patients can be meaningful in the context of the ED triage. Our findings reiterate the potential of machine learning-based scores like SERP+ in supporting accurate, data-driven triage decisions at the ED.

Exploring the Impact of Artificial Intelligence on Global Health and Enhancing Healthcare in Developing Nations

BACKGROUND

Artificial intelligence (AI), which combines computer science with extensive datasets, seeks to mimic human-like intelligence. Subsets of AI are being applied in almost all fields of medicine and surgery.

AIM

This review focuses on the applications of AI in healthcare settings in developing countries, designed to underscore its significance by comprehensively outlining the advancements made thus far, the shortcomings encountered in AI applications, the present status of AI integration, persistent challenges, and innovative strategies to surmount them.

METHODOLOGY

Articles from PubMed, Google Scholar, and Cochrane were searched from 2000 to 2023 with keywords including AI and healthcare, focusing on multiple medical specialties.

RESULTS

The increasing role of AI in diagnosis, prognosis prediction, and patient management, as well as hospital management and community healthcare, has made the overall healthcare system more efficient, especially in the high patient load setups and resource-limited areas of developing countries where patient care is often compromised. However, challenges, including low adoption rates and the absence of standardized guidelines, high installation and maintenance costs of equipment, poor transportation and connectivvity issues hinder AI's full use in healthcare.

CONCLUSION

Despite these challenges, AI holds a promising future in healthcare. Adequate knowledge and expertise of healthcare professionals for the use of AI technology in healthcare is imperative in developing nations.

Application of Machine Learning Techniques to Development of Emergency Medical Rapid Triage Prediction Models in Acute Care

Given the critical and complex features of medical emergencies, it is essential to develop models that enable prompt and suitable clinical decision-making based on considerable information. Emergency nurses are responsible for categorizing and prioritizing injuries and illnesses on the frontlines of the emergency room. This study aims to create an Emergency Medical Rapid Triage and Prediction Assistance model using electronic medical records and machine learning techniques. Patient information was retrieved from the emergency department of a large regional teaching hospital in Taiwan, and five supervised learning techniques were used to construct classification models for predicting critical outcomes. Of these models, the model using logistic regression had superior prediction performance, with an F1 score of 0.861 and an area under the receiver operating characteristic curve of 0.855. The Emergency Medical Rapid Triage and Prediction Assistance model demonstrated superior performance in predicting intensive care and hospitalization outcomes compared with the Taiwan Triage and Acuity Scale and three clinical early warning tools. The proposed model has the potential to assist emergency nurses in executing challenging triage assessments and emergency teams in treating critically ill patients promptly, leading to improved clinical care and efficient utilization of medical resources.

Machine learning for predicting hepatitis B or C virus infection in diabetic patients

Highly prevalent hepatitis B and hepatitis C virus (HBV and HCV) infections have been reported among individuals with diabetes. Given the frequently asymptomatic nature of hepatitis and the challenges associated with screening in some vulnerable populations such as diabetes patients, we conducted an investigation into the performance of various machine learning models for the identification of hepatitis in diabetic patients while also evaluating the significance of features. Analyzing NHANES data from 2013 to 2018, machine learning models were evaluated; random forest (RF), support vector machine (SVM), eXtreme Gradient Boosting (XGBoost), and least absolute shrinkage and selection operator (LASSO) along with stacked ensemble model. We performed hyperparameter tuning to improve the performance of the model, and selected important predictors using the best performance model. LASSO showed the highest predictive performance (AUC-ROC = 0.810) rather than other models. Illicit drug use, poverty, and race were highly ranked as predictive factors for developing hepatitis in diabetes patients. Our study demonstrated that a machine-learning-based model performed optimally in the detection of hepatitis among diabetes patients, achieving high performance. Furthermore, models and predictors evaluated from the current study, we expect, could be supportive information for developing screening or treatment methods for hepatitis care in diabetes patients.

Using emergency department triage for machine learning-based admission and mortality prediction

AIMS

Patient admission is a decision relying on sparsely available data. This study aims to provide prediction models for discharge versus admission for ward observation or intensive care, and 30 day-mortality for patients triaged with the Manchester Triage System.

METHODS

This is a single-centre, observational, retrospective cohort study from data within ten minutes of patient presentation at the interdisciplinary emergency department of the Kepler University Hospital, Linz, Austria. We trained machine learning models including Random Forests and Neural Networks individually to predict discharge versus ward observation or intensive care admission, and 30 day-mortality. For analysis of the features' relevance, we used permutation feature importance.

RESULTS

A total of 58323 adult patients between 1 December 2015 and 31 August 2020 were included. Neural Networks and Random Forests predicted admission to ward observation with an AUC-ROC of 0.842 ± 0.00 with the most important features being age and chief complaint. For admission to intensive care, the models had an AUC-ROC of 0.819 ± 0.002 with the most important features being the Manchester Triage category and heart rate, and for the outcome 30 day-mortality an AUC-ROC of 0.925 ± 0.001. The most important features for the prediction of 30 day-mortality were age and general ward admission.

CONCLUSION

Machine learning can provide prediction on discharge versus admission to general wards and intensive care and inform about risk on 30 day-mortality for patients in the emergency department.

Advancing prognostic precision in pulmonary embolism: A clinical and laboratory-based artificial intelligence approach for enhanced early mortality risk stratification

BACKGROUND

Acute pulmonary embolism (PE) is a critical medical emergency that necessitates prompt identification and intervention. Accurate prognostication of early mortality is vital for recognizing patients at elevated risk for unfavourable outcomes and administering suitable therapy. Machine learning (ML) algorithms hold promise for enhancing the precision of early mortality prediction in PE patients.

OBJECTIVE

To devise an ML algorithm for early mortality prediction in PE patients by employing clinical and laboratory variables.

METHODS

This study utilized diverse oversampling techniques to improve the performance of various machine learning models including ANN, SVM, DT, RF, and AdaBoost for early mortality prediction. Appropriate oversampling methods were chosen for each model based on algorithm characteristics and dataset properties. Predictor variables included four lab tests, eight physiological time series indicators, and two general descriptors. Evaluation used metrics like accuracy, F1_score, precision, recall, Area Under the Curve (AUC) and Receiver Operating Characteristic (ROC) curves, providing a comprehensive view of models' predictive abilities.

RESULTS

The findings indicated that the RF model with random oversampling exhibited superior performance among the five models assessed, achieving elevated accuracy and precision alongside high recall for predicting the death class. The oversampling approaches effectively equalized the sample distribution among the classes and enhanced the models' performance.

CONCLUSIONS

The suggested ML technique can efficiently prognosticate mortality in patients afflicted with acute PE. The RF model with random oversampling can aid healthcare professionals in making well-informed decisions regarding the treatment of patients with acute PE. The study underscores the significance of oversampling methods in managing imbalanced data and emphasizes the potential of ML algorithms in refining early mortality prediction for PE patients.

Exploring the impact of missingness on racial disparities in predictive performance of a machine learning model for emergency department triage

Objective

To investigate how missing data in the patient problem list may impact racial disparities in the predictive performance of a machine learning (ML) model for emergency department (ED) triage.

Materials and Methods

Racial disparities may exist in the missingness of EHR data (eg, systematic differences in access, testing, and/or treatment) that can impact model predictions across racialized patient groups. We use an ML model that predicts patients' risk for adverse events to produce triage-level recommendations, patterned after a clinical decision support tool deployed at multiple EDs. We compared the model's predictive performance on sets of observed (problem list data at the point of triage) versus manipulated (updated to the more complete problem list at the end of the encounter) test data. These differences were compared between Black and non-Hispanic White patient groups using multiple performance measures relevant to health equity.

Results

There were modest, but significant, changes in predictive performance comparing the observed to manipulated models across both Black and non-Hispanic White patient groups; c-statistic improvement ranged between 0.027 and 0.058. The manipulation produced no between-group differences in c-statistic by race. However, there were small between-group differences in other performance measures, with greater change for non-Hispanic White patients.

Discussion

Problem list missingness impacted model performance for both patient groups, with marginal differences detected by race.

Conclusion

Further exploration is needed to examine how missingness may contribute to racial disparities in clinical model predictions across settings. The novel manipulation method demonstrated may aid future research.

Prediction of outpatient waiting time: using machine learning in a tertiary children's hospital

Background

Accurately predicting waiting time for patients is crucial for effective hospital management. The present study examined the prediction of outpatient waiting time in a Chinese pediatric hospital through the use of machine learning algorithms. If patients are informed about their waiting time in advance, they can make more informed decisions and better plan their visit on the day of admission.

Methods

First, a novel classification method for the outpatient clinic in the Chinese pediatric hospital was proposed, which was based on medical knowledge and statistical analysis. Subsequently, four machine learning algorithms [linear regression (LR), random forest (RF), gradient boosting decision tree (GBDT), and K-nearest neighbor (KNN)] were used to construct prediction models of the waiting time of patients in four department categories.

Results

The three machine learning algorithms outperformed LR in the four department categories. The optimal model for Internal Medicine Department I was the RF model, with a mean absolute error (MAE) of 5.03 minutes, which was 47.60% lower than that of the LR model. The optimal model for the other three categories was the GBDT model. The MAE of the GBDT model was decreased by 28.26%, 35.86%, and 33.10%, respectively compared to that of the LR model.

Conclusions

Machine learning can predict the outpatient waiting time of pediatric hospitals well and ease patient anxiety when waiting in line without medical appointments. This study offers key insights into enhancing healthcare services and reaffirms the dedication of Chinese pediatric hospitals to providing efficient and patient-centric care.

Application of exponential smoothing method and SARIMA model in predicting the number of admissions in a third-class hospital in Zhejiang Province

OBJECTIVE

To establish the exponential smoothing prediction model and SARIMA model to predict the number of inpatients in a third-class hospital in Zhejiang Province, and evaluate the prediction effect of the two models, and select the best number prediction model.

METHODS

The data of hospital admissions from January 2019 to September 2022 were selected to establish the exponential smoothing prediction model and the SARIMA model respectively. Then compare the fitting parameters of different models: R2_adjusted, R2, Root Mean Square Error (RMSE)、Mean Absolute Percentage Error (MAPE)、Mean Absolute Error(MAE) and standardized BIC to select the best model. Finally, the established model was used to predict the number of hospital admissions from October to December 2022, and the prediction effect of the average relative error judgment model was compared.

RESULTS

The best fitting exponential smoothing prediction model was Winters Addition model, whose R2_adjusted was 0.533, R2 was 0.817, MAPE was 6.133, MAE was 447.341. The best SARIMA model is SARIMA(2,2,2)(0,1,1)12 model, whose R2_adjusted is 0.449, R2 is 0.199, MAPE is 8.240, MAE is 718.965. The Winters addition model and SARIMA(2,2,2)(0,1,1)12 model were used to predict the number of hospital admissions in October-December 2022, respectively. The results showed that the average relative error was 0.038 and 0.015, respectively. The SARIMA(2,2,2)(0,1,1)12 model had a good prediction effect.

CONCLUSION

Both models can better fit the number of admissions, and SARIMA model has better prediction effect.