Machine learning versus traditional methods for the development of risk stratification scores: a case study using original Canadian Syncope Risk Score data

Role of Artificial Intelligence in Improving Syncope Management

Syncope is common in the general population and a common presenting symptom in acute care settings. Substantial costs are attributed to the care of patients with syncope. Current challenges include differentiating syncope from its mimickers, identifying serious underlying conditions that caused the syncope, and wide variations in current management. Although validated risk tools exist, especially for short-term prognosis, there is inconsistent application, and the current approach does not meet patient needs and expectations. Artificial intelligence (AI) techniques, such as machine learning methods including natural language processing, can potentially address the current challenges in syncope management. Preliminary evidence from published studies indicates that it is possible to accurately differentiate syncope from its mimickers and predict short-term prognosis and hospitalisation. More recently, AI analysis of electrocardiograms has shown promise in detection of serious structural and functional cardiac abnormalities, which has the potential to improve syncope care. Future AI studies have the potential to address current issues in syncope management. AI can automatically prognosticate risk in real time by accessing traditional and nontraditional data. However, steps to mitigate known problems such as generalisability, patient privacy, data protection, and liability will be needed. In the past AI has had limited impact due to underdeveloped analytical methods, lack of computing power, poor access to powerful computing systems, and availability of reliable high-quality data. All impediments except data have been solved. AI will live up to its promise to transform syncope care if the health care system can satisfy AI requirement of large scale, robust, accurate, and reliable data.

Assessing the efficacy of machine learning algorithms for syncope classification: A systematic review

Syncope is a transient loss of consciousness with rapid onset. The aims of the study were to systematically evaluate available machine learning (ML) algorithm for supporting syncope diagnosis to determine their performance compared to existing point scoring protocols. We systematically searched IEEE Xplore, Web of Science, and Elsevier for English articles (Jan 2011 - Sep 2021) on individuals aged five and above, employing ML algorithms in syncope detection with Head-up titl table test (HUTT)-monitored hemodynamic parameters and reported metrics. Extracted data encompassed subject count, age range, syncope protocols, ML type, hemodynamic parameters, and performance metrics. Of the 6301 studies initially identified, 10 studies, involving 1205 participants aged 5 to 82 years, met the inclusion criteria, and formed the basis for it. Selected studies must use ML algorithms in syncope detection with hemodynamic parameters recorded throughout HUTT. The overall ML algorithm performance achieved a sensitivity of 88.8% (95% CI: 79.4-96.1%), specificity of 81.5% (95% CI: 69.8-92.8%) and accuracy of 85.8% (95% CI: 78.6-92.8%). Machine learning improves syncope diagnosis compared to traditional scoring, requiring fewer parameters. Future enhancements with larger databases are anticipated. Integrating ML can curb needless admissions, refine diagnostics, and enhance the quality of life for syncope patients.

An all-inclusive model for predicting invasive bacterial infection in febrile infants age 7-60 days

BACKGROUND

Invasive bacterial infections (IBIs) in febrile infants are rare but potentially devastating. We aimed to derive and validate a predictive model for IBI among febrile infants age 7-60 days.

METHODS

Data were abstracted retrospectively from electronic records of 37 emergency departments (EDs) for infants with a measured temperature >=100.4 F who underwent an ED evaluation with blood and urine cultures. Models to predict IBI were developed and validated respectively using a random 80/20 dataset split, including 10-fold cross-validation. We used precision recall curves as the classification metric.

RESULTS

Of 4411 eligible infants with a mean age of 37 days, 29% had characteristics that would likely have excluded them from existing risk stratification protocols. There were 196 patients with IBI (4.4%), including 43 (1.0%) with bacterial meningitis. Analytic approaches varied in performance characteristics (precision recall range 0.04-0.29, area under the curve range 0.5-0.84), with the XGBoost model demonstrating the best performance (0.29, 0.84). The five most important variables were serum white blood count, maximum temperature, absolute neutrophil count, absolute band count, and age in days.

CONCLUSION

A machine learning model (XGBoost) demonstrated the best performance in predicting a rare outcome among febrile infants, including those excluded from existing algorithms.

IMPACT

Several models for the risk stratification of febrile infants have been developed. There is a need for a preferred comprehensive model free from limitations and algorithm exclusions that accurately predicts IBIs. This is the first study to derive an all-inclusive predictive model for febrile infants aged 7-60 days in a community ED sample with IBI as a primary outcome. This machine learning model demonstrates potential for clinical utility in predicting IBI.

A Hybrid Model for 30-Day Syncope Prognosis Prediction in the Emergency Department

Syncope is a challenging problem in the emergency department (ED) as the available risk prediction tools have suboptimal predictive performances. Predictive models based on machine learning (ML) are promising tools whose application in the context of syncope remains underexplored. The aim of the present study was to develop and compare the performance of ML-based models in predicting the risk of clinically significant outcomes in patients presenting to the ED for syncope. We enrolled 266 consecutive patients (age 73, IQR 58-83; 52% males) admitted for syncope at three tertiary centers. We collected demographic and clinical information as well as the occurrence of clinically significant outcomes at a 30-day telephone follow-up. We implemented an XGBoost model based on the best-performing candidate predictors. Subsequently, we integrated the XGboost predictors with knowledge-based rules. The obtained hybrid model outperformed the XGboost model (AUC = 0.81 vs. 0.73, p < 0.001) with acceptable calibration. In conclusion, we developed an ML-based model characterized by a commendable capability to predict adverse events within 30 days post-syncope evaluation in the ED. This model relies solely on clinical data routinely collected during a patient's initial syncope evaluation, thus obviating the need for laboratory tests or syncope experienced clinical judgment.

Can Artificial Intelligence Enhance Syncope Management?: A JACC: Advances Multidisciplinary Collaborative Statement

Syncope, a form of transient loss of consciousness, remains a complex medical condition for which adverse cardiovascular outcomes, including death, are of major concern but rarely occur. Current risk stratification algorithms have not completely delineated which patients benefit from hospitalization and specific interventions. Patients are often admitted unnecessarily and at high cost. Artificial intelligence (AI) and machine learning may help define the transient loss of consciousness event, diagnose the cause, assess short- and long-term risks, predict recurrence, and determine need for hospitalization and therapeutic intervention; however, several challenges remain, including medicolegal and ethical concerns. This collaborative statement, from a multidisciplinary group of clinicians, investigators, and scientists, focuses on the potential role of AI in syncope management with a goal to inspire creation of AI-derived clinical decision support tools that may improve patient outcomes, streamline diagnostics, and reduce health-care costs.

Machine learning and LACE index for predicting 30-day readmissions after heart failure hospitalization in elderly patients

Machine learning (ML) techniques may improve readmission prediction performance in heart failure (HF) patients. This study aimed to assess the ability of ML algorithms to predict unplanned all-cause 30-day readmissions in HF elderly patients, and to compare them with conventional LACE (Length of hospitalization, Acuity, Comorbidities, Emergency department visits) index. All patients aged ≥ 65 years discharged alive between 2010 and 2019 after a hospitalization for acute HF were included in this retrospective cohort study. We applied MICE (Multivariate Imputation via Chained Equations) method to obtain a balanced, fully valued dataset and LASSO (Least Absolute Shrinkage and Selection Operator) algorithm to get the most significant features. Training (80% of records) and test (20%) cohorts were randomly selected. Study population: 3079 patients, 394 (12.8%) presented at least one readmission within 30 days, and 2685 (87.2%) did not. In the test cohort AUCs (IC95%) of XGBoost, Ada Boost Classifier, Random forest, and Gradient Boosting, and LACE Index were: 0.803 (0.734-0.872), 0.782 (0.711-0.854), 0.776 (0.703-0.848), 0.786 (0.715-0.857), and 0.504 (0.414-0.594), respectively, for predicting readmissions. A SHAP analysis was performed to offer a breakdown of the ML variables associated with readmission. Positive and negative predicting values estimates of the different ML models and LACE index were also provided, for several values of readmission rate prevalence. Among elderly patients, the rate of all-cause unplanned 30-day readmissions after hospitalization due to an acute HF was high. ML models performed better than the conventional LACE index for predicting readmissions. ML models can be proposed as promising tools for the identification of subjects at high risk of hospitalization in this clinical setting, enabling care teams to target interventions for improving overall clinical outcomes.

Risk stratification of syncope: Current syncope guidelines and beyond

Syncope is an alarming event carrying the possibility of serious outcomes, including sudden cardiac death (SCD). Therefore, immediate risk stratification should be applied whenever syncope occurs, especially in the Emergency Department, where most dramatic presentations occur. It has long been known that short- and long-term syncope prognosis is affected not only by its mechanism but also by presence of concomitant conditions, especially cardiovascular disease. Over the last two decades, several syncope prediction tools have been developed to refine patient stratification and triage patients who need expert in-hospital care from those who may receive nonurgent expert care in the community. However, despite promising results, prognostic tools for syncope remain challenging and often poorly effective. Current European Society of Cardiology syncope guidelines recommend an initial syncope workup based on detailed patient's history, physical examination supine and standing blood pressure, resting ECG, and laboratory tests, including cardiac biomarkers, where appropriate. Subsequent risk stratification based on screening of features aims to identify three groups: high-, intermediate- and low-risk. The first should immediately be hospitalized and appropriately investigated; intermediate group, with recurrent or medium-risk events, requires systematic evaluation by syncope experts; low-risk group, sporadic reflex syncope, merits education about its benign nature, and discharge. Thus, initial syncope risk stratification is crucial as it determines how and by whom syncope patients are managed. This review summarizes the crucial elements of syncope risk stratification, pros and cons of proposed risk evaluation scores, major challenges in initial syncope management, and how risk stratification impacts management of high-risk/recurrent syncope.