Comparison of Machine Learning Methods for Predicting Outcomes After In-Hospital Cardiac Arrest

Development and Validation of a Machine Learning COVID-19 Veteran (COVet) Deterioration Risk Score

BACKGROUND AND OBJECTIVE

To develop the COVid Veteran (COVet) score for clinical deterioration in Veterans hospitalized with COVID-19 and further validate this model in both Veteran and non-Veteran samples. No such score has been derived and validated while incorporating a Veteran sample.

DERIVATION COHORT

Adults (age ≥ 18 yr) hospitalized outside the ICU with a diagnosis of COVID-19 for model development to the Veterans Health Administration (VHA) (n = 80 hospitals).

VALIDATION COHORT

External validation occurred in a VHA cohort of 34 hospitals, as well as six non-Veteran health systems for further external validation (n = 21 hospitals) between 2020 and 2023.

PREDICTION MODEL

eXtreme Gradient Boosting machine learning methods were used, and performance was assessed using the area under the receiver operating characteristic curve and compared with the National Early Warning Score (NEWS). The primary outcome was transfer to the ICU or death within 24 hours of each new variable observation. Model predictor variables included demographics, vital signs, structured flowsheet data, and laboratory values.

RESULTS

A total of 96,908 admissions occurred during the study period, of which 59,897 were in the Veteran sample and 37,011 were in the non-Veteran sample. During external validation in the Veteran sample, the model demonstrated excellent discrimination, with an area under the receiver operating characteristic curve of 0.88. This was significantly higher than NEWS (0.79; p < 0.01). In the non-Veteran sample, the model also demonstrated excellent discrimination (0.86 vs. 0.79 for NEWS; p < 0.01). The top three variables of importance were eosinophil percentage, mean oxygen saturation in the prior 24-hour period, and worst mental status in the prior 24-hour period.

CONCLUSIONS

We used machine learning methods to develop and validate a highly accurate early warning score in both Veterans and non-Veterans hospitalized with COVID-19. The model could lead to earlier identification and therapy, which may improve outcomes.

Neuromonitoring in the ICU - what, how and why?

PURPOSE OF REVIEW

We selectively review emerging noninvasive neuromonitoring techniques and the evidence that supports their use in the ICU setting. The focus is on neuromonitoring research in patients with acute brain injury.

RECENT FINDINGS

Noninvasive intracranial pressure evaluation with optic nerve sheath diameter measurements, transcranial Doppler waveform analysis, or skull mechanical extensometer waveform recordings have potential safety and resource-intensity advantages when compared to standard invasive monitors, however each of these techniques has limitations. Quantitative electroencephalography can be applied for detection of cerebral ischemia and states of covert consciousness. Near-infrared spectroscopy may be leveraged for cerebral oxygenation and autoregulation computation. Automated quantitative pupillometry and heart rate variability analysis have been shown to have diagnostic and/or prognostic significance in selected subtypes of acute brain injury. Finally, artificial intelligence is likely to transform interpretation and deployment of neuromonitoring paradigms individually and when integrated in multimodal paradigms.

SUMMARY

The ability to detect brain dysfunction and injury in critically ill patients is being enriched thanks to remarkable advances in neuromonitoring data acquisition and analysis. Studies are needed to validate the accuracy and reliability of these new approaches, and their feasibility and implementation within existing intensive care workflows.

Development of early prediction model of in-hospital cardiac arrest based on laboratory parameters

BACKGROUND

In-hospital cardiac arrest (IHCA) is an acute disease with a high fatality rate that burdens individuals, society, and the economy. This study aimed to develop a machine learning (ML) model using routine laboratory parameters to predict the risk of IHCA in rescue-treated patients.

METHODS

This retrospective cohort study examined all rescue-treated patients hospitalized at the First Medical Center of the PLA General Hospital in Beijing, China, from January 2016 to December 2020. Five machine learning algorithms, including support vector machine, random forest, extra trees classifier (ETC), decision tree, and logistic regression algorithms, were trained to develop models for predicting IHCA. We included blood counts, biochemical markers, and coagulation markers in the model development. We validated model performance using fivefold cross-validation and used the SHapley Additive exPlanation (SHAP) for model interpretation.

RESULTS

A total of 11,308 participants were included in the study, of which 7779 patients remained. Among these patients, 1796 (23.09%) cases of IHCA occurred. Among five machine learning models for predicting IHCA, the ETC algorithm exhibited better performance, with an AUC of 0.920, compared with the other four machine learning models in the fivefold cross-validation. The SHAP showed that the top ten factors accounting for cardiac arrest in rescue-treated patients are prothrombin activity, platelets, hemoglobin, N-terminal pro-brain natriuretic peptide, neutrophils, prothrombin time, serum albumin, sodium, activated partial thromboplastin time, and potassium.

CONCLUSIONS

We developed a reliable machine learning-derived model that integrates readily available laboratory parameters to predict IHCA in patients treated with rescue therapy.

The value of a machine learning algorithm to predict adverse short-term outcome during resuscitation of patients with in-hospital cardiac arrest: a retrospective study

Background and importance Guidelines recommend that hospital emergency teams locally validate criteria for termination of cardiopulmonary resuscitation in patients with in-hospital cardiac arrest (IHCA). Objective To determine the value of a machine learning algorithm to predict failure to achieve return of spontaneous circulation (ROSC) and unfavourable functional outcome from IHCA using only data readily available at emergency team arrival. Design Retrospective cohort study. Setting and participants Adults who experienced an IHCA were attended to by the emergency team. Outcome measures and analysis Demographic and clinical data typically available at the arrival of the emergency team were extracted from the institutional IHCA database. In addition, outcome data including the Cerebral Performance Category (CPC) score count at hospital discharge were collected. A model selection procedure for random forests with a hyperparameter search was employed to develop two classification algorithms to predict failure to achieve ROSC and unfavourable (CPC 3-5) functional outcomes. Main results Six hundred thirty patients were included, of which 390 failed to achieve ROSC (61.9%). The final classification model to predict failure to achieve ROSC had an area under the receiver operating characteristic curve of 0.9 [95% confidence interval (CI), 0.89-0.9], a balanced accuracy of 0.77 (95% CI, 0.75-0.79), an F1-score of 0.78 (95% CI, 0.76-0.79), a positive predictive value of 0.88 (0.86-0.91), a negative predictive value of 0.61 (0.6-0.63), a sensitivity of 0.69 (0.66-0.72), and a specificity of 0.84 (0.8-0.88). Five hundred fifty-nine subjects experienced an unfavourable outcome (88.7%). The final classification model to predict unfavourable functional outcomes from IHCA at hospital discharge had an area under the receiver operating characteristic curve of 0.93 (95% CI, 0.92-0.93), a balanced accuracy of 0.59 (95% CI, 0.57-0.61), an F1-score of 0.94 (95% CI, 0.94-0.95), a positive predictive value of 0.91 (0.9-0.91), a negative predictive value of 0.57 (0.48-0.66), a sensitivity of 0.98 (0.97-0.99), and a specificity of 0.2 (0.16-0.24). Conclusion Using data readily available at emergency team arrival, machine learning algorithms had a high predictive power to forecast failure to achieve ROSC and unfavourable functional outcomes from IHCA while cardiopulmonary resuscitation was still ongoing; however, the positive predictive value of both models was not high enough to allow for early termination of resuscitation efforts.

Using machine learning to study the effect of medication adherence in Opioid Use Disorder

BACKGROUND

Opioid Use Disorder (OUD) and opioid overdose (OD) impose huge social and economic burdens on society and health care systems. Research suggests that Medication for Opioid Use Disorder (MOUD) is effective in the treatment of OUD. We use machine learning to investigate the association between patient's adherence to prescribed MOUD along with other risk factors in patients diagnosed with OUD and potential OD following the treatment.

METHODS

We used longitudinal Medicaid claims for two selected US states to subset a total of 26,685 patients with OUD diagnosis and appropriate Medicaid coverage between 2015 and 2018. We considered patient age, sex, region level socio-economic data, past comorbidities, MOUD prescription type and other selected prescribed medications along with the Proportion of Days Covered (PDC) as a proxy for adherence to MOUD as predictive variables for our model, and overdose events as the dependent variable. We applied four different machine learning classifiers and compared their performance, focusing on the importance and effect of PDC as a variable. We also calculated results based on risk stratification, where our models separate high risk individuals from low risk, to assess usefulness in clinical decision-making.

RESULTS

Among the selected classifiers, the XGBoost classifier has the highest AUC (0.77) closely followed by the Logistic Regression (LR). The LR has the best stratification result: patients in the top 10% of risk scores account for 35.37% of overdose events over the next 12 month observation period. PDC score calculated over the treatment window is one of the most important features, with better PDC lowering risk of OD, as expected. In terms of risk stratification results, of the 35.37% of overdose events that the predictive model could detect within the top 10% of risk scores, 72.3% of these cases were non-adherent in terms of their medication (PDC <0.8). Targeting the top 10% outcome of the predictive model could decrease the total number of OD events by 10.4%.

CONCLUSIONS

The best performing models allow identification of, and focus on, those at high risk of opioid overdose. With MOUD being included for the first time as a factor of interest, and being identified as a significant factor, outreach activities related to MOUD can be targeted at those at highest risk.

Artificial intelligence's role in vascular surgery decision-making

Artificial intelligence (AI) is the next great advance informing medical science. Several disciplines, including vascular surgery, use AI-based decision-making tools to improve clinical performance. Although applied widely, AI functions best when confronted with voluminous, accurate data. Consistent, predictable analytic technique selection also challenges researchers. This article contextualizes AI analyses within evidence-based medicine, focusing on "big data" and health services research, as well as discussing opportunities to improve data collection and realize AI's promise.