1
|
Govindan S, Spicer A, Bearce M, Schaefer RS, Uhl A, Alterovitz G, Kim MJ, Carey KA, Shah NS, Winslow C, Gilbert E, Stey A, Weiss AM, Amin D, Karway G, Martin J, Edelson DP, Churpek MM. Development and Validation of a Machine Learning COVID-19 Veteran (COVet) Deterioration Risk Score. Crit Care Explor 2024; 6:e1116. [PMID: 39028867 PMCID: PMC11262818 DOI: 10.1097/cce.0000000000001116] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/21/2024] Open
Abstract
BACKGROUND AND OBJECTIVE To develop the COVid Veteran (COVet) score for clinical deterioration in Veterans hospitalized with COVID-19 and further validate this model in both Veteran and non-Veteran samples. No such score has been derived and validated while incorporating a Veteran sample. DERIVATION COHORT Adults (age ≥ 18 yr) hospitalized outside the ICU with a diagnosis of COVID-19 for model development to the Veterans Health Administration (VHA) (n = 80 hospitals). VALIDATION COHORT External validation occurred in a VHA cohort of 34 hospitals, as well as six non-Veteran health systems for further external validation (n = 21 hospitals) between 2020 and 2023. PREDICTION MODEL eXtreme Gradient Boosting machine learning methods were used, and performance was assessed using the area under the receiver operating characteristic curve and compared with the National Early Warning Score (NEWS). The primary outcome was transfer to the ICU or death within 24 hours of each new variable observation. Model predictor variables included demographics, vital signs, structured flowsheet data, and laboratory values. RESULTS A total of 96,908 admissions occurred during the study period, of which 59,897 were in the Veteran sample and 37,011 were in the non-Veteran sample. During external validation in the Veteran sample, the model demonstrated excellent discrimination, with an area under the receiver operating characteristic curve of 0.88. This was significantly higher than NEWS (0.79; p < 0.01). In the non-Veteran sample, the model also demonstrated excellent discrimination (0.86 vs. 0.79 for NEWS; p < 0.01). The top three variables of importance were eosinophil percentage, mean oxygen saturation in the prior 24-hour period, and worst mental status in the prior 24-hour period. CONCLUSIONS We used machine learning methods to develop and validate a highly accurate early warning score in both Veterans and non-Veterans hospitalized with COVID-19. The model could lead to earlier identification and therapy, which may improve outcomes.
Collapse
Affiliation(s)
- Sushant Govindan
- MInDSET Service Line, Kansas City Veterans Affairs Hospital, Kansas City, MO
| | - Alexandra Spicer
- Division of Allergy, Pulmonary, and Critical Care Division, University of Wisconsin-Madison, Madison, WI
| | - Matthew Bearce
- MInDSET Service Line, Kansas City Veterans Affairs Hospital, Kansas City, MO
| | - Richard S. Schaefer
- MInDSET Service Line, Kansas City Veterans Affairs Hospital, Kansas City, MO
| | - Andrea Uhl
- MInDSET Service Line, Kansas City Veterans Affairs Hospital, Kansas City, MO
| | - Gil Alterovitz
- Harvard Medical School, Boston, MA
- Office of Research and Development, Department of Veterans Affairs, Washington, DC
| | - Michael J. Kim
- Office of Research and Development, Department of Veterans Affairs, Washington, DC
| | - Kyle A. Carey
- Section of General Internal Medicine, University of Chicago, Chicago, IL
| | - Nirav S. Shah
- Department of Medicine, NorthShore University HealthSystem, Evanston, IL
| | | | - Emily Gilbert
- Department of Medicine, Loyola University Medical Center, Maywood, IL
| | - Anne Stey
- Department of Surgery, Northwestern University School of Medicine, Chicago, IL
| | - Alan M. Weiss
- Section of Critical Care, Baycare Health System, Clearwater, FL
| | - Devendra Amin
- Section of Critical Care, Baycare Health System, Clearwater, FL
| | - George Karway
- Division of Allergy, Pulmonary, and Critical Care Division, University of Wisconsin-Madison, Madison, WI
| | - Jennie Martin
- Division of Allergy, Pulmonary, and Critical Care Division, University of Wisconsin-Madison, Madison, WI
| | - Dana P. Edelson
- Section of Hospital Medicine, University of Chicago, Chicago, IL
| | - Matthew M. Churpek
- Division of Allergy, Pulmonary, and Critical Care Division, University of Wisconsin-Madison, Madison, WI
- Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, Madison, WI
| |
Collapse
|
2
|
Mathur R, Meyfroidt G, Robba C, Stevens RD. Neuromonitoring in the ICU - what, how and why? Curr Opin Crit Care 2024; 30:99-105. [PMID: 38441121 DOI: 10.1097/mcc.0000000000001138] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 03/12/2024]
Abstract
PURPOSE OF REVIEW We selectively review emerging noninvasive neuromonitoring techniques and the evidence that supports their use in the ICU setting. The focus is on neuromonitoring research in patients with acute brain injury. RECENT FINDINGS Noninvasive intracranial pressure evaluation with optic nerve sheath diameter measurements, transcranial Doppler waveform analysis, or skull mechanical extensometer waveform recordings have potential safety and resource-intensity advantages when compared to standard invasive monitors, however each of these techniques has limitations. Quantitative electroencephalography can be applied for detection of cerebral ischemia and states of covert consciousness. Near-infrared spectroscopy may be leveraged for cerebral oxygenation and autoregulation computation. Automated quantitative pupillometry and heart rate variability analysis have been shown to have diagnostic and/or prognostic significance in selected subtypes of acute brain injury. Finally, artificial intelligence is likely to transform interpretation and deployment of neuromonitoring paradigms individually and when integrated in multimodal paradigms. SUMMARY The ability to detect brain dysfunction and injury in critically ill patients is being enriched thanks to remarkable advances in neuromonitoring data acquisition and analysis. Studies are needed to validate the accuracy and reliability of these new approaches, and their feasibility and implementation within existing intensive care workflows.
Collapse
Affiliation(s)
- Rohan Mathur
- Department of Anesthesiology & Critical Care Medicine, Johns Hopkins University School of Medicine, Baltimore, MD
- Department of Neurology, Johns Hopkins University School of Medicine, Baltimore, MD, USA
| | - Geert Meyfroidt
- Department of Intensive Care Medicine, University Hospitals Leuven, Belgium and Laboratory of Intensive Care Medicine, Department of Cellular and Molecular Medicine, KU Leuven, Belgium
| | - Chiara Robba
- IRCCS Ospedale Policlinico San Martino, Genova, Italy
- Dipartimento di Scienze Chirurgiche e Diagnostiche Integrate, Università degli Studi di Genova, Genova, Italy
| | - Robert D Stevens
- Department of Anesthesiology & Critical Care Medicine, Johns Hopkins University School of Medicine, Baltimore, MD
- Department of Neurology, Johns Hopkins University School of Medicine, Baltimore, MD, USA
- Department of Biomedical Engineering, Johns Hopkins University, Baltimore MD, USA
| |
Collapse
|
3
|
Ding X, Wang Y, Ma W, Peng Y, Huang J, Wang M, Zhu H. Development of early prediction model of in-hospital cardiac arrest based on laboratory parameters. Biomed Eng Online 2023; 22:116. [PMID: 38057823 DOI: 10.1186/s12938-023-01178-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2023] [Accepted: 11/23/2023] [Indexed: 12/08/2023] Open
Abstract
BACKGROUND In-hospital cardiac arrest (IHCA) is an acute disease with a high fatality rate that burdens individuals, society, and the economy. This study aimed to develop a machine learning (ML) model using routine laboratory parameters to predict the risk of IHCA in rescue-treated patients. METHODS This retrospective cohort study examined all rescue-treated patients hospitalized at the First Medical Center of the PLA General Hospital in Beijing, China, from January 2016 to December 2020. Five machine learning algorithms, including support vector machine, random forest, extra trees classifier (ETC), decision tree, and logistic regression algorithms, were trained to develop models for predicting IHCA. We included blood counts, biochemical markers, and coagulation markers in the model development. We validated model performance using fivefold cross-validation and used the SHapley Additive exPlanation (SHAP) for model interpretation. RESULTS A total of 11,308 participants were included in the study, of which 7779 patients remained. Among these patients, 1796 (23.09%) cases of IHCA occurred. Among five machine learning models for predicting IHCA, the ETC algorithm exhibited better performance, with an AUC of 0.920, compared with the other four machine learning models in the fivefold cross-validation. The SHAP showed that the top ten factors accounting for cardiac arrest in rescue-treated patients are prothrombin activity, platelets, hemoglobin, N-terminal pro-brain natriuretic peptide, neutrophils, prothrombin time, serum albumin, sodium, activated partial thromboplastin time, and potassium. CONCLUSIONS We developed a reliable machine learning-derived model that integrates readily available laboratory parameters to predict IHCA in patients treated with rescue therapy.
Collapse
Affiliation(s)
- Xinhuan Ding
- Medical School of Chinese PLA, Beijing, 100853, China
- Department of Emergency, The First Medical Center, Chinese PLA General Hospital, Fuxing Road 28, Beijing, 100853, People's Republic of China
| | - Yingchan Wang
- Department of Emergency, The First Medical Center, Chinese PLA General Hospital, Fuxing Road 28, Beijing, 100853, People's Republic of China
| | - Weiyi Ma
- Medical School of Chinese PLA, Beijing, 100853, China
- Department of Emergency, The First Medical Center, Chinese PLA General Hospital, Fuxing Road 28, Beijing, 100853, People's Republic of China
| | - Yaojun Peng
- Medical School of Chinese PLA, Beijing, 100853, China
- Department of Emergency, The First Medical Center, Chinese PLA General Hospital, Fuxing Road 28, Beijing, 100853, People's Republic of China
| | - Jingjing Huang
- The Second School of Clinical Medicine, Southern Medical University, Guangzhou, 510000, Guangdong, China
- Department of Emergency, Hainan Hospital of PLA General Hospital, Sanya, 572013, Hainan, China
| | - Meng Wang
- Medical School of Chinese PLA, Beijing, 100853, China
- Department of Emergency, The First Medical Center, Chinese PLA General Hospital, Fuxing Road 28, Beijing, 100853, People's Republic of China
| | - Haiyan Zhu
- Medical School of Chinese PLA, Beijing, 100853, China.
- Department of Emergency, The First Medical Center, Chinese PLA General Hospital, Fuxing Road 28, Beijing, 100853, People's Republic of China.
| |
Collapse
|
4
|
Dünser MW, Hirschl D, Weh B, Meier J, Tschoellitsch T. The value of a machine learning algorithm to predict adverse short-term outcome during resuscitation of patients with in-hospital cardiac arrest: a retrospective study. Eur J Emerg Med 2023; 30:252-259. [PMID: 37115946 DOI: 10.1097/mej.0000000000001031] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/30/2023]
Abstract
Background and importance Guidelines recommend that hospital emergency teams locally validate criteria for termination of cardiopulmonary resuscitation in patients with in-hospital cardiac arrest (IHCA). Objective To determine the value of a machine learning algorithm to predict failure to achieve return of spontaneous circulation (ROSC) and unfavourable functional outcome from IHCA using only data readily available at emergency team arrival. Design Retrospective cohort study. Setting and participants Adults who experienced an IHCA were attended to by the emergency team. Outcome measures and analysis Demographic and clinical data typically available at the arrival of the emergency team were extracted from the institutional IHCA database. In addition, outcome data including the Cerebral Performance Category (CPC) score count at hospital discharge were collected. A model selection procedure for random forests with a hyperparameter search was employed to develop two classification algorithms to predict failure to achieve ROSC and unfavourable (CPC 3-5) functional outcomes. Main results Six hundred thirty patients were included, of which 390 failed to achieve ROSC (61.9%). The final classification model to predict failure to achieve ROSC had an area under the receiver operating characteristic curve of 0.9 [95% confidence interval (CI), 0.89-0.9], a balanced accuracy of 0.77 (95% CI, 0.75-0.79), an F1-score of 0.78 (95% CI, 0.76-0.79), a positive predictive value of 0.88 (0.86-0.91), a negative predictive value of 0.61 (0.6-0.63), a sensitivity of 0.69 (0.66-0.72), and a specificity of 0.84 (0.8-0.88). Five hundred fifty-nine subjects experienced an unfavourable outcome (88.7%). The final classification model to predict unfavourable functional outcomes from IHCA at hospital discharge had an area under the receiver operating characteristic curve of 0.93 (95% CI, 0.92-0.93), a balanced accuracy of 0.59 (95% CI, 0.57-0.61), an F1-score of 0.94 (95% CI, 0.94-0.95), a positive predictive value of 0.91 (0.9-0.91), a negative predictive value of 0.57 (0.48-0.66), a sensitivity of 0.98 (0.97-0.99), and a specificity of 0.2 (0.16-0.24). Conclusion Using data readily available at emergency team arrival, machine learning algorithms had a high predictive power to forecast failure to achieve ROSC and unfavourable functional outcomes from IHCA while cardiopulmonary resuscitation was still ongoing; however, the positive predictive value of both models was not high enough to allow for early termination of resuscitation efforts.
Collapse
Affiliation(s)
- Martin W Dünser
- Department of Anesthesiology and Intensive Care Medicine, Kepler University Hospital and Johannes Kepler University, Linz, Austria
| | | | | | | | | |
Collapse
|
5
|
Jennings JB. Can machine learning predict recurrent cardiac arrest? Resuscitation 2023; 184:109704. [PMID: 36708944 DOI: 10.1016/j.resuscitation.2023.109704] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/13/2023] [Accepted: 01/18/2023] [Indexed: 01/26/2023]
Affiliation(s)
- Jason B Jennings
- Department of Emergency Medicine, VCU Health, 1250 E Marshall St, Richmond, VA 23298, United States.
| |
Collapse
|
6
|
Warren D, Marashi A, Siddiqui A, Eijaz AA, Pradhan P, Lim D, Call G, Dras M. Using machine learning to study the effect of medication adherence in Opioid Use Disorder. PLoS One 2022; 17:e0278988. [PMID: 36520864 PMCID: PMC9754174 DOI: 10.1371/journal.pone.0278988] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/15/2021] [Accepted: 11/29/2022] [Indexed: 12/23/2022] Open
Abstract
BACKGROUND Opioid Use Disorder (OUD) and opioid overdose (OD) impose huge social and economic burdens on society and health care systems. Research suggests that Medication for Opioid Use Disorder (MOUD) is effective in the treatment of OUD. We use machine learning to investigate the association between patient's adherence to prescribed MOUD along with other risk factors in patients diagnosed with OUD and potential OD following the treatment. METHODS We used longitudinal Medicaid claims for two selected US states to subset a total of 26,685 patients with OUD diagnosis and appropriate Medicaid coverage between 2015 and 2018. We considered patient age, sex, region level socio-economic data, past comorbidities, MOUD prescription type and other selected prescribed medications along with the Proportion of Days Covered (PDC) as a proxy for adherence to MOUD as predictive variables for our model, and overdose events as the dependent variable. We applied four different machine learning classifiers and compared their performance, focusing on the importance and effect of PDC as a variable. We also calculated results based on risk stratification, where our models separate high risk individuals from low risk, to assess usefulness in clinical decision-making. RESULTS Among the selected classifiers, the XGBoost classifier has the highest AUC (0.77) closely followed by the Logistic Regression (LR). The LR has the best stratification result: patients in the top 10% of risk scores account for 35.37% of overdose events over the next 12 month observation period. PDC score calculated over the treatment window is one of the most important features, with better PDC lowering risk of OD, as expected. In terms of risk stratification results, of the 35.37% of overdose events that the predictive model could detect within the top 10% of risk scores, 72.3% of these cases were non-adherent in terms of their medication (PDC <0.8). Targeting the top 10% outcome of the predictive model could decrease the total number of OD events by 10.4%. CONCLUSIONS The best performing models allow identification of, and focus on, those at high risk of opioid overdose. With MOUD being included for the first time as a factor of interest, and being identified as a significant factor, outreach activities related to MOUD can be targeted at those at highest risk.
Collapse
Affiliation(s)
| | - Amir Marashi
- Macquarie University, Sydney, NSW, Australia
- Digital Health Cooperative Research Centre, Sydney, NSW, Australia
| | | | | | - Pooja Pradhan
- Western Sydney University, Campbelltown, NSW, Australia
| | - David Lim
- Western Sydney University, Campbelltown, NSW, Australia
| | - Gary Call
- Gainwell Technologies, Tysons, VA, United States of America
| | - Mark Dras
- Macquarie University, Sydney, NSW, Australia
- * E-mail:
| |
Collapse
|
7
|
Zarkowsky DS, Stonko DP. Artificial intelligence's role in vascular surgery decision-making. Semin Vasc Surg 2021; 34:260-267. [PMID: 34911632 DOI: 10.1053/j.semvascsurg.2021.10.005] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2021] [Revised: 10/17/2021] [Accepted: 10/18/2021] [Indexed: 12/28/2022]
Abstract
Artificial intelligence (AI) is the next great advance informing medical science. Several disciplines, including vascular surgery, use AI-based decision-making tools to improve clinical performance. Although applied widely, AI functions best when confronted with voluminous, accurate data. Consistent, predictable analytic technique selection also challenges researchers. This article contextualizes AI analyses within evidence-based medicine, focusing on "big data" and health services research, as well as discussing opportunities to improve data collection and realize AI's promise.
Collapse
Affiliation(s)
- Devin S Zarkowsky
- Division of Vascular Surgery and Endovascular Therapy, University of Colorado School of Medicine, 12615 E 17(th) Place, AO1, Aurora, CO, 80045.
| | - David P Stonko
- Department of Surgery, The Johns Hopkins Hospital, Baltimore, MD
| |
Collapse
|