1
|
Zeng J, Zhang D, Lin S, Su X, Wang P, Zhao Y, Zheng Z. Comparative analysis of machine learning vs. traditional modeling approaches for predicting in-hospital mortality after cardiac surgery: temporal and spatial external validation based on a nationwide cardiac surgery registry. EUROPEAN HEART JOURNAL. QUALITY OF CARE & CLINICAL OUTCOMES 2024; 10:121-131. [PMID: 37218710 DOI: 10.1093/ehjqcco/qcad028] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/11/2023] [Revised: 05/12/2023] [Accepted: 05/21/2023] [Indexed: 05/24/2023]
Abstract
AIMS Preoperative risk assessment is crucial for cardiac surgery. Although previous studies suggested machine learning (ML) may improve in-hospital mortality predictions after cardiac surgery compared to traditional modeling approaches, the validity is doubted due to lacking external validation, limited sample sizes, and inadequate modeling considerations. We aimed to assess predictive performance between ML and traditional modelling approaches, while addressing these major limitations. METHODS AND RESULTS Adult cardiac surgery cases (n = 168 565) between 2013 and 2018 in the Chinese Cardiac Surgery Registry were used to develop, validate, and compare various ML vs. logistic regression (LR) models. The dataset was split for temporal (2013-2017 for training, 2018 for testing) and spatial (geographically-stratified random selection of 83 centers for training, 22 for testing) experiments, respectively. Model performances were evaluated in testing sets for discrimination and calibration. The overall in-hospital mortality was 1.9%. In the temporal testing set (n = 32 184), the best-performing ML model demonstrated a similar area under the receiver operating characteristic curve (AUC) of 0.797 (95% CI 0.779-0.815) to the LR model (AUC 0.791 [95% CI 0.775-0.808]; P = 0.12). In the spatial experiment (n = 28 323), the best ML model showed a statistically better but modest performance improvement (AUC 0.732 [95% CI 0.710-0.754]) than LR (AUC 0.713 [95% CI 0.691-0.737]; P = 0.002). Varying feature selection methods had relatively smaller effects on ML models. Most ML and LR models were significantly miscalibrated. CONCLUSION ML provided only marginal improvements over traditional modelling approaches in predicting cardiac surgery mortality with routine preoperative variables, which calls for more judicious use of ML in practice.
Collapse
Affiliation(s)
- Juntong Zeng
- National Clinical Research Center of Cardiovascular Diseases, Fuwai Hospital, National Center for Cardiovascular Diseases, 167 Beilishi Road, Xicheng, Beijing, 100037, People's Republic of China
- State Key Laboratory of Cardiovascular Disease, Fuwai Hospital, National Center for Cardiovascular Diseases, 167 Beilishi Road, Xicheng, Beijing, 100037, People's Republic of China
- Chinese Academy of Medical Sciences and Peking Union Medical College, 9 Dongdansantiao, Dongcheng, Beijing, 100730, People's Republic of China
| | - Danwei Zhang
- National Clinical Research Center of Cardiovascular Diseases, Fuwai Hospital, National Center for Cardiovascular Diseases, 167 Beilishi Road, Xicheng, Beijing, 100037, People's Republic of China
- State Key Laboratory of Cardiovascular Disease, Fuwai Hospital, National Center for Cardiovascular Diseases, 167 Beilishi Road, Xicheng, Beijing, 100037, People's Republic of China
- Chinese Academy of Medical Sciences and Peking Union Medical College, 9 Dongdansantiao, Dongcheng, Beijing, 100730, People's Republic of China
- Department of Cardiac Surgery, Fujian Children's Hospital (Fujian Branch of Shanghai Children's Medical Center), College of Clinical Medicine for Obstetrics & Gynecology and Pediatrics, Fujian Medical University, 966 Hengyu Road, Jinan, Fuzhou, 350014, People's Republic of China
| | - Shen Lin
- National Clinical Research Center of Cardiovascular Diseases, Fuwai Hospital, National Center for Cardiovascular Diseases, 167 Beilishi Road, Xicheng, Beijing, 100037, People's Republic of China
- State Key Laboratory of Cardiovascular Disease, Fuwai Hospital, National Center for Cardiovascular Diseases, 167 Beilishi Road, Xicheng, Beijing, 100037, People's Republic of China
- Chinese Academy of Medical Sciences and Peking Union Medical College, 9 Dongdansantiao, Dongcheng, Beijing, 100730, People's Republic of China
- Department of Cardiovascular Surgery, Fuwai Hospital, National Center for Cardiovascular Diseases, 167 Beilishi Road, Xicheng, Beijing, 100037, People's Republic of China
| | - Xiaoting Su
- National Clinical Research Center of Cardiovascular Diseases, Fuwai Hospital, National Center for Cardiovascular Diseases, 167 Beilishi Road, Xicheng, Beijing, 100037, People's Republic of China
- State Key Laboratory of Cardiovascular Disease, Fuwai Hospital, National Center for Cardiovascular Diseases, 167 Beilishi Road, Xicheng, Beijing, 100037, People's Republic of China
- Chinese Academy of Medical Sciences and Peking Union Medical College, 9 Dongdansantiao, Dongcheng, Beijing, 100730, People's Republic of China
| | - Peng Wang
- National Clinical Research Center of Cardiovascular Diseases, Fuwai Hospital, National Center for Cardiovascular Diseases, 167 Beilishi Road, Xicheng, Beijing, 100037, People's Republic of China
- State Key Laboratory of Cardiovascular Disease, Fuwai Hospital, National Center for Cardiovascular Diseases, 167 Beilishi Road, Xicheng, Beijing, 100037, People's Republic of China
- Chinese Academy of Medical Sciences and Peking Union Medical College, 9 Dongdansantiao, Dongcheng, Beijing, 100730, People's Republic of China
| | - Yan Zhao
- National Clinical Research Center of Cardiovascular Diseases, Fuwai Hospital, National Center for Cardiovascular Diseases, 167 Beilishi Road, Xicheng, Beijing, 100037, People's Republic of China
- State Key Laboratory of Cardiovascular Disease, Fuwai Hospital, National Center for Cardiovascular Diseases, 167 Beilishi Road, Xicheng, Beijing, 100037, People's Republic of China
| | - Zhe Zheng
- National Clinical Research Center of Cardiovascular Diseases, Fuwai Hospital, National Center for Cardiovascular Diseases, 167 Beilishi Road, Xicheng, Beijing, 100037, People's Republic of China
- State Key Laboratory of Cardiovascular Disease, Fuwai Hospital, National Center for Cardiovascular Diseases, 167 Beilishi Road, Xicheng, Beijing, 100037, People's Republic of China
- Chinese Academy of Medical Sciences and Peking Union Medical College, 9 Dongdansantiao, Dongcheng, Beijing, 100730, People's Republic of China
- Department of Cardiovascular Surgery, Fuwai Hospital, National Center for Cardiovascular Diseases, 167 Beilishi Road, Xicheng, Beijing, 100037, People's Republic of China
- Key Laboratory of Coronary Heart Disease Risk Prediction and Precision Therapy, Chinese Academy of Medical Sciences and Peking Union Medical College, 167 Beilishi Road, Xicheng, Beijing, 100037, People's Republic of China
| |
Collapse
|
2
|
Weiss AJ, Yadaw AS, Meretzky DL, Levin MA, Adams DH, McCardle K, Pandey G, Iyengar R. Machine learning using institution-specific multi-modal electronic health records improves mortality risk prediction for cardiac surgery patients. JTCVS OPEN 2023; 14:214-251. [PMID: 37425442 PMCID: PMC10328834 DOI: 10.1016/j.xjon.2023.03.010] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 10/27/2022] [Revised: 02/04/2023] [Accepted: 03/16/2023] [Indexed: 07/11/2023]
Abstract
Background The Society of Thoracic Surgeons risk scores are widely used to assess risk of morbidity and mortality in specific cardiac surgeries but may not perform optimally in all patients. In a cohort of patients undergoing cardiac surgery, we developed a data-driven, institution-specific machine learning-based model inferred from multi-modal electronic health records and compared the performance with the Society of Thoracic Surgeons models. Methods All adult patients undergoing cardiac surgery between 2011 and 2016 were included. Routine electronic health record administrative, demographic, clinical, hemodynamic, laboratory, pharmacological, and procedural data features were extracted. The outcome was postoperative mortality. The database was randomly split into training (development) and test (evaluation) cohorts. Models developed using 4 classification algorithms were compared using 6 evaluation metrics. The performance of the final model was compared with the Society of Thoracic Surgeons models for 7 index surgical procedures. Results A total of 6392 patients were included and described by 4016 features. Overall mortality was 3.0% (n = 193). The XGBoost algorithm using only features with no missing data (336 features) yielded the best-performing predictor. When applied to the test set, the predictor performed well (F-measure = 0.775; precision = 0.756; recall = 0.795; accuracy = 0.986; area under the receiver operating characteristic curve = 0.978; area under the precision-recall curve = 0.804). eXtreme Gradient Boosting consistently demonstrated improved performance over the Society of Thoracic Surgeons models when evaluated on index procedures within the test set. Conclusions Machine learning models using institution-specific multi-modal electronic health records may improve performance in predicting mortality for individual patients undergoing cardiac surgery compared with the standard-of-care, population-derived Society of Thoracic Surgeons models. Institution-specific models may provide insights complementary to population-derived risk predictions to aid patient-level decision making.
Collapse
Affiliation(s)
- Aaron J. Weiss
- Department of Cardiovascular Surgery, Icahn School of Medicine at Mount Sinai, New York, NY
- Department of Thoracic and Cardiovascular Surgery, Heart, Vascular & Thoracic Institute, Cleveland Clinic, Cleveland, Ohio
- Department of Pharmacological Sciences and Institute for Systems Biomedicine, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Arjun S. Yadaw
- Department of Pharmacological Sciences and Institute for Systems Biomedicine, Icahn School of Medicine at Mount Sinai, New York, NY
| | - David L. Meretzky
- Department of Pharmacological Sciences and Institute for Systems Biomedicine, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Matthew A. Levin
- Division of Cardiothoracic Anesthesia, Department of Anesthesiology and Critical Care, Icahn School of Medicine at Mount Sinai, New York, NY
| | - David H. Adams
- Department of Cardiovascular Surgery, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Ken McCardle
- Department of Clinical Operations, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Gaurav Pandey
- Department of Genetics and Genomic Sciences, Icahn School of Medicine at Mount Sinai, New York, NY
| | - Ravi Iyengar
- Department of Pharmacological Sciences and Institute for Systems Biomedicine, Icahn School of Medicine at Mount Sinai, New York, NY
| |
Collapse
|
3
|
Rellum SR, Schuurmans J, van der Ven WH, Eberl S, Driessen AHG, Vlaar APJ, Veelo DP. Machine learning methods for perioperative anesthetic management in cardiac surgery patients: a scoping review. J Thorac Dis 2022; 13:6976-6993. [PMID: 35070381 PMCID: PMC8743411 DOI: 10.21037/jtd-21-765] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2021] [Accepted: 08/27/2021] [Indexed: 12/27/2022]
Abstract
Background Machine learning (ML) is developing fast with promising prospects within medicine and already has several applications in perioperative care. We conducted a scoping review to examine the extent and potential limitations of ML implementation in perioperative anesthetic care, specifically in cardiac surgery patients. Methods We mapped the current literature by searching three databases: MEDLINE (Ovid), EMBASE (Ovid), and Cochrane Library. Articles were eligible if they reported on perioperative ML use in the field of cardiac surgery with relevance to anesthetic practices. Data on the applicability of ML and comparability to conventional statistical methods were extracted. Results Forty-six articles on ML relevant to the work of the anesthesiologist in cardiac surgery were identified. Three main categories emerged: (I) event and risk prediction, (II) hemodynamic monitoring, and (III) automation of echocardiography. Prediction models based on ML tend to behave similarly to conventional statistical methods. Using dynamic hemodynamic or ultrasound data in ML models, however, shifts the potential to promising results. Conclusions ML in cardiac surgery is increasingly used in perioperative anesthetic management. The majority is used for prediction purposes similar to conventional clinical scores. Remarkable ML model performances are achieved when using real-time dynamic parameters. However, beneficial clinical outcomes of ML integration have yet to be determined. Nonetheless, the first steps introducing ML in perioperative anesthetic care for cardiac surgery have been taken.
Collapse
Affiliation(s)
- Santino R Rellum
- Department of Anesthesiology, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands.,Department of Intensive Care, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| | - Jaap Schuurmans
- Department of Anesthesiology, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands.,Department of Intensive Care, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| | - Ward H van der Ven
- Department of Anesthesiology, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands.,Department of Intensive Care, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| | - Susanne Eberl
- Department of Anesthesiology, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| | - Antoine H G Driessen
- Department of Cardiothoracic Surgery, Heart Center, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| | - Alexander P J Vlaar
- Department of Intensive Care, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| | - Denise P Veelo
- Department of Anesthesiology, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| |
Collapse
|
4
|
Atilgan K, Onuk BE, Köksal Coşkun P, Yeşi L FG, Aslan C, Çolak A, Çelebi AS, Bozbaş H. Remote patient monitoring after cardiac surgery: The utility of a novel telemedicine system. J Card Surg 2021; 36:4226-4234. [PMID: 34478205 PMCID: PMC9292885 DOI: 10.1111/jocs.15962] [Citation(s) in RCA: 15] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2021] [Accepted: 08/02/2021] [Indexed: 11/28/2022]
Abstract
Objective We examined cardiac surgery patients who underwent monitoring of postoperative vital parameters using medical monitoring devices which transferred data to a mobile application and a web‐based software. Methods From November 2017 to November 2020, a total of 2340 patients were enrolled in the remote patient monitoring system after undergoing cardiac surgery. The medical devices recorded vital parameters, such as blood pressure, pulse rate, saturation, body temperature, blood glucose, and electrocardiography were measured via the Health Monitor DakikApp and Holter ECG DakikApp devices which reported data to web‐based software and a mobile application (DakikApp Mobile Systems, Remscheid, Germany). During the follow‐up period, patients were contacted daily through text and voice messages, and video conferences. Remote Medical Evaluations (RMEs) concerning patients' medical states were performed. Medication reminders, daily treatment were communicated to the patients with the DakikApp Mobile Systems Software. Results During a mean follow‐up period of 78.9 ± 107.1 (10–395) days, a total of 135,786 patient contacts were recorded (782 video conferences, 2805 voice messaging, and 132,199 text correspondence). The number of RMEs handled by the Telemedicine Team was 79,560. A total of 105,335 vital parameter measurements were performed and 5024 hospital application requests (6.3% per RME) were addressed successfully and hospitalization was avoided. A total of 144 (6.1%) potentially life‐threatening complications were found to have been diagnosed early using the Telemedicine System. Conclusion Remote Patient Monitoring Systems combined with professional medical devices are feasible, effective, and safe for the purpose of improving postoperative outcomes.
Collapse
Affiliation(s)
- Kıvanç Atilgan
- Department of Cardiovascular Surgery, TOBB ETU Hospital, Ankara, Turkey
| | - Burak E Onuk
- Department of Cardiovascular Surgery, TOBB ETU Hospital, Ankara, Turkey
| | | | - Fahri G Yeşi L
- Department of Cardiovascular Surgery, TOBB ETU Hospital, Ankara, Turkey
| | - Cemal Aslan
- Department of Cardiovascular Surgery, TOBB ETU Hospital, Ankara, Turkey
| | - Abdullah Çolak
- Department of Cardiovascular Surgery, TOBB ETU Hospital, Ankara, Turkey
| | | | - Hüseyin Bozbaş
- Department of Cardiology, TOBB ETU Hospital, Ankara, Turkey
| |
Collapse
|
5
|
Benedetto U, Dimagli A, Sinha S, Cocomello L, Gibbison B, Caputo M, Gaunt T, Lyon M, Holmes C, Angelini GD. Machine learning improves mortality risk prediction after cardiac surgery: Systematic review and meta-analysis. J Thorac Cardiovasc Surg 2020; 163:2075-2087.e9. [PMID: 32900480 DOI: 10.1016/j.jtcvs.2020.07.105] [Citation(s) in RCA: 33] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/23/2020] [Revised: 07/16/2020] [Accepted: 07/30/2020] [Indexed: 02/01/2023]
Abstract
BACKGROUND Interest in the usefulness of machine learning (ML) methods for outcomes prediction has continued to increase in recent years. However, the advantage of advanced ML model over traditional logistic regression (LR) remains controversial. We performed a systematic review and meta-analysis of studies comparing the discrimination accuracy between ML models versus LR in predicting operative mortality following cardiac surgery. METHODS The present systematic review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analysis statement. Discrimination ability was assessed using the C-statistic. Pooled C-statistics and its 95% credibility interval for ML models and LR were obtained were obtained using a Bayesian framework. Pooled estimates for ML models and LR were compared to inform on difference between the 2 approaches. RESULTS We identified 459 published citations of which 15 studies met inclusion criteria and were used for the quantitative and qualitative analysis. When the best ML model from individual study was used, meta-analytic estimates showed that ML were associated with a significantly higher C-statistic (ML, 0.88; 95% credibility interval, 0.83-0.93 vs LR, 0.81; 95% credibility interval, 0.77-0.85; P = .03). When individual ML algorithms were instead selected, we found a nonsignificant trend toward better prediction with each of ML algorithms. We found no evidence of publication bias (P = .70). CONCLUSIONS The present findings suggest that when compared with LR, ML models provide better discrimination in mortality prediction after cardiac surgery. However, the magnitude and clinical influence of such an improvement remains uncertain.
Collapse
Affiliation(s)
- Umberto Benedetto
- Department of Translational Health Sciences, Bristol Heart Institute, University of Bristol, London, United Kingdom.
| | - Arnaldo Dimagli
- Department of Translational Health Sciences, Bristol Heart Institute, University of Bristol, London, United Kingdom
| | - Shubhra Sinha
- Department of Translational Health Sciences, Bristol Heart Institute, University of Bristol, London, United Kingdom
| | - Lucia Cocomello
- Department of Translational Health Sciences, Bristol Heart Institute, University of Bristol, London, United Kingdom
| | - Ben Gibbison
- Department of Translational Health Sciences, Bristol Heart Institute, University of Bristol, London, United Kingdom
| | - Massimo Caputo
- Department of Translational Health Sciences, Bristol Heart Institute, University of Bristol, London, United Kingdom
| | - Tom Gaunt
- Population Health Sciences, University of Bristol, London, United Kingdom
| | - Matt Lyon
- Population Health Sciences, University of Bristol, London, United Kingdom
| | - Chris Holmes
- Department of Statistics, University of Oxford, Oxford, United Kingdom
| | - Gianni D Angelini
- Department of Translational Health Sciences, Bristol Heart Institute, University of Bristol, London, United Kingdom
| |
Collapse
|
6
|
Abstract
Commercial applications of artificial intelligence and machine learning have made remarkable progress recently, particularly in areas such as image recognition, natural speech processing, language translation, textual analysis, and self-learning. Progress had historically languished in these areas, such that these skills had come to seem ineffably bound to intelligence. However, these commercial advances have performed best at single-task applications in which imperfect outputs and occasional frank errors can be tolerated.The practice of anesthesiology is different. It embodies a requirement for high reliability, and a pressured cycle of interpretation, physical action, and response rather than any single cognitive act. This review covers the basics of what is meant by artificial intelligence and machine learning for the practicing anesthesiologist, describing how decision-making behaviors can emerge from simple equations. Relevant clinical questions are introduced to illustrate how machine learning might help solve them-perhaps bringing anesthesiology into an era of machine-assisted discovery.
Collapse
|
7
|
Mejía OAV, Sá MPBO, Deininger MO, Dallan LRP, Segalote RC, Oliveira MAPD, Atik FA, Santos MAD, Silva PGMDBE, Milani RM, Hueb AC, Monteiro R, Lima RC, Lisboa LAF, Dallan LAO, Puskas J, Jatene FB. Off-pump versus On-pump Coronary Artery Bypass Grafting in Frail Patients: Study Protocol for the FRAGILE Multicenter Randomized Controlled Trial. Braz J Cardiovasc Surg 2018; 32:428-434. [PMID: 29211225 PMCID: PMC5701109 DOI: 10.21470/1678-9741-2017-0196] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2017] [Accepted: 10/09/2017] [Indexed: 11/24/2022] Open
Abstract
Introduction Advances in modern medicine have led to people living longer and healthier
lives. Frailty is an emerging concept in medicine yet to be explored as a
risk factor in cardiac surgery. When it comes to CABG surgery, randomized
controlled clinical trials have primarily focused on low-risk (ROOBY,
CORONARY), elevated-risk (GOPCABE) or high-risk patients (BBS), but not on
frail patients. Therefore, we believe that off-pump CABG could be an
important technique in patients with limited functional capacity to respond
to surgical stress. In this study, the authors introduce the new national,
multicenter, randomized, controlled trial "FRAGILE", to be developed in the
main cardiac surgery centers of Brazil, to clarify the potential benefit of
off-pump CABG in frail patients. Methods FRAGILE is a two-arm, parallel-group, multicentre, individually randomized
(1:1) controlled trial which will enroll 630 patients with blinded outcome
assessment (at 30 days, 6 months, 1 year, 2 years and 3 years), which aims
to compare adverse cardiac and cerebrovascular events after off-pump
versus on-pump CABG in pre-frail and frail patients.
Primary outcomes will be all-cause mortality, acute myocardial infarction,
cardiac arrest with successful resuscitation, low cardiac output
syndrome/cardiogenic shock, stroke, and coronary reintervention. Secondary
outcomes will be major adverse cardiac and cerebrovascular events, operative
time, mechanical ventilation time, hyperdynamic shock, new onset of atrial
fibrillation, renal replacement therapy, reoperation for bleeding,
pneumonia, length of stay in intensive care unit, length of stay in
hospital, number of units of blood transfused, graft patency, rate of
complete revascularization, neurobehavioral outcomes after cardiac surgery,
quality of life after cardiac surgery and costs. Discussion FRAGILE trial will determine whether off-pump CABG is superior to
conventional on-pump CABG in the surgical treatment of pre-frail and frail
patients. Trial registration ClinicalTrials.gov, ID: NCT02338947. Registered on August 29th
2014; last updated on March 21st 2016.
Collapse
Affiliation(s)
- Omar Asdrúbal Vilca Mejía
- Instituto do Coração do Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo (InCor-HCFMUSP), São Paulo, SP, Brazil
| | - Michel Pompeu Barros Oliveira Sá
- Instituto do Coração do Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo (InCor-HCFMUSP), São Paulo, SP, Brazil.,Universidade de Pernambuco (UPE), Recife, PE, Brazil
| | | | - Luís Roberto Palma Dallan
- Instituto do Coração do Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo (InCor-HCFMUSP), São Paulo, SP, Brazil
| | | | | | | | | | | | | | - Alexandre Ciappina Hueb
- Instituto do Coração do Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo (InCor-HCFMUSP), São Paulo, SP, Brazil.,Hospital das Clinicas Samuel Libânio, Pouso Alegre, Minas Gerais, Brazil
| | - Rosangela Monteiro
- Instituto do Coração do Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo (InCor-HCFMUSP), São Paulo, SP, Brazil
| | - Ricardo Carvalho Lima
- Division of Cardiovascular Surgery, Pronto-Socorro Cardiológico de Pernambuco (PROCAPE), Recife, PE, Brazil.,Universidade de Pernambuco (UPE), Recife, PE, Brazil
| | - Luiz Augusto Ferreira Lisboa
- Instituto do Coração do Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo (InCor-HCFMUSP), São Paulo, SP, Brazil
| | - Luís Alberto Oliveira Dallan
- Instituto do Coração do Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo (InCor-HCFMUSP), São Paulo, SP, Brazil
| | - John Puskas
- Mount Sinai Heart at Mount Sinai Saint Luke's New York, NY, USA
| | - Fabio Biscegli Jatene
- Instituto do Coração do Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo (InCor-HCFMUSP), São Paulo, SP, Brazil
| |
Collapse
|
8
|
Complementarity of Clinician Judgment and Evidence Based Models in Medical Decision Making: Antecedents, Prospects, and Challenges. BIOMED RESEARCH INTERNATIONAL 2016; 2016:1425693. [PMID: 27642588 PMCID: PMC5013221 DOI: 10.1155/2016/1425693] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 03/10/2016] [Accepted: 07/25/2016] [Indexed: 01/02/2023]
Abstract
Early accounts of the development of modern medicine suggest that the clinical skills, scientific competence, and doctors' judgment were the main impetus for treatment decision, diagnosis, prognosis, therapy assessment, and medical progress. Yet, clinician judgment has its own critics and is sometimes harshly described as notoriously fallacious and an irrational and unfathomable black box with little transparency. With the rise of contemporary medical research, the reputation of clinician judgment has undergone significant reformation in the last century as its fallacious aspects are increasingly emphasized relative to the evidence based options. Within the last decade, however, medical forecasting literature has seen tremendous change and new understanding is emerging on best ways of sharing medical information to complement the evidence based medicine practices. This review revisits and highlights the core debate on clinical judgments and its interrelations with evidence based medicine. It outlines the key empirical results of clinician judgments relative to evidence based models and identifies its key strengths and prospects, the key limitations and conditions for the effective use of clinician judgment, and the extent to which it can be optimized and professionalized for medical use.
Collapse
|
9
|
LaFaro RJ, Pothula S, Kubal KP, Inchiosa ME, Pothula VM, Yuan SC, Maerz DA, Montes L, Oleszkiewicz SM, Yusupov A, Perline R, Inchiosa MA. Neural Network Prediction of ICU Length of Stay Following Cardiac Surgery Based on Pre-Incision Variables. PLoS One 2015; 10:e0145395. [PMID: 26710254 PMCID: PMC4692524 DOI: 10.1371/journal.pone.0145395] [Citation(s) in RCA: 42] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/28/2015] [Accepted: 12/03/2015] [Indexed: 11/29/2022] Open
Abstract
Background Advanced predictive analytical techniques are being increasingly applied to clinical risk assessment. This study compared a neural network model to several other models in predicting the length of stay (LOS) in the cardiac surgical intensive care unit (ICU) based on pre-incision patient characteristics. Methods Thirty six variables collected from 185 cardiac surgical patients were analyzed for contribution to ICU LOS. The Automatic Linear Modeling (ALM) module of IBM-SPSS software identified 8 factors with statistically significant associations with ICU LOS; these factors were also analyzed with the Artificial Neural Network (ANN) module of the same software. The weighted contributions of each factor (“trained” data) were then applied to data for a “new” patient to predict ICU LOS for that individual. Results Factors identified in the ALM model were: use of an intra-aortic balloon pump; O2 delivery index; age; use of positive cardiac inotropic agents; hematocrit; serum creatinine ≥ 1.3 mg/deciliter; gender; arterial pCO2. The r2 value for ALM prediction of ICU LOS in the initial (training) model was 0.356, p <0.0001. Cross validation in prediction of a “new” patient yielded r2 = 0.200, p <0.0001. The same 8 factors analyzed with ANN yielded a training prediction r2 of 0.535 (p <0.0001) and a cross validation prediction r2 of 0.410, p <0.0001. Two additional predictive algorithms were studied, but they had lower prediction accuracies. Our validated neural network model identified the upper quartile of ICU LOS with an odds ratio of 9.8(p <0.0001). Conclusions ANN demonstrated a 2-fold greater accuracy than ALM in prediction of observed ICU LOS. This greater accuracy would be presumed to result from the capacity of ANN to capture nonlinear effects and higher order interactions. Predictive modeling may be of value in early anticipation of risks of post-operative morbidity and utilization of ICU facilities.
Collapse
Affiliation(s)
- Rocco J. LaFaro
- Department of Surgery, New York Medical College, Valhalla, New York, United States of America
| | - Suryanarayana Pothula
- Department of Anesthesiology, New York Medical College, Valhalla, New York, United States of America
| | - Keshar Paul Kubal
- Department of Pharmacology, New York Medical College, Valhalla, New York, United States of America
| | - Mario Emil Inchiosa
- Revolution Analytics, Inc., Mountain View, California, United States of America
| | - Venu M. Pothula
- Department of Pharmacology, New York Medical College, Valhalla, New York, United States of America
| | - Stanley C. Yuan
- Department of Anesthesiology, New York Medical College, Valhalla, New York, United States of America
| | - David A. Maerz
- Department of Pharmacology, New York Medical College, Valhalla, New York, United States of America
| | - Lucresia Montes
- Department of Pharmacology, New York Medical College, Valhalla, New York, United States of America
| | - Stephen M. Oleszkiewicz
- Department of Pharmacology, New York Medical College, Valhalla, New York, United States of America
| | - Albert Yusupov
- Department of Anesthesiology, New York Medical College, Valhalla, New York, United States of America
| | - Richard Perline
- The SAS Institute, Cary, North Carolina, United States of America
| | - Mario Anthony Inchiosa
- Department of Pharmacology, New York Medical College, Valhalla, New York, United States of America
- * E-mail:
| |
Collapse
|
10
|
Mendes RG, de Souza CR, Machado MN, Correa PR, Di Thommazo-Luporini L, Arena R, Myers J, Pizzolato EB, Borghi-Silva A. Predicting reintubation, prolonged mechanical ventilation and death in post-coronary artery bypass graft surgery: a comparison between artificial neural networks and logistic regression models. Arch Med Sci 2015; 11:756-63. [PMID: 26322087 PMCID: PMC4548023 DOI: 10.5114/aoms.2015.48145] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/08/2013] [Revised: 08/29/2013] [Accepted: 10/07/2013] [Indexed: 11/17/2022] Open
Abstract
INTRODUCTION In coronary artery bypass (CABG) surgery, the common complications are the need for reintubation, prolonged mechanical ventilation (PMV) and death. Thus, a reliable model for the prognostic evaluation of those particular outcomes is a worthwhile pursuit. The existence of such a system would lead to better resource planning, cost reductions and an increased ability to guide preventive strategies. The aim of this study was to compare different methods - logistic regression (LR) and artificial neural networks (ANNs) - in accomplishing this goal. MATERIAL AND METHODS Subjects undergoing CABG (n = 1315) were divided into training (n = 1053) and validation (n = 262) groups. The set of independent variables consisted of age, gender, weight, height, body mass index, diabetes, creatinine level, cardiopulmonary bypass, presence of preserved ventricular function, moderate and severe ventricular dysfunction and total number of grafts. The PMV was also an input for the prediction of death. The ability of ANN to discriminate outcomes was assessed using receiver-operating characteristic (ROC) analysis and the results were compared using a multivariate LR. RESULTS The ROC curve areas for LR and ANN models, respectively, were: for reintubation 0.62 (CI: 0.50-0.75) and 0.65 (CI: 0.53-0.77); for PMV 0.67 (CI: 0.57-0.78) and 0.72 (CI: 0.64-0.81); and for death 0.86 (CI: 0.79-0.93) and 0.85 (CI: 0.80-0.91). No differences were observed between models. CONCLUSIONS The ANN has similar discriminating power in predicting reintubation, PMV and death outcomes. Thus, both models may be applicable as a predictor for these outcomes in subjects undergoing CABG.
Collapse
Affiliation(s)
- Renata G Mendes
- Department of Physical Therapy, Federal University of Sao Carlos, Sao Carlos, SP, Brazil
| | - César R de Souza
- Computer Department, Federal University of Sao Carlos, Sao Carlos, SP, Brazil
| | - Maurício N Machado
- Hospital de Base of São José do Rio Preto, Faculty of Medicine, São José do Rio Preto, SP, Brazil
| | - Paulo R Correa
- Hospital de Base of São José do Rio Preto, Faculty of Medicine, São José do Rio Preto, SP, Brazil
| | | | - Ross Arena
- Department of Physical Therapy, College of Applied Health Sciences, University of Illinois, Chicago, USA
| | - Jonathan Myers
- Cardiology Division, Department of Veterans Affairs (VA) Palo Alto Health Care System, Palo Alto, CA, USA
| | - Ednaldo B Pizzolato
- Computer Department, Federal University of Sao Carlos, Sao Carlos, SP, Brazil
| | - Audrey Borghi-Silva
- Department of Physical Therapy, Federal University of Sao Carlos, Sao Carlos, SP, Brazil
| |
Collapse
|
11
|
Sündermann SH, Dademasch A, Seifert B, Rodriguez Cetina Biefer H, Emmert MY, Walther T, Jacobs S, Mohr FW, Falk V, Starck CT. Frailty is a predictor of short- and mid-term mortality after elective cardiac surgery independently of age. Interact Cardiovasc Thorac Surg 2014; 18:580-5. [DOI: 10.1093/icvts/ivu006] [Citation(s) in RCA: 91] [Impact Index Per Article: 9.1] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/19/2023] Open
|
12
|
Mortality predicted accuracy for hepatocellular carcinoma patients with hepatic resection using artificial neural network. ScientificWorldJournal 2013; 2013:201976. [PMID: 23737707 PMCID: PMC3659648 DOI: 10.1155/2013/201976] [Citation(s) in RCA: 29] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/03/2013] [Accepted: 04/03/2013] [Indexed: 12/15/2022] Open
Abstract
The aim of this present study is firstly to compare significant predictors of mortality for hepatocellular carcinoma (HCC) patients undergoing resection between artificial neural network (ANN) and logistic regression (LR) models and secondly to evaluate the predictive accuracy of ANN and LR in different survival year estimation models. We constructed a prognostic model for 434 patients with 21 potential input variables by Cox regression model. Model performance was measured by numbers of significant predictors and predictive accuracy. The results indicated that ANN had double to triple numbers of significant predictors at 1-, 3-, and 5-year survival models as compared with LR models. Scores of accuracy, sensitivity, specificity, and area under the receiver operating characteristic curve (AUROC) of 1-, 3-, and 5-year survival estimation models using ANN were superior to those of LR in all the training sets and most of the validation sets. The study demonstrated that ANN not only had a great number of predictors of mortality variables but also provided accurate prediction, as compared with conventional methods. It is suggested that physicians consider using data mining methods as supplemental tools for clinical decision-making and prognostic evaluation.
Collapse
|
13
|
Addition of Frailty and Disability to Cardiac Surgery Risk Scores Identifies Elderly Patients at High Risk of Mortality or Major Morbidity. Circ Cardiovasc Qual Outcomes 2012; 5:222-8. [DOI: 10.1161/circoutcomes.111.963157] [Citation(s) in RCA: 301] [Impact Index Per Article: 25.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
Background—
Cardiac surgery risk scores perform poorly in elderly patients, in part because they do not take into account frailty and disability which are critical determinants of health status with advanced age. There is an unmet need to combine established cardiac surgery risk scores with measures of frailty and disability to provide a more complete model for risk prediction in elderly patients undergoing cardiac surgery.
Methods and Results—
This was a prospective, multicenter cohort study of elderly patients (≥70 years) undergoing coronary artery bypass and/or valve surgery in the United States and Canada. Four different frailty scales, 3 disability scales, and 5 cardiac surgery risk scores were measured in all patients. The primary outcome was the STS composite end point of in-hospital postoperative mortality or major morbidity. A total of 152 patients were enrolled, with a mean age of 75.9±4.4 years and 34% women. Depending on the scale used, 20–46% of patients were found to be frail, and 5–76% were found to have at least 1 disability. The most predictive scale in each domain was: 5-meter gait speed ≥6 seconds as a measure of frailty (odds ratio [OR], 2.63; 95% confidence interval [CI], 1.17–5.90), ≥3 impairments in the Nagi scale as a measure of disability (OR, 2.98; 95% CI, 1.35–6.56) and either the Parsonnet score (OR, 1.08; 95% CI, 1.04–1.13) or Society of Thoracic Surgeons Predicted Risk of Mortality or Major Morbidity (STS-PROMM) (OR, 1.05; 95% CI, 1.01–1.09) as a cardiac surgery risk score. Compared with the Parsonnet score or STS-PROMM alone, (area under the curve, 0.68–0.72), addition of frailty and disability provided incremental value and improved model discrimination (area under the curve, 0.73–0.76).
Conclusions—
Clinicians should use an integrative approach combining frailty, disability, and risk scores to better characterize elderly patients referred for cardiac surgery and identify those that are at increased risk.
Collapse
|