1
|
Nikolova-Simons M, Keldermann R, Peters Y, Compagner W, Montenij L, de Jong Y, Bouwman RA. Predictive analytics for cardio-thoracic surgery duration as a stepstone towards data-driven capacity management. NPJ Digit Med 2023; 6:205. [PMID: 37935901 PMCID: PMC10630382 DOI: 10.1038/s41746-023-00938-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/25/2023] [Accepted: 09/29/2023] [Indexed: 11/09/2023] Open
Abstract
Effective capacity management of operation rooms is key to avoid surgery cancellations and prevent long waiting lists that negatively affect clinical and financial outcomes as well as patient and staff satisfaction. This requires optimal surgery scheduling, leveraging essential parameters like surgery duration, post-operative bed type and hospital length-of-stay. Common clinical practice is to use the surgeon's average procedure time of the last N patients as a planned surgery duration for the next patient. A discrepancy between the actual and planned surgery duration may lead to suboptimal surgery schedule. We used deidentified data from 2294 cardio-thoracic surgeries to first calculate the discrepancy of the current model and second to develop new predictive models based on linear regression, random forest, and extreme gradient boosting. The new ensamble models reduced the RMSE for elective and acute surgeries by 19% (0.99 vs 0.80, p = 0.002) and 52% (1.87 vs 0.89, p < 0.001), respectively. Also, the elective and acute surgeries "behind schedule" were reduced by 28% (60% vs. 32%, p < 0.001) and 9% (37% vs. 28%, p = 0.003), respectively. These improvements were fueled by the patient and surgery features added to the models. Surgery planners can benefit from these predictive models as a patient flow AI decision support tool to optimize OR utilization.
Collapse
Affiliation(s)
| | | | - Yvon Peters
- Philips Research, Eindhoven, the Netherlands
| | | | | | | | - R Arthur Bouwman
- Catharina Hospital, Eindhoven, the Netherlands
- Eindhoven University of Technology, Department of Electrical Engineering, Eindhoven, the Netherlands
| |
Collapse
|
2
|
Xavier J, Seringa J, Pinto FJ, Magalhães T. Decision-making support systems on extended hospital length of stay: Validation and recalibration of a model for patients with AMI. Front Med (Lausanne) 2023; 10:907310. [PMID: 36844231 PMCID: PMC9946108 DOI: 10.3389/fmed.2023.907310] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/29/2022] [Accepted: 01/23/2023] [Indexed: 02/11/2023] Open
Abstract
Background Cardiovascular diseases are still a significant cause of death and hospitalization. In 2019, circulatory diseases were responsible for 29.9% of deaths in Portugal. These diseases have a significant impact on the hospital length of stay. Length of stay predictive models is an efficient way to aid decision-making in health. This study aimed to validate a predictive model on the extended length of stay in patients with acute myocardial infarction at the time of admission. Methods An analysis was conducted to test and recalibrate a previously developed model in the prediction of prolonged length of stay, for a new set of population. The study was conducted based on administrative and laboratory data of patients admitted for acute myocardial infarction events from a public hospital in Portugal from 2013 to 2015. Results Comparable performance measures were observed upon the validation and recalibration of the predictive model of extended length of stay. Comorbidities such as shock, diabetes with complications, dysrhythmia, pulmonary edema, and respiratory infections were the common variables found between the previous model and the validated and recalibrated model for acute myocardial infarction. Conclusion Predictive models for the extended length of stay can be applied in clinical practice since they are recalibrated and modeled to the relevant population characteristics.
Collapse
Affiliation(s)
- Joana Xavier
- NOVA National School of Public Health, Nova University Lisbon, Lisbon, Portugal
| | - Joana Seringa
- NOVA National School of Public Health, Nova University Lisbon, Lisbon, Portugal
| | - Fausto José Pinto
- Serviço de Cardiologia do Centro Hospitalar de Lisboa Norte, Faculdade de Medicina, Universidade de Lisboa, Lisbon, Portugal
| | - Teresa Magalhães
- NOVA National School of Public Health, Nova University Lisbon, Lisbon, Portugal,Public Health Research Centre, Comprehensive Health Research Center, CHRC, Nova University Lisbon, Lisbon, Portugal,*Correspondence: Teresa Magalhães ✉
| |
Collapse
|
3
|
Multilayer dynamic ensemble model for intensive care unit mortality prediction of neonate patients. J Biomed Inform 2022; 135:104216. [DOI: 10.1016/j.jbi.2022.104216] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/24/2021] [Revised: 09/25/2022] [Accepted: 09/28/2022] [Indexed: 12/26/2022]
|
4
|
Risk Management In Intensive Care Units With Artificial Intelligence Technologies: Systematic Review of Prediction Models Using Electronic Health Records. JOURNAL OF BASIC AND CLINICAL HEALTH SCIENCES 2022. [DOI: 10.30621/jbachs.993798] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/18/2022]
Abstract
Background and aim: Clinical risk assessments should be made to protect patients from negative outcomes, and the definition, frequency and severity of the risk should be determined. The information contained in the electronic health records (EHRs) can use in different areas such as risk prediction, estimation of treatment effect ect. Many prediction models using artificial intelligence (AI) technologies that can be used in risk assessment have been developed. The aim of this study is to bring together the researches on prediction models developed with AI technologies using the EHRs of patients hospitalized in the intensive care unit (ICU) and to evaluate them in terms of risk management in healthcare.
Methods: The study restricted the search to the Web of Science, Pubmed, Science Direct, and Medline databases to retrieve research articles published in English in 2010 and after. Studies with a prediction model using data obtained from EHRs in the ICU are included. The study focused solely on research conducted in ICU to predict a health condition that poses a significant risk to patient safety using artificial intellegence (AI) technologies.
Results: Recognized prediction subcategories were mortality (n=6), sepsis (n=4), pressure ulcer (n=4), acute kidney injury (n=3), and other areas (n=10). It has been found that EHR-based prediction models are good risk management and decision support tools and adoption of such models in ICUs may reduce the prevalence of adverse conditions.
Conclusions: The article results remarks that developed models was found to have higher performance and better selectivity than previously developed risk models, so they are better at predicting risks and serious adverse events in ICU. It is recommended to use AI based prediction models developed using EHRs in risk management studies. Future work is still needed to researches to predict different health conditions risks.
Collapse
|
5
|
Rellum SR, Schuurmans J, van der Ven WH, Eberl S, Driessen AHG, Vlaar APJ, Veelo DP. Machine learning methods for perioperative anesthetic management in cardiac surgery patients: a scoping review. J Thorac Dis 2022; 13:6976-6993. [PMID: 35070381 PMCID: PMC8743411 DOI: 10.21037/jtd-21-765] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2021] [Accepted: 08/27/2021] [Indexed: 12/27/2022]
Abstract
Background Machine learning (ML) is developing fast with promising prospects within medicine and already has several applications in perioperative care. We conducted a scoping review to examine the extent and potential limitations of ML implementation in perioperative anesthetic care, specifically in cardiac surgery patients. Methods We mapped the current literature by searching three databases: MEDLINE (Ovid), EMBASE (Ovid), and Cochrane Library. Articles were eligible if they reported on perioperative ML use in the field of cardiac surgery with relevance to anesthetic practices. Data on the applicability of ML and comparability to conventional statistical methods were extracted. Results Forty-six articles on ML relevant to the work of the anesthesiologist in cardiac surgery were identified. Three main categories emerged: (I) event and risk prediction, (II) hemodynamic monitoring, and (III) automation of echocardiography. Prediction models based on ML tend to behave similarly to conventional statistical methods. Using dynamic hemodynamic or ultrasound data in ML models, however, shifts the potential to promising results. Conclusions ML in cardiac surgery is increasingly used in perioperative anesthetic management. The majority is used for prediction purposes similar to conventional clinical scores. Remarkable ML model performances are achieved when using real-time dynamic parameters. However, beneficial clinical outcomes of ML integration have yet to be determined. Nonetheless, the first steps introducing ML in perioperative anesthetic care for cardiac surgery have been taken.
Collapse
Affiliation(s)
- Santino R Rellum
- Department of Anesthesiology, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands.,Department of Intensive Care, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| | - Jaap Schuurmans
- Department of Anesthesiology, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands.,Department of Intensive Care, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| | - Ward H van der Ven
- Department of Anesthesiology, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands.,Department of Intensive Care, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| | - Susanne Eberl
- Department of Anesthesiology, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| | - Antoine H G Driessen
- Department of Cardiothoracic Surgery, Heart Center, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| | - Alexander P J Vlaar
- Department of Intensive Care, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| | - Denise P Veelo
- Department of Anesthesiology, Amsterdam UMC, Location AMC, Amsterdam, The Netherlands
| |
Collapse
|
6
|
Zhang X, Yan C, Malin BA, Patel MB, Chen Y. Predicting next-day discharge via electronic health record access logs. J Am Med Inform Assoc 2021; 28:2670-2680. [PMID: 34592753 DOI: 10.1093/jamia/ocab211] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/18/2021] [Revised: 08/21/2021] [Accepted: 09/15/2021] [Indexed: 02/06/2023] Open
Abstract
OBJECTIVE Hospital capacity management depends on accurate real-time estimates of hospital-wide discharges. Estimation by a clinician requires an excessively large amount of effort and, even when attempted, accuracy in forecasting next-day patient-level discharge is poor. This study aims to support next-day discharge predictions with machine learning by incorporating electronic health record (EHR) audit log data, a resource that captures EHR users' granular interactions with patients' records by communicating various semantics and has been neglected in outcome predictions. MATERIALS AND METHODS This study focused on the EHR data for all adults admitted to Vanderbilt University Medical Center in 2019. We learned multiple advanced models to assess the value that EHR audit log data adds to the daily prediction of discharge likelihood within 24 h and to compare different representation strategies. We applied Shapley additive explanations to identify the most influential types of user-EHR interactions for discharge prediction. RESULTS The data include 26 283 inpatient stays, 133 398 patient-day observations, and 819 types of user-EHR interactions. The model using the count of each type of interaction in the recent 24 h and other commonly used features, including demographics and admission diagnoses, achieved the highest area under the receiver operating characteristics (AUROC) curve of 0.921 (95% CI: 0.919-0.923). By contrast, the model lacking user-EHR interactions achieved a worse AUROC of 0.862 (0.860-0.865). In addition, 10 of the 20 (50%) most influential factors were user-EHR interaction features. CONCLUSION EHR audit log data contain rich information such that it can improve hospital-wide discharge predictions.
Collapse
Affiliation(s)
- Xinmeng Zhang
- Department of Computer Science, Vanderbilt University, Nashville, Tennessee, USA
| | - Chao Yan
- Department of Computer Science, Vanderbilt University, Nashville, Tennessee, USA
| | - Bradley A Malin
- Department of Computer Science, Vanderbilt University, Nashville, Tennessee, USA.,Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA.,Department of Biostatistics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| | - Mayur B Patel
- Section of Surgical Sciences, Departments of Surgery & Neurosurgery, Division of Trauma, Surgical Critical Care, and Emergency General Surgery, Nashville, Tennessee, USA.,Geriatric Research and Education Clinical Center, Surgical Services, Veteran Affairs Tennessee Valley Healthcare System, Nashville, Tennessee, USA
| | - You Chen
- Department of Computer Science, Vanderbilt University, Nashville, Tennessee, USA.,Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, Tennessee, USA
| |
Collapse
|
7
|
Nguyen D, Ngo B, vanSonnenberg E. AI in the Intensive Care Unit: Up-to-Date Review. J Intensive Care Med 2020; 36:1115-1123. [PMID: 32985324 DOI: 10.1177/0885066620956620] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
AI is the latest technologic trend that likely will have a huge impact in medicine. AI's potential lies in its ability to process large volumes of data and perform complex pattern analyses. The ICU is an area of medicine that is particularly conducive to AI applications. Much AI ICU research currently is focused on improving high volumes of data on high-risk patients and making clinical workflow more efficient. Emerging topics of AI medicine in the ICU include AI sensors, sepsis prediction, AI in the NICU or SICU, and the legal role of AI in medicine. This review will cover the current applications of AI medicine in the ICU, potential pitfalls, and other AI medicine-related topics relevant for the ICU.
Collapse
Affiliation(s)
- Diep Nguyen
- University of Arizona College of Medicine Phoenix, AZ, USA
| | - Brandon Ngo
- University of Arizona College of Medicine Phoenix, AZ, USA
| | | |
Collapse
|
8
|
Data-driven ICU management: Using Big Data and algorithms to improve outcomes. J Crit Care 2020; 60:300-304. [PMID: 32977139 DOI: 10.1016/j.jcrc.2020.09.002] [Citation(s) in RCA: 15] [Impact Index Per Article: 3.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/11/2020] [Revised: 08/17/2020] [Accepted: 09/02/2020] [Indexed: 12/13/2022]
Abstract
The digitalization of the Intensive Care Unit (ICU) led to an increasing amount of clinical data being collected at the bedside. The term "Big Data" can be used to refer to the analysis of these datasets that collect enormous amount of data of different origin and format. Complexity and variety define the value of Big Data. In fact, the retrospective analysis of these datasets allows to generate new knowledge, with consequent potential improvements in the clinical practice. Despite the promising start of Big Data analysis in medical research, which has seen a rising number of peer-reviewed articles, very limited applications have been used in ICU clinical practice. A close future effort should be done to validate the knowledge extracted from clinical Big Data and implement it in the clinic. In this article, we provide an introduction to Big Data in the ICU, from data collection and data analysis, to the main successful examples of prognostic, predictive and classification models based on ICU data. In addition, we focus on the main challenges that these models face to reach the bedside and effectively improve ICU care.
Collapse
|
9
|
Tang L, Li K, Wu CJJ. Thirty-day readmission, length of stay and self-management behaviour among patients with acute coronary syndrome and type 2 diabetes mellitus: A scoping review. J Clin Nurs 2019; 29:320-329. [PMID: 31698508 DOI: 10.1111/jocn.15087] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/31/2019] [Revised: 09/16/2019] [Accepted: 10/20/2019] [Indexed: 01/22/2023]
Abstract
AIMS AND OBJECTIVES To summarise the current evidence on comorbid type 2 diabetes mellitus (T2DM) related to 30-day readmission and hospital length of stay (LOS) among patients with acute coronary syndrome (ACS) and evidence on the effectiveness of self-management programmes for patients with both conditions. BACKGROUND Acute coronary syndrome and T2DM remain two major diseases leading to serious consequences. Thirty-day readmission and LOS were considered indicators of the quality of care, with the understanding that the potential significant effects of these outcomes could be varied. DESIGN This scoping review followed the methodology described by Arksey and O'Malley. METHODS Five databases including PubMed, Embase, Cochrane Library, Web of Science and CINAHL were searched, and a total of 20 articles involving 913,807 patients were included. Results were reported in accordance with PRISMA-ScR guidelines. RESULTS The results indicated that patients with both ACS and T2DM have prolonged LOS and increased 30-day readmission rates. The findings supported that improvements in patient self-management behaviour for optimal health outcomes were partially successful by effective self-management programmes; however, few articles on intervention programmes specifically designed for patients with two conditions were found. CONCLUSION Prolonged LOS and increased 30-day readmission rates are found among patients with ACS and T2DM. Based on few pilot studies building on each other, the effectiveness of self-management programmes in promoting self-care behaviour, self-efficacy and knowledge for patients with ACS and T2DM cannot be concluded. RELEVANCE TO CLINICAL PRACTICE Findings from this review provide valuable information on and a better understanding of readmissions and LOS among patients with ACS and T2DM for healthcare providers. Future developments and implementations of effective self-management programmes should target patients with dual diagnoses to improve health behaviour and reduce readmission and LOS.
Collapse
Affiliation(s)
- Liya Tang
- School of Nursing, Jilin University, Changchun, China
| | - Kun Li
- School of Nursing, Jilin University, Changchun, China
| | - Chiung-Jung Jo Wu
- School of Nursing, Midwifery and Paramedicine, University of the Sunshine Coast (USC), Sippy Downs, QLD, Australia.,Royal Brisbane and Women's Hospital (RBWH), Brisbane, QLD, Australia.,Mater Medical Research Institute-University of Queensland (MMRI-UQ), Brisbane, QLD, Australia
| |
Collapse
|
10
|
Pakbin A, Rafi P, Hurley N, Schulz W, Harlan Krumholz M, Bobak Mortazavi J. Prediction of ICU Readmissions Using Data at Patient Discharge. ANNUAL INTERNATIONAL CONFERENCE OF THE IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. IEEE ENGINEERING IN MEDICINE AND BIOLOGY SOCIETY. ANNUAL INTERNATIONAL CONFERENCE 2019; 2018:4932-4935. [PMID: 30441449 DOI: 10.1109/embc.2018.8513181] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Unplanned readmissions to ICU contribute to high health care costs and poor patient outcomes. 6-7% of all ICU cases see a readmission within 72 hours. Machine learning models on electronic health record data can help identify these cases, providing more information about short and long-term risks to clinicians at the time of ICU discharge. While time-toevent techniques have been used in clinical care, models that identify risks over time using higher-dimensional, non-linear machine learning models need to be developed to present changes in risk with non-linear techniques. This work identifies risks of ICU readmissions at 24 hours, 72 hours, 7 days, 30 days, and bounceback readmissions in the same hospital admission with an AUROC for 72 hours of 0.76 and for bounceback of 0.84.
Collapse
|
11
|
Shillan D, Sterne JAC, Champneys A, Gibbison B. Use of machine learning to analyse routinely collected intensive care unit data: a systematic review. CRITICAL CARE : THE OFFICIAL JOURNAL OF THE CRITICAL CARE FORUM 2019; 23:284. [PMID: 31439010 PMCID: PMC6704673 DOI: 10.1186/s13054-019-2564-9] [Citation(s) in RCA: 97] [Impact Index Per Article: 19.4] [Reference Citation Analysis] [Abstract] [Key Words] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 06/12/2019] [Accepted: 08/09/2019] [Indexed: 01/30/2023]
Abstract
BACKGROUND Intensive care units (ICUs) face financial, bed management, and staffing constraints. Detailed data covering all aspects of patients' journeys into and through intensive care are now collected and stored in electronic health records: machine learning has been used to analyse such data in order to provide decision support to clinicians. METHODS Systematic review of the applications of machine learning to routinely collected ICU data. Web of Science and MEDLINE databases were searched to identify candidate articles: those on image processing were excluded. The study aim, the type of machine learning used, the size of dataset analysed, whether and how the model was validated, and measures of predictive accuracy were extracted. RESULTS Of 2450 papers identified, 258 fulfilled eligibility criteria. The most common study aims were predicting complications (77 papers [29.8% of studies]), predicting mortality (70 [27.1%]), improving prognostic models (43 [16.7%]), and classifying sub-populations (29 [11.2%]). Median sample size was 488 (IQR 108-4099): 41 studies analysed data on > 10,000 patients. Analyses focused on 169 (65.5%) papers that used machine learning to predict complications, mortality, length of stay, or improvement of health. Predictions were validated in 161 (95.2%) of these studies: the area under the ROC curve (AUC) was reported by 97 (60.2%) but only 10 (6.2%) validated predictions using independent data. The median AUC was 0.83 in studies of 1000-10,000 patients, rising to 0.94 in studies of > 100,000 patients. The most common machine learning methods were neural networks (72 studies [42.6%]), support vector machines (40 [23.7%]), and classification/decision trees (34 [20.1%]). Since 2015 (125 studies [48.4%]), the most common methods were support vector machines (37 studies [29.6%]) and random forests (29 [23.2%]). CONCLUSIONS The rate of publication of studies using machine learning to analyse routinely collected ICU data is increasing rapidly. The sample sizes used in many published studies are too small to exploit the potential of these methods. Methodological and reporting guidelines are needed, particularly with regard to the choice of method and validation of predictions, to increase confidence in reported findings and aid in translating findings towards routine use in clinical practice.
Collapse
Affiliation(s)
- Duncan Shillan
- NIHR Bristol Biomedical Research Centre, University of Bristol, Bristol, UK.,Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Jonathan A C Sterne
- NIHR Bristol Biomedical Research Centre, University of Bristol, Bristol, UK.,Population Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK
| | - Alan Champneys
- Department of Engineering Mathematics, University of Bristol, Bristol, UK
| | - Ben Gibbison
- NIHR Bristol Biomedical Research Centre, University of Bristol, Bristol, UK. .,Translational Health Sciences, Bristol Medical School, University of Bristol, Bristol, UK. .,Department of Anaesthesia, Bristol Royal Infirmary, Level 7 Queens Building, Upper Maudlin St, Bristol, BS2 8HW, UK.
| |
Collapse
|
12
|
Sai A, Vivas-Valencia C, Imperiale TF, Kong N. Multiobjective Calibration of Disease Simulation Models Using Gaussian Processes. Med Decis Making 2019; 39:540-552. [PMID: 31375053 DOI: 10.1177/0272989x19862560] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/12/2022]
Abstract
Background. Developing efficient procedures of model calibration, which entails matching model predictions to observed outcomes, has gained increasing attention. With faithful but complex simulation models established for cancer diseases, key parameters of cancer natural history can be investigated for possible fits, which can subsequently inform optimal prevention and treatment strategies. When multiple calibration targets exist, one approach to identifying optimal parameters relies on the Pareto frontier. However, computational burdens associated with higher-dimensional parameter spaces require a metamodeling approach. The goal of this work is to explore multiobjective calibration using Gaussian process regression (GPR) with an eye toward how multiple goodness-of-fit (GOF) criteria identify Pareto-optimal parameters. Methods. We applied GPR, a metamodeling technique, to estimate colorectal cancer (CRC)-related prevalence rates simulated from a microsimulation model of CRC natural history, known as the Colon Modeling Open Source Tool (CMOST). We embedded GPR metamodels within a Pareto optimization framework to identify best-fitting parameters for age-, adenoma-, and adenoma staging-dependent transition probabilities and risk factors. The Pareto frontier approach is demonstrated using genetic algorithms with both sum-of-squared errors (SSEs) and Poisson deviance GOF criteria. Results. The GPR metamodel is able to approximate CMOST outputs accurately on 2 separate parameter sets. Both GOF criteria are able to identify different best-fitting parameter sets on the Pareto frontier. The SSE criterion emphasizes the importance of age-specific adenoma progression parameters, while the Poisson criterion prioritizes adenoma-specific progression parameters. Conclusion. Different GOF criteria assert different components of the CRC natural history. The combination of multiobjective optimization and nonparametric regression, along with diverse GOF criteria, can advance the calibration process by identifying optimal regions of the underlying parameter landscape.
Collapse
Affiliation(s)
- Aditya Sai
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN, USA
| | | | - Thomas F Imperiale
- Indiana University School of Medicine, Indiana University, Indianapolis, IN, USA.,Richard A. Roudebush VA Medical Center, Indianapolis, IN, USA.,Regenstrief Institute, Indianapolis, IN, USA
| | - Nan Kong
- Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN, USA
| |
Collapse
|
13
|
Schetinin V, Jakaite L, Krzanowski W. Bayesian averaging over decision tree models: An application for estimating uncertainty in trauma severity scoring. Int J Med Inform 2018; 112:6-14. [PMID: 29500023 DOI: 10.1016/j.ijmedinf.2018.01.009] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/21/2017] [Revised: 01/05/2018] [Accepted: 01/10/2018] [Indexed: 12/01/2022]
Abstract
INTRODUCTION For making reliable decisions, practitioners need to estimate uncertainties that exist in data and decision models. In this paper we analyse uncertainties of predicting survival probability for patients in trauma care. The existing prediction methodology employs logistic regression modelling of Trauma and Injury Severity Score (TRISS), which is based on theoretical assumptions. These assumptions limit the capability of TRISS methodology to provide accurate and reliable predictions. METHODS We adopt the methodology of Bayesian model averaging and show how this methodology can be applied to decision trees in order to provide practitioners with new insights into the uncertainty. The proposed method has been validated on a large set of 447,176 cases registered in the US National Trauma Data Bank in terms of discrimination ability evaluated with receiver operating characteristic (ROC) and precision-recall (PRC) curves. RESULTS Areas under curves were improved for ROC from 0.951 to 0.956 (p = 3.89 × 10-18) and for PRC from 0.564 to 0.605 (p = 3.89 × 10-18). The new model has significantly better calibration in terms of the Hosmer-Lemeshow Hˆ statistic, showing an improvement from 223.14 (the standard method) to 11.59 (p = 2.31 × 10-18). CONCLUSION The proposed Bayesian method is capable of improving the accuracy and reliability of survival prediction. The new method has been made available for evaluation purposes as a web application.
Collapse
Affiliation(s)
| | - L Jakaite
- University of Bedfordshire, United Kingdom
| | | |
Collapse
|
14
|
Howitt SH, Grant SW, Riding DM, Malagon I, McCollum CN. Risk Models That Use Postoperative Patient Monitoring Data to Predict Outcomes in Adult Cardiac Surgery: A Systematic Review. J Cardiothorac Vasc Anesth 2017; 31:1865-1877. [DOI: 10.1053/j.jvca.2016.10.002] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/30/2016] [Indexed: 11/11/2022]
|
15
|
Desautels T, Das R, Calvert J, Trivedi M, Summers C, Wales DJ, Ercole A. Prediction of early unplanned intensive care unit readmission in a UK tertiary care hospital: a cross-sectional machine learning approach. BMJ Open 2017; 7:e017199. [PMID: 28918412 PMCID: PMC5640090 DOI: 10.1136/bmjopen-2017-017199] [Citation(s) in RCA: 52] [Impact Index Per Article: 7.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 01/08/2023] Open
Abstract
OBJECTIVES Unplanned readmissions to the intensive care unit (ICU) are highly undesirable, increasing variance in care, making resource planning difficult and potentially increasing length of stay and mortality in some settings. Identifying patients who are likely to suffer unplanned ICU readmission could reduce the frequency of this adverse event. SETTING A single academic, tertiary care hospital in the UK. PARTICIPANTS A set of 3326 ICU episodes collected between October 2014 and August 2016. All records were of patients who visited an ICU at some point during their stay. We excluded patients who were ≤16 years of age; visited ICUs other than the general and neurosciences ICU; were missing crucial electronic patient record measurements; or had indeterminate ICU discharge outcomes or very early or extremely late discharge times. After exclusion, 2018 outcome-labelled episodes remained. PRIMARY AND SECONDARY OUTCOME MEASURES Area under the receiver operating characteristic curve (AUROC) for prediction of unplanned ICU readmission or in-hospital death within 48 hours of first ICU discharge. RESULTS In 10-fold cross-validation, an ensemble predictor was trained on data from both the target hospital and the Medical Information Mart for Intensive Care (MIMIC-III) database and tested on the target hospital's data. This predictor discriminated between patients with the unplanned ICU readmission or death outcome and those without this outcome, attaining mean AUROC of 0.7095 (SE 0.0260), superior to the purpose-built Stability and Workload Index for Transfer (SWIFT) score (AUROC=0.6082, SE 0.0249; p=0.014, pairwise t-test). CONCLUSIONS Despite the inherent difficulties, we demonstrate that a novel machine learning algorithm based on transfer learning could achieve good discrimination, over and above that of the treating clinicians or the value added by the SWIFT score. Accurate prediction of unplanned readmission could be used to target resources more efficiently.
Collapse
Affiliation(s)
| | | | | | - Monica Trivedi
- John V Farman Intensive Care Unit, Addenbrooke's Hospital, Cambridge, UK
| | - Charlotte Summers
- Division of Anaesthesia, University of Cambridge, Addenbrooke's Hospital, Cambridge, UK
| | - David J Wales
- Department of Chemistry, University of Cambridge, Cambridge, UK
| | - Ari Ercole
- Division of Anaesthesia, University of Cambridge, Addenbrooke's Hospital, Cambridge, UK
| |
Collapse
|
16
|
Random Survival Forests for Predicting the Bed Occupancy in the Intensive Care Unit. COMPUTATIONAL AND MATHEMATICAL METHODS IN MEDICINE 2016; 2016:7087053. [PMID: 27818706 PMCID: PMC5081505 DOI: 10.1155/2016/7087053] [Citation(s) in RCA: 14] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 05/27/2016] [Revised: 09/08/2016] [Accepted: 09/22/2016] [Indexed: 11/21/2022]
Abstract
Predicting the bed occupancy of an intensive care unit (ICU) is a daunting task. The uncertainty associated with the prognosis of critically ill patients and the random arrival of new patients can lead to capacity problems and the need for reactive measures. In this paper, we work towards a predictive model based on Random Survival Forests which can assist physicians in estimating the bed occupancy. As input data, we make use of the Sequential Organ Failure Assessment (SOFA) score collected and calculated from 4098 patients at two ICU units of Ghent University Hospital over a time period of four years. We compare the performance of our system with a baseline performance and a standard Random Forest regression approach. Our results indicate that Random Survival Forests can effectively be used to assist in the occupancy prediction problem. Furthermore, we show that a group based approach, such as Random Survival Forests, performs better compared to a setting in which the length of stay of a patient is individually assessed.
Collapse
|
17
|
Ghosh S, Feng M, Nguyen H, Li J. Hypotension Risk Prediction via Sequential Contrast Patterns of ICU Blood Pressure. IEEE J Biomed Health Inform 2016; 20:1416-1426. [PMID: 26168449 PMCID: PMC5219944 DOI: 10.1109/jbhi.2015.2453478] [Citation(s) in RCA: 27] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/22/2022]
Abstract
Acute hypotension is a significant risk factor for in-hospital mortality at intensive care units. Prolonged hypotension can cause tissue hypoperfusion, leading to cellular dysfunction and severe injuries to multiple organs. Prompt medical interventions are thus extremely important for dealing with acute hypotensive episodes (AHE). Population level prognostic scoring systems for risk stratification of patients are suboptimal in such scenarios. However, the design of an efficient risk prediction system can significantly help in the identification of critical care patients, who are at risk of developing an AHE within a future time span. Toward this objective, a pattern mining algorithm is employed to extract informative sequential contrast patterns from hemodynamic data, for the prediction of hypotensive episodes. The hypotensive and normotensive patient groups are extracted from the MIMIC-II critical care research database, following an appropriate clinical inclusion criteria. The proposed method consists of a data preprocessing step to convert the blood pressure time series into symbolic sequences, using a symbolic aggregate approximation algorithm. Then, distinguishing subsequences are identified using the sequential contrast mining algorithm. These subsequences are used to predict the occurrence of an AHE in a future time window separated by a user-defined gap interval. Results indicate that the method performs well in terms of the prediction performance as well as in the generation of sequential patterns of clinical significance. Hence, the novelty of sequential patterns is in their usefulness as potential physiological biomarkers for building optimal patient risk stratification systems and for further clinical investigation of interesting patterns in critical care patients.
Collapse
|
18
|
Goldstein BA, Navar AM, Pencina MJ, Ioannidis JPA. Opportunities and challenges in developing risk prediction models with electronic health records data: a systematic review. J Am Med Inform Assoc 2016; 24:198-208. [PMID: 27189013 DOI: 10.1093/jamia/ocw042] [Citation(s) in RCA: 424] [Impact Index Per Article: 53.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/27/2015] [Revised: 01/25/2016] [Accepted: 02/20/2016] [Indexed: 12/23/2022] Open
Abstract
OBJECTIVE Electronic health records (EHRs) are an increasingly common data source for clinical risk prediction, presenting both unique analytic opportunities and challenges. We sought to evaluate the current state of EHR based risk prediction modeling through a systematic review of clinical prediction studies using EHR data. METHODS We searched PubMed for articles that reported on the use of an EHR to develop a risk prediction model from 2009 to 2014. Articles were extracted by two reviewers, and we abstracted information on study design, use of EHR data, model building, and performance from each publication and supplementary documentation. RESULTS We identified 107 articles from 15 different countries. Studies were generally very large (median sample size = 26 100) and utilized a diverse array of predictors. Most used validation techniques (n = 94 of 107) and reported model coefficients for reproducibility (n = 83). However, studies did not fully leverage the breadth of EHR data, as they uncommonly used longitudinal information (n = 37) and employed relatively few predictor variables (median = 27 variables). Less than half of the studies were multicenter (n = 50) and only 26 performed validation across sites. Many studies did not fully address biases of EHR data such as missing data or loss to follow-up. Average c-statistics for different outcomes were: mortality (0.84), clinical prediction (0.83), hospitalization (0.71), and service utilization (0.71). CONCLUSIONS EHR data present both opportunities and challenges for clinical risk prediction. There is room for improvement in designing such studies.
Collapse
Affiliation(s)
- Benjamin A Goldstein
- Department of Biostatistics & Bioinformatics, Duke University, Durham, NC 27710, USA .,Center for Predictive Medicine, Duke Clinical Research Institute, Duke University, Durham, NC 27710, USA
| | - Ann Marie Navar
- Center for Predictive Medicine, Duke Clinical Research Institute, Duke University, Durham, NC 27710, USA.,Division of Cardiology at Duke University Medical Center, Duhram, NC 27710, USA
| | - Michael J Pencina
- Department of Biostatistics & Bioinformatics, Duke University, Durham, NC 27710, USA.,Center for Predictive Medicine, Duke Clinical Research Institute, Duke University, Durham, NC 27710, USA
| | - John P A Ioannidis
- Department of Medicine, Stanford University, Palo Alto, CA 94305, USA.,Department of Health Research and Policy, and Statistics and Meta-Research Innovation Center at Stanford, Stanford University, Palo Alto, CA 94305, USA
| |
Collapse
|
19
|
The Predictive Factors on Extended Hospital Length of Stay in Patients with AMI: Laboratory and Administrative Data. J Med Syst 2015; 40:2. [DOI: 10.1007/s10916-015-0363-7] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/26/2015] [Accepted: 09/30/2015] [Indexed: 11/25/2022]
|
20
|
Dexter F, Wachtel RE. Strategies for Net Cost Reductions with the Expanded Role and Expertise of Anesthesiologists in the Perioperative Surgical Home. Anesth Analg 2014; 118:1062-71. [DOI: 10.1213/ane.0000000000000173] [Citation(s) in RCA: 73] [Impact Index Per Article: 7.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/05/2022]
|
21
|
Wu YC, Lee WC. Alternative performance measures for prediction models. PLoS One 2014; 9:e91249. [PMID: 24608868 PMCID: PMC3946724 DOI: 10.1371/journal.pone.0091249] [Citation(s) in RCA: 27] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/27/2013] [Accepted: 02/10/2014] [Indexed: 11/19/2022] Open
Abstract
As a performance measure for a prediction model, the area under the receiver operating characteristic curve (AUC) is insensitive to the addition of strong markers. A number of measures sensitive to performance change have recently been proposed; however, these relative-performance measures may lead to self-contradictory conclusions. This paper examines alternative performance measures for prediction models: the Lorenz curve-based Gini and Pietra indices, and a standardized version of the Brier score, the scaled Brier. Computer simulations are performed in order to study the sensitivity of these measures to performance change when a new marker is added to a baseline model. When the discrimination power of the added marker is concentrated in the gray zone of the baseline model, the AUC and the Gini show minimal performance improvements. The Pietra and the scaled Brier show more significant improvements in the same situation, comparatively. The Pietra and the scaled Brier indices are therefore recommended for prediction model performance measurement, in light of their ease of interpretation, clinical relevance and sensitivity to gray-zone resolving markers.
Collapse
Affiliation(s)
- Yun-Chun Wu
- Institute of Epidemiology and Preventive Medicine, College of Public Health, National Taiwan University, Taipei, Taiwan
| | - Wen-Chung Lee
- Institute of Epidemiology and Preventive Medicine, College of Public Health, National Taiwan University, Taipei, Taiwan
- Research Center for Genes, Environment and Human Health, College of Public Health, National Taiwan University, Taipei, Taiwan
- * E-mail:
| |
Collapse
|
22
|
Novel methods to predict increased intracranial pressure during intensive care and long-term neurologic outcome after traumatic brain injury: development and validation in a multicenter dataset. Crit Care Med 2013; 41:554-64. [PMID: 23263587 DOI: 10.1097/ccm.0b013e3182742d0a] [Citation(s) in RCA: 70] [Impact Index Per Article: 6.4] [Reference Citation Analysis] [Abstract] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/25/2022]
Abstract
OBJECTIVE Intracranial pressure monitoring is standard of care after severe traumatic brain injury. Episodes of increased intracranial pressure are secondary injuries associated with poor outcome. We developed a model to predict increased intracranial pressure episodes 30 mins in advance, by using the dynamic characteristics of continuous intracranial pressure and mean arterial pressure monitoring. In addition, we hypothesized that performance of current models to predict long-term neurologic outcome could be substantially improved by adding dynamic characteristics of continuous intracranial pressure and mean arterial pressure monitoring during the first 24 hrs in the ICU. DESIGN Prognostic modeling. Noninterventional, observational, retrospective study. SETTING AND PATIENTS The Brain Monitoring with Information Technology dataset consisted of 264 traumatic brain injury patients admitted to 22 neuro-ICUs from 11 European countries. INTERVENTIONS None. MEASUREMENTS Predictive models were built with multivariate logistic regression and Gaussian processes, a machine learning technique. Predictive attributes were Corticosteroid Randomisation After Significant Head Injury-basic and International Mission for Prognosis and Clinical Trial design in TBI-core predictors, together with time-series summary statistics of minute-by-minute mean arterial pressure and intracranial pressure. MAIN RESULTS Increased intracranial pressure episodes could be predicted 30 mins ahead with good calibration (Hosmer-Lemeshow p value 0.12, calibration slope 1.02, calibration-in-the-large -0.02) and discrimination (area under the receiver operating curve = 0.87) on an external validation dataset. Models for prediction of poor neurologic outcome at six months (Glasgow Outcome Score 1-2) based only on static admission data had 0.72 area under the receiver operating curve; adding dynamic information of intracranial pressure and mean arterial pressure during the first 24 hrs increased performance to 0.90. Similarly, prediction of Glasgow Outcome Score 1-3 was improved from 0.68 to 0.87 when including dynamic information. CONCLUSION The dynamic information in continuous mean arterial pressure and intracranial pressure monitoring allows to accurately predict increased intracranial pressure in the neuro-ICU. Adding information of the first 24 hrs of intracranial pressure and mean arterial pressure monitoring to known baseline risk factors allows very accurate prediction of long-term neurologic outcome at 6 months.
Collapse
|
23
|
Abstract
The widespread implementation of computerized medical files in intensive care units (ICUs) over recent years has made available large databases of clinical data for the purpose of developing clinical prediction models. The typical intensive care unit has several information sources from which data is electronically collected as time series of varying time resolutions. We present an overview of research questions studied in the ICU setting that have been addressed through the automatic analysis of these large databases. We focus on automatic learning methods, specifically data mining approaches for predictive modeling based on these time series of clinical data. On the one hand we examine short and medium term predictions, which have as ultimate goal the development of early warning or decision support systems. On the other hand we examine long term outcome prediction models and evaluate their performance with respect to established scoring systems based on static admission and demographic data.
Collapse
|
24
|
Takada M, Sugimoto M, Naito Y, Moon HG, Han W, Noh DY, Kondo M, Kuroi K, Sasano H, Inamoto T, Tomita M, Toi M. Prediction of axillary lymph node metastasis in primary breast cancer patients using a decision tree-based model. BMC Med Inform Decis Mak 2012; 12:54. [PMID: 22695278 PMCID: PMC3407483 DOI: 10.1186/1472-6947-12-54] [Citation(s) in RCA: 32] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2011] [Accepted: 06/13/2012] [Indexed: 01/17/2023] Open
Abstract
Background The aim of this study was to develop a new data-mining model to predict axillary lymph node (AxLN) metastasis in primary breast cancer. To achieve this, we used a decision tree-based prediction method—the alternating decision tree (ADTree). Methods Clinical datasets for primary breast cancer patients who underwent sentinel lymph node biopsy or AxLN dissection without prior treatment were collected from three institutes (institute A, n = 148; institute B, n = 143; institute C, n = 174) and were used for variable selection, model training and external validation, respectively. The models were evaluated using area under the receiver operating characteristics (ROC) curve analysis to discriminate node-positive patients from node-negative patients. Results The ADTree model selected 15 of 24 clinicopathological variables in the variable selection dataset. The resulting area under the ROC curve values were 0.770 [95% confidence interval (CI), 0.689–0.850] for the model training dataset and 0.772 (95% CI: 0.689–0.856) for the validation dataset, demonstrating high accuracy and generalization ability of the model. The bootstrap value of the validation dataset was 0.768 (95% CI: 0.763–0.774). Conclusions Our prediction model showed high accuracy for predicting nodal metastasis in patients with breast cancer using commonly recorded clinical variables. Therefore, our model might help oncologists in the decision-making process for primary breast cancer patients before starting treatment.
Collapse
Affiliation(s)
- Masahiro Takada
- Department of Breast Surgery, Graduate School of Medicine, Kyoto University, Kyoto, Japan
| | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|