1
|
Vagliano I, Dormosh N, Rios M, Luik TT, Buonocore TM, Elbers PWG, Dongelmans DA, Schut MC, Abu-Hanna A. Prognostic models of in-hospital mortality of intensive care patients using neural representation of unstructured text: A systematic review and critical appraisal. J Biomed Inform 2023; 146:104504. [PMID: 37742782 DOI: 10.1016/j.jbi.2023.104504] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/09/2023] [Revised: 08/29/2023] [Accepted: 09/21/2023] [Indexed: 09/26/2023]
Abstract
OBJECTIVE To review and critically appraise published and preprint reports of prognostic models of in-hospital mortality of patients in the intensive-care unit (ICU) based on neural representations (embeddings) of clinical notes. METHODS PubMed and arXiv were searched up to August 1, 2022. At least two reviewers independently selected the studies that developed a prognostic model of in-hospital mortality of intensive-care patients using free-text represented as embeddings and extracted data using the CHARMS checklist. Risk of bias was assessed using PROBAST. Reporting on the model was assessed with the TRIPOD guideline. To assess the machine learning components that were used in the models, we present a new descriptive framework based on different techniques to represent text and provide predictions from text. The study protocol was registered in the PROSPERO database (CRD42022354602). RESULTS Eighteen studies out of 2,825 were included. All studies used the publicly-available MIMIC dataset. Context-independent word embeddings are widely used. Model discrimination was provided by all studies (AUROC 0.75-0.96), but measures of calibration were scarce. Seven studies used both structural clinical variables and notes. Model discrimination improved when adding clinical notes to variables. None of the models was externally validated and often a simple train/test split was used for internal validation. Our critical appraisal demonstrated a high risk of bias in all studies and concerns regarding their applicability in clinical practice. CONCLUSION All studies used a neural architecture for prediction and were based on one publicly available dataset. Clinical notes were reported to improve predictive performance when used in addition to only clinical variables. Most studies had methodological, reporting, and applicability issues. We recommend reporting both model discrimination and calibration, using additional data sources, and using more robust evaluation strategies, including prospective and external validation. Finally, sharing data and code is encouraged to improve study reproducibility.
Collapse
Affiliation(s)
- I Vagliano
- Dept. of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands; Amsterdam Public Health (APH), Amsterdam, the Netherlands.
| | - N Dormosh
- Dept. of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands; Amsterdam Public Health (APH), Amsterdam, the Netherlands
| | - M Rios
- Centre for Translation Studies, University of Vienna, Vienna, Austria. https://twitter.com/zhizhid
| | - T T Luik
- Amsterdam Public Health (APH), Amsterdam, the Netherlands; Dept. of Medical Biology, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
| | - T M Buonocore
- Dept. of Electrical, Computer and Biomedical Engineering, University of Pavia, Pavia, Italy
| | - P W G Elbers
- Amsterdam Public Health (APH), Amsterdam, the Netherlands; Dept. of Intensive Care Medicine, Center for Critical Care Computational Intelligence (C4I), Amsterdam Medical Data Science (AMDS), Amsterdam Institute for Infection and Immunity (AII), Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands. https://twitter.com/zhizhid
| | - D A Dongelmans
- Amsterdam Public Health (APH), Amsterdam, the Netherlands; National Intensive Care Evaluation (NICE) Foundation, Amsterdam, the Netherlands; Dept. of Intensive Care Medicine, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
| | - M C Schut
- Dept. of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands; Amsterdam Public Health (APH), Amsterdam, the Netherlands; Dept. of Clinical Chemistry, Amsterdam UMC, Vrije Universiteit Amsterdam, Amsterdam, the Netherlands
| | - A Abu-Hanna
- Dept. of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands; Amsterdam Public Health (APH), Amsterdam, the Netherlands
| |
Collapse
|
2
|
Bajwa MS, Sohail M, Ali H, Nazir U, Bashir MM. Predicting Thermal Injury Patient Outcomes in a Tertiary-Care Burn Center, Pakistan. J Surg Res 2022; 279:575-585. [DOI: 10.1016/j.jss.2022.06.044] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/21/2021] [Revised: 06/15/2022] [Accepted: 06/17/2022] [Indexed: 12/01/2022]
|
3
|
Theophanous S, Lønne PI, Choudhury A, Berbee M, Dekker A, Dennis K, Dewdney A, Gambacorta MA, Gilbert A, Guren MG, Holloway L, Jadon R, Kochhar R, Mohamed AA, Muirhead R, Parés O, Raszewski L, Roy R, Scarsbrook A, Sebag-Montefiore D, Spezi E, Spindler KLG, van Triest B, Vassiliou V, Malinen E, Wee L, Appelt AL. Development and validation of prognostic models for anal cancer outcomes using distributed learning: protocol for the international multi-centre atomCAT2 study. Diagn Progn Res 2022; 6:14. [PMID: 35922837 PMCID: PMC9351222 DOI: 10.1186/s41512-022-00128-8] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/11/2022] [Accepted: 06/09/2022] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND Anal cancer is a rare cancer with rising incidence. Despite the relatively good outcomes conferred by state-of-the-art chemoradiotherapy, further improving disease control and reducing toxicity has proven challenging. Developing and validating prognostic models using routinely collected data may provide new insights for treatment development and selection. However, due to the rarity of the cancer, it can be difficult to obtain sufficient data, especially from single centres, to develop and validate robust models. Moreover, multi-centre model development is hampered by ethical barriers and data protection regulations that often limit accessibility to patient data. Distributed (or federated) learning allows models to be developed using data from multiple centres without any individual-level patient data leaving the originating centre, therefore preserving patient data privacy. This work builds on the proof-of-concept three-centre atomCAT1 study and describes the protocol for the multi-centre atomCAT2 study, which aims to develop and validate robust prognostic models for three clinically important outcomes in anal cancer following chemoradiotherapy. METHODS This is a retrospective multi-centre cohort study, investigating overall survival, locoregional control and freedom from distant metastasis after primary chemoradiotherapy for anal squamous cell carcinoma. Patient data will be extracted and organised at each participating radiotherapy centre (n = 18). Candidate prognostic factors have been identified through literature review and expert opinion. Summary statistics will be calculated and exchanged between centres prior to modelling. The primary analysis will involve developing and validating Cox proportional hazards models across centres for each outcome through distributed learning. Outcomes at specific timepoints of interest and factor effect estimates will be reported, allowing for outcome prediction for future patients. DISCUSSION The atomCAT2 study will analyse one of the largest available cross-institutional cohorts of patients with anal cancer treated with chemoradiotherapy. The analysis aims to provide information on current international clinical practice outcomes and may aid the personalisation and design of future anal cancer clinical trials through contributing to a better understanding of patient risk stratification.
Collapse
Affiliation(s)
- Stelios Theophanous
- Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK.
| | - Per-Ivar Lønne
- Department of Medical Physics, Oslo University Hospital, Oslo, Norway
| | - Ananya Choudhury
- MAASTRO (Dept of Radiotherapy), GROW School of Oncology and Developmental Biology, Maastricht University and Maastricht University Medical Centre+, P. Debyelaan 25, 6229, Maastricht, Netherlands
| | - Maaike Berbee
- MAASTRO (Dept of Radiotherapy), GROW School of Oncology and Developmental Biology, Maastricht University and Maastricht University Medical Centre+, P. Debyelaan 25, 6229, Maastricht, Netherlands
| | - Andre Dekker
- MAASTRO (Dept of Radiotherapy), GROW School of Oncology and Developmental Biology, Maastricht University and Maastricht University Medical Centre+, P. Debyelaan 25, 6229, Maastricht, Netherlands
| | | | | | | | - Alexandra Gilbert
- Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
| | - Marianne Grønlie Guren
- Department of Oncology, Oslo University Hospital, and Institute of Clinical Medicine, University of Oslo, Oslo, Norway
| | - Lois Holloway
- Ingham Research Institute and Liverpool Hospital, Liverpool, New South Wales, Australia
| | | | | | | | | | | | | | - Rajarshi Roy
- Hull University Teaching Hospitals NHS Trust, Hull, UK
| | - Andrew Scarsbrook
- Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
- Leeds Teaching Hospitals NHS Trust, Leeds, UK
| | | | | | | | - Baukelien van Triest
- The Netherlands Cancer Institute-Antoni van Leeuwenhoek (NKI-AVL), Amsterdam, The Netherlands
| | | | - Eirik Malinen
- Department of Medical Physics, Oslo University Hospital, Oslo, Norway
| | - Leonard Wee
- MAASTRO (Dept of Radiotherapy), GROW School of Oncology and Developmental Biology, Maastricht University and Maastricht University Medical Centre+, P. Debyelaan 25, 6229, Maastricht, Netherlands
| | - Ane L Appelt
- Leeds Institute of Medical Research at St James's, University of Leeds, Leeds, UK
- Leeds Teaching Hospitals NHS Trust, Leeds, UK
| |
Collapse
|
4
|
The GERtality Score: The Development of a Simple Tool to Help Predict in-Hospital Mortality in Geriatric Trauma Patients. J Clin Med 2021; 10:jcm10071362. [PMID: 33806240 PMCID: PMC8037079 DOI: 10.3390/jcm10071362] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/08/2021] [Revised: 03/11/2021] [Accepted: 03/22/2021] [Indexed: 12/19/2022] Open
Abstract
Feasible and predictive scoring systems for severely injured geriatric patients are lacking. Therefore, the aim of this study was to develop a scoring system for the prediction of in-hospital mortality in severely injured geriatric trauma patients. The TraumaRegister DGU® (TR-DGU) was utilized. European geriatric patients (≥65 years) admitted between 2008 and 2017 were included. Relevant patient variables were implemented in the GERtality score. By conducting a receiver operating characteristic (ROC) analysis, a comparison with the Geriatric Trauma Outcome Score (GTOS) and the Revised Injury Severity Classification II (RISC-II) Score was performed. A total of 58,055 geriatric trauma patients (mean age: 77 years) were included. Univariable analysis led to the following variables: age ≥ 80 years, need for packed red blood cells (PRBC) transfusion prior to intensive care unit (ICU), American Society of Anesthesiologists (ASA) score ≥ 3, Glasgow Coma Scale (GCS) ≤ 13, Abbreviated Injury Scale (AIS) in any body region ≥ 4. The maximum GERtality score was 5 points. A mortality rate of 72.4% was calculated in patients with the maximum GERtality score. Mortality rates of 65.1 and 47.5% were encountered in patients with GERtality scores of 4 and 3 points, respectively. The area under the curve (AUC) of the novel GERtality score was 0.803 (GTOS: 0.784; RISC-II: 0.879). The novel GERtality score is a simple and feasible score that enables an adequate prediction of the probability of mortality in polytraumatized geriatric patients by using only five specific parameters.
Collapse
|
5
|
Nazari M, Shiri I, Zaidi H. Radiomics-based machine learning model to predict risk of death within 5-years in clear cell renal cell carcinoma patients. Comput Biol Med 2020; 129:104135. [PMID: 33254045 DOI: 10.1016/j.compbiomed.2020.104135] [Citation(s) in RCA: 39] [Impact Index Per Article: 9.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/02/2020] [Revised: 10/21/2020] [Accepted: 11/11/2020] [Indexed: 12/13/2022]
Abstract
PURPOSE The aim of this study was to develop radiomics-based machine learning models based on extracted radiomic features and clinical information to predict the risk of death within 5 years for prognosis of clear cell renal cell carcinoma (ccRCC) patients. METHODS According to image quality and clinical data availability, we eventually selected 70 ccRCC patients that underwent CT scans. Manual volume-of-interest (VOI) segmentation of each image was performed by an experienced radiologist using the 3D slicer software package. Prior to feature extraction, image pre-processing was performed on CT images to extract different image features, including wavelet, Laplacian of Gaussian, and resampling of the intensity values to 32, 64 and 128 bin levels. Overall, 2544 3D radiomics features were extracted from each VOI for each patient. Minimum Redundancy Maximum Relevance (MRMR) algorithm was used as feature selector. Four classification algorithms were used, including Generalized Linear Model (GLM), Support Vector Machine (SVM), K-nearest Neighbor (KNN) and XGBoost. We used the Bootstrap resampling method to create validation sets. Area under the receiver operating characteristic (ROC) curve (AUROC), accuracy, sensitivity, and specificity were used to assess the performance of the classification models. RESULTS The best single performance among 8 different models was achieved by the XGBoost model using a combination of radiomic features and clinical information (AUROC, accuracy, sensitivity, and specificity with 95% confidence interval were 0.95-0.98, 0.93-0.98, 0.93-0.96 and ~1.0, respectively). CONCLUSIONS We developed a robust radiomics-based classifier that is capable of accurately predicting overall survival of RCC patients for prognosis of ccRCC patients. This signature may help identifying high-risk patients who require additional treatment and follow up regimens.
Collapse
Affiliation(s)
- Mostafa Nazari
- Department of Biomedical Engineering and Medical Physics, Shahid Beheshti University of Medical Sciences, Tehran, Iran
| | - Isaac Shiri
- Division of Nuclear Medicine and Molecular Imaging, Geneva University Hospital, CH-1211, Geneva 4, Switzerland
| | - Habib Zaidi
- Division of Nuclear Medicine and Molecular Imaging, Geneva University Hospital, CH-1211, Geneva 4, Switzerland; Geneva University Neurocenter, Geneva University, Geneva, Switzerland; Department of Nuclear Medicine and Molecular Imaging, University of Groningen, University Medical Center Groningen, Groningen, Netherlands; Department of Nuclear Medicine, University of Southern Denmark, Odense, Denmark.
| |
Collapse
|
6
|
Wang KM, Wang KJ, Makond B. Survivability modelling using Bayesian network for patients with first and secondary primary cancers. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2020; 196:105686. [PMID: 32777652 DOI: 10.1016/j.cmpb.2020.105686] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/30/2020] [Accepted: 07/29/2020] [Indexed: 06/11/2023]
Abstract
BACKGROUND AND OBJECTIVE Multiple primary cancers significantly threat patient survivability. Predicting the survivability of patients with two cancers is challenging because its stochastic pattern relates with numerous variables. METHODS In this study, a Bayesian network (BN) model was proposed to describe the occurrence of two primary cancers and predict the five-year survivability of patients using probabilistic evidence. Eleven types of major primary cancers and contingent occurrences of secondary cancers were investigated. A nationwide two-cancer database involving 7,845 patients in Taiwan was investigated. The BN topology is rigorously examined and imbalanced dataset is processed by the synthetic minority oversampling technique. The proposed BN survivability prognosis model was compared with benchmark approaches. RESULTS The proposed model significantly outperformed the back-propagation neural network, logistic regression, support vector machine, and naïve Bayes in terms of sensitivity, which is a critical performance index for the non-survival group. CONCLUSIONS Using the proposed BN model, one can estimate the posterior probabilities for every query provided appropriate prior evidences. The potential survivability information of patients, treatment effects, and socio-demographics factor effects predicted by the proposed model can help in cancer treatment assessment and cancer development monitoring.
Collapse
Affiliation(s)
- Kung-Min Wang
- Department of Surgery, Shin-Kong Wu Ho-Su Memorial Hospital, Taipei, Taiwan, ROC.
| | - Kung-Jeng Wang
- Department of Industrial Management, National Taiwan University of Science and Technology, Taipei, 106, Taiwan, ROC.
| | - Bunjira Makond
- Faculty of Commerce and Management, Prince of Songkla University, Trang, Thailand
| |
Collapse
|
7
|
Alaka SA, Menon BK, Brobbey A, Williamson T, Goyal M, Demchuk AM, Hill MD, Sajobi TT. Functional Outcome Prediction in Ischemic Stroke: A Comparison of Machine Learning Algorithms and Regression Models. Front Neurol 2020; 11:889. [PMID: 32982920 PMCID: PMC7479334 DOI: 10.3389/fneur.2020.00889] [Citation(s) in RCA: 31] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/06/2020] [Accepted: 07/13/2020] [Indexed: 01/02/2023] Open
Abstract
Background and Purpose: Stroke-related functional risk scores are used to predict patients' functional outcomes following a stroke event. We evaluate the predictive accuracy of machine-learning algorithms for predicting functional outcomes in acute ischemic stroke patients after endovascular treatment. Methods: Data were from the Precise and Rapid Assessment of Collaterals with Multi-phase CT Angiography (PROVE-IT), an observational study of 614 ischemic stroke patients. Regression and machine learning models, including random forest (RF), classification and regression tree (CART), C5.0 decision tree (DT), support vector machine (SVM), adaptive boost machine (ABM), least absolute shrinkage and selection operator (LASSO) logistic regression, and logistic regression models were used to train and predict the 90-day functional impairment risk, which is measured by the modified Rankin scale (mRS) score > 2. The models were internally validated using split-sample cross-validation and externally validated in the INTERRSeCT cohort study. The accuracy of these models was evaluated using the area under the receiver operating characteristic curve (AUC), Matthews Correlation Coefficient (MCC), and Brier score. Results: Of the 614 patients included in the training data, 249 (40.5%) had 90-day functional impairment (i.e., mRS > 2). The median and interquartile range (IQR) of age and baseline NIHSS scores were 77 years (IQR = 69–83) and 17 (IQR = 11–22), respectively. Both logistic regression and machine learning models had comparable predictive accuracy when validated internally (AUC range = [0.65–0.72]; MCC range = [0.29–0.42]) and externally (AUC range = [0.66–0.71]; MCC range = [0.34–0.42]). Conclusions: Machine learning algorithms and logistic regression had comparable predictive accuracy for predicting stroke-related functional impairment in stroke patients.
Collapse
Affiliation(s)
- Shakiru A Alaka
- Department of Community Health Sciences, O'Brien Institute for Public Health, University of Calgary, Calgary, AB, Canada
| | - Bijoy K Menon
- Department of Community Health Sciences, O'Brien Institute for Public Health, University of Calgary, Calgary, AB, Canada.,Department of Clinical Neurosciences, Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada.,Department of Radiology, University of Calgary, Calgary, AB, Canada
| | - Anita Brobbey
- Department of Community Health Sciences, O'Brien Institute for Public Health, University of Calgary, Calgary, AB, Canada
| | - Tyler Williamson
- Department of Community Health Sciences, O'Brien Institute for Public Health, University of Calgary, Calgary, AB, Canada
| | - Mayank Goyal
- Department of Clinical Neurosciences, Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada.,Department of Radiology, University of Calgary, Calgary, AB, Canada
| | - Andrew M Demchuk
- Department of Clinical Neurosciences, Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada
| | - Michael D Hill
- Department of Community Health Sciences, O'Brien Institute for Public Health, University of Calgary, Calgary, AB, Canada.,Department of Clinical Neurosciences, Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada.,Department of Radiology, University of Calgary, Calgary, AB, Canada
| | - Tolulope T Sajobi
- Department of Community Health Sciences, O'Brien Institute for Public Health, University of Calgary, Calgary, AB, Canada.,Department of Clinical Neurosciences, Hotchkiss Brain Institute, University of Calgary, Calgary, AB, Canada
| |
Collapse
|
8
|
Kyrimi E, Neves MR, McLachlan S, Neil M, Marsh W, Fenton N. Medical idioms for clinical Bayesian network development. J Biomed Inform 2020; 108:103495. [PMID: 32619692 DOI: 10.1016/j.jbi.2020.103495] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/09/2019] [Revised: 05/07/2020] [Accepted: 06/24/2020] [Indexed: 01/17/2023]
Abstract
Bayesian Networks (BNs) are graphical probabilistic models that have proven popular in medical applications. While numerous medical BNs have been published, most are presented fait accompli without explanation of how the network structure was developed or justification of why it represents the correct structure for the given medical application. This means that the process of building medical BNs from experts is typically ad hoc and offers little opportunity for methodological improvement. This paper proposes generally applicable and reusable medical reasoning patterns to aid those developing medical BNs. The proposed method complements and extends the idiom-based approach introduced by Neil, Fenton, and Nielsen in 2000. We propose instances of their generic idioms that are specific to medical BNs. We refer to the proposed medical reasoning patterns as medical idioms. In addition, we extend the use of idioms to represent interventional and counterfactual reasoning. We believe that the proposed medical idioms are logical reasoning patterns that can be combined, reused and applied generically to help develop medical BNs. All proposed medical idioms have been illustrated using medical examples on coronary artery disease. The method has also been applied to other ongoing BNs being developed with medical experts. Finally, we show that applying the proposed medical idioms to published BN models results in models with a clearer structure.
Collapse
Affiliation(s)
- Evangelia Kyrimi
- Risk and Information Management Research Group, School of Electronic Engineering and Computer Science, Queen Mary University of London, London E1 4NS, UK.
| | - Mariana Raniere Neves
- Risk and Information Management Research Group, School of Electronic Engineering and Computer Science, Queen Mary University of London, London E1 4NS, UK
| | - Scott McLachlan
- Risk and Information Management Research Group, School of Electronic Engineering and Computer Science, Queen Mary University of London, London E1 4NS, UK
| | - Martin Neil
- Risk and Information Management Research Group, School of Electronic Engineering and Computer Science, Queen Mary University of London, London E1 4NS, UK
| | - William Marsh
- Risk and Information Management Research Group, School of Electronic Engineering and Computer Science, Queen Mary University of London, London E1 4NS, UK
| | - Norman Fenton
- Risk and Information Management Research Group, School of Electronic Engineering and Computer Science, Queen Mary University of London, London E1 4NS, UK
| |
Collapse
|
9
|
Kasatkin DS, Bogomolov YV, Spirin NN. [Steps to personalized therapy of multiple sclerosis: predicting safety of treatment using mathematical modeling]. Zh Nevrol Psikhiatr Im S S Korsakova 2019; 118:70-76. [PMID: 30160671 DOI: 10.17116/jnevro201811808270] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/17/2022]
Abstract
AIM To construct a mathematical model capable of predicting the drug safety of a patient receiving multiple sclerosis disease modifying drugs (DMD), on a model of flu-like syndrome (FLS). MATERIAL AND METHODS The study included 457 patients with multiple sclerosis (MS), aged from 18 to 68 years, mean 38.79 years, the mean duration of disease 122.58 months. All patients received first-line injections drug (interferon-beta). The sample included data from a three-year prospective dynamic observation with a frequency of observation of 1 every 6 months, with only the data of those examinations for which the presence or absence of FLS was known for the next 6 months (1203 cases). At the first step, the frequency of factors in the compared groups using the W Wald-Wolkovitz test, then the prognostic coefficients (PC) and the Kulbak informativity coefficient (CI) were calculated for each factor gradation. To determine the predictive ability of signs, the Spearman's R criterion was used. At the second step, a model of a two-layer neural network was constructed based on the data obtained. RESULTS A simple static model and algorithm were developed to assess the risks of the onset and persistence of FLS during the next 6 months of interferon beta therapy. An attempt was also made to create an active model using neural network technology. Both models showed good sensitivity and specificity - 81.2% and 80.6% for the neural network, and 73.4 and 71.6% for the static model. CONCLUSION Using of these algorithms allows to significantly increase the possibility of predicting the occurrence of AE at the time of drug prescribing. From the mathematical point of view, for the first time the mechanism and possibilities of using a neural network in conditions of incomplete initial information were determined.
Collapse
Affiliation(s)
- D S Kasatkin
- Yaroslavl State Medical University, Yaroslavl, Russia
| | | | - N N Spirin
- Yaroslavl State Medical University, Yaroslavl, Russia
| |
Collapse
|
10
|
Gracia CG, Chagin K, Kattan MW, Ji X, Kattan MG, Crotty L, Najm I, Gonzalez-Martinez J, Bingaman W, Jehi L. Predicting seizure freedom after epilepsy surgery, a challenge in clinical practice. Epilepsy Behav 2019; 95:124-130. [PMID: 31035104 PMCID: PMC6546523 DOI: 10.1016/j.yebeh.2019.03.047] [Citation(s) in RCA: 25] [Impact Index Per Article: 5.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/24/2019] [Revised: 03/08/2019] [Accepted: 03/27/2019] [Indexed: 11/19/2022]
Abstract
OBJECTIVE The objective of this study was to compare the accuracy of clinical judgment in predicting seizure outcome after resective epilepsy surgery relative to two recently published statistical tools [the Epilepsy Surgery Nomogram (ESN) and the modified Seizure-Freedom score (m-SFS)]. METHODS Details of presurgical evaluations of 20 patients who underwent epilepsy surgery were presented to 20 epilepsy experts. The final surgical treatment was also disclosed. The clinicians were asked to predict the likelihood of a good outcome (Engel 1) at 2 and 5 years in each case. The ESN and the m-SFS predictions were calculated with the data provided to the clinicians. The discriminative ability of clinical judgment, ESN, and m-SFS was assessed by calculating a concordance index (C-index). Expert opinion, the m-SFS and the ESN performances were compared using a Receiver Operating Characteristic (ROC) curve analysis. RESULTS The mean age at surgery was 29 years (standard deviation [SD] = 14); 40% were male; 70% were right-handed, and thirteen (65%) had an Engel outcome 1 at 2 and 5 years. The mean C-index for the mean physician's prediction was 0.478 with a variance of 0.012. The ESN had an area under the curve (AUC) of 0.528 and 0.533 for the 2-year and 5-year predictions in comparison with the clinicians' predictions that was 0.476, and 0.466, respectively. For the m-SFS, the AUC at 2 years and 5 years was 0.539 and 0.539, respectively. No statistical difference was noted between the ESN and the clinicians or between m-SFS and the ESN, but there is a moderate statistical difference favoring the m-SFS to the clinicians (p 0.0960 and 0.0514, for 2 and 5 years). SIGNIFICANCE Clinical judgment was not superior to the ESN nor to the m-SFS. Together with the interphysician's prediction variability, our findings reinforce the need for better tools to predict postoperative outcomes.
Collapse
Affiliation(s)
- Camilo Garcia Gracia
- Cleveland Clinic Epilepsy Center, Neurological Institute, Cleveland Clinic, Cleveland, OH 44195, United States of America
| | - Kevin Chagin
- Quantitative Health Sciences, Cleveland Clinic, Cleveland, OH 44195, United States of America
| | - Michael W Kattan
- Quantitative Health Sciences, Cleveland Clinic, Cleveland, OH 44195, United States of America
| | - Xinge Ji
- Quantitative Health Sciences, Cleveland Clinic, Cleveland, OH 44195, United States of America
| | - Madeleine G Kattan
- Quantitative Health Sciences, Cleveland Clinic, Cleveland, OH 44195, United States of America
| | - Lizzie Crotty
- Quantitative Health Sciences, Cleveland Clinic, Cleveland, OH 44195, United States of America
| | - Imad Najm
- Cleveland Clinic Epilepsy Center, Neurological Institute, Cleveland Clinic, Cleveland, OH 44195, United States of America
| | - Jorge Gonzalez-Martinez
- Cleveland Clinic Epilepsy Center, Neurological Institute, Cleveland Clinic, Cleveland, OH 44195, United States of America
| | - William Bingaman
- Cleveland Clinic Epilepsy Center, Neurological Institute, Cleveland Clinic, Cleveland, OH 44195, United States of America
| | - Lara Jehi
- Cleveland Clinic Epilepsy Center, Neurological Institute, Cleveland Clinic, Cleveland, OH 44195, United States of America.
| |
Collapse
|
11
|
Kraisangka J, Druzdzel MJ. A Bayesian Network Interpretation of the Cox's Proportional Hazard Model. Int J Approx Reason 2019; 103:195-211. [PMID: 31130777 DOI: 10.1016/j.ijar.2018.09.007] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/28/2022]
Abstract
Cox's proportional hazards (CPH) model is quite likely the most popular modeling technique in survival analysis. While the CPH model is able to represent a relationship between a collection of risks and their common effect, Bayesian networks have become an attractive alternative with an increased modeling power and far broader applications. Our paper focuses on a Bayesian network interpretation of the CPH model (BN-Cox). We provide a method of encoding knowledge from existing CPH models in the process of knowledge engineering for Bayesian networks. This is important because in practice we often have CPH models available in the literature and no access to the original data from which they have been derived. We compare the accuracy of the resulting BN-Cox model to the original CPH model, Kaplan-Meier estimate, and Bayesian networks learned from data, including Naive Bayes, Tree Augmented Naive Bayes, Noisy-Max, and parameter learning by means of the EM algorithm. BN-Cox model came out as the most accurate of all BN approaches and very close to the original CPH model. We study two approaches for simplifying the BN-Cox model for the sake of representational and computational efficiency: (1) parent divorcing and (2) removing less important risk factors. We show that removing less important risk factors leads to smaller loss of accuracy.
Collapse
Affiliation(s)
- Jidapa Kraisangka
- Decision System Laboratory, School of Computing and Information, University of Pittsburgh, Pittsburgh, PA, 15260, USA
| | - Marek J Druzdzel
- Decision System Laboratory, School of Computing and Information, University of Pittsburgh, Pittsburgh, PA, 15260, USA.,Faculty of Computer Science, Białystok University of Technology, Wiejska 45A, 15-351, Białystok, Poland
| |
Collapse
|
12
|
Rahman MS, Rumana AS. A model-based concordance-type index for evaluating the added predictive ability of novel risk factors and markers in the logistic regression models. J Appl Stat 2019. [DOI: 10.1080/02664763.2019.1580253] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/13/2022]
Affiliation(s)
- M. Shafiqur Rahman
- Institute of Statistical Research and Training, University of Dhaka, Dhaka, Bangladesh
| | - Afrin Sadia Rumana
- Department of Accounting and Information Systems, Bangladesh University of Professionals, Dhaka, Bangladesh
| |
Collapse
|
13
|
Jenkins DA, Sperrin M, Martin GP, Peek N. Dynamic models to predict health outcomes: current status and methodological challenges. Diagn Progn Res 2018; 2:23. [PMID: 31093570 PMCID: PMC6460710 DOI: 10.1186/s41512-018-0045-2] [Citation(s) in RCA: 50] [Impact Index Per Article: 8.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/09/2018] [Accepted: 11/19/2018] [Indexed: 12/14/2022] Open
Abstract
BACKGROUND Disease populations, clinical practice, and healthcare systems are constantly evolving. This can result in clinical prediction models quickly becoming outdated and less accurate over time. A potential solution is to develop 'dynamic' prediction models capable of retaining accuracy by evolving over time in response to observed changes. Our aim was to review the literature in this area to understand the current state-of-the-art in dynamic prediction modelling and identify unresolved methodological challenges. METHODS MEDLINE, Embase and Web of Science were searched for papers which used or developed dynamic clinical prediction models. Information was extracted on methods for model updating, choice of update windows and decay factors and validation of models. We also extracted reported limitations of methods and recommendations for future research. RESULTS We identified eleven papers that discussed seven dynamic clinical prediction modelling methods which split into three categories. The first category uses frequentist methods to update models in discrete steps, the second uses Bayesian methods for continuous updating and the third, based on varying coefficients, explicitly describes the relationship between predictors and outcome variable as a function of calendar time. These methods have been applied to a limited number of healthcare problems, and few empirical comparisons between them have been made. CONCLUSION Dynamic prediction models are not well established but they overcome one of the major issues with static clinical prediction models, calibration drift. However, there are challenges in choosing decay factors and in dealing with sudden changes. The validation of dynamic prediction models is still largely unexplored terrain.
Collapse
Affiliation(s)
- David A. Jenkins
- 0000000121662407grid.5379.8Health e-Research Centre, Farr Institute, Faculty of Biology, Medicine and Health, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK
- 0000000121662407grid.5379.8NIHR Greater Manchester Patient Safety Translational Research Centre, The University of Manchester, Manchester, UK
- 0000000121662407grid.5379.8Faculty of Biology, Medicine and Health, University of Manchester, City Labs 1.0, Nelson Street, Manchester, M13 9NQ UK
| | - Matthew Sperrin
- 0000000121662407grid.5379.8Health e-Research Centre, Farr Institute, Faculty of Biology, Medicine and Health, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK
| | - Glen P. Martin
- 0000000121662407grid.5379.8Health e-Research Centre, Farr Institute, Faculty of Biology, Medicine and Health, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK
| | - Niels Peek
- 0000000121662407grid.5379.8Health e-Research Centre, Farr Institute, Faculty of Biology, Medicine and Health, Manchester Academic Health Science Centre, University of Manchester, Manchester, UK
- 0000000121662407grid.5379.8NIHR Greater Manchester Patient Safety Translational Research Centre, The University of Manchester, Manchester, UK
| |
Collapse
|
14
|
Lemos DRQ, Franco AR, de Oliveira Garcia MH, Pastor D, Bravo-Alcantara P, de Moraes JC, Domingues C, Pamplona de Goes Cavalcanti L. Risk analysis for the reintroduction and transmission of measles in the post-elimination period in the Americas. Rev Panam Salud Publica 2017; 41:e157. [PMID: 31391839 PMCID: PMC6660859 DOI: 10.26633/rpsp.2017.157] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/09/2017] [Accepted: 06/12/2017] [Indexed: 11/24/2022] Open
Abstract
Objective. To propose and test a model for analyzing municipalities’ level of risk of reintroduction and transmission of the measles virus in the post-elimination period in the Americas. Methods. An ecological-analytical study was conducted using data on the measles epidemic that occurred in 2013–2015 in northeastern Brazil. The variables for analysis were selected after an extensive review of scientific literature on the risk of importation of measles cases. A univariate analysis considering the presence or absence of confirmed cases of measles in 184 municipalities in the state of Ceará, Brazil, was carried out to evaluate the association between the dependent variable and 23 independent variables, grouped into four categories: 1) characteristics of the municipalities; 2) quality indicators for immunization programs and epidemiological surveillance; 3) organizational structure for the public health response; and 4) selected impact indicators. A P value < 0.05 was considered significant. All variables with P < 0.200 were analyzed using multivariate logistic regression. Based on the results, the municipalities were categorized by four levels of risk (“low,” “medium,” “high,” and “very high”). Results. The model sensitivity was 95% for concordance between municipalities classified as “high risk” and “very high risk” and those that had an epidemic between 2013 and 2015 in Ceará. Of the 38 municipalities that had an epidemic, 76% (29/38) were classified as “high risk” and “very high risk”; 146 municipalities did not report cases (P < 0.0002). Conclusions. Given the imminent risk of reintroduction of measles circulation in the post-elimination period in the Americas, this model may be useful in identifying areas at greater risk for reintroduction and continued transmission of measles. Knowledge of vulnerable areas could trigger appropriate surveillance and monitoring to prevent sustained transmission.
Collapse
Affiliation(s)
- Daniele Rocha Queiroz Lemos
- Faculty of Medicin Centro Universitário Christus Fortaleza, CE Brazil Faculty of Medicine, Centro Universitário Christus, Fortaleza, CE, Brazil
| | - Aidee Ramirez Franco
- Pan American Health Organization Pan American Health Organization Washington, D.C. United States of America Pan American Health Organization, Washington, D.C., United States of America
| | | | - Desiree Pastor
- Pan American Health Organization Pan American Health Organization Washington, D.C. United States of America Pan American Health Organization, Washington, D.C., United States of America
| | - Pamela Bravo-Alcantara
- Pan American Health Organization Pan American Health Organization Washington, D.C. United States of America Pan American Health Organization, Washington, D.C., United States of America
| | - Jose Cassio de Moraes
- Faculty of Medical Sciences Santa Casa de Misericórdia São Paulo, SP Brazil Faculty of Medical Sciences, Santa Casa de Misericórdia, São Paulo, SP, Brazil
| | - Carla Domingues
- Ministry of Health Ministry of Health Brasília, DF Brazil Ministry of Health, Brasília, DF, Brazil
| | - Luciano Pamplona de Goes Cavalcanti
- Faculty of Medicine Universidade Federal do Fortaleza Fortaleza, CE Brazil Faculty of Medicine, Universidade Federal do Fortaleza, Fortaleza, CE, Brazil
| |
Collapse
|
15
|
Rahman MS, Sultana M. Performance of Firth-and logF-type penalized methods in risk prediction for small or sparse binary data. BMC Med Res Methodol 2017; 17:33. [PMID: 28231767 PMCID: PMC5324225 DOI: 10.1186/s12874-017-0313-9] [Citation(s) in RCA: 39] [Impact Index Per Article: 5.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/25/2016] [Accepted: 02/16/2017] [Indexed: 12/23/2022] Open
Abstract
BACKGROUND When developing risk models for binary data with small or sparse data sets, the standard maximum likelihood estimation (MLE) based logistic regression faces several problems including biased or infinite estimate of the regression coefficient and frequent convergence failure of the likelihood due to separation. The problem of separation occurs commonly even if sample size is large but there is sufficient number of strong predictors. In the presence of separation, even if one develops the model, it produces overfitted model with poor predictive performance. Firth-and logF-type penalized regression methods are popular alternative to MLE, particularly for solving separation-problem. Despite the attractive advantages, their use in risk prediction is very limited. This paper evaluated these methods in risk prediction in comparison with MLE and other commonly used penalized methods such as ridge. METHODS The predictive performance of the methods was evaluated through assessing calibration, discrimination and overall predictive performance using an extensive simulation study. Further an illustration of the methods were provided using a real data example with low prevalence of outcome. RESULTS The MLE showed poor performance in risk prediction in small or sparse data sets. All penalized methods offered some improvements in calibration, discrimination and overall predictive performance. Although the Firth-and logF-type methods showed almost equal amount of improvement, Firth-type penalization produces some bias in the average predicted probability, and the amount of bias is even larger than that produced by MLE. Of the logF(1,1) and logF(2,2) penalization, logF(2,2) provides slight bias in the estimate of regression coefficient of binary predictor and logF(1,1) performed better in all aspects. Similarly, ridge performed well in discrimination and overall predictive performance but it often produces underfitted model and has high rate of convergence failure (even the rate is higher than that for MLE), probably due to the separation problem. CONCLUSIONS The logF-type penalized method, particularly logF(1,1) could be used in practice when developing risk model for small or sparse data sets.
Collapse
Affiliation(s)
- M Shafiqur Rahman
- Institute of Statistical Research and Training, University of Dhaka, Dhaka, Bangladesh.
| | - Mahbuba Sultana
- Institute of Statistical Research and Training, University of Dhaka, Dhaka, Bangladesh
| |
Collapse
|
16
|
|
17
|
Makond B, Wang KJ, Wang KM. Probabilistic modeling of short survivability in patients with brain metastasis from lung cancer. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2015; 119:142-162. [PMID: 25804445 DOI: 10.1016/j.cmpb.2015.02.005] [Citation(s) in RCA: 7] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/03/2014] [Revised: 02/07/2015] [Accepted: 02/10/2015] [Indexed: 06/04/2023]
Abstract
The prediction of substantially short survivability in patients is extremely risky. In this study, we proposed a probabilistic model using Bayesian network (BN) to predict the short survivability of patients with brain metastasis from lung cancer. A nationwide cancer patient database from 1996 to 2010 in Taiwan was used. The cohort consisted of 438 patients with brain metastasis from lung cancer. We utilized synthetic minority over-sampling technique (SMOTE) to solve the imbalanced property embedded in the problem. The proposed BN was compared with three competitive models, namely, naive Bayes (NB), logistic regression (LR), and support vector machine (SVM). Statistical analysis showed that performances of BN, LR, NB, and SVM were statistically the same in terms of all indices with low sensitivity when these models were applied on an imbalanced data set. Results also showed that SMOTE can improve the performance of the four models in terms of sensitivity, while keeping high accuracy and specificity. Further, the proposed BN is more effective as compared with NB, LR, and SVM from two perspectives: the transparency and ability to show the relation of factors affecting brain metastasis from lung cancer; it allows decision makers to find the probability despite incomplete evidence and information; and the sensitivity of the proposed BN is the highest among all standard machine learning methods.
Collapse
Affiliation(s)
- Bunjira Makond
- Faculty of Commerce and Management, Prince of Songkla University, Trang, Thailand.
| | - Kung-Jeng Wang
- Department of Industrial Management, National Taiwan University of Science and Technology, Taipei 106, Taiwan, ROC.
| | - Kung-Min Wang
- Department of Surgery, Shin-Kong Wu Ho-Su Memorial Hospital, Taipei, Taiwan, ROC.
| |
Collapse
|
18
|
Black J, Hickson L, Black B, Khan A. Paediatric cochlear implantation: Adverse prognostic factors and trends from a review of 174 cases. Cochlear Implants Int 2013; 15:62-77. [DOI: 10.1179/1754762813y.0000000045] [Citation(s) in RCA: 27] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/31/2022]
|
19
|
Bernegger G, Musalek M, Rehmann-Sutter C. An alternative view on the task of prognosis. Crit Rev Oncol Hematol 2013; 84 Suppl 2:S17-24. [PMID: 23347414 DOI: 10.1016/s1040-8428(13)70005-2] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/24/2023] Open
Abstract
Prognosis is central to clinical medical practice. In oncology, an accurate prognosis is a key requirement for making the right therapeutical decisions. Prognosis is even more important when medicine extends its predictive capacities using genetic data. Based on statistics and probability, indices, survival curves and prognostic scores are established. Intending to be an objective and neutral description of reality, this kind of prognosis and the corresponding practices of medical care however carry and imply a particular conception of human life and destiny. Medical Humanities can help to develop reflexivity regarding prognostic practices. This paper intends to clarify which particular assumptions are implied in the current way of doing prognosis in medicine, and highlights its advantages and limitations. Prognosis not only describes but also affects present and future patient experience. An alternative view on the task of prognosis is then developed with a broader understanding of the prognostic act. Based on a phenomenology of time, distinguishing between a quantitative chronological understanding of time (chronos) and an experienced qualitative time (kairos), the article contrasts a conception of the prognosis as forecast ('probabilistic prognosis') with a conception of prognosis as perspective ('hermeneutic prognosis'). In probabilistic prognosis, the future is seen as something that can be read from presently accessible signs, as something that we can anticipate within ranges of uncertainty. The patient's lifetime, which has been open toward the future, becomes closed through this kind of prognosis. In a hermeneutic conception of prognosis as a perspective on the future, which is thought of as an open space of possibilities, the future as not-yet is what can be envisioned without being known. A hermeneutic approach emphasizes the meanings of the experience of illness for the patient, helps to improve the practice of prognosis today, and offers caregivers and patients an opportunity for living better.
Collapse
Affiliation(s)
- Guenda Bernegger
- University of Applied Sciences and Arts of Southern Switzerland, SUPSI, Palazzo E, 6928 Manno, Switzerland.
| | | | | |
Collapse
|
20
|
Assessing and combining repeated prognosis of physicians and temporal models in the intensive care. Artif Intell Med 2013; 57:111-7. [DOI: 10.1016/j.artmed.2012.08.005] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/09/2011] [Revised: 07/12/2012] [Accepted: 08/26/2012] [Indexed: 11/18/2022]
|
21
|
Debray TPA, Koffijberg H, Vergouwe Y, Moons KGM, Steyerberg EW. Aggregating published prediction models with individual participant data: a comparison of different approaches. Stat Med 2012; 31:2697-712. [PMID: 22733546 DOI: 10.1002/sim.5412] [Citation(s) in RCA: 43] [Impact Index Per Article: 3.6] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/07/2011] [Accepted: 03/16/2012] [Indexed: 01/07/2023]
Abstract
During the recent decades, interest in prediction models has substantially increased, but approaches to synthesize evidence from previously developed models have failed to keep pace. This causes researchers to ignore potentially useful past evidence when developing a novel prediction model with individual participant data (IPD) from their population of interest. We aimed to evaluate approaches to aggregate previously published prediction models with new data. We consider the situation that models are reported in the literature with predictors similar to those available in an IPD dataset. We adopt a two-stage method and explore three approaches to calculate a synthesis model, hereby relying on the principles of multivariate meta-analysis. The former approach employs a naive pooling strategy, whereas the latter accounts for within-study and between-study covariance. These approaches are applied to a collection of 15 datasets of patients with traumatic brain injury, and to five previously published models for predicting deep venous thrombosis. Here, we illustrated how the generally unrealistic assumption of consistency in the availability of evidence across included studies can be relaxed. Results from the case studies demonstrate that aggregation yields prediction models with an improved discrimination and calibration in a vast majority of scenarios, and result in equivalent performance (compared with the standard approach) in a small minority of situations. The proposed aggregation approaches are particularly useful when few participant data are at hand. Assessing the degree of heterogeneity between IPD and literature findings remains crucial to determine the optimal approach in aggregating previous evidence into new prediction models.
Collapse
Affiliation(s)
- Thomas P A Debray
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands.
| | | | | | | | | |
Collapse
|
22
|
Minne L, Eslami S, de Keizer N, de Jonge E, de Rooij SE, Abu-Hanna A. Effect of changes over time in the performance of a customized SAPS-II model on the quality of care assessment. Intensive Care Med 2011; 38:40-6. [PMID: 22042520 PMCID: PMC3233667 DOI: 10.1007/s00134-011-2390-2] [Citation(s) in RCA: 51] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/27/2011] [Accepted: 06/28/2011] [Indexed: 01/29/2023]
Abstract
Purpose The aim of our study was to explore, using an innovative method, the effect of temporal changes in the mortality prediction performance of an existing model on the quality of care assessment. The prognostic model (rSAPS-II) was a recalibrated Simplified Acute Physiology Score-II model developed for very elderly Intensive Care Unit (ICU) patients. Methods The study population comprised all 12,143 consecutive patients aged 80 years and older admitted between January 2004 and July 2009 to one of the ICUs of 21 Dutch hospitals. The prospective dataset was split into 30 equally sized consecutive subsets. Per subset, we measured the model’s discrimination [area under the curve (AUC)], accuracy (Brier score), and standardized mortality ratio (SMR), both without and after repeated recalibration. All performance measures were considered to be stable if <2 consecutive points fell outside the green zone [mean ± 2 standard deviation (SD)] and none fell outside the yellow zone (mean ± 4SD) of pre-control charts. We compared proportions of hospitals with SMR>1 without and after repeated recalibration for the year 2009. Results For all subsets, the AUCs were stable, but the Brier scores and SMRs were not. The SMR was downtrending, achieving levels significantly below 1. Repeated recalibration rendered it stable again. The proportions of hospitals with SMR>1 and SMR<1 changed from 15 versus 85% to 35 versus 65%. Conclusions Variability over time may markedly vary among different performance measures, and infrequent model recalibration can result in improper assessment of the quality of care in many hospitals. We stress the importance of the timely recalibration and repeated validation of prognostic models over time. Electronic supplementary material The online version of this article (doi:10.1007/s00134-011-2390-2) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Lilian Minne
- Department of Medical Informatics, Academic Medical Center Amsterdam, Room J1b-124, PO Box 22660, 1100 DD Amsterdam, The Netherlands.
| | | | | | | | | | | |
Collapse
|
23
|
Minne L, Eslami S, de Keizer N, de Jonge E, de Rooij SE, Abu-Hanna A. Statistical process control for validating a classification tree model for predicting mortality--a novel approach towards temporal validation. J Biomed Inform 2011; 45:37-44. [PMID: 21907826 DOI: 10.1016/j.jbi.2011.08.015] [Citation(s) in RCA: 12] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/06/2011] [Revised: 08/08/2011] [Accepted: 08/25/2011] [Indexed: 11/29/2022]
Abstract
Prediction models are postulated as useful tools to support tasks such as clinical decision making and benchmarking. In particular, classification tree models have enjoyed much interest in the Biomedical Informatics literature. However, their prospective predictive performance over the course of time has not been investigated. In this paper we suggest and apply statistical process control methods to monitor over more than 5 years the prospective predictive performance of TM80+, one of the few classification-tree models published in the clinical literature. TM80+ is a model for predicting mortality among very elderly patients in the intensive care based on a multi-center dataset. We also inspect the predictive performance at the tree's leaves. This study provides important insights into patterns of (in)stability of the tree's performance and its "shelf life". The study underlies the importance of continuous validation of prognostic models over time using statistical tools and the timely recalibration of tree models.
Collapse
Affiliation(s)
- Lilian Minne
- Academic Medical Center, Department of Medical Informatics, PO Box 22660, 1100 DD Amsterdam, The Netherlands.
| | | | | | | | | | | |
Collapse
|
24
|
|
25
|
Prediction of clinical conditions after coronary bypass surgery using dynamic data analysis. J Med Syst 2010; 34:229-39. [PMID: 20503607 DOI: 10.1007/s10916-008-9234-9] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/21/2022]
Abstract
This work studies the impact of using dynamic information as features in a machine learning algorithm for the prediction task of classifying critically ill patients in two classes according to the time they need to reach a stable state after coronary bypass surgery: less or more than 9 h. On the basis of five physiological variables (heart rate, systolic arterial blood pressure, systolic pulmonary pressure, blood temperature and oxygen saturation), different dynamic features were extracted, namely the means and standard deviations at different moments in time, coefficients of multivariate autoregressive models and cepstral coefficients. These sets of features served subsequently as inputs for a Gaussian process and the prediction results were compared with the case where only admission data was used for the classification. The dynamic features, especially the cepstral coefficients (aROC: 0.749, Brier score: 0.206), resulted in higher performances when compared to static admission data (aROC: 0.547, Brier score: 0.247). The differences in performance are shown to be significant. In all cases, the Gaussian process classifier outperformed to logistic regression.
Collapse
|
26
|
Learning patient-specific predictive models from clinical data. J Biomed Inform 2010; 43:669-85. [PMID: 20450985 DOI: 10.1016/j.jbi.2010.04.009] [Citation(s) in RCA: 30] [Impact Index Per Article: 2.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/05/2009] [Revised: 04/14/2010] [Accepted: 04/29/2010] [Indexed: 11/21/2022]
Abstract
We introduce an algorithm for learning patient-specific models from clinical data to predict outcomes. Patient-specific models are influenced by the particular history, symptoms, laboratory results, and other features of the patient case at hand, in contrast to the commonly used population-wide models that are constructed to perform well on average on all future cases. The patient-specific algorithm uses Markov blanket (MB) models, carries out Bayesian model averaging over a set of models to predict the outcome for the patient case at hand, and employs a patient-specific heuristic to locate a set of suitable models to average over. We evaluate the utility of using a local structure representation for the conditional probability distributions in the MB models that captures additional independence relations among the variables compared to the typically used representation that captures only the global structure among the variables. In addition, we compare the performance of Bayesian model averaging to that of model selection. The patient-specific algorithm and its variants were evaluated on two clinical datasets for two outcomes. Our results provide support that the performance of an algorithm for learning patient-specific models can be improved by using a local structure representation for MB models and by performing Bayesian model averaging.
Collapse
|
27
|
Soini EJ, Rissanen T, Tiihonen J, Hodgins S, Eronen M, Ryynänen OP. Predicting forensic admission among the mentally ill in a multinational setting: A Bayesian modelling approach. DATA KNOWL ENG 2009. [DOI: 10.1016/j.datak.2009.07.005] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/20/2022]
|
28
|
Huang ML, Chen HY. Glaucoma Classification Model Based on GDx VCC Measured Parameters by Decision Tree. J Med Syst 2009; 34:1141-7. [DOI: 10.1007/s10916-009-9333-2] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/26/2009] [Accepted: 06/11/2009] [Indexed: 11/28/2022]
|
29
|
Farion K, Michalowski W, Wilk S, O'Sullivan D, Matwin S. A tree-based decision model to support prediction of the severity of asthma exacerbations in children. J Med Syst 2009; 34:551-62. [PMID: 20703909 DOI: 10.1007/s10916-009-9268-7] [Citation(s) in RCA: 26] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2008] [Accepted: 02/19/2009] [Indexed: 11/26/2022]
Abstract
This paper describes the development of a tree-based decision model to predict the severity of pediatric asthma exacerbations in the emergency department (ED) at 2 h following triage. The model was constructed from retrospective patient data abstracted from the ED charts. The original data was preprocessed to eliminate questionable patient records and to normalize values of age-dependent clinical attributes. The model uses attributes routinely collected in the ED and provides predictions even for incomplete observations. Its performance was verified on independent validating data (split-sample validation) where it demonstrated AUC (area under ROC curve) of 0.83, sensitivity of 84%, specificity of 71% and the Brier score of 0.18. The model is intended to supplement an asthma clinical practice guideline, however, it can be also used as a stand-alone decision tool.
Collapse
Affiliation(s)
- Ken Farion
- Department of Pediatrics and Emergency Medicine, University of Ottawa, Ottawa, ON, Canada
| | | | | | | | | |
Collapse
|
30
|
Guenther T, Mueller I, Preuss M, Kruse R, Sabel B. A Treatment Outcome Prediction Model of Visual Field Recovery Using Self-Organizing Maps. IEEE Trans Biomed Eng 2009; 56:572-81. [DOI: 10.1109/tbme.2008.2009995] [Citation(s) in RCA: 12] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
|
31
|
Predicting the Need to Perform Life-Saving Interventions in Trauma Patients by Using New Vital Signs and Artificial Neural Networks. Artif Intell Med 2009. [DOI: 10.1007/978-3-642-02976-9_55] [Citation(s) in RCA: 6] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/17/2023]
|
32
|
Luaces O, Taboada F, Albaiceta GM, Domínguez LA, Enríquez P, Bahamonde A. Predicting the probability of survival in intensive care unit patients from a small number of variables and training examples. Artif Intell Med 2009; 45:63-76. [DOI: 10.1016/j.artmed.2008.11.005] [Citation(s) in RCA: 14] [Impact Index Per Article: 0.9] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/07/2007] [Revised: 10/09/2008] [Accepted: 11/05/2008] [Indexed: 11/29/2022]
|
33
|
Suebnukarn S, Rungcharoenporn N, Sangsuratham S. A Bayesian decision support model for assessment of endodontic treatment outcome. ACTA ACUST UNITED AC 2008; 106:e48-58. [DOI: 10.1016/j.tripleo.2008.05.011] [Citation(s) in RCA: 16] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/15/2008] [Revised: 04/06/2008] [Accepted: 05/06/2008] [Indexed: 11/29/2022]
|
34
|
Toma T, Abu-Hanna A, Bosman RJ. Discovery and integration of univariate patterns from daily individual organ-failure scores for intensive care mortality prediction. Artif Intell Med 2008; 43:47-60. [PMID: 18394871 DOI: 10.1016/j.artmed.2008.01.002] [Citation(s) in RCA: 17] [Impact Index Per Article: 1.1] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/19/2007] [Revised: 01/10/2008] [Accepted: 01/20/2008] [Indexed: 11/27/2022]
Abstract
OBJECTIVES The current established mortality predictive models in the intensive care rely only on patient information gathered within the first 24 hours of admission. Recent research demonstrated the added prognostic value residing in the sequential organ-failure assessment (SOFA) score which quantifies on each day the cumulative patient organ derangement. The objective of this paper is to develop and study predictive models that also incorporate univariate patterns of the six individual organ systems underlining the SOFA score. A model for a given day d predicts the probability of in-hospital mortality. MATERIALS AND METHODS We use the logistic framework to combine a summary statistic of the historic SOFA information for a patient together with selected dummy variables indicating the occurrence of univariate frequent temporal patterns of individual organ system functioning. We demonstrate the application of our method to a large real-life data set from an intensive care unit (ICU) in a teaching hospital. Model performance is tested in terms of the AUC and the Brier score. RESULTS An algorithm for categorization, discovery, and selection of univariate patterns of individual organ scores and the induction of predictive models. The case-study resulted in six daily models corresponding to days 2-7. Their AUC ranged between 0.715 and 0.794 and the Brier scores between 0.161 and 0.216. Models using only admission data but recalibrated for days 2-7 generated AUC ranging between 0.643 and 0.761 and Brier scores ranged between 0.175 and 0.230. CONCLUSIONS The results show that temporal organ-failure episodes improve predictions' quality in terms of both discrimination and calibration. In addition, they enhance the interpretability of models. Our approach should be applicable to many other medical domains where severity scores and sub-scores are collected.
Collapse
Affiliation(s)
- Tudor Toma
- Academic Medical Center, Universiteit van Amsterdam, Department of Medical Informatics, P.O. Box 22700, 1100 DE Amsterdam, The Netherlands.
| | | | | |
Collapse
|
35
|
van Gerven MAJ, Taal BG, Lucas PJF. Dynamic Bayesian networks as prognostic models for clinical patient management. J Biomed Inform 2008; 41:515-29. [PMID: 18337188 DOI: 10.1016/j.jbi.2008.01.006] [Citation(s) in RCA: 67] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2007] [Revised: 01/08/2008] [Accepted: 01/21/2008] [Indexed: 12/31/2022]
Abstract
Prognostic models in medicine are usually been built using simple decision rules, proportional hazards models, or Markov models. Dynamic Bayesian networks (DBNs) offer an approach that allows for the incorporation of the causal and temporal nature of medical domain knowledge as elicited from domain experts, thereby allowing for detailed prognostic predictions. The aim of this paper is to describe the considerations that must be taken into account when constructing a DBN for complex medical domains and to demonstrate their usefulness in practice. To this end, we focus on the construction of a DBN for prognosis of carcinoid patients, compare performance with that of a proportional hazards model, and describe predictions for three individual patients. We show that the DBN can make detailed predictions, about not only patient survival, but also other variables of interest, such as disease progression, the effect of treatment, and the development of complications. Strengths and limitations of our approach are discussed and compared with those offered by traditional methods.
Collapse
Affiliation(s)
- Marcel A J van Gerven
- Radboud University Nijmegen, Institute for Computing and Information Sciences, Toernooiveld 1, 6525 ED Nijmegen, The Netherlands.
| | | | | |
Collapse
|
36
|
Toma T, Abu-Hanna A, Bosman RJ. Discovery and inclusion of SOFA score episodes in mortality prediction. J Biomed Inform 2007; 40:649-60. [PMID: 17485242 DOI: 10.1016/j.jbi.2007.03.007] [Citation(s) in RCA: 37] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/19/2006] [Revised: 02/19/2007] [Accepted: 03/09/2007] [Indexed: 01/31/2023]
Abstract
Predicting the survival status of Intensive Care patients at the end of their hospital stay is useful for various clinical and organizational tasks. Current models for predicting mortality use logistic regression models that rely solely on data collected during the first 24h of patient admission. These models do not exploit information contained in daily organ failure scores which nowadays are being routinely collected in many Intensive Care Units. We propose a novel method for mortality prediction that, in addition to admission-related data, takes advantage of daily data as well. The method is characterized by the data-driven discovery of temporal patterns, called episodes, of the organ failure scores and by embedding them in the familiar logistic regression framework for prediction. Our method results in a set of D logistic regression models, one for each of the first D days of Intensive Care Unit stay. A model for day d<or=D is trained on the patient subpopulation that stayed at least d days in the Intensive Care Unit and predicts the probability of death at the end of hospital stay for such patients. We implemented our method, with a specific form of episodes, called aligned episodes, on a large dataset of Intensive Care Unit patients for the first 5 days of stay (D=5) in the unit. We compared our models with ones that were developed on the same patient subpopulations but which did not use the episodes. The new models show improved performance on each of the five days. They also provide insight in the effect of the various selected episodes on mortality.
Collapse
Affiliation(s)
- Tudor Toma
- Department of Medical Informatics, Academic Medical Center, Universiteit van Amsterdam, P.O. Box 22700, 1100 DE Amsterdam, The Netherlands.
| | | | | |
Collapse
|
37
|
Kazmierska J, Malicki J. Application of the Naïve Bayesian Classifier to optimize treatment decisions. Radiother Oncol 2007; 86:211-6. [PMID: 18022719 DOI: 10.1016/j.radonc.2007.10.019] [Citation(s) in RCA: 51] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/01/2007] [Revised: 10/08/2007] [Accepted: 10/11/2007] [Indexed: 11/18/2022]
Abstract
BACKGROUND AND PURPOSE To study the accuracy, specificity and sensitivity of the Naïve Bayesian Classifier (NBC) in the assessment of individual risk of cancer relapse or progression after radiotherapy (RT). MATERIALS AND METHODS Data of 142 brain tumour patients irradiated from 2000 to 2005 were analyzed. Ninety-six attributes related to disease, patient and treatment were chosen. Attributes in binary form consisted of the training set for NBC learning. NBC calculated an individual conditional probability of being assigned to: relapse or progression (1), or no relapse or progression (0) group. Accuracy, attribute selection and quality of classifier were determined by comparison with actual treatment results, leave-one-out and cross validation methods, respectively. Clinical setting test utilized data of 35 patients. Treatment results at classification were unknown and were compared with classification results after 3 months. RESULTS High classification accuracy (84%), specificity (0.87) and sensitivity (0.80) were achieved, both for classifier training and in progressive clinical evaluation. CONCLUSIONS NBC is a useful tool to support the assessment of individual risk of relapse or progression in patients diagnosed with brain tumour undergoing RT postoperatively.
Collapse
Affiliation(s)
- Joanna Kazmierska
- Department of Radiotherapy, Great Poland Cancer Centre, Poznan, Poland.
| | | |
Collapse
|
38
|
Discovery and Integration of Organ-Failure Episodes in Mortality Prediction. Artif Intell Med 2007. [DOI: 10.1007/978-3-540-73599-1_11] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
39
|
Verduijn M, Sacchi L, Peek N, Bellazzi R, de Jonge E, de Mol BAJM. Temporal abstraction for feature extraction: a comparative case study in prediction from intensive care monitoring data. Artif Intell Med 2007; 41:1-12. [PMID: 17698331 DOI: 10.1016/j.artmed.2007.06.003] [Citation(s) in RCA: 27] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/30/2006] [Revised: 06/02/2007] [Accepted: 06/06/2007] [Indexed: 01/31/2023]
Abstract
OBJECTIVES To compare two temporal abstraction procedures for the extraction of meta features from monitoring data. Feature extraction prior to predictive modeling is a common strategy in prediction from temporal data. A fundamental dilemma in this strategy, however, is the extent to which the extraction should be guided by domain knowledge, and to which extent it should be guided by the available data. The two temporal abstraction procedures compared in this case study differ in this respect. METHODS AND MATERIAL The first temporal abstraction procedure derives symbolic descriptions from the data that are predefined using existing concepts from the medical language. In the second procedure, a large space of numerical meta features is searched through to discover relevant features from the data. These procedures were applied to a prediction problem from intensive care monitoring data. The predictive value of the resulting meta features were compared, and based on each type of features, a class probability tree model was developed. RESULTS The numerical meta features extracted by the second procedure were found to be more informative than the symbolic meta features of the first procedure in the case study, and a superior predictive performance was observed for the associated tree model. CONCLUSION The findings indicate that for prediction from monitoring data, induction of numerical meta features from data is preferable to extraction of symbolic meta features using existing clinical concepts.
Collapse
Affiliation(s)
- Marion Verduijn
- Department of Medical Informatics, Academic Medical Center, P.O. Box 22700, 1100 DE Amsterdam, The Netherlands.
| | | | | | | | | | | |
Collapse
|
40
|
Verduijn M, Peek N, Rosseel PMJ, de Jonge E, de Mol BAJM. Prognostic Bayesian networks I: rationale, learning procedure, and clinical use. J Biomed Inform 2007; 40:609-18. [PMID: 17704008 DOI: 10.1016/j.jbi.2007.07.003] [Citation(s) in RCA: 31] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/03/2006] [Revised: 06/28/2007] [Accepted: 07/05/2007] [Indexed: 10/23/2022]
Abstract
Prognostic models are tools to predict the future outcome of disease and disease treatment, one of the fundamental tasks in clinical medicine. This article presents the prognostic Bayesian network (PBN) as a new type of prognostic model that builds on the Bayesian network methodology, and implements a dynamic, process-oriented view on prognosis. A PBN describes the mutual relationships between variables that come into play during subsequent stages of a care process and a clinical outcome. A dedicated procedure for inducing these networks from clinical data is presented. In this procedure, the network is composed of a collection of local supervised learning models that are recursively learned from the data. The procedure optimizes performance of the network's primary task, outcome prediction, and handles the fact that patients may drop out of the process in earlier stages. Furthermore, the article describes how PBNs can be applied to solve a number of information problems that are related to medical prognosis.
Collapse
Affiliation(s)
- Marion Verduijn
- Department of Medical Informatics, Academic Medical Center (AMC), P.O. box 22700, 1100 DE Amsterdam, The Netherlands.
| | | | | | | | | |
Collapse
|
41
|
Peek N, Verduijn M, Sjoe-Sjoe WGT, Rosseel PJM, de Jonge E, de Mol BAJM. ProCarSur: A System for Dynamic Prognostic Reasoning in Cardiac Surgery. Artif Intell Med 2007. [DOI: 10.1007/978-3-540-73599-1_45] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/22/2022]
|
42
|
Matthes G, Seifert J, Bogatzki S, Steinhage K, Ekkernkamp A, Stengel D. [Age and survival likelihood of polytrauma patients. "Local tailoring" of the DGU prognosis model]. Unfallchirurg 2005; 108:288-92. [PMID: 15812668 DOI: 10.1007/s00113-005-0929-9] [Citation(s) in RCA: 8] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Abstract
INTRODUCTION Age is one of five prognostic parameters identified based on data of the trauma registry of the German Association for Trauma Surgery (DGU). We asked ourselves if the suggested prognostic model provides the same predictive power of data from an independent hospital. Furthermore, we investigated whether age itself or age-associated comorbidity causes an unfavorable prognostic effect. METHODS The investigation was based on data of 103 multiply injured patients (67 male, 36 female, mean age 35,4+/-SD 19,0 years, ISS 36,8+/-10,9). Data were collected prospectively following the guidelines of the trauma registry of the German Association for Trauma Surgery. Based on documented comorbidities, a risk calculation was performed using the ASA classification. Correlation between age and ASA was analyzed using Spearman's method. The prognostic value of the original model in our patient pool with or without ASA classification, possible interactions, and the discriminatory power of the model were estimated using logistic regression. RESULTS Attributable mortality was 31,7% (95% CI 22,7-41,7%). Age, ISS, GCS and ASA were included into the final logistic model. Odds ratios of the origin model were reproducible nearly identical in our patinet pool (OR: age 1,048; ISS 1,066; GCS 0,822). In spite of the fact that we have found a strong correlation between age and ASA-Classification (rho=0,60, p<0,0001) there was no prognostic value of comorbidity. CONCLUSION The suggested prognostic model based on multicenter data evaluation can be applied to a single center with only minimal loss of discriminatory power. In this context, age seems to have a prognostic value independent of comorbidity.
Collapse
Affiliation(s)
- G Matthes
- Abteilung für Unfallchirurgie, Klinik und Poliklinik für Chirurgie, Ernst-Moritz-Arndt-Universität, Greifswald.
| | | | | | | | | | | |
Collapse
|
43
|
Silva A, Cortez P, Santos MF, Gomes L, Neves J. Mortality assessment in intensive care units via adverse events using artificial neural networks. Artif Intell Med 2005; 36:223-34. [PMID: 16213693 DOI: 10.1016/j.artmed.2005.07.006] [Citation(s) in RCA: 31] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/14/2005] [Revised: 07/29/2005] [Accepted: 07/30/2005] [Indexed: 10/25/2022]
Abstract
OBJECTIVE This work presents a novel approach for the prediction of mortality in intensive care units (ICUs) based on the use of adverse events, which are defined from four bedside alarms, and artificial neural networks (ANNs). This approach is compared with two logistic regression (LR) models: the prognostic model used in most of the European ICUs, based on the simplified acute physiology score (SAPS II), and a LR that uses the same input variables of the ANN model. MATERIALS AND METHODS A large dataset was considered, encompassing forty two ICUs of nine European countries. The recorded features of each patient include the final outcome, the case mix (e.g. age) and the intermediate outcomes, defined as the daily averages of the out of range values of four biometrics (e.g. heart rate). The SAPS II score requires 17 static variables (e.g. serum sodium), which are collected within the first day of the patient's admission. A nonlinear least squares method was used to calibrate the LR models while the ANNs are made up of multilayer perceptrons trained by the RPROP algorithm. A total of 13,164 adult patients were randomly divided into training (66%) and test (33%) sets. The two methods were evaluated in terms of receiver operator characteristic (ROC) curves. RESULTS The event based models predicted the outcome more accurately than the currently used SAPS II model (P<0.05), with ROC areas within the ranges 83.9-87.1% (ANN) and 82.6-85.2% (LR) versus 80% (LR SAPS II). When using the same inputs, the ANNs outperform the LR (improvement of 1.3-2%). CONCLUSION Better prognostic models can be achieved by adopting low cost and real-time intermediate outcomes rather than static data.
Collapse
Affiliation(s)
- Alvaro Silva
- Serviço de Cuidados Intensivos, Hospital Geral de Santo António, Porto, Portugal
| | | | | | | | | |
Collapse
|
44
|
Abstract
OBJECTIVE Handling time-related concepts is essential in medicine. During diagnosis it can make a substantial difference to know the temporal order in which some symptoms occurred or for how long they lasted. During prognosis the potential evolutions of a disease are conceived as a description of events unfolding in time. In therapy planning the different steps of treatment must be applied in a precise order, with a given frequency and for a certain span of time in order to be effective. This article offers a survey on the use of temporal reasoning for decision support-related tasks in medicine. MATERIAL AND METHODS Key publications of the area, mainly circumscribed to the latest two decades, are reviewed and classified according to three important stages of patient treatment requiring decision support: diagnosis, prognosis and therapy planning/management. Other complementary publications, like those on time-centered information storage and retrieval, are also considered as they provide valuable support to the above mentioned three stages. RESULTS Key areas are highlighted and used to organize the latest contributions. The survey of previous research is followed by an analysis of what can still be improved and what is needed to make the next generation of decision support systems for medicine more effective. CONCLUSIONS It can be observed that although the area has been considerably developed, there are still areas where more research is needed to make time-based systems of widespread use in decision support-related areas of medicine. Several suggestions for further exploration are proposed as a result of the survey.
Collapse
Affiliation(s)
- Juan Carlos Augusto
- School of Computing and Mathematics, University of Ulster at Jordanstown, Newtownabbey, Co. Antrim BT37 0QB, UK.
| |
Collapse
|
45
|
Abu-Hanna A, de Keizer N. Integrating classification trees with local logistic regression in Intensive Care prognosis. Artif Intell Med 2003; 29:5-23. [PMID: 12957778 DOI: 10.1016/s0933-3657(03)00047-2] [Citation(s) in RCA: 41] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022]
Abstract
Health care effectiveness and efficiency are under constant scrutiny especially when treatment is quite costly as in the Intensive Care (IC). Currently there are various international quality of care programs for the evaluation of IC. At the heart of such quality of care programs lie prognostic models whose prediction of patient mortality can be used as a norm to which actual mortality is compared. The current generation of prognostic models in IC are statistical parametric models based on logistic regression. Given a description of a patient at admission, these models predict the probability of his or her survival. Typically, this patient description relies on an aggregate variable, called a score, that quantifies the severity of illness of the patient. The use of a parametric model and an aggregate score form adequate means to develop models when data is relatively scarce but it introduces the risk of bias. This paper motivates and suggests a method for studying and improving the performance behavior of current state-of-the-art IC prognostic models. Our method is based on machine learning and statistical ideas and relies on exploiting information that underlies a score variable. In particular, this underlying information is used to construct a classification tree whose nodes denote patient sub-populations. For these sub-populations, local models, most notably logistic regression ones, are developed using only the total score variable. We compare the performance of this hybrid model to that of a traditional global logistic regression model. We show that the hybrid model not only provides more insight into the data but also has a better performance. We pay special attention to the precision aspect of model performance and argue why precision is more important than discrimination ability.
Collapse
Affiliation(s)
- Ameen Abu-Hanna
- Department of Medical Informatics, AMC-University of Amsterdam, Meibergdreef 15, 1105 AZ Amsterdam, The Netherlands.
| | | |
Collapse
|
46
|
Abu-Hanna A, de Keizer N. A Classification-Tree Hybrid Method for Studying Prognostic Models in Intensive Care. Artif Intell Med 2001. [DOI: 10.1007/3-540-48229-6_13] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
|