1
|
Chung A, Opoku-Pare GA, Tibble H. Cause of death coding in asthma. BMC Med Res Methodol 2024; 24:129. [PMID: 38840045 PMCID: PMC11151540 DOI: 10.1186/s12874-024-02238-x] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2024] [Accepted: 05/03/2024] [Indexed: 06/07/2024] Open
Abstract
BACKGROUND While clinical coding is intended to be an objective and standardized practice, it is important to recognize that it is not entirely the case. The clinical and bureaucratic practices from event of death to a case being entered into a research dataset are important context for analysing and interpreting this data. Variation in practices can influence the accuracy of the final coded record in two different stages: the reporting of the death certificate, and the International Classification of Diseases (Version 10; ICD-10) coding of that certificate. METHODS This study investigated 91,022 deaths recorded in the Scottish Asthma Learning Healthcare System dataset between 2000 and 2017. Asthma-related deaths were identified by the presence of any of ICD-10 codes J45 or J46, in any position. These codes were categorized either as relating to asthma attacks specifically (status asthmatic; J46) or generally to asthma diagnosis (J45). RESULTS We found that one in every 200 deaths in this were coded as being asthma related. Less than 1% of asthma-related mortality records used both J45 and J46 ICD-10 codes as causes. Infection (predominantly pneumonia) was more commonly reported as a contributing cause of death when J45 was the primary coded cause, compared to J46, which specifically denotes asthma attacks. CONCLUSION Further inspection of patient history can be essential to validate deaths recorded as caused by asthma, and to identify potentially mis-recorded non-asthma deaths, particularly in those with complex comorbidities.
Collapse
Affiliation(s)
| | | | - Holly Tibble
- Usher Institute, University of Edinburgh, Edinburgh, Scotland.
- Asthma UK Centre for Applied Research, Edinburgh, Scotland.
| |
Collapse
|
2
|
Kigo J, Kamau S, Mawji A, Mwaniki P, Dunsmuir D, Pillay Y, Zhang C, Pallot K, Ogero M, Kimutai D, Ouma M, Mohamed I, Chege M, Thuranira L, Kissoon N, Ansermino JM, Akech S. External validation of a paediatric Smart triage model for use in resource limited facilities. PLOS DIGITAL HEALTH 2024; 3:e0000293. [PMID: 38905166 PMCID: PMC11192416 DOI: 10.1371/journal.pdig.0000293] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/05/2023] [Accepted: 04/24/2024] [Indexed: 06/23/2024]
Abstract
Models for digital triage of sick children at emergency departments of hospitals in resource poor settings have been developed. However, prior to their adoption, external validation should be performed to ensure their generalizability. We externally validated a previously published nine-predictor paediatric triage model (Smart Triage) developed in Uganda using data from two hospitals in Kenya. Both discrimination and calibration were assessed, and recalibration was performed by optimizing the intercept for classifying patients into emergency, priority, or non-urgent categories based on low-risk and high-risk thresholds. A total of 2539 patients were eligible at Hospital 1 and 2464 at Hospital 2, and 5003 for both hospitals combined; admission rates were 8.9%, 4.5%, and 6.8%, respectively. The model showed good discrimination, with area under the receiver-operator curve (AUC) of 0.826, 0.784 and 0.821, respectively. The pre-calibrated model at a low-risk threshold of 8% achieved a sensitivity of 93% (95% confidence interval, (CI):89%-96%), 81% (CI:74%-88%), and 89% (CI:85%-92%), respectively, and at a high-risk threshold of 40%, the model achieved a specificity of 86% (CI:84%-87%), 96% (CI:95%-97%), and 91% (CI:90%-92%), respectively. Recalibration improved the graphical fit, but new risk thresholds were required to optimize sensitivity and specificity.The Smart Triage model showed good discrimination on external validation but required recalibration to improve the graphical fit of the calibration plot. There was no change in the order of prioritization of patients following recalibration in the respective triage categories. Recalibration required new site-specific risk thresholds that may not be needed if prioritization based on rank is all that is required. The Smart Triage model shows promise for wider application for use in triage for sick children in different settings.
Collapse
Affiliation(s)
- Joyce Kigo
- Health Service Unit, Kenya Medical Research Institute (KEMRI)-Wellcome Trust Research Programme, Nairobi, Kenya
| | - Stephen Kamau
- Health Service Unit, Kenya Medical Research Institute (KEMRI)-Wellcome Trust Research Programme, Nairobi, Kenya
| | - Alishah Mawji
- Centre for International Child Health, BC Children’s Hospital Research Institute, Vancouver, British Columbia, Canada
| | - Paul Mwaniki
- Health Service Unit, Kenya Medical Research Institute (KEMRI)-Wellcome Trust Research Programme, Nairobi, Kenya
| | - Dustin Dunsmuir
- Centre for International Child Health, BC Children’s Hospital Research Institute, Vancouver, British Columbia, Canada
- Department of Anesthesiology, Pharmacology & Therapeutics, University of British Columbia, Vancouver, British Columbia, Canada
| | - Yashodani Pillay
- Department of Anesthesiology, Pharmacology & Therapeutics, University of British Columbia, Vancouver, British Columbia, Canada
| | - Cherri Zhang
- Department of Anesthesiology, Pharmacology & Therapeutics, University of British Columbia, Vancouver, British Columbia, Canada
| | - Katija Pallot
- Department of Anesthesiology, Pharmacology & Therapeutics, University of British Columbia, Vancouver, British Columbia, Canada
| | - Morris Ogero
- Health Service Unit, Kenya Medical Research Institute (KEMRI)-Wellcome Trust Research Programme, Nairobi, Kenya
| | - David Kimutai
- Department of Pediatrics, Mbagathi County Hospital, Nairobi, Kenya
| | - Mary Ouma
- Department of Pediatrics, Mbagathi County Hospital, Nairobi, Kenya
| | - Ismael Mohamed
- Department of Pediatrics, Mbagathi County Hospital, Nairobi, Kenya
| | - Mary Chege
- Department of Pediatrics, Kiambu County Referral Hospital, Kiambu, Kenya
| | - Lydia Thuranira
- Department of Pediatrics, Kiambu County Referral Hospital, Kiambu, Kenya
| | - Niranjan Kissoon
- Department of Pediatrics, University of British Columbia, Vancouver, British Columbia, Canada
| | - J. Mark Ansermino
- Centre for International Child Health, BC Children’s Hospital Research Institute, Vancouver, British Columbia, Canada
- Department of Anesthesiology, Pharmacology & Therapeutics, University of British Columbia, Vancouver, British Columbia, Canada
| | - Samuel Akech
- Health Service Unit, Kenya Medical Research Institute (KEMRI)-Wellcome Trust Research Programme, Nairobi, Kenya
| |
Collapse
|
3
|
Kore A, Abbasi Bavil E, Subasri V, Abdalla M, Fine B, Dolatabadi E, Abdalla M. Empirical data drift detection experiments on real-world medical imaging data. Nat Commun 2024; 15:1887. [PMID: 38424096 PMCID: PMC10904813 DOI: 10.1038/s41467-024-46142-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/31/2023] [Accepted: 02/14/2024] [Indexed: 03/02/2024] Open
Abstract
While it is common to monitor deployed clinical artificial intelligence (AI) models for performance degradation, it is less common for the input data to be monitored for data drift - systemic changes to input distributions. However, when real-time evaluation may not be practical (eg., labeling costs) or when gold-labels are automatically generated, we argue that tracking data drift becomes a vital addition for AI deployments. In this work, we perform empirical experiments on real-world medical imaging to evaluate three data drift detection methods' ability to detect data drift caused (a) naturally (emergence of COVID-19 in X-rays) and (b) synthetically. We find that monitoring performance alone is not a good proxy for detecting data drift and that drift-detection heavily depends on sample size and patient features. Our work discusses the need and utility of data drift detection in various scenarios and highlights gaps in knowledge for the practical application of existing methods.
Collapse
Affiliation(s)
- Ali Kore
- Vector Institute, Toronto, Canada
| | | | - Vallijah Subasri
- Peter Munk Cardiac Center, University Health Network, Toronto, ON, Canada
| | - Moustafa Abdalla
- Department of Surgery, Harvard Medical School, Massachusetts General Hospital, Boston, USA
| | - Benjamin Fine
- Institute for Better Health, Trillium Health Partners, Mississauga, Canada
- Department of Medical Imaging, University of Toronto, Toronto, Canada
| | - Elham Dolatabadi
- Vector Institute, Toronto, Canada
- School of Health Policy and Management, Faculty of Health, York University, Toronto, Canada
| | - Mohamed Abdalla
- Institute for Better Health, Trillium Health Partners, Mississauga, Canada.
| |
Collapse
|
4
|
Collins GS, Dhiman P, Ma J, Schlussel MM, Archer L, Van Calster B, Harrell FE, Martin GP, Moons KGM, van Smeden M, Sperrin M, Bullock GS, Riley RD. Evaluation of clinical prediction models (part 1): from development to external validation. BMJ 2024; 384:e074819. [PMID: 38191193 PMCID: PMC10772854 DOI: 10.1136/bmj-2023-074819] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Accepted: 09/04/2023] [Indexed: 01/10/2024]
Affiliation(s)
- Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| | - Paula Dhiman
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| | - Jie Ma
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| | - Michael M Schlussel
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford OX3 7LD, UK
| | - Lucinda Archer
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, UK
| | - Ben Van Calster
- KU Leuven, Department of Development and Regeneration, Leuven, Belgium
- Department of Biomedical Data Sciences, Leiden University Medical Centre, Leiden, Netherlands
- EPI-Centre, KU Leuven, Belgium
| | - Frank E Harrell
- Department of Biostatistics, Vanderbilt University, Nashville, TN, USA
| | - Glen P Martin
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Karel G M Moons
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Maarten van Smeden
- Julius Centre for Health Sciences and Primary Care, University Medical Centre Utrecht, Utrecht University, Utrecht, Netherlands
| | - Matthew Sperrin
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Garrett S Bullock
- Department of Orthopaedic Surgery, Wake Forest School of Medicine, Winston-Salem, NC, USA
- Centre for Sport, Exercise and Osteoarthritis Research Versus Arthritis, University of Oxford, Oxford, UK
| | - Richard D Riley
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
- National Institute for Health and Care Research (NIHR) Birmingham Biomedical Research Centre, UK
| |
Collapse
|
5
|
Argus L, Taylor M, Ouzounian M, Venkateswaran R, Grant SW. Risk Prediction Models for Long-Term Survival after Cardiac Surgery: A Systematic Review. Thorac Cardiovasc Surg 2024; 72:29-39. [PMID: 36750201 DOI: 10.1055/s-0043-1760747] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/09/2023]
Abstract
BACKGROUND The reporting of alternative postoperative measures of quality after cardiac surgery is becoming increasingly important as in-hospital mortality rates continue to decline. This study aims to systematically review and assess risk models designed to predict long-term outcomes after cardiac surgery. METHODS The MEDLINE and Embase databases were searched for articles published between 1990 and 2020. Studies developing or validating risk prediction models for long-term outcomes after cardiac surgery were included. Data were extracted using checklists for critical appraisal and systematic review of prediction modeling studies. RESULTS Eleven studies were identified for inclusion in the review, of which nine studies described the development of long-term risk prediction models after cardiac surgery and two were external validation studies. A total of 70 predictors were included across the nine models. The most frequently used predictors were age (n = 9), peripheral vascular disease (n = 8), renal disease (n = 8), and pulmonary disease (n = 8). Despite all models demonstrating acceptable performance on internal validation, only two models underwent external validation, both of which performed poorly. CONCLUSION Nine risk prediction models predicting long-term mortality after cardiac surgery have been identified in this review. Statistical issues with model development, limited inclusion of outcomes beyond 5 years of follow-up, and a lack of external validation studies means that none of the models identified can be recommended for use in contemporary cardiac surgery. Further work is needed either to successfully externally validate existing models or to develop new models. Newly developed models should aim to use standardized long-term specific reproducible outcome measures.
Collapse
Affiliation(s)
- Leah Argus
- The University of Manchester, Manchester, United Kingdom
| | - Marcus Taylor
- Department of Cardiothoracic Surgery, Manchester University NHS Foundation Trust, Manchester, United Kingdom
| | - Maral Ouzounian
- Division of Cardiovascular Surgery, Peter Munk Cardiac Centre, University Health Network, Toronto, Ontario, Canada
| | - Rajamiyer Venkateswaran
- Department of Cardiothoracic Surgery, Manchester University NHS Foundation Trust, Manchester, United Kingdom
| | - Stuart W Grant
- Division of Cardiovascular Sciences, University of Manchester, Manchester, United Kingdom
- Academic Cardiovascular Unit, South Tees Hospitals NHS Foundation Trust, Middlesborough, United Kingdom
| |
Collapse
|
6
|
Booth S, Mozumder SI, Archer L, Ensor J, Riley RD, Lambert PC, Rutherford MJ. Using temporal recalibration to improve the calibration of risk prediction models in competing risk settings when there are trends in survival over time. Stat Med 2023; 42:5007-5024. [PMID: 37705296 PMCID: PMC10946485 DOI: 10.1002/sim.9898] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2021] [Revised: 07/31/2023] [Accepted: 08/23/2023] [Indexed: 09/15/2023]
Abstract
We have previously proposed temporal recalibration to account for trends in survival over time to improve the calibration of predictions from prognostic models for new patients. This involves first estimating the predictor effects using data from all individuals (full dataset) and then re-estimating the baseline using a subset of the most recent data whilst constraining the predictor effects to remain the same. In this article, we demonstrate how temporal recalibration can be applied in competing risk settings by recalibrating each cause-specific (or subdistribution) hazard model separately. We illustrate this using an example of colon cancer survival with data from the Surveillance Epidemiology and End Results (SEER) program. Data from patients diagnosed in 1995-2004 were used to fit two models for deaths due to colon cancer and other causes respectively. We discuss considerations that need to be made in order to apply temporal recalibration such as the choice of data used in the recalibration step. We also demonstrate how to assess the calibration of these models in new data for patients diagnosed subsequently in 2005. Comparison was made to a standard analysis (when improvements over time are not taken into account) and a period analysis which is similar to temporal recalibration but differs in the data used to estimate the predictor effects. The 10-year calibration plots demonstrated that using the standard approach over-estimated the risk of death due to colon cancer and the total risk of death and that calibration was improved using temporal recalibration or period analysis.
Collapse
Affiliation(s)
- Sarah Booth
- Biostatistics Research Group, Department of Population Health SciencesUniversity of LeicesterLeicesterUK
| | - Sarwar I. Mozumder
- Biostatistics Research Group, Department of Population Health SciencesUniversity of LeicesterLeicesterUK
- Oncology Biometrics Statistical Innovation, AstraZenecaCambridgeUK
| | - Lucinda Archer
- Institute of Applied Health Research, College of Medical and Dental SciencesUniversity of BirminghamBirminghamUK
| | - Joie Ensor
- Institute of Applied Health Research, College of Medical and Dental SciencesUniversity of BirminghamBirminghamUK
| | - Richard D. Riley
- Institute of Applied Health Research, College of Medical and Dental SciencesUniversity of BirminghamBirminghamUK
| | - Paul C. Lambert
- Biostatistics Research Group, Department of Population Health SciencesUniversity of LeicesterLeicesterUK
- Department of Medical Epidemiology and BiostatisticsKarolinska InstitutetStockholmSweden
| | - Mark J. Rutherford
- Biostatistics Research Group, Department of Population Health SciencesUniversity of LeicesterLeicesterUK
| |
Collapse
|
7
|
Gunaydin S, McCusker K, Nicotra W. Redosing of long acting cardioplegic solutions in adult cardiac surgery: A comparative study. Perfusion 2023:2676591231216315. [PMID: 37965876 DOI: 10.1177/02676591231216315] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/16/2023]
Abstract
INTRODUCTION Despite promising results regarding using long-acting cardioplegia in the adult population, little data exists specifically for operations requiring prolonged aortic cross-clamp needing additional doses. In this pilot study, we evaluated the outcomes of patients undergoing surgery with prolonged cross-clamp time based on four different redosing compositions. METHODS During the period from January 2019 until June 2022, 288 patients undergoing cardiac surgery with an expected cross-clamp time over 60 min were prospectively randomized regarding the type of the cardioplegia used: Group 1 (N = 150)- single-dose del Nido antegrade cardioplegia and Group 2 (N = 138)- single-dose Histidine-Tryptophane-Ketoglutarate (HTK) antegrade cardioplegia. In patients with ischemic time over 60 min, needing a redosing were further analyzed separately in four subgroups: (A) Cold whole blood (CWB) (4:1) (N = 95); (A1: DN-CWB; A2: HTK-CWB) and (B) St Thomas Solution (N = 92) (B1: DN-St Thomas; B2: HTK-St Thomas. Control groups were C1 (DN redosed by DN) and C2 (HTK by HTK). RESULTS Troponin levels in A1 and B1 groups were significantly lower than in DN-control. Respiratory support time and incidence of atrial fibrillation were significantly lower in Group A1 versus DN-control. CONCLUSIONS Long-acting cardioplegic techniques are becoming widely utilized in the adult population, with minimal data on redosing methods/compositions for prolonged cases. Due to the small patient population, further investigation is needed to delineate optimal redosing methods, but this report brings to attention the initial success of multiple strategies.
Collapse
Affiliation(s)
- Serdar Gunaydin
- Department of Cardiac Surgery, University of Health Sciences, Ankara, Turkey
| | - Kevin McCusker
- Department of Clinical Perfusion, Lawrence Technological University, Southfield, Michigan
| | - William Nicotra
- Department of Cardiac Surgery, St Clair Hospital, Pittsburgh, PA, USA
| |
Collapse
|
8
|
Abbasciano RG, Tomassini S, Roman MA, Rizzello A, Pathak S, Ramzi J, Lucarelli C, Layton G, Butt A, Lai F, Kumar T, Wozniak MJ, Murphy GJ. Effects of interventions targeting the systemic inflammatory response to cardiac surgery on clinical outcomes in adults. Cochrane Database Syst Rev 2023; 10:CD013584. [PMID: 37873947 PMCID: PMC10594589 DOI: 10.1002/14651858.cd013584.pub2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 10/25/2023]
Abstract
BACKGROUND Organ injury is a common and severe complication of cardiac surgery that contributes to the majority of deaths. There are no effective treatment or prevention strategies. It has been suggested that innate immune system activation may have a causal role in organ injury. A wide range of organ protection interventions targeting the innate immune response have been evaluated in randomised controlled trials (RCTs) in adult cardiac surgery patients, with inconsistent results in terms of effectiveness. OBJECTIVES The aim of the review was to summarise the results of RCTs of organ protection interventions targeting the innate immune response in adult cardiac surgery. The review considered whether the interventions had a treatment effect on inflammation, important clinical outcomes, or both. SEARCH METHODS CENTRAL, MEDLINE, Embase, conference proceedings and two trial registers were searched on October 2022 together with reference checking to identify additional studies. SELECTION CRITERIA RCTs comparing organ protection interventions targeting the innate immune response versus placebo or no treatment in adult patients undergoing cardiac surgery where the treatment effect on innate immune activation and on clinical outcomes of interest were reported. DATA COLLECTION AND ANALYSIS Searches, study selection, quality assessment, and data extractions were performed independently by pairs of authors. The primary inflammation outcomes were peak IL-6 and IL-8 concentrations in blood post-surgery. The primary clinical outcome was in-hospital or 30-day mortality. Treatment effects were expressed as risk ratios (RR) and standardised mean difference (SMD) with 95% confidence intervals (CI). Meta-analyses were performed using random effects models, and heterogeneity was assessed using I2. MAIN RESULTS A total of 40,255 participants from 328 RCTs were included in the synthesis. The effects of treatments on IL-6 (SMD -0.77, 95% CI -0.97 to -0.58, I2 = 92%) and IL-8 (SMD -0.92, 95% CI -1.20 to -0.65, I2 = 91%) were unclear due to heterogeneity. Heterogeneity for inflammation outcomes persisted across multiple sensitivity and moderator analyses. The pooled treatment effect for in-hospital or 30-day mortality was RR 0.78, 95% CI 0.68 to 0.91, I2 = 0%, suggesting a significant clinical benefit. There was little or no treatment effect on mortality when analyses were restricted to studies at low risk of bias. Post hoc analyses failed to demonstrate consistent treatment effects on inflammation and clinical outcomes. Levels of certainty for pooled treatment effects on the primary outcomes were very low. AUTHORS' CONCLUSIONS A systematic review of RCTs of organ protection interventions targeting innate immune system activation did not resolve uncertainty as to the effectiveness of these treatments, or the role of innate immunity in organ injury following cardiac surgery.
Collapse
Affiliation(s)
| | | | - Marius A Roman
- Department of Cardiovascular Sciences, University of Leicester, Leicester, UK
| | - Angelica Rizzello
- Department of Cardiovascular Sciences, University of Leicester, Leicester, UK
| | - Suraj Pathak
- Department of Cardiovascular Sciences, University of Leicester, Leicester, UK
| | - Joussi Ramzi
- Leicester Medical School, University of Leicester, Leicester, UK
| | - Carla Lucarelli
- Department of Cardiac Surgery, University of Verona, Verona, Italy
| | - Georgia Layton
- University Hospitals of Leicester NHS Trust, Leicester, UK
| | - Ayesha Butt
- Department of Cardiovascular Sciences, University of Leicester, Leicester, UK
| | - Florence Lai
- Department of Cardiovascular Sciences, University of Leicester, Leicester, UK
| | - Tracy Kumar
- Leicester Clinical Trials Unit, University of Leicester, Leicester, UK
| | - Marcin J Wozniak
- Department of Cardiovascular Sciences, University of Leicester, Leicester, UK
| | - Gavin J Murphy
- Department of Cardiovascular Sciences, University of Leicester, Leicester, UK
| |
Collapse
|
9
|
van Dijk WB, Leeuwenberg AM, Grobbee DE, Siregar S, Houterman S, Daeter EJ, de Vries MC, Groenwold RHH, Schuit E. Dynamics in cardiac surgery: trends in population characteristics and the performance of the EuroSCORE II over time. Eur J Cardiothorac Surg 2023; 64:ezad301. [PMID: 37672025 PMCID: PMC10504469 DOI: 10.1093/ejcts/ezad301] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/27/2022] [Revised: 06/21/2023] [Accepted: 09/05/2023] [Indexed: 09/07/2023] Open
Abstract
OBJECTIVES The aim of this study was to investigate the performance of the EuroSCORE II over time and dynamics in values of predictors included in the model. METHODS A cohort study was performed using data from the Netherlands Heart Registration. All cardiothoracic surgical procedures performed between 1 January 2013 and 31 December 2019 were included for analysis. Performance of the EuroSCORE II was assessed across 3-month intervals in terms of calibration and discrimination. For subgroups of major surgical procedures, performance of the EuroSCORE II was assessed across 12-month time intervals. Changes in values of individual EuroSCORE II predictors over time were assessed graphically. RESULTS A total of 103 404 cardiothoracic surgical procedures were included. Observed mortality risk ranged between 1.9% [95% confidence interval (CI) 1.6-2.4] and 3.6% (95% CI 2.6-4.4) across 3-month intervals, while the mean predicted mortality risk ranged between 3.4% (95% CI 3.3-3.6) and 4.2% (95% CI 3.9-4.6). The corresponding observed:expected ratios ranged from 0.50 (95% CI 0.46-0.61) to 0.95 (95% CI 0.74-1.16). Discriminative performance in terms of the c-statistic ranged between 0.82 (95% CI 0.78-0.89) and 0.89 (95% CI 0.87-0.93). The EuroSCORE II consistently overestimated mortality compared to observed mortality. This finding was consistent across all major cardiothoracic surgical procedures. Distributions of values of individual predictors varied broadly across predictors over time. Most notable trends were a decrease in elective surgery from 75% to 54% and a rise in patients with no or New York Heart Association I class heart failure from 27% to 33%. CONCLUSIONS The EuroSCORE II shows good discriminative performance, but consistently overestimates mortality risks of all types of major cardiothoracic surgical procedures in the Netherlands.
Collapse
Affiliation(s)
- Wouter B van Dijk
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Artuur M Leeuwenberg
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Diederick E Grobbee
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| | - Sabrina Siregar
- Department of Cardiothoracic Surgery, Erasmus Medical Center, Erasmus University, Rotterdam, Netherlands
| | | | - Edgar J Daeter
- Netherlands Heart Registration, Utrecht, Netherlands
- Department of Cardiothoracic Surgery, St. Antonius Hospital, Nieuwegein, Netherlands
| | - Martine C de Vries
- Department of Medical Ethics and Health Law, Leiden University Medical Center, Leiden University, Leiden, Netherlands
| | - Rolf H H Groenwold
- Department of Clinical Epidemiology, Leiden University Medical Center, Leiden University, Leiden, Netherlands
- Department of Biomedical Data Sciences, Leiden University Medical Center, Leiden University, Leiden, Netherlands
| | - Ewoud Schuit
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, Netherlands
| |
Collapse
|
10
|
Pollack J, Yang W, Schnellinger EM, Arnaoutakis GJ, Kallan MJ, Kimmel SE. Dynamic prediction modeling of postoperative mortality among patients undergoing surgical aortic valve replacement in a statewide cohort over a 12-year period. JTCVS OPEN 2023; 15:94-112. [PMID: 37808034 PMCID: PMC10556941 DOI: 10.1016/j.xjon.2023.07.011] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 03/13/2023] [Revised: 06/07/2023] [Accepted: 06/21/2023] [Indexed: 10/10/2023]
Abstract
Objective Clinical prediction models for surgical aortic valve replacement mortality, are valuable decision tools but are often limited in their ability to account for changes in medical practice, patient selection, and the risk of outcomes over time. Recent research has identified methods to update models as new data accrue, but their effect on model performance has not been rigorously tested. Methods The study population included 44,546 adults who underwent an isolated surgical aortic valve replacement from January 1, 1999, to December 31, 2018, statewide in Pennsylvania. After chronologically splitting the data into training and validation sets, we compared calibration, discrimination, and accuracy measures amongst a nonupdating model to 2 methods of model updating: calibration regression and the novel dynamic logistic state space model. Results The risk of mortality decreased significantly during the validation period (P < .01) and the nonupdating model demonstrated poor calibration and reduced accuracy over time. Both updating models maintained better calibration (Hosmer-Lemeshow χ2 statistic) than the nonupdating model: nonupdating (156.5), calibration regression (4.9), and dynamic logistic state space model (8.0). Overall accuracy (Brier score) was consistently better across both updating models: dynamic logistic state space model (0.0252), calibration regression (0.0253), and nonupdating (0.0256). Discrimination improved with the dynamic logistic state space model (area under the curve, 0.696) compared with the nonupdating model (area under the curve, 0.685) and calibration regression method (area under the curve, 0.687). Conclusions Dynamic model updating can improve model accuracy, discrimination, and calibration. The decision as to which method to use may depend on which measure is most important in each clinical context. Because competing therapies have emerged for valve replacement models, updating may guide clinical decision making.
Collapse
Affiliation(s)
- Jackie Pollack
- Department of Epidemiology, College of Public Health and Health Professions and College of Medicine, University of Florida, Gainesville, Fla
| | - Wei Yang
- Department of Biostatistics, Epidemiology, and Informatics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pa
| | | | - George J. Arnaoutakis
- Division of Cardiovascular and Thoracic Surgery, University of Texas at Austin Dell Medical School, Austin, Tex
| | - Michael J. Kallan
- Center for Clinical Epidemiology and Biostatistics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pa
| | - Stephen E. Kimmel
- Department of Epidemiology, College of Public Health and Health Professions and College of Medicine, University of Florida, Gainesville, Fla
| |
Collapse
|
11
|
Yadava OP. Predicting the unpredictable in cardiothoracic surgery. Indian J Thorac Cardiovasc Surg 2023; 39:109-111. [PMID: 36785611 PMCID: PMC9918619 DOI: 10.1007/s12055-023-01478-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/27/2023] Open
|
12
|
Abstract
BACKGROUND Clinical prediction models should be validated before implementation in clinical practice. But is favorable performance at internal validation or one external validation sufficient to claim that a prediction model works well in the intended clinical context? MAIN BODY We argue to the contrary because (1) patient populations vary, (2) measurement procedures vary, and (3) populations and measurements change over time. Hence, we have to expect heterogeneity in model performance between locations and settings, and across time. It follows that prediction models are never truly validated. This does not imply that validation is not important. Rather, the current focus on developing new models should shift to a focus on more extensive, well-conducted, and well-reported validation studies of promising models. CONCLUSION Principled validation strategies are needed to understand and quantify heterogeneity, monitor performance over time, and update prediction models when appropriate. Such strategies will help to ensure that prediction models stay up-to-date and safe to support clinical decision-making.
Collapse
|
13
|
Castro MDL, Alves M, Martins A, Luísa Papoila A, Botelho MA, Fragata J. Reproducibility and validity of the Portuguese Edmonton Frail Scale version in cardiac surgery patients. Rev Port Cardiol 2023; 42:295-304. [PMID: 36738965 DOI: 10.1016/j.repc.2022.03.009] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/14/2021] [Accepted: 03/18/2022] [Indexed: 02/05/2023] Open
Abstract
INTRODUCTION Frailty is a multidimensional syndrome characterized by the loss of functional reserve, associated with higher mortality and less functional survival in cardiac surgery patients. The Edmonton Frail Scale (EFS) is a comprehensive tool devised for brief frailty detection. To the best of our knowledge, there are no culturally adapted and validated frailty screening tools that enable the identification of vulnerability domains suited for use in the preoperative setting in Portugal. This was the motivation for this study. OBJECTIVES To assess the validity and reproducibility of the Portuguese version of the EFS. METHODS Prospective observational study, in a sample of elective cardiac surgery patients. The Edmonton Frail Scale (EFS) translation and backtranslation were performed. Demographic and clinical data were collected, and the translated EFS translated, Geriatric Depression Scale, and Mini Mental State Examination Portuguese versions, Katz and Clinical Frailty Scales were administered. To assess validity Mann-Whitney test, Spearman's correlation coefficient, marginal homogeneity test and Kappa coefficient were employed. Reproducibility was assessed estimating kappa coefficient for the frailty diagnosis and the 11 EFS items. Intra-class correlation coefficients and the corresponding 95% confidence interval were estimated using linear mixed effects model. RESULTS The EFS Portuguese version revealed construct validity for frailty identification, as well as criterion validity for cognition and mood domains. Reproducibility was demonstrated, with k=0.62 (95% confidence interval (CI) 0.42-0.82) and intraclass correlation (ICC)=0.94 (95% CI 0.89-0.97) in inter-observer test and k=0.48 (95% CI 0.26-0.70) and ICC=0.85 (95% CI 0.72-0.92) in intra-observer test. CONCLUSIONS The EFS Portuguese version is valid and reproducible for use, suiting pre-operative frailty screening in a cardiac surgery setting.
Collapse
Affiliation(s)
- Maria de Lurdes Castro
- Anesthesiology Department, Hospital de Santa Marta, Centro Hospitalar Universitário Lisboa Central, Lisbon, Portugal.
| | - Marta Alves
- Epidemiology and Statistics Unit, Research Center, Centro Hospitalar Universitário Lisboa Central, Lisbon, Portugal
| | - Ana Martins
- Anesthesiology Department, Hospital de Santa Marta, Centro Hospitalar Universitário Lisboa Central, Lisbon, Portugal
| | - Ana Luísa Papoila
- Epidemiology and Statistics Unit, Research Center, Centro Hospitalar Universitário Lisboa Central, Lisbon, Portugal
| | - Maria-Amália Botelho
- Chronic Diseases Research Center (CEDOC), Nova Medical School, Universidade Nova de Lisboa, Lisbon, Portugal
| | - José Fragata
- Cardiothoracic University Clinic and Department, Hospital de Santa Marta, Centro Hospitalar Universitário Lisboa Central, Lisbon, Portugal
| |
Collapse
|
14
|
Dong T, Sinha S, Zhai B, Fudulu DP, Chan J, Narayan P, Judge A, Caputo M, Dimagli A, Benedetto U, Angelini GD. Cardiac surgery risk prediction using ensemble machine learning to incorporate legacy risk scores: A benchmarking study. Digit Health 2023; 9:20552076231187605. [PMID: 37492033 PMCID: PMC10363892 DOI: 10.1177/20552076231187605] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/23/2023] [Accepted: 06/23/2023] [Indexed: 07/27/2023] Open
Abstract
Objective The introduction of new clinical risk scores (e.g. European System for Cardiac Operative Risk Evaluation (EuroSCORE) II) superseding original scores (e.g. EuroSCORE I) with different variable sets typically result in disparate datasets due to high levels of missingness for new score variables prior to time of adoption. Little is known about the use of ensemble learning to incorporate disparate data from legacy scores. We tested the hypothesised that Homogenenous and Heterogeneous Machine Learning (ML) ensembles will have better performance than ensembles of Dynamic Model Averaging (DMA) for combining knowledge from EuroSCORE I legacy data with EuroSCORE II data to predict cardiac surgery risk. Methods Using the National Adult Cardiac Surgery Audit dataset, we trained 12 different base learner models, based on two different variable sets from either EuroSCORE I (LogES) or EuroScore II (ES II), partitioned by the time of score adoption (1996-2016 or 2012-2016) and evaluated on holdout set (2017-2019). These base learner models were ensembled using nine different combinations of six ML algorithms to produce homogeneous or heterogeneous ensembles. Performance was assessed using a consensus metric. Results Xgboost homogenous ensemble (HE) was the highest performing model (clinical effectiveness metric (CEM) 0.725) with area under the curve (AUC) (0.8327; 95% confidence interval (CI) 0.8323-0.8329) followed by Random Forest HE (CEM 0.723; AUC 0.8325; 95%CI 0.8320-0.8326). Across different heterogenous ensembles, significantly better performance was obtained by combining siloed datasets across time (CEM 0.720) than building ensembles of either 1996-2011 (t-test adjusted, p = 1.67×10-6) or 2012-2019 (t-test adjusted, p = 1.35×10-193) datasets alone. Conclusions Both homogenous and heterogenous ML ensembles performed significantly better than DMA ensemble of Bayesian Update models. Time-dependent ensemble combination of variables, having differing qualities according to time of score adoption, enabled previously siloed data to be combined, leading to increased power, clinical interpretability of variables and usage of data.
Collapse
Affiliation(s)
- Tim Dong
- Translational Health Sciences, Bristol Heart Institute, University of Bristol, Bristol, UK
| | - Shubhra Sinha
- Translational Health Sciences, Bristol Heart Institute, University of Bristol, Bristol, UK
| | - Ben Zhai
- School of Computing Science, Northumbria University, Newcastle upon Tyne, UK
| | - Daniel P Fudulu
- Translational Health Sciences, Bristol Heart Institute, University of Bristol, Bristol, UK
| | - Jeremy Chan
- Translational Health Sciences, Bristol Heart Institute, University of Bristol, Bristol, UK
| | - Pradeep Narayan
- Department of Cardiac Surgery, Rabindranath Tagore International Institute of Cardiac Sciences, Kolkata, India
| | - Andy Judge
- Translational Health Sciences, Bristol Heart Institute, University of Bristol, Bristol, UK
| | - Massimo Caputo
- Translational Health Sciences, Bristol Heart Institute, University of Bristol, Bristol, UK
| | - Arnaldo Dimagli
- Translational Health Sciences, Bristol Heart Institute, University of Bristol, Bristol, UK
| | - Umberto Benedetto
- Translational Health Sciences, Bristol Heart Institute, University of Bristol, Bristol, UK
| | - Gianni D Angelini
- Translational Health Sciences, Bristol Heart Institute, University of Bristol, Bristol, UK
| |
Collapse
|
15
|
Dimagli A, Benedetto U. Commentary: "Do or do not. There is no try": Which role for minimally invasive direct coronary artery bypass? J Thorac Cardiovasc Surg 2023; 165:131-132. [PMID: 33642104 DOI: 10.1016/j.jtcvs.2021.01.130] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/29/2021] [Revised: 01/29/2021] [Accepted: 01/29/2021] [Indexed: 12/16/2022]
Affiliation(s)
- Arnaldo Dimagli
- Bristol Heart Institute, University of Bristol, Bristol, United Kingdom
| | - Umberto Benedetto
- Bristol Heart Institute, University of Bristol, Bristol, United Kingdom.
| |
Collapse
|
16
|
Sperrin M, Riley RD, Collins GS, Martin GP. Targeted validation: validating clinical prediction models in their intended population and setting. Diagn Progn Res 2022; 6:24. [PMID: 36550534 PMCID: PMC9773429 DOI: 10.1186/s41512-022-00136-8] [Citation(s) in RCA: 14] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/24/2022] [Accepted: 11/14/2022] [Indexed: 12/24/2022] Open
Abstract
Clinical prediction models must be appropriately validated before they can be used. While validation studies are sometimes carefully designed to match an intended population/setting of the model, it is common for validation studies to take place with arbitrary datasets, chosen for convenience rather than relevance. We call estimating how well a model performs within the intended population/setting "targeted validation". Use of this term sharpens the focus on the intended use of a model, which may increase the applicability of developed models, avoid misleading conclusions, and reduce research waste. It also exposes that external validation may not be required when the intended population for the model matches the population used to develop the model; here, a robust internal validation may be sufficient, especially if the development dataset was large.
Collapse
Affiliation(s)
- Matthew Sperrin
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, University of Manchester, Manchester, UK.
| | - Richard D Riley
- Institute of Applied Health Research, College of Medical and Dental Sciences, University of Birmingham, Birmingham, UK
| | - Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Glen P Martin
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, University of Manchester, Manchester, UK
| |
Collapse
|
17
|
Lin Y, Shao H, Shi L, Anderson AH, Fonseca V. Predicting incident heart failure among patients with type 2 diabetes mellitus: The DM-CURE risk score. Diabetes Obes Metab 2022; 24:2203-2211. [PMID: 35801340 DOI: 10.1111/dom.14806] [Citation(s) in RCA: 6] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/11/2022] [Revised: 06/20/2022] [Accepted: 07/01/2022] [Indexed: 11/30/2022]
Abstract
AIM Early identification and prediction of incident heart failure (HF) is important because of severe morbidity and mortality. This study aimed to predict onset of HF among patients with diabetes. METHODS A time-varying Cox model was derived from ACCORD clinical trial to predict the risk of incident HF, defined by hospitalization for HF (HHF). External validation was performed on patient-level data from the Harmony Outcome trial and Chronic Renal Insufficiency Cohort (CRIC) study. The model was transformed into an integer-based scoring algorithm for 10-year risk evaluation. A stepwise algorithm identified and selected predictors from demographic characteristics, physical examination, laboratory results, medical history, medication and health care utilization, to develop a risk prediction model. The main outcome was incident HF, defined by HHF. The C statistic and Brier score were used to assess model performance. RESULTS In total, 9649 patients with diabetes free of HF were used, with median follow-up of 4 years and 299 incident hospitalization of HF events. The model identified several predictors for the 10-year HF incidence risk score 'DM-CURE': socio-Demographic [education, age at type 2 diabetes (T2DM) diagnosis], Metabolic (glycated haemoglobin, systolic blood pressure, body mass index, high-density lipoproteins), diabetes-related Complications (myocardial infarction, revascularization, cardiovascular medications, neuropathy, hypertension duration, albuminuria, urine albumin-to-creatinine ratio, End Stage Kidney Disease), and health care Utilization (all-cause hospitalization, emergency room visits) for Risk Evaluation. Among them, the strongest impact factors for future HF were age at T2DM diagnosis, health care utilization and cardiovascular disease-related variables. The model showed good discrimination (C statistic: 0.838, 95% CI: 0.821-0.855) and calibration (Brier score: 0.006, 95% CI: 0.006-0.007) in the ACCORD data and good performance in the validation data (Harmony: C statistic: 0.881, 95% CI: 0.863-0.899; CRIC: C statistic: 0.813, 95% CI: 0.794-0.833). The 10-year risk of incident HF increased in a graded fashion, from ≤1% in quintile 1 (score ≤14), 1%-5% in quintile 2 (score 15-23), 5%-10% in quintile 3 (score 24-27), 10%-20% in quintile 4 (score 28-33) and ≥20% in quintile 5 (score >33). CONCLUSIONS The DM-CURE model and score were useful for population risk stratification of incident HHF among patients with T2DM and can be easily applied in clinical practice.
Collapse
Affiliation(s)
- Yilu Lin
- Department of Health Policy and Management, School of Public Health and Tropical Medicine, Tulane University, New Orleans, Louisiana, United States
| | - Hui Shao
- Department of Pharmaceutical Outcomes and Policy, College of Pharmacy, University of Florida, Gainesville, Florida, United States
| | - Lizheng Shi
- Department of Health Policy and Management, School of Public Health and Tropical Medicine, Tulane University, New Orleans, Louisiana, United States
| | - Amanda H Anderson
- Department of Epidemiology, School of Public Health and Tropical Medicine, Tulane University, New Orleans, Louisiana, United States
| | - Vivian Fonseca
- Department of Medicine and Pharmacology, School of Medicine, Tulane University, New Orleans, Louisiana, United States
| |
Collapse
|
18
|
Mejia OAV, Borgomoni GB, Palma Dallan LR, Mioto BM, Duenhas Accorsi TA, Lima EG, de Matos Soeiro A, Lima FG, Manuel de Almeida Brandão C, Alberto Pomerantzeff PM, Oliveira Dallan LA, Ferreira Lisboa LA, Jatene FB. Quality improvement program at Latin America. Int J Surg 2022; 106:106931. [PMID: 36126857 DOI: 10.1016/j.ijsu.2022.106931] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/18/2022] [Revised: 09/06/2022] [Accepted: 09/07/2022] [Indexed: 11/15/2022]
Abstract
BACKGROUND The current challenge of cardiac surgery (CS) is to improve outcomes in adverse scenarios. The aim of this study was to assess the impact of a quality improvement program (QIP) on hospital mortality in the largest CS center in Latin America. METHODS Patients were divided into two groups: before (Jan 2013-Dec 2015, n = 3534) and after establishment of the QIP (Jan 2017-Dec 2019, n = 3544). The QIP consisted of the implementation of 10 central initiatives during 2016. The procedures evaluated were isolated coronary artery bypass grafting surgery (CABG), mitral valve surgery, aortic valve surgery, combined mitral and aortic valve surgery, and CABG associated with heart valve surgery. Propensity Score Matching (PSM) was used to adjust for inequality in patients' preoperative characteristics before and after the implementation of QIP. A multivariate logistic regression model was built to predict hospital mortality and validated using discrimination and calibration metrics. RESULTS The PMS paired two groups using 5 variables, obtaining 858 patients operated before (non-QIP) and 858 patients operated after the implementation of the QIP. When comparing the QIP versus Non-QIP group, there was a shorter length of stay in all phases of hospitalization. In addition, the patients evolved with less anemia (P = 0.001), use of intra-aortic balloon pump (P = 0.003), atrial fibrillation (P = 0.001), acute kidney injury (P < 0.001), cardiogenic shock (P = 0.011), sepsis (P = 0.046), and hospital mortality (P = 0.001). In the multiple model, among the predictors of hospital mortality, the lack of QIP increased the chances of mortality by 2.09 times. CONCLUSION The implementation of a first CS QIP in Latin America was associated with a reduction in length of hospital stay, complications and mortality after the cardiac surgeries analyzed.
Collapse
Affiliation(s)
- Omar Asdrúbal Vilca Mejia
- Department of Cardiopneumology, Instituto do Coração, Hospital das Clinicas, Faculdade de Medicina, Universidade de São Paulo (HCFMUSP), São Paulo, SP, Brazil.
| | - Gabrielle Barbosa Borgomoni
- Department of Cardiopneumology, Instituto do Coração, Hospital das Clinicas, Faculdade de Medicina, Universidade de São Paulo (HCFMUSP), São Paulo, SP, Brazil
| | - Luís Roberto Palma Dallan
- Department of Cardiopneumology, Instituto do Coração, Hospital das Clinicas, Faculdade de Medicina, Universidade de São Paulo (HCFMUSP), São Paulo, SP, Brazil
| | - Bruno Mahler Mioto
- Department of Cardiopneumology, Instituto do Coração, Hospital das Clinicas, Faculdade de Medicina, Universidade de São Paulo (HCFMUSP), São Paulo, SP, Brazil
| | - Tarso Augusto Duenhas Accorsi
- Department of Cardiopneumology, Instituto do Coração, Hospital das Clinicas, Faculdade de Medicina, Universidade de São Paulo (HCFMUSP), São Paulo, SP, Brazil
| | - Eduardo Gomes Lima
- Department of Cardiopneumology, Instituto do Coração, Hospital das Clinicas, Faculdade de Medicina, Universidade de São Paulo (HCFMUSP), São Paulo, SP, Brazil
| | - Alexandre de Matos Soeiro
- Department of Cardiopneumology, Instituto do Coração, Hospital das Clinicas, Faculdade de Medicina, Universidade de São Paulo (HCFMUSP), São Paulo, SP, Brazil
| | - Felipe Gallego Lima
- Department of Cardiopneumology, Instituto do Coração, Hospital das Clinicas, Faculdade de Medicina, Universidade de São Paulo (HCFMUSP), São Paulo, SP, Brazil
| | - Carlos Manuel de Almeida Brandão
- Department of Cardiopneumology, Instituto do Coração, Hospital das Clinicas, Faculdade de Medicina, Universidade de São Paulo (HCFMUSP), São Paulo, SP, Brazil
| | - Pablo Maria Alberto Pomerantzeff
- Department of Cardiopneumology, Instituto do Coração, Hospital das Clinicas, Faculdade de Medicina, Universidade de São Paulo (HCFMUSP), São Paulo, SP, Brazil
| | - Luís Alberto Oliveira Dallan
- Department of Cardiopneumology, Instituto do Coração, Hospital das Clinicas, Faculdade de Medicina, Universidade de São Paulo (HCFMUSP), São Paulo, SP, Brazil
| | - Luiz Augusto Ferreira Lisboa
- Department of Cardiopneumology, Instituto do Coração, Hospital das Clinicas, Faculdade de Medicina, Universidade de São Paulo (HCFMUSP), São Paulo, SP, Brazil
| | - Fábio Biscegli Jatene
- Department of Cardiopneumology, Instituto do Coração, Hospital das Clinicas, Faculdade de Medicina, Universidade de São Paulo (HCFMUSP), São Paulo, SP, Brazil
| |
Collapse
|
19
|
Davis SE, Walsh CG, Matheny ME. Open questions and research gaps for monitoring and updating AI-enabled tools in clinical settings. Front Digit Health 2022; 4:958284. [PMID: 36120717 PMCID: PMC9478183 DOI: 10.3389/fdgth.2022.958284] [Citation(s) in RCA: 4] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/31/2022] [Accepted: 08/11/2022] [Indexed: 11/15/2022] Open
Abstract
As the implementation of artificial intelligence (AI)-enabled tools is realized across diverse clinical environments, there is a growing understanding of the need for ongoing monitoring and updating of prediction models. Dataset shift-temporal changes in clinical practice, patient populations, and information systems-is now well-documented as a source of deteriorating model accuracy and a challenge to the sustainability of AI-enabled tools in clinical care. While best practices are well-established for training and validating new models, there has been limited work developing best practices for prospective validation and model maintenance. In this paper, we highlight the need for updating clinical prediction models and discuss open questions regarding this critical aspect of the AI modeling lifecycle in three focus areas: model maintenance policies, performance monitoring perspectives, and model updating strategies. With the increasing adoption of AI-enabled tools, the need for such best practices must be addressed and incorporated into new and existing implementations. This commentary aims to encourage conversation and motivate additional research across clinical and data science stakeholders.
Collapse
Affiliation(s)
- Sharon E. Davis
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States,Correspondence: Sharon E. Davis
| | - Colin G. Walsh
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States,Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, United States,Department of Psychiatry, Vanderbilt University Medical Center, Nashville, TN, United States
| | - Michael E. Matheny
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, United States,Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, United States,Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, United States,Tennessee Valley Healthcare System VA Medical Center, Veterans Health Administration, Nashville, TN, United States
| |
Collapse
|
20
|
Davis SE, Brown JR, Dorn C, Westerman D, Solomon RJ, Matheny ME. Maintaining a National Acute Kidney Injury Risk Prediction Model to Support Local Quality Benchmarking. Circ Cardiovasc Qual Outcomes 2022; 15:e008635. [PMID: 35959674 PMCID: PMC9388604 DOI: 10.1161/circoutcomes.121.008635] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/16/2022]
Abstract
BACKGROUND The utility of quality dashboards to inform decision-making and improve clinical outcomes is tightly linked to the accuracy of the information they provide and, in turn, accuracy of underlying prediction models. Despite recognition of the need to update prediction models to maintain accuracy over time, there is limited guidance on updating strategies. We compare predefined and surveillance-based updating strategies applied to a model supporting quality evaluations among US veterans. METHODS We evaluated the performance of a US Department of Veterans Affairs-specific model for postcardiac catheterization acute kidney injury using routinely collected observational data over the 6 years following model development (n=90 295 procedures in 2013-2019). Predicted probabilities were generated from the original model, an annually retrained model, and a surveillance-based approach that monitored performance to inform the timing and method of updates. We evaluated how updating the national model impacted regional quality profiles. We compared observed-to-expected outcome ratios, where values above and below 1 indicated more and fewer adverse outcomes than expected, respectively. RESULTS The original model overpredicted risk at the national level (observed-to-expected outcome ratio, 0.75 [0.74-0.77]). Annual retraining updated the model 5×; surveillance-based updating retrained once and recalibrated twice. While both strategies improved performance, the surveillance-based approach provided superior calibration (observed-to-expected outcome ratio, 1.01 [0.99-1.03] versus 0.94 [0.92-0.96]). Overprediction by the original model led to optimistic quality assessments, incorrectly indicating most of the US Department of Veterans Affairs' 18 regions observed fewer acute kidney injury events than predicted. Both updating strategies revealed 16 regions performed as expected and 2 regions increasingly underperformed, having more acute kidney injury events than predicted. CONCLUSIONS Miscalibrated clinical prediction models provide inaccurate pictures of performance across clinical units, and degrading calibration further complicates our understanding of quality. Updating strategies tailored to health system needs and capacity should be incorporated into model implementation plans to promote the utility and longevity of quality reporting tools.
Collapse
Affiliation(s)
- Sharon E. Davis
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN
| | - Jeremiah R. Brown
- Departments of Epidemiology and Biomedical Data Science, Dartmouth Geisel School of Medicine, Hanover, NH
| | - Chad Dorn
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN
| | - Dax Westerman
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN
| | - Richard J. Solomon
- Department of Medicine, Larner College of Medicine, University of Vermont, Burlington, VT
| | - Michael E. Matheny
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, USA
- Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, USA
- Tennessee Valley Healthcare System VA Medical Center, Veterans Health Administration, Nashville, TN, USA
| |
Collapse
|
21
|
Reynard C, Jenkins D, Martin GP, Kontopantelis E, Body R. Is your clinical prediction model past its sell by date? Arch Emerg Med 2022; 39:956-958. [DOI: 10.1136/emermed-2021-212224] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/07/2021] [Accepted: 06/22/2022] [Indexed: 11/04/2022]
|
22
|
Kurniawaty J, Setianto BY, Supomo, Widyastuti Y, Boom CE. The Effect of Low Preoperative Ejection Fraction on Mortality After Cardiac Surgery in Indonesia. Vasc Health Risk Manag 2022; 18:131-137. [PMID: 35356550 PMCID: PMC8959716 DOI: 10.2147/vhrm.s350671] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/01/2021] [Accepted: 03/10/2022] [Indexed: 12/02/2022] Open
Abstract
Background Among cardiac surgery patients, low preoperative left ventricular ejection fraction (LVEF) is common and has been associated with poor outcomes. The objective of this study was to assess the association between LVEF and postoperative mortality in patients undergoing open-heart surgery in several hospitals in Indonesia. Methods We conducted a multicenter study with the retrospective design using data from patients undergoing open-heart surgery in 4 institutions in Indonesia. Data regarding LVEF and other potential risk factors were extracted from medical records and compiled in one datasheet. Statistical analyses were performed to assess if low LVEF was associated with postoperative mortality and identify other potential risk factors. Results A total of 4789 patients underwent cardiac surgery in participating centers during the study period. Of these, 189 subjects (3.9%) had poor preoperative LVEF. Poor LVEF was associated with postoperative mortality (adjusted OR 2.761, 95% CI 1.763–4.323, p < 0.001). Based on types of surgery, LVEF had a significant association with mortality only in CABG patients, while there was no such association in valve surgery and inconclusive in congenital surgery patients. Other significant independent predictors of in-hospital mortality included age more than 65 years old, non-elective surgery, the complexity of procedures, history of cardiac surgery, organ failure, CARE score ≥ 3, NYHA class ≥ III, and poor right ventricular function. Conclusion Patients with low preoperative LVEF undergoing open-heart surgery had a higher risk of postoperative mortality. Cardiac surgery can be performed with acceptable mortality rates. Accurate selection of patients, risk/benefit evaluation, and planning of surgical and anesthesiological management are mandatory to improve outcomes.
Collapse
Affiliation(s)
- Juni Kurniawaty
- Faculty of Medicine, Public Health, and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia
| | - Budi Yuli Setianto
- Faculty of Medicine, Public Health, and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia
| | - Supomo
- Faculty of Medicine, Public Health, and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia
| | - Yunita Widyastuti
- Faculty of Medicine, Public Health, and Nursing, Universitas Gadjah Mada, Yogyakarta, Indonesia
| | - Cindy E Boom
- Harapan Kita National Heart Center, Jakarta, Indonesia
| |
Collapse
|
23
|
Chi S, Tian Y, Wang F, Zhou T, Jin S, Li J. A novel lifelong machine learning-based method to eliminate calibration drift in clinical prediction models. Artif Intell Med 2022; 125:102256. [DOI: 10.1016/j.artmed.2022.102256] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2021] [Revised: 01/14/2022] [Accepted: 02/09/2022] [Indexed: 02/03/2023]
|
24
|
Franeková J, Hošková L, Jabor A. Galectin-3 as an independent prognostic factor after heart transplantation. Clin Transplant 2022; 36:e14592. [PMID: 35029311 DOI: 10.1111/ctr.14592] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/14/2021] [Revised: 01/07/2022] [Accepted: 01/10/2022] [Indexed: 11/30/2022]
Abstract
BACKGROUND Galectin-3 (GAL3) is linked to the prognosis of patients with heart failure and after heart transplantation (HTx). We assessed the prognostic role of GAL3 in a long-term follow-up after HTx. METHODS HTx patients (N = 121) were evaluated in a single-center, noninterventional, prospective, observational study. The median follow-up was 96 months (2,942 days, interquartile range (IQR) 2,408-3,264 days), and 40 patients died. GAL3 was measured before HTx, +10 days after HTx, and during the first posttransplant year. Survival analysis (all-cause mortality) was performed with adjustments for clinical and laboratory variables. RESULTS The median pretransplant GAL3 level was 18.0 μg/L (IQR 14.0-25.9), and higher values were associated with older age, worse kidney function, left ventricular assist device use before HTx, a higher IMPACT score and mortality. Increased pretransplant GAL3 predicted shorter survival time (HR 2.05, 95% CI 1.09-3.85, p<0.05). Similar prognostic power had GAL3 on the 10th posttransplant day (HR 2.03, 95% CI 1.08-3.82, p<0.05). GAL3 was an independent predictor of death after adjustment for clinical variables (age, infection, diabetes, smoking, IMPACT score, troponin). CONCLUSIONS GAL3 was significantly associated with all-cause mortality after adjusting for clinical and laboratory variables and may serve as an additional prognostic biomarker. This article is protected by copyright. All rights reserved.
Collapse
Affiliation(s)
- Janka Franeková
- Department of Laboratory Methods, Institute for Clinical and Experimental Medicine, Prague, Czech Republic.,3rd Faculty of Medicine, Charles University, Prague, Czech Republic
| | - Lenka Hošková
- Heart Centre, Institute for Clinical and Experimental Medicine, Prague, Czech Republic
| | - Antonín Jabor
- Department of Laboratory Methods, Institute for Clinical and Experimental Medicine, Prague, Czech Republic.,3rd Faculty of Medicine, Charles University, Prague, Czech Republic
| |
Collapse
|
25
|
D'Angelo AM, Nemeth S, Wang C, Kossar AP, Takeda K, Takayama H, Bapat V, Naka Y, Argenziano M, Smith CR, Beck J, Spellman J, Kurlansky P, George I. Re-dosing of del Nido cardioplegia in adult cardiac surgery requiring prolonged aortic cross-clamp. Interact Cardiovasc Thorac Surg 2021; 34:556-563. [PMID: 34788429 PMCID: PMC8972223 DOI: 10.1093/icvts/ivab310] [Citation(s) in RCA: 6] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/06/2020] [Revised: 08/24/2021] [Accepted: 09/20/2021] [Indexed: 11/18/2022] Open
Affiliation(s)
- Alex M D'Angelo
- Department of Surgery, Columbia University Medical Center, New York, NY, USA
| | - Samantha Nemeth
- Center for Innovation and Outcomes Research, Columbia University, New York, NY, USA
| | - Catherine Wang
- Department of Surgery, Columbia University Medical Center, New York, NY, USA
| | - Alexander P Kossar
- Department of Surgery, Columbia University Medical Center, New York, NY, USA
| | - Koji Takeda
- Department of Surgery, Columbia University Medical Center, New York, NY, USA
| | - Hiroo Takayama
- Department of Surgery, Columbia University Medical Center, New York, NY, USA
| | - Vinayak Bapat
- Department of Cardiothoracic Surgery, Abbott Northwestern Hospital, Minneapolis, MN, USA
| | - Yoshifumi Naka
- Department of Surgery, Columbia University Medical Center, New York, NY, USA
| | - Michael Argenziano
- Department of Surgery, Columbia University Medical Center, New York, NY, USA
| | - Craig R Smith
- Department of Surgery, Columbia University Medical Center, New York, NY, USA
| | - James Beck
- Department of Surgery, Columbia University Medical Center, New York, NY, USA
| | - Jessica Spellman
- Department of Anesthesiology, Columbia University Medical Center, New York, NY, USA
| | - Paul Kurlansky
- Department of Surgery, Columbia University Medical Center, New York, NY, USA
| | - Isaac George
- Department of Surgery, Columbia University Medical Center, New York, NY, USA
| |
Collapse
|
26
|
Antoniou T, Mamdani M. Évaluation des solutions fondées sur l’apprentissage machine en santé. CMAJ 2021; 193:E1720-E1724. [PMID: 34750185 PMCID: PMC8584374 DOI: 10.1503/cmaj.210036-f] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/23/2022] Open
Affiliation(s)
- Tony Antoniou
- Centre de recherche et de formation en analytique des soins de santé Li Ka Shing (Antoniou, Mamdani), Réseau hospitalier Unity Health de Toronto; Institut du savoir Li Ka Shing (Antoniou, Mamdani), Réseau hospitalier Unity Health de Toronto; Département de médecine de famille et communautaire (Antoniou), Réseau hospitalier Unity Health deToronto et Université de Toronto; Faculté de médecine Temerty (Mamdani) et Faculté de pharmacie Leslie Dan (Mamdani), Université de Toronto; Institut des politiques, de la gestion et de l'évaluation de la santé (Mamdani), Université de Toronto, Toronto, Ont.
| | - Muhammad Mamdani
- Centre de recherche et de formation en analytique des soins de santé Li Ka Shing (Antoniou, Mamdani), Réseau hospitalier Unity Health de Toronto; Institut du savoir Li Ka Shing (Antoniou, Mamdani), Réseau hospitalier Unity Health de Toronto; Département de médecine de famille et communautaire (Antoniou), Réseau hospitalier Unity Health deToronto et Université de Toronto; Faculté de médecine Temerty (Mamdani) et Faculté de pharmacie Leslie Dan (Mamdani), Université de Toronto; Institut des politiques, de la gestion et de l'évaluation de la santé (Mamdani), Université de Toronto, Toronto, Ont
| |
Collapse
|
27
|
Reynard C, Martin GP, Kontopantelis E, Jenkins DA, Heagerty A, McMillan B, Jafar A, Garlapati R, Body R. Advanced cardiovascular risk prediction in the emergency department: updating a clinical prediction model - a large database study protocol. Diagn Progn Res 2021; 5:16. [PMID: 34620253 PMCID: PMC8499458 DOI: 10.1186/s41512-021-00105-7] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 07/27/2021] [Accepted: 09/27/2021] [Indexed: 12/23/2022] Open
Abstract
BACKGROUND Patients presenting with chest pain represent a large proportion of attendances to emergency departments. In these patients clinicians often consider the diagnosis of acute myocardial infarction (AMI), the timely recognition and treatment of which is clinically important. Clinical prediction models (CPMs) have been used to enhance early diagnosis of AMI. The Troponin-only Manchester Acute Coronary Syndromes (T-MACS) decision aid is currently in clinical use across Greater Manchester. CPMs have been shown to deteriorate over time through calibration drift. We aim to assess potential calibration drift with T-MACS and compare methods for updating the model. METHODS We will use routinely collected electronic data from patients who were treated using TMACS at two large NHS hospitals. This is estimated to include approximately 14,000 patient episodes spanning June 2016 to October 2020. The primary outcome of acute myocardial infarction will be sourced from NHS Digital's admitted patient care dataset. We will assess the calibration drift of the existing model and the benefit of updating the CPM by model recalibration, model extension and dynamic updating. These models will be validated by bootstrapping and one step ahead prequential testing. We will evaluate predictive performance using calibrations plots and c-statistics. We will also examine the reclassification of predicted probability with the updated TMACS model. DISCUSSION CPMs are widely used in modern medicine, but are vulnerable to deteriorating calibration over time. Ongoing refinement using routinely collected electronic data will inevitably be more efficient than deriving and validating new models. In this analysis we will seek to exemplify methods for updating CPMs to protect the initial investment of time and effort. If successful, the updating methods could be used to continually refine the algorithm used within TMACS, maintaining or even improving predictive performance over time. TRIAL REGISTRATION ISRCTN number: ISRCTN41008456.
Collapse
Affiliation(s)
- Charles Reynard
- grid.5379.80000000121662407Division of Cardiovascular Sciences, University of Manchester, Manchester, UK
- grid.498924.aEmergency Department, Manchester University NHS Foundation Trust, Manchester, UK
| | - Glen P. Martin
- grid.5379.80000000121662407Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Evangelos Kontopantelis
- grid.5379.80000000121662407Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - David A. Jenkins
- grid.5379.80000000121662407Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Anthony Heagerty
- grid.5379.80000000121662407Division of Cardiovascular Sciences, University of Manchester, Manchester, UK
| | - Brian McMillan
- Centre for Primary Care and Health Services Research Division of Population Health, Health Services Research and Primary Care School of Health Sciences Faculty of Biology, Medicine and Health University of Manchestern, Manchester, UK
| | - Anisa Jafar
- grid.5379.80000000121662407Humanitarian and Conflict Response Institute, University of Manchester, Manchester, UK
| | - Rajendar Garlapati
- grid.439642.e0000 0004 0489 3782Emergency Department, Royal Blackburn Hospital, East Lancashire Hospitals NHS Trust, Burnley, UK
| | - Richard Body
- grid.5379.80000000121662407Division of Cardiovascular Sciences, University of Manchester, Manchester, UK
- grid.498924.aEmergency Department, Manchester University NHS Foundation Trust, Manchester, UK
| |
Collapse
|
28
|
Affiliation(s)
- Tony Antoniou
- Li Ka Shing Centre for Healthcare Analytics Research & Training (Antoniou, Mamdani), Unity Health Toronto; Li Ka Shing Knowledge Institute (Antoniou, Mamdani), Unity Health Toronto; Department of Family and Community Medicine (Antoniou), Unity Health Toronto and University of Toronto; Temerty Faculty of Medicine (Mamdani) and Leslie Dan Faculty of Pharmacy (Mamdani), University of Toronto; Institute of Health Policy, Management, and Evaluation (Mamdani), University of Toronto, Toronto, Ont.
| | - Muhammad Mamdani
- Li Ka Shing Centre for Healthcare Analytics Research & Training (Antoniou, Mamdani), Unity Health Toronto; Li Ka Shing Knowledge Institute (Antoniou, Mamdani), Unity Health Toronto; Department of Family and Community Medicine (Antoniou), Unity Health Toronto and University of Toronto; Temerty Faculty of Medicine (Mamdani) and Leslie Dan Faculty of Pharmacy (Mamdani), University of Toronto; Institute of Health Policy, Management, and Evaluation (Mamdani), University of Toronto, Toronto, Ont
| |
Collapse
|
29
|
Abbasciano RG, Lai FY, Roman MA, Rizzello A, Pathak S, Ramzi J, Lucarelli C, Layton GR, Kumar T, Wozniak MJ, Eagle-Hemming B, Akowuah E, Rogers CA, Angelini GD, Murphy GJ. Activation of the innate immune response and organ injury after cardiac surgery: a systematic review and meta-analysis of randomised trials and analysis of individual patient data from randomised and non-randomised studies. Br J Anaesth 2021; 127:365-375. [PMID: 34229833 DOI: 10.1016/j.bja.2021.04.032] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/18/2020] [Revised: 04/14/2021] [Accepted: 04/15/2021] [Indexed: 10/20/2022] Open
Abstract
BACKGROUND It is unclear whether the innate immune response represents a therapeutic target for organ protection strategies in cardiac surgery. METHODS A systematic review of trials of interventions targeting the inflammatory response to cardiac surgery reporting treatment effects on both innate immune system cytokines and organ injury was performed. The protocol was registered at the International Prospective Register of Systematic Reviews: CRD42020187239. Searches of the Cochrane Central Register of Controlled Trials, MEDLINE, and Embase were performed. Random-effects meta-analyses were used for the primary analysis. A separate analysis of individual patient data from six studies (n=785) explored sources of heterogeneity for treatment effects on cytokine levels. RESULTS Searches to May 2020 identified 251 trials evaluating 24 interventions with 20 582 participants for inclusion. Most trials had important limitations. Methodological limitations of the included trials and heterogeneity of the treatment effects on cytokine levels between trials limited interpretation. The primary analysis demonstrated inconsistency in the direction of the treatment effects on innate immunity and organ failure or death between interventions. Analyses restricted to important subgroups or trials with fewer limitations showed similar results. Meta-regression, pooling available data from all trials, demonstrated no association between the direction of the treatment effects on inflammatory cytokines and organ injury or death. The analysis of individual patient data demonstrated heterogeneity in the association between the cytokine response and organ injury after cardiac surgery for people >75 yr old and those with some chronic diseases. CONCLUSIONS The certainty of the evidence for a causal relationship between innate immune system activation and organ injury after cardiac surgery is low.
Collapse
Affiliation(s)
| | - Florence Y Lai
- Department of Cardiovascular Sciences, University of Leicester, Leicester, UK
| | - Marius A Roman
- Department of Cardiovascular Sciences, University of Leicester, Leicester, UK
| | - Angelica Rizzello
- Department of Cardiovascular Sciences, University of Leicester, Leicester, UK
| | - Suraj Pathak
- Department of Cardiovascular Sciences, University of Leicester, Leicester, UK
| | - Joussi Ramzi
- Leicester Medical School, University of Leicester, Leicester, UK
| | - Carla Lucarelli
- Department of Cardiac Surgery, University of Verona, Verona, Italy
| | | | - Tracy Kumar
- Department of Cardiovascular Sciences, University of Leicester, Leicester, UK
| | - Marcin J Wozniak
- Department of Cardiovascular Sciences, University of Leicester, Leicester, UK
| | | | - Enoch Akowuah
- South Tees Hospitals NHS Foundation Trust, Middlesbrough, UK
| | - Chris A Rogers
- Clinical Trials and Evaluation Unit, Bristol Trials Centre, University of Bristol, Bristol, UK
| | | | - Gavin J Murphy
- Department of Cardiovascular Sciences, University of Leicester, Leicester, UK
| |
Collapse
|
30
|
Ohri SK, Benedetto U, Luthra S, Grant SW, Goodwin AT, Trivedi U, Kendall S, Jenkins DP. Coronary artery bypass surgery in the UK, trends in activity and outcomes from a 15-year complete national series. Eur J Cardiothorac Surg 2021; 61:449-456. [PMID: 34448848 DOI: 10.1093/ejcts/ezab391] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/20/2021] [Revised: 07/07/2021] [Accepted: 07/27/2021] [Indexed: 11/14/2022] Open
Abstract
OBJECTIVES The aim of this study was to review the UK national trends in activity and outcome in coronary artery bypass graft (CABG) over a 15-year period (2002-2016). METHODS Validated data collected (2002-2016) and uploaded to National Institute for Cardiovascular Outcomes Research were used to generate summary data from the National Adult Cardiac Surgery Audit Database for the analysis. Logistic European System of Cardiac Operative Risk Evaluation was used for risk stratification with recalibration applied for governance. Data were analysed by financial year and presented as numerical, categorical, %, mean and standard deviation where appropriate. Mortality was recorded as death in hospital at any time after index CABG operation. RESULTS A total of 347 626 CABG procedures (282 883 isolated CABG, 61 109 CABG and valve and 4132 redo CABG) were recorded. Over this period annual activity reduced from 66.6% of workload to 41.7%. The mean age for isolated CABG was 65.7 years. The mean log European System of Cardiac Operative Risk Evaluation was 3.1, 5.9 and 23.2 for elective, urgent and emergency isolated CABG, respectively. There was a decline in the observed mortality for all procedures. Overall mortality for isolated CABG surgery is now 1.0% and only 0.6% for elective operations. CONCLUSIONS Quality of care and risk-adjusted mortality rates have consistently improved over the last 15 years despite the increasing risk profile of patients. There have been a consistent decline in overall case volumes and a three-fold increase in elderly cases.
Collapse
Affiliation(s)
- Sunil K Ohri
- Division of Cardiac Surgery, University Hospital Southampton NHS Foundation Trust, Southampton, UK
| | - Umberto Benedetto
- Division of Cardiac Surgery, University of Bristol and Bristol Royal Infirmary, Bristol, UK
| | - Suvitesh Luthra
- Division of Cardiac Surgery, University Hospital Southampton NHS Foundation Trust, Southampton, UK
| | - Stuart W Grant
- Society for Cardiothoracic Surgery in Great Britain and Ireland, London, UK.,Division of Cardiovascular Sciences, University of Manchester, Manchester, UK
| | - Andrew T Goodwin
- Society for Cardiothoracic Surgery in Great Britain and Ireland, London, UK.,Division of Cardiac Surgery, James Cook University Hospital, Middlesborough, UK
| | - Uday Trivedi
- Society for Cardiothoracic Surgery in Great Britain and Ireland, London, UK.,Division of Cardiac Surgery, Brighton and Sussex University Hospitals NHS Trust, Brighton, UK
| | - Simon Kendall
- Society for Cardiothoracic Surgery in Great Britain and Ireland, London, UK.,Division of Cardiac Surgery, James Cook University Hospital, Middlesborough, UK
| | - David P Jenkins
- Society for Cardiothoracic Surgery in Great Britain and Ireland, London, UK.,Division of Cardiac Surgery, Royal Papworth Hospital NHS Foundation Trust, Cambridge, UK
| |
Collapse
|
31
|
Taylor M, Hashmi SF, Martin GP, Shackcloth M, Shah R, Booton R, Grant SW. A systematic review of risk prediction models for perioperative mortality after thoracic surgery. Interact Cardiovasc Thorac Surg 2021; 32:333-342. [PMID: 33257987 DOI: 10.1093/icvts/ivaa273] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/02/2020] [Revised: 10/05/2020] [Accepted: 10/13/2020] [Indexed: 11/13/2022] Open
Abstract
OBJECTIVES Guidelines advocate that patients being considered for thoracic surgery should undergo a comprehensive preoperative risk assessment. Multiple risk prediction models to estimate the risk of mortality after thoracic surgery have been developed, but their quality and performance has not been reviewed in a systematic way. The objective was to systematically review these models and critically appraise their performance. METHODS The Cochrane Library and the MEDLINE database were searched for articles published between 1990 and 2019. Studies that developed or validated a model predicting perioperative mortality after thoracic surgery were included. Data were extracted based on the checklist for critical appraisal and data extraction for systematic reviews of prediction modelling studies. RESULTS A total of 31 studies describing 22 different risk prediction models were identified. There were 20 models developed specifically for thoracic surgery with two developed in other surgical specialties. A total of 57 different predictors were included across the identified models. Age, sex and pneumonectomy were the most frequently included predictors in 19, 13 and 11 models, respectively. Model performance based on either discrimination or calibration was inadequate for all externally validated models. The most recent data included in validation studies were from 2018. Risk of bias (assessed using Prediction model Risk Of Bias ASsessment Tool) was high for all except two models. CONCLUSIONS Despite multiple risk prediction models being developed to predict perioperative mortality after thoracic surgery, none could be described as appropriate for contemporary thoracic surgery. Contemporary validation of available models or new model development is required to ensure that appropriate estimates of operative risk are available for contemporary thoracic surgical practice.
Collapse
Affiliation(s)
- Marcus Taylor
- Department of Cardiothoracic Surgery, Wythenshawe Hospital, Manchester University Hospital Foundation Trust, Manchester, UK
| | - Syed F Hashmi
- Department of Cardiothoracic Surgery, Wythenshawe Hospital, Manchester University Hospital Foundation Trust, Manchester, UK
| | - Glen P Martin
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, Manchester Academic Heath Science Centre, University of Manchester, Manchester, UK
| | - Michael Shackcloth
- Department of Cardiothoracic Surgery, Liverpool Heart and Chest Hospital, Liverpool, UK
| | - Rajesh Shah
- Department of Cardiothoracic Surgery, Wythenshawe Hospital, Manchester University Hospital Foundation Trust, Manchester, UK
| | - Richard Booton
- Department of Respiratory Medicine, Wythenshawe Hospital, Manchester University Hospital Foundation Trust, Manchester, UK
| | - Stuart W Grant
- Division of Cardiovascular Sciences, University of Manchester, ERC, Manchester University Hospitals Foundation Trust, Manchester, UK
| |
Collapse
|
32
|
Montgomery C, Stelfox H, Norris C, Rolfson D, Meyer S, Zibdawi M, Bagshaw S. Association between preoperative frailty and outcomes among adults undergoing cardiac surgery: a prospective cohort study. CMAJ Open 2021; 9:E777-E787. [PMID: 34285057 PMCID: PMC8313095 DOI: 10.9778/cmajo.20200034] [Citation(s) in RCA: 11] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/13/2022] Open
Abstract
BACKGROUND The identification of frailty before complex and invasive procedures may have relevance for prognostic and recovery purposes, to optimally inform patients, caregivers and clinicians about perioperative risk and postoperative care needs. The aim of this study was to estimate the prevalence of frailty and describe the associated clinical course and outcomes of patients referred for nonemergent cardiac surgery. METHODS A prospective cohort of patients aged 50 years and older referred for nonemergent cardiac surgery in Alberta, Canada, from November 2011 to March 2014 were screened preoperatively for frailty, defined as a Clinical Frailty Scale (CFS) score of 5 or greater. Postoperatively, patients were followed by telephone to assess CFS score, health services use and vital status. The primary outcome was all-cause hospital mortality. Secondary outcomes included health services use, hospital discharge disposition, 1-year health-related quality of life and all-cause 5-year mortality. RESULTS The cohort (n = 529) had a mean age of 67 (standard deviation [SD] 9) years; 25.9% were female, and the prevalence of frailty was 9.6% (n = 51; 95% confidence interval [CI] 7.3%-12.5%). Frail patients were older (median age 75, interquartile range [IQR] 65-80 v. 67, IQR 60-73, yr; p < 0.001), were more likely to be female (51.0% v. 23.2%; p < 0.001), had a higher mean EuroSCORE II (8, SD 3 v. 5, SD 3; p < 0.001) and received combined coronary artery bypass grafting and valve procedures more frequently (29.4% v. 15.9%; p = 0.02) than nonfrail patients. Postoperatively, frail patients had a longer median duration of stay in the cardiovascular intensive care unit (median difference 2.2, 95% CI 1.60-2.79) and hospital (median difference 9.3, 95% CI 8.2-10.3). Hospital mortality was 9.8% among frail patients and 1.0% among nonfrail patients (adjusted hazard ratio 3.84, 95% CI 0.90-16.34). INTERPRETATION Preoperative frailty was present in 10% of patients and was associated with a higher risk of morbidity and greater health services use. Preoperative frailty has important implications for the postoperative clinical course and resource utilization of patients undergoing cardiac surgery.
Collapse
Affiliation(s)
- Carmel Montgomery
- Department of Critical Care Medicine (Montgomery, Zibdawi, Bagshaw), Faculty of Medicine & Dentistry, University of Alberta, Edmonton, Alta.; Department of Critical Care Medicine (Stelfox), Cumming School of Medicine, and Department of Community Health Sciences (Stelfox), University of Calgary, Calgary, Alta.; Faculty of Nursing and School of Public Health (Norris, Bagshaw), University of Alberta; Strategic Clinical Networks (Norris, Bagshaw), Alberta Health Services; Division of Geriatric Medicine (Rolfson), Department of Medicine, and Division of Cardiac Surgery, (Meyer) Department of Surgery, Faculty of Medicine & Dentistry, University of Alberta, Edmonton, Alta.
| | - Henry Stelfox
- Department of Critical Care Medicine (Montgomery, Zibdawi, Bagshaw), Faculty of Medicine & Dentistry, University of Alberta, Edmonton, Alta.; Department of Critical Care Medicine (Stelfox), Cumming School of Medicine, and Department of Community Health Sciences (Stelfox), University of Calgary, Calgary, Alta.; Faculty of Nursing and School of Public Health (Norris, Bagshaw), University of Alberta; Strategic Clinical Networks (Norris, Bagshaw), Alberta Health Services; Division of Geriatric Medicine (Rolfson), Department of Medicine, and Division of Cardiac Surgery, (Meyer) Department of Surgery, Faculty of Medicine & Dentistry, University of Alberta, Edmonton, Alta
| | - Colleen Norris
- Department of Critical Care Medicine (Montgomery, Zibdawi, Bagshaw), Faculty of Medicine & Dentistry, University of Alberta, Edmonton, Alta.; Department of Critical Care Medicine (Stelfox), Cumming School of Medicine, and Department of Community Health Sciences (Stelfox), University of Calgary, Calgary, Alta.; Faculty of Nursing and School of Public Health (Norris, Bagshaw), University of Alberta; Strategic Clinical Networks (Norris, Bagshaw), Alberta Health Services; Division of Geriatric Medicine (Rolfson), Department of Medicine, and Division of Cardiac Surgery, (Meyer) Department of Surgery, Faculty of Medicine & Dentistry, University of Alberta, Edmonton, Alta
| | - Darryl Rolfson
- Department of Critical Care Medicine (Montgomery, Zibdawi, Bagshaw), Faculty of Medicine & Dentistry, University of Alberta, Edmonton, Alta.; Department of Critical Care Medicine (Stelfox), Cumming School of Medicine, and Department of Community Health Sciences (Stelfox), University of Calgary, Calgary, Alta.; Faculty of Nursing and School of Public Health (Norris, Bagshaw), University of Alberta; Strategic Clinical Networks (Norris, Bagshaw), Alberta Health Services; Division of Geriatric Medicine (Rolfson), Department of Medicine, and Division of Cardiac Surgery, (Meyer) Department of Surgery, Faculty of Medicine & Dentistry, University of Alberta, Edmonton, Alta
| | - Steven Meyer
- Department of Critical Care Medicine (Montgomery, Zibdawi, Bagshaw), Faculty of Medicine & Dentistry, University of Alberta, Edmonton, Alta.; Department of Critical Care Medicine (Stelfox), Cumming School of Medicine, and Department of Community Health Sciences (Stelfox), University of Calgary, Calgary, Alta.; Faculty of Nursing and School of Public Health (Norris, Bagshaw), University of Alberta; Strategic Clinical Networks (Norris, Bagshaw), Alberta Health Services; Division of Geriatric Medicine (Rolfson), Department of Medicine, and Division of Cardiac Surgery, (Meyer) Department of Surgery, Faculty of Medicine & Dentistry, University of Alberta, Edmonton, Alta
| | - Mohamad Zibdawi
- Department of Critical Care Medicine (Montgomery, Zibdawi, Bagshaw), Faculty of Medicine & Dentistry, University of Alberta, Edmonton, Alta.; Department of Critical Care Medicine (Stelfox), Cumming School of Medicine, and Department of Community Health Sciences (Stelfox), University of Calgary, Calgary, Alta.; Faculty of Nursing and School of Public Health (Norris, Bagshaw), University of Alberta; Strategic Clinical Networks (Norris, Bagshaw), Alberta Health Services; Division of Geriatric Medicine (Rolfson), Department of Medicine, and Division of Cardiac Surgery, (Meyer) Department of Surgery, Faculty of Medicine & Dentistry, University of Alberta, Edmonton, Alta
| | - Sean Bagshaw
- Department of Critical Care Medicine (Montgomery, Zibdawi, Bagshaw), Faculty of Medicine & Dentistry, University of Alberta, Edmonton, Alta.; Department of Critical Care Medicine (Stelfox), Cumming School of Medicine, and Department of Community Health Sciences (Stelfox), University of Calgary, Calgary, Alta.; Faculty of Nursing and School of Public Health (Norris, Bagshaw), University of Alberta; Strategic Clinical Networks (Norris, Bagshaw), Alberta Health Services; Division of Geriatric Medicine (Rolfson), Department of Medicine, and Division of Cardiac Surgery, (Meyer) Department of Surgery, Faculty of Medicine & Dentistry, University of Alberta, Edmonton, Alta
| |
Collapse
|
33
|
de Jong VMT, Moons KGM, Eijkemans MJC, Riley RD, Debray TPA. Developing more generalizable prediction models from pooled studies and large clustered data sets. Stat Med 2021; 40:3533-3559. [PMID: 33948970 PMCID: PMC8252590 DOI: 10.1002/sim.8981] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/03/2019] [Revised: 02/16/2021] [Accepted: 03/22/2021] [Indexed: 12/14/2022]
Abstract
Prediction models often yield inaccurate predictions for new individuals. Large data sets from pooled studies or electronic healthcare records may alleviate this with an increased sample size and variability in sample characteristics. However, existing strategies for prediction model development generally do not account for heterogeneity in predictor‐outcome associations between different settings and populations. This limits the generalizability of developed models (even from large, combined, clustered data sets) and necessitates local revisions. We aim to develop methodology for producing prediction models that require less tailoring to different settings and populations. We adopt internal‐external cross‐validation to assess and reduce heterogeneity in models' predictive performance during the development. We propose a predictor selection algorithm that optimizes the (weighted) average performance while minimizing its variability across the hold‐out clusters (or studies). Predictors are added iteratively until the estimated generalizability is optimized. We illustrate this by developing a model for predicting the risk of atrial fibrillation and updating an existing one for diagnosing deep vein thrombosis, using individual participant data from 20 cohorts (N = 10 873) and 11 diagnostic studies (N = 10 014), respectively. Meta‐analysis of calibration and discrimination performance in each hold‐out cluster shows that trade‐offs between average and heterogeneity of performance occurred. Our methodology enables the assessment of heterogeneity of prediction model performance during model development in multiple or clustered data sets, thereby informing researchers on predictor selection to improve the generalizability to different settings and populations, and reduce the need for model tailoring. Our methodology has been implemented in the R package metamisc.
Collapse
Affiliation(s)
- Valentijn M T de Jong
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.,Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Karel G M Moons
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.,Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Marinus J C Eijkemans
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Richard D Riley
- Centre for Prognosis Research, School of Medicine, Keele University, Staffordshire, UK
| | - Thomas P A Debray
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands.,Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| |
Collapse
|
34
|
Li J, Zhou Z, Dong J, Fu Y, Li Y, Luan Z, Peng X. Predicting breast cancer 5-year survival using machine learning: A systematic review. PLoS One 2021; 16:e0250370. [PMID: 33861809 PMCID: PMC8051758 DOI: 10.1371/journal.pone.0250370] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/21/2021] [Accepted: 04/06/2021] [Indexed: 12/24/2022] Open
Abstract
BACKGROUND Accurately predicting the survival rate of breast cancer patients is a major issue for cancer researchers. Machine learning (ML) has attracted much attention with the hope that it could provide accurate results, but its modeling methods and prediction performance remain controversial. The aim of this systematic review is to identify and critically appraise current studies regarding the application of ML in predicting the 5-year survival rate of breast cancer. METHODS In accordance with the PRISMA guidelines, two researchers independently searched the PubMed (including MEDLINE), Embase, and Web of Science Core databases from inception to November 30, 2020. The search terms included breast neoplasms, survival, machine learning, and specific algorithm names. The included studies related to the use of ML to build a breast cancer survival prediction model and model performance that can be measured with the value of said verification results. The excluded studies in which the modeling process were not explained clearly and had incomplete information. The extracted information included literature information, database information, data preparation and modeling process information, model construction and performance evaluation information, and candidate predictor information. RESULTS Thirty-one studies that met the inclusion criteria were included, most of which were published after 2013. The most frequently used ML methods were decision trees (19 studies, 61.3%), artificial neural networks (18 studies, 58.1%), support vector machines (16 studies, 51.6%), and ensemble learning (10 studies, 32.3%). The median sample size was 37256 (range 200 to 659820) patients, and the median predictor was 16 (range 3 to 625). The accuracy of 29 studies ranged from 0.510 to 0.971. The sensitivity of 25 studies ranged from 0.037 to 1. The specificity of 24 studies ranged from 0.008 to 0.993. The AUC of 20 studies ranged from 0.500 to 0.972. The precision of 6 studies ranged from 0.549 to 1. All of the models were internally validated, and only one was externally validated. CONCLUSIONS Overall, compared with traditional statistical methods, the performance of ML models does not necessarily show any improvement, and this area of research still faces limitations related to a lack of data preprocessing steps, the excessive differences of sample feature selection, and issues related to validation. Further optimization of the performance of the proposed model is also needed in the future, which requires more standardization and subsequent validation.
Collapse
Affiliation(s)
- Jiaxin Li
- School of Nursing, Jilin University, Jilin, China
| | - Zijun Zhou
- Breast Surgery, Jilin Province Tumor Hospital, Jilin, China
| | - Jianyu Dong
- School of Nursing, Jilin University, Jilin, China
| | - Ying Fu
- School of Nursing, Jilin University, Jilin, China
| | - Yuan Li
- School of Nursing, Jilin University, Jilin, China
| | - Ze Luan
- School of Nursing, Jilin University, Jilin, China
| | - Xin Peng
- School of Nursing, Jilin University, Jilin, China
- * E-mail:
| |
Collapse
|
35
|
Ensor J, Snell KIE, Debray TPA, Lambert PC, Look MP, Mamas MA, Moons KGM, Riley RD. Individual participant data meta-analysis for external validation, recalibration, and updating of a flexible parametric prognostic model. Stat Med 2021; 40:3066-3084. [PMID: 33768582 DOI: 10.1002/sim.8959] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/18/2019] [Revised: 03/04/2021] [Accepted: 03/05/2021] [Indexed: 12/14/2022]
Abstract
Individual participant data (IPD) from multiple sources allows external validation of a prognostic model across multiple populations. Often this reveals poor calibration, potentially causing poor predictive performance in some populations. However, rather than discarding the model outright, it may be possible to modify the model to improve performance using recalibration techniques. We use IPD meta-analysis to identify the simplest method to achieve good model performance. We examine four options for recalibrating an existing time-to-event model across multiple populations: (i) shifting the baseline hazard by a constant, (ii) re-estimating the shape of the baseline hazard, (iii) adjusting the prognostic index as a whole, and (iv) adjusting individual predictor effects. For each strategy, IPD meta-analysis examines (heterogeneity in) model performance across populations. Additionally, the probability of achieving good performance in a new population can be calculated allowing ranking of recalibration methods. In an applied example, IPD meta-analysis reveals that the existing model had poor calibration in some populations, and large heterogeneity across populations. However, re-estimation of the intercept substantially improved the expected calibration in new populations, and reduced between-population heterogeneity. Comparing recalibration strategies showed that re-estimating both the magnitude and shape of the baseline hazard gave the highest predicted probability of good performance in a new population. In conclusion, IPD meta-analysis allows a prognostic model to be externally validated in multiple settings, and enables recalibration strategies to be compared and ranked to decide on the least aggressive recalibration strategy to achieve acceptable external model performance without discarding existing model information.
Collapse
Affiliation(s)
- Joie Ensor
- Centre for Prognosis Research, School of Medicine, Keele University, Keele, UK
| | - Kym I E Snell
- Centre for Prognosis Research, School of Medicine, Keele University, Keele, UK
| | - Thomas P A Debray
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands.,Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Paul C Lambert
- Biostatistics Research Group, Department of Health Sciences, University of Leicester, Centre for Medicine, Leicester, UK.,Department of Medical Epidemiology and Biostatistics, Karolinska Institutet, Stockholm, Sweden
| | - Maxime P Look
- Department of Medical Oncology, Erasmus MC Cancer Institute, Erasmus University Medical Center, Rotterdam, The Netherlands
| | - Mamas A Mamas
- Centre for Prognosis Research, School of Medicine, Keele University, Keele, UK.,Department of Cardiology, Royal Stoke University Hospital, Stoke-on-Trent, UK
| | - Karel G M Moons
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands.,Cochrane Netherlands, Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht, The Netherlands
| | - Richard D Riley
- Centre for Prognosis Research, School of Medicine, Keele University, Keele, UK
| |
Collapse
|
36
|
Al-Farra H, de Mol BAJM, Ravelli ACJ, Ter Burg WJPP, Houterman S, Henriques JPS, Abu-Hanna A, Vis MM, Vos J, Timmers L, Tonino WAL, Schotborgh CE, Roolvink V, Porta F, Stoel MG, Kats S, Amoroso G, van der Werf HW, Stella PR, de Jaegere P. Update and, internal and temporal-validation of the FRANCE-2 and ACC-TAVI early-mortality prediction models for Transcatheter Aortic Valve Implantation (TAVI) using data from the Netherlands heart registration (NHR). IJC HEART & VASCULATURE 2021; 32:100716. [PMID: 33537406 PMCID: PMC7843396 DOI: 10.1016/j.ijcha.2021.100716] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/08/2020] [Revised: 12/30/2020] [Accepted: 01/04/2021] [Indexed: 01/08/2023]
Abstract
Background The predictive performance of the models FRANCE-2 and ACC-TAVI for early-mortality after Transcatheter Aortic Valve Implantation (TAVI) can decline over time and can be enhanced by updating them on new populations. We aim to update and internally and temporally validate these models using a recent TAVI-cohort from the Netherlands Heart Registration (NHR). Methods We used data of TAVI-patients treated in 2013-2017. For each original-model, the best update-method (model-intercept, model-recalibration, or model-revision) was selected by a closed-testing procedure. We internally validated both updated models with 1000 bootstrap samples. We also updated the models on the 2013-2016 dataset and temporally validated them on the 2017-dataset. Performance measures were the Area-Under ROC-curve (AU-ROC), Brier-score, and calibration graphs. Results We included 6177 TAVI-patients, with 4.5% observed early-mortality. The selected update-method for FRANCE-2 was model-intercept-update. Internal validation showed an AU-ROC of 0.63 (95%CI 0.62-0.66) and Brier-score of 0.04 (0.04-0.05). Calibration graphs show that it overestimates early-mortality. In temporal-validation, the AU-ROC was 0.61 (0.53-0.67).The selected update-method for ACC-TAVI was model-revision. In internal-validation, the AU-ROC was 0.63 (0.63-0.66) and Brier-score was 0.04 (0.04-0.05). The updated ACC-TAVI calibrates well up to a probability of 20%, and subsequently underestimates early-mortality. In temporal-validation the AU-ROC was 0.65 (0.58-0.72). Conclusion Internal-validation of the updated models FRANCE-2 and ACC-TAVI with data from the NHR demonstrated improved performance, which was better than in external-validation studies and comparable to the original studies. In temporal-validation, ACC-TAVI outperformed FRANCE-2 because it suffered less from changes over time.
Collapse
Key Words
- ACC-TAVI (ACC TVT), American College of Cardiology Transcatheter Valve Therapy
- AU-PRC, Area Under the Precision-Recall Curve
- AU-ROC, Area Under the Receiver Operating-Characteristic Curve
- Amsterdam UMC, Amsterdam University Medical Center - location AMC (Academic Medical Center)
- BSS, Brier-skill score
- Closed-testing procedure
- EuroSCORE, European System for Cardiac Operative Risk Evaluation
- External Validation
- FRANCE-2, French Aortic National CoreValve and Edwards [15]
- LVEF, Left Ventricular Ejection Fraction
- MPM, Mortality Prediction Models
- Model recalibration
- Model updating
- NHR, Netherlands Heart Registration (“Nederlandse Hart Registratie in Dutch”)
- NYHA, New York Heart Association
- Prediction models
- SAVR, Surgical Aortic Valve Replacement
- TAVI (TAVR), Transcatheter Aortic Valve Implantation (Replacement)
- Transcatheter Aortic Valve Implantation (TAVI)
Collapse
Affiliation(s)
- Hatem Al-Farra
- Department of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands.,Heart Center, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
| | - Bas A J M de Mol
- Heart Center, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
| | - Anita C J Ravelli
- Department of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
| | - W J P P Ter Burg
- Department of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
| | | | - José P S Henriques
- Heart Center, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
| | - Ameen Abu-Hanna
- Department of Medical Informatics, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
| | - M M Vis
- Heart Center, Amsterdam UMC, University of Amsterdam, Amsterdam, the Netherlands
| | - J Vos
- Amphia Hospital, the Netherlands
| | - L Timmers
- St. Antonius Hospital, the Netherlands
| | | | | | | | - F Porta
- Leeuwarden Medical Center, the Netherlands
| | - M G Stoel
- Medisch Spectrum Twente, the Netherlands
| | - S Kats
- Maastricht University Medical Center, the Netherlands
| | - G Amoroso
- Onze Lieve Vrouwe Gasthuis, the Netherlands
| | | | - P R Stella
- University Medical Center Utrecht, the Netherlands
| | - P de Jaegere
- Erasmus University Medical Center, the Netherlands
| | | |
Collapse
|
37
|
Benedetto U, Dimagli A, Gibbison B, Sinha S, Pufulete M, Fudulu D, Cocomello L, Bryan AJ, Ohri S, Caputo M, Cooper G, Dong T, Akowuah E, Angelini GD. Disparity in clinical outcomes after cardiac surgery between private and public (NHS) payers in England. THE LANCET REGIONAL HEALTH. EUROPE 2021; 1:100003. [PMID: 35104303 PMCID: PMC8454835 DOI: 10.1016/j.lanepe.2020.100003] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/14/2023]
Abstract
BACKGROUND There is little known about how payer status impacts clinical outcomes in a universal single-payer system such as the UK National Health Service (NHS). The aim of this study was to evaluate the relationship between payer status (private or public) and clinical outcomes following cardiac surgery from NHS providers in England. METHODS The National Adult Cardiac Surgery Audit (NACSA) registry was interrogated for patients who underwent adult cardiac surgery in England from 2009 to 2018. Information on socioeconomic status were provided by linkage with the Iteration of the English Indices of Deprivation (IoD). The primary outcome was in-hospital mortality. Secondary outcomes included incidence of in-hospital postoperative cerebrovascular accident (CVA), renal dialysis, sternal wound infection, and re-exploration. To assess whether payer status was an independent predictor of in-hospital mortality, binomial generalized linear mixed models (GLMM) were fitted along with 17 items forming the EuroSCORE and the IoD domains. FINDINGS The final sample consisted of 280,209 patients who underwent surgery in 31 NHS hospitals in England from 2009 to 2018. Of them, 5,967 (2.1%) and 274,242 (97.9%) were private and NHS payers respectively. Private payer status was associated with a lower risk of in-hospital mortality (OR 0.79; 95%CI 0.65 - 0.97;P = 0.026), CVA (OR 0.77; 95%CI 0.60 - 0.99; P = 0.039), need for re-exploration (OR 0.84; 95%CI 0.72 - 0.97; P = 0.017) and with non-significant lower risk of dialysis (OR 0.84; 95%CI 0.69 - 1.02; P = 0.074). Private payer status was found to be independently associated with lower risk of in-hospital mortality in the elective subgroup (OR 0.76; 95%CI 0.61 - 0.96; P = 0.020) but not in the non-elective subgroup (OR 1.01; 95%CI 0.64 - 1.58; P = 0.976). INTERPRETATION In conclusion, using a national database, we have found evidence of significant beneficial effect of payer status on hospital outcomes following cardiac surgery in favour of private payers regardless their socioeconomic factors.
Collapse
Affiliation(s)
- Umberto Benedetto
- Bristol Heart Institute, University of Bristol, Bristol Royal Infirmary, Bristol, UK
| | - Arnaldo Dimagli
- Bristol Heart Institute, University of Bristol, Bristol Royal Infirmary, Bristol, UK
| | - Ben Gibbison
- Bristol Heart Institute, University of Bristol, Bristol Royal Infirmary, Bristol, UK
| | - Shubhra Sinha
- Bristol Heart Institute, University of Bristol, Bristol Royal Infirmary, Bristol, UK
| | - Maria Pufulete
- Clinical Trials and Evaluation Unit, University of Bristol, Bristol, UK
| | - Daniel Fudulu
- Bristol Heart Institute, University of Bristol, Bristol Royal Infirmary, Bristol, UK
| | - Lucia Cocomello
- Bristol Heart Institute, University of Bristol, Bristol Royal Infirmary, Bristol, UK
| | - Alan J. Bryan
- Bristol Heart Institute, University of Bristol, Bristol Royal Infirmary, Bristol, UK
| | - Sunil Ohri
- Wessex Cardiothoracic Centre, Southampton University Hospitals NHS Trust, Southampton, UK
| | - Massimo Caputo
- Bristol Heart Institute, University of Bristol, Bristol Royal Infirmary, Bristol, UK
| | - Graham Cooper
- Department of Cardiothoracic Surgery, Northern General Hospital, Sheffield, UK
| | - Tim Dong
- Bristol Heart Institute, University of Bristol, Bristol Royal Infirmary, Bristol, UK
| | - Enoch Akowuah
- James Cook Hospital, South Tees Hospitals NHS Foundation Trust, Middlesbrough, UK
| | - Gianni D. Angelini
- Bristol Heart Institute, University of Bristol, Bristol Royal Infirmary, Bristol, UK
| |
Collapse
|
38
|
Mejia OAV, Borgomoni GB, Lima EG, Guerreiro GP, Dallan LR, de Barros E Silva P, Nakazone MA, Junior OP, Gomes WJ, de Oliveira MAP, Sousa A, Campagnucci VP, Tiveron MG, Rodrigues AJ, Tineli RÂ, Rocha E Silva R, Lisboa LAF, Jatene FB. Most deaths in low-risk cardiac surgery could be avoidable. Sci Rep 2021; 11:1045. [PMID: 33441748 PMCID: PMC7806717 DOI: 10.1038/s41598-020-80175-7] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/19/2020] [Accepted: 12/14/2020] [Indexed: 01/09/2023] Open
Abstract
It is observed that death rates in cardiac surgery has decreased, however, root causes that behave like triggers of potentially avoidable deaths (AD), especially in low-risk patients (less bias) are often unknown and underexplored, Phase of Care Mortality Analysis (POCMA) can be a valuable tool to identify seminal events (SE), providing valuable information where it is possible to make improvements in the quality and safety of future procedures. Our results show that in São Paul State, only one third of AD in low-risk cardiac surgery was related to specific surgical problems. After a revisited analysis, 75% of deaths could have been avoided, which in the pre-operative phase, the SE was related judgment, patient evaluation and preparation. In the intra-operative phase, most occurrences could have been avoided if other surgical technique had been used. Sepsis was responsible for 75% of AD in the intensive care unit. In the ward phase, the recognition/management of clinical decompensations and sepsis were the contributing factors. Logistic regression model identified age, previous coronary stent implantation, coronary artery bypass grafting + heart valve surgery, ≥ 2 combined heart valve surgery and hospital-acquired infection as independent predictors of AD.
Collapse
Affiliation(s)
- Omar Asdrúbal Vilca Mejia
- Department of Cardiovascular Surgery, Universidade de São Paulo Instituto do Coração (INCOR), São Paulo, São Paulo, Brazil. .,Department of Cardiovascular Surgery, Hospital Samaritano Paulista, São Paulo, São Paulo, Brazil.
| | - Gabrielle Barbosa Borgomoni
- Department of Cardiovascular Surgery, Universidade de São Paulo Instituto do Coração (INCOR), São Paulo, São Paulo, Brazil
| | - Eduardo Gomes Lima
- Department of Cardiovascular Surgery, Universidade de São Paulo Instituto do Coração (INCOR), São Paulo, São Paulo, Brazil
| | - Gustavo Pampolha Guerreiro
- Department of Cardiovascular Surgery, Universidade de São Paulo Instituto do Coração (INCOR), São Paulo, São Paulo, Brazil
| | - Luís Roberto Dallan
- Department of Cardiovascular Surgery, Universidade de São Paulo Instituto do Coração (INCOR), São Paulo, São Paulo, Brazil
| | - Pedro de Barros E Silva
- Department of Cardiovascular Surgery, Hospital Samaritano Paulista, São Paulo, São Paulo, Brazil
| | - Marcelo Arruda Nakazone
- Department of Cardiovascular Surgery, Hospital De Base de São José do Rio Preto, São José de Rio Preto, São Paulo, Brazil
| | - Orlando Petrucci Junior
- Department of Cardiovascular Surgery, Universidade Estadual de Campinas (UNICAMP), Campinas, São Paulo, Brazil
| | - Walter José Gomes
- Department of Cardiovascular Surgery, Universidade Federal de São Paulo (UNIFESP), São Paulo, São Paulo, Brazil
| | | | - Alexandre Sousa
- Department of Cardiovascular Surgery, Beneficência Portuguesa de São Paulo, São Paulo, São Paulo, Brazil
| | - Valquíria Pelisser Campagnucci
- Department of Cardiovascular Surgery, Irmandade da Santa Casa de Misericórdia de São Paulo, São Paulo, São Paulo, Brazil
| | - Marcos Gradim Tiveron
- Department of Cardiovascular Surgery, Irmandade da Santa Casa de Misericórdia de Marília, Marília, São Paulo, Brazil
| | - Alfredo José Rodrigues
- Departament of Cardiovascular Surgery, Universidade de São Paulo Hospital das Clínicas da Faculdade de Medicina de Ribeirão Preto, São Paulo, Brazil
| | - Rafael Ângelo Tineli
- Department of Cardiovascular Surgery, Irmandade da Santa Casa de Misericórdia de Piracicaba, Piracicaba, São Paulo, Brazil
| | - Roberto Rocha E Silva
- Department of Cardiovascular Surgery, Hospital Paulo Sacramento, Jundiaí, São Paulo, Brazil
| | - Luiz Augusto Ferreira Lisboa
- Department of Cardiovascular Surgery, Universidade de São Paulo Instituto do Coração (INCOR), São Paulo, São Paulo, Brazil
| | - Fabio Biscegli Jatene
- Department of Cardiovascular Surgery, Universidade de São Paulo Instituto do Coração (INCOR), São Paulo, São Paulo, Brazil
| |
Collapse
|
39
|
Jenkins DA, Martin GP, Sperrin M, Riley RD, Debray TPA, Collins GS, Peek N. Continual updating and monitoring of clinical prediction models: time for dynamic prediction systems? Diagn Progn Res 2021; 5:1. [PMID: 33431065 PMCID: PMC7797885 DOI: 10.1186/s41512-020-00090-3] [Citation(s) in RCA: 46] [Impact Index Per Article: 15.3] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/14/2020] [Accepted: 12/08/2020] [Indexed: 01/01/2023] Open
Abstract
Clinical prediction models (CPMs) have become fundamental for risk stratification across healthcare. The CPM pipeline (development, validation, deployment, and impact assessment) is commonly viewed as a one-time activity, with model updating rarely considered and done in a somewhat ad hoc manner. This fails to address the fact that the performance of a CPM worsens over time as natural changes in populations and care pathways occur. CPMs need constant surveillance to maintain adequate predictive performance. Rather than reactively updating a developed CPM once evidence of deteriorated performance accumulates, it is possible to proactively adapt CPMs whenever new data becomes available. Approaches for validation then need to be changed accordingly, making validation a continuous rather than a discrete effort. As such, "living" (dynamic) CPMs represent a paradigm shift, where the analytical methods dynamically generate updated versions of a model through time; one then needs to validate the system rather than each subsequent model revision.
Collapse
Affiliation(s)
- David A Jenkins
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK.
- NIHR Greater Manchester Patient Safety Translational Research Centre, The University of Manchester, Manchester, UK.
| | - Glen P Martin
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Matthew Sperrin
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| | - Richard D Riley
- Centre for Prognosis Research, School of Primary, Community and Social Care, Keele University, Staffordshire, UK
| | - Thomas P A Debray
- Julius Center for Health Sciences and Primary Care, University Medical Center Utrecht, Utrecht University, Utrecht, The Netherlands
| | - Gary S Collins
- Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology and Musculoskeletal Sciences, University of Oxford, Oxford, UK
| | - Niels Peek
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
- NIHR Greater Manchester Patient Safety Translational Research Centre, The University of Manchester, Manchester, UK
- NIHR Manchester Biomedical Research Centre, The University of Manchester, Manchester Academic Health Science Centre, Manchester, UK
| |
Collapse
|
40
|
Kogan A, Kassif Y, Frogel J, Levin S, Ram E, Peled Y, Raanani E, Sternik L. The Impact of Initiation of an Intensivist-Led Patient Management Protocol on Outcomes After Cardiac Surgery. J Cardiothorac Vasc Anesth 2021; 35:2370-2376. [PMID: 33483270 DOI: 10.1053/j.jvca.2020.12.048] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/08/2020] [Revised: 12/26/2020] [Accepted: 12/28/2020] [Indexed: 11/11/2022]
Abstract
OBJECTIVES Prolonged intensive care unit stay after cardiac surgery is associated with high mortality. The aim of this study was to evaluate the impact of the introduction of a quality improvement program under the supervision of an intensivist on the long-term mortality of high-risk patients with prolonged intensive care unit (ICU) stay after cardiac surgery. DESIGN Retrospective study of prospectively collected data. SETTING Cardiac surgery ICU. PARTICIPANTS A total of 7,549 patients after cardiac surgery. INTERVENTIONS Patients were divided into two periods: 2004 to 2007, before introducing the quality improvement program (3,315 patients), and 2009 to 2014, after introduction of the program (4,234 patients). In the period from 2004 to 2007, patients were divided into group I (ICU stay ≥ seven days), which included 242 patients, and group III (ICU stay < seven days), which included 3,073 patients. Also, in the period from 2009 to 2014 patients, were divided into group II (ICU stay ≥seven days), which included 326 patients, and group IV (ICU stay < seven days), which included 3,908 patients. Patient outcomes were compared. Follow-up was five years for each group. MEASUREMENTS AND MAIN RESULTS The European System for Cardiac Operative Risk Evaluation did not differ significantly among the groups. When comparing between group I and group II, 30-day mortality decreased significantly from 24.8% to 16.6%, six-month mortality from 27.3% to 19.3%, one-year mortality from 42.1% to 32.2%, 3-year mortality from 54.5% to 43.3%, and 5-year mortality from 61.2% to 51.8%. In comparing between group III and group IV, the authors did not observe a statistically significant decrease of mortality. CONCLUSIONS Intensivist-led patient management protocol is associated with decreased long-term mortality in high-risk patients with a prolonged ICU stay.
Collapse
Affiliation(s)
- Alexander Kogan
- Department of Cardiac Surgery, Sheba Medical Center, Tel Aviv, Israel; Cardiac Surgery Intensive Care Unit, Sheba Medical Center, Tel Aviv, Israel.
| | - Yigal Kassif
- Department of Cardiac Surgery, Sheba Medical Center, Tel Aviv, Israel; Cardiac Surgery Intensive Care Unit, Sheba Medical Center, Tel Aviv, Israel
| | - Jonathan Frogel
- Department of Anesthesiology, Sheba Medical Center, Tel Aviv, Israel
| | - Shany Levin
- Department of Cardiac Surgery, Sheba Medical Center, Tel Aviv, Israel
| | - Eilon Ram
- Department of Cardiac Surgery, Sheba Medical Center, Tel Aviv, Israel
| | - Yael Peled
- Heart Transplantation Unit, Leviev Cardiothoracic and Vascular Center, Sheba Medical Center, Tel Hashomer, affiliated to the Sackler School of Medicine, Tel Aviv University, Tel Aviv, Israel
| | - Ehud Raanani
- Department of Cardiac Surgery, Sheba Medical Center, Tel Aviv, Israel
| | - Leonid Sternik
- Department of Cardiac Surgery, Sheba Medical Center, Tel Aviv, Israel
| |
Collapse
|
41
|
Davis SE, Greevy RA, Lasko TA, Walsh CG, Matheny ME. Detection of calibration drift in clinical prediction models to inform model updating. J Biomed Inform 2020; 112:103611. [PMID: 33157313 PMCID: PMC8627243 DOI: 10.1016/j.jbi.2020.103611] [Citation(s) in RCA: 45] [Impact Index Per Article: 11.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/14/2020] [Revised: 10/21/2020] [Accepted: 10/29/2020] [Indexed: 10/23/2022]
Abstract
Model calibration, critical to the success and safety of clinical prediction models, deteriorates over time in response to the dynamic nature of clinical environments. To support informed, data-driven model updating strategies, we present and evaluate a calibration drift detection system. Methods are developed for maintaining dynamic calibration curves with optimized online stochastic gradient descent and for detecting increasing miscalibration with adaptive sliding windows. These methods are generalizable to support diverse prediction models developed using a variety of learning algorithms and customizable to address the unique needs of clinical use cases. In both simulation and case studies, our system accurately detected calibration drift. When drift is detected, our system further provides actionable alerts by including information on a window of recent data that may be appropriate for model updating. Simulations showed these windows were primarily composed of data accruing after drift onset, supporting the potential utility of the windows for model updating. By promoting model updating as calibration deteriorates rather than on pre-determined schedules, implementations of our drift detection system may minimize interim periods of insufficient model accuracy and focus analytic resources on those models most in need of attention.
Collapse
Affiliation(s)
- Sharon E Davis
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA.
| | - Robert A Greevy
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, USA.
| | - Thomas A Lasko
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA.
| | - Colin G Walsh
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA; Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, USA; Department of Psychiatry, Vanderbilt University Medical Center, Nashville, TN, USA.
| | - Michael E Matheny
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA; Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, USA; Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, USA; Geriatrics Research, Education, and Clinical Care, Tennessee Valley Healthcare System VA, Nashville, TN, USA.
| |
Collapse
|
42
|
Cocomello L, Caputo M, Cornish R, Lawlor D. External validation of the improving partial risk adjustment in surgery (PRAIS-2) model for 30-day mortality after paediatric cardiac surgery. BMJ Open 2020; 10:e039236. [PMID: 33247009 PMCID: PMC7703410 DOI: 10.1136/bmjopen-2020-039236] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 12/31/2022] Open
Abstract
OBJECTIVE Independent temporal external validation of the improving partial risk adjustment in surgery model (PRAIS-2) to predict 30-day mortality in patients undergoing paediatric cardiac surgery. DESIGN Retrospective analysis of prospectively collected data. SETTING Paediatric cardiac surgery. INTERVENTION PRAIS-2 validation was carried out using a two temporally different single centre (Bristol, UK) cohorts: Cohort 1 surgery undertaken from April 2004 to March 2009 and Cohort 2 from April 2015 to July 2019. For each subject PRAIS-2 score was calculated according to the original formula. PARTICIPANTS A total of 1352 (2004-2009) and 1197 (2015-2019) paediatric cardiac surgical procedures were included in the Cohort 1 and Cohort 2, respectively (median age at the procedure 6.3 and 7.1 months). PRIMARY AND SECONDARY OUTCOME MEASURES PRAIS-2 performance was assessed in terms of discrimination by means of ROC (receiver operating characteristic) curve analysis and calibration by using the calibration belt method. RESULTS PRAIS-2 score showed excellent discrimination for both cohorts (AUC 0.72 (95%CI: 0.65 to 0.80) and 0.88 (95%CI: 0.82 to 0.93), respectively). While PRAIS-2 was only marginally calibrated in Cohort 1, with a tendency to underestimate risk in lowrisk and overestimate risk in high risk procedures (P-value = 0.033), validation in Cohort 2 showed good calibration with the 95% confidence belt containing the bisector for predicted mortality (P-value = 0.143). We also observed good prediction accuracy in the non-elective procedures (N = 483;AUC 0.78 (95%CI 0.68 to 0.87); Calibration belt containing the bisector (P-value=0.589). CONCLUSIONS In a single centre UK-based cohort, PRAIS-2 showed excellent discrimination and calibration in predicting 30-day mortality in paediatric cardiac surgery including in those undergoing non-elective procedures. Our results support a wider adoption of PRAIS-2 score in the clinical practice.
Collapse
Affiliation(s)
- Lucia Cocomello
- MRC Integrative Epidemiology Unit, University of Bristol, Bristol, UK
| | - Massimo Caputo
- Bristol Heart Institute, University of Bristol, Bristol, UK
| | - Rosie Cornish
- Population Health Science, Bristol Medical School University of Bristol, Bristol, United Kingdom
| | - Deborah Lawlor
- MRC Integrative Epidemiology Unit, Department of Social Medicine, University of Bristol, Bristol, UK
| |
Collapse
|
43
|
Taylor M, Szafron B, Martin GP, Abah U, Smith M, Shackcloth M, Granato F, Shah R, Grant SW, Eadington T, Argus L, Michael S, Mason S, Bhullar D, Obale E, Fritsch NC, Shah R, Krysiak P, Rammohan K, Fontaine E, Granato F, Page R, Woolley S, Shackcloth M, Assante-Siaw J, Mediratta N. External validation of six existing multivariable clinical prediction models for short-term mortality in patients undergoing lung resection. Eur J Cardiothorac Surg 2020; 59:1030-1036. [DOI: 10.1093/ejcts/ezaa422] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/28/2020] [Revised: 10/16/2020] [Accepted: 10/21/2020] [Indexed: 12/23/2022] Open
Abstract
Abstract
OBJECTIVES
National guidelines advocate the use of clinical prediction models to estimate perioperative mortality for patients undergoing lung resection. Several models have been developed that may potentially be useful but contemporary external validation studies are lacking. The aim of this study was to validate existing models in a multicentre patient cohort.
METHODS
The Thoracoscore, Modified Thoracoscore, Eurolung, Modified Eurolung, European Society Objective Score and Brunelli models were validated using a database of 6600 patients who underwent lung resection between 2012 and 2018. Models were validated for in-hospital or 30-day mortality (depending on intended outcome of each model) and also for 90-day mortality. Model calibration (calibration intercept, calibration slope, observed to expected ratio and calibration plots) and discrimination (area under receiver operating characteristic curve) were assessed as measures of model performance.
RESULTS
Mean age was 66.8 years (±10.9 years) and 49.7% (n = 3281) of patients were male. In-hospital, 30-day, perioperative (in-hospital or 30-day) and 90-day mortality were 1.5% (n = 99), 1.4% (n = 93), 1.8% (n = 121) and 3.1% (n = 204), respectively. Model area under the receiver operating characteristic curves ranged from 0.67 to 0.73. Calibration was inadequate in five models and mortality was significantly overestimated in five models. No model was able to adequately predict 90-day mortality.
CONCLUSIONS
Five of the validated models were poorly calibrated and had inadequate discriminatory ability. The modified Eurolung model demonstrated adequate statistical performance but lacked clinical validity. Development of accurate models that can be used to estimate the contemporary risk of lung resection is required.
Collapse
Affiliation(s)
- Marcus Taylor
- Department of Cardiothoracic Surgery, Wythenshawe Hospital, Manchester University Hospital Foundation Trust, Manchester, UK
| | - Bartłomiej Szafron
- Department of Cardiothoracic Surgery, Wythenshawe Hospital, Manchester University Hospital Foundation Trust, Manchester, UK
| | - Glen P Martin
- Division of Informatics, Imaging and Data Science, Faculty of Biology, Medicine and Health, Manchester Academic Heath Science Centre, University of Manchester, Manchester, UK
| | - Udo Abah
- Department of Cardiothoracic Surgery, Liverpool Heart & Chest Hospital, Liverpool, UK
| | - Matthew Smith
- Department of Cardiothoracic Surgery, Liverpool Heart & Chest Hospital, Liverpool, UK
| | - Michael Shackcloth
- Department of Cardiothoracic Surgery, Liverpool Heart & Chest Hospital, Liverpool, UK
| | - Felice Granato
- Department of Cardiothoracic Surgery, Wythenshawe Hospital, Manchester University Hospital Foundation Trust, Manchester, UK
| | - Rajesh Shah
- Department of Cardiothoracic Surgery, Wythenshawe Hospital, Manchester University Hospital Foundation Trust, Manchester, UK
| | - Stuart W Grant
- Division of Cardiovascular Sciences, University of Manchester, ERC, Manchester University Hospitals Foundation Trust, Manchester, UK
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | |
Collapse
|
44
|
Validation and quality measurements for STS, EuroSCORE II and a regional risk model in Brazilian patients. PLoS One 2020; 15:e0238737. [PMID: 32911513 PMCID: PMC7482975 DOI: 10.1371/journal.pone.0238737] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/16/2020] [Accepted: 08/22/2020] [Indexed: 12/13/2022] Open
Abstract
OBJECTIVES The objectives of this study were to describe a novel statewide registry for cardiac surgery in Brazil (REPLICCAR), to compare a regional risk model (SPScore) with EuroSCORE II and STS, and to understand where quality improvement and safety initiatives can be implemented. METHODS A total of 11 sites in the state of São Paulo, Brazil, formed an online registry platform to capture information on risk factors and outcomes after cardiac surgery procedures for all consecutive patients. EuroSCORE II and STS values were calculated for each patient. An SPScore model was designed and compared with EuroSCORE II and STS to predict 30-day outcomes: death, reoperation, readmission, and any morbidity. RESULTS A total of 5222 patients were enrolled in this study between November 2013 and December 2017. The observed 30-day mortality rate was 7.6%. Most patients were older, overweight, and classified as New York Heart Association (NYHA) functional class III; 14.5% of the patient population had a positive diagnosis of rheumatic heart disease, 10.9% had insulin-dependent diabetes, and 19 individuals had a positive diagnosis of Chagas disease. When evaluating the prediction performance, we found that SPScore outperformed EuroSCORE II and STS in the prediction of mortality (0.90 vs. 0.76 and 0.77), reoperation (0.84 vs. 0.60 and 0.56), readmission (0.84 vs. 0.55 and 0.51), and any morbidity (0.80 vs. 0.65 and 0.64), respectively (p<0.001). CONCLUSIONS The REPLICCAR registry might stimulate the creation of other cardiac surgery registries in developing countries, ultimately improving the regional quality of care provided to patients.
Collapse
|
45
|
Wolff G, Shamekhi J, Al-Kassou B, Tabata N, Parco C, Klein K, Maier O, Sedaghat A, Polzin A, Sugiura A, Jung C, Grube E, Westenfeld R, Icks A, Zeus T, Sinning JM, Baldus S, Nickenig G, Kelm M, Veulemans V. Risk modeling in transcatheter aortic valve replacement remains unsolved: an external validation study in 2946 German patients. Clin Res Cardiol 2020; 110:368-376. [PMID: 32851491 PMCID: PMC7907023 DOI: 10.1007/s00392-020-01731-9] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 06/13/2020] [Accepted: 08/12/2020] [Indexed: 12/31/2022]
Abstract
Background Surgical risk prediction models are routinely used to guide decision-making for transcatheter aortic valve replacement (TAVR). New and updated TAVR-specific models have been developed to improve risk stratification; however, the best option remains unknown. Objective To perform a comparative validation study of six risk models for the prediction of 30-day mortality in TAVR Methods and results A total of 2946 patients undergoing transfemoral (TF, n = 2625) or transapical (TA, n = 321) TAVR from 2008 to 2018 from the German Rhine Transregio Aortic Diseases cohort were included. Six surgical and TAVR-specific risk scoring models (LogES I, ES II, STS PROM, FRANCE-2, OBSERVANT, GAVS-II) were evaluated for the prediction of 30-day mortality. Observed 30-day mortality was 3.7% (TF 3.2%; TA 7.5%), mean 30-day mortality risk prediction varied from 5.8 ± 5.0% (OBSERVANT) to 23.4 ± 15.9% (LogES I). Discrimination performance (ROC analysis, c-indices) ranged from 0.60 (OBSERVANT) to 0.67 (STS PROM), without significant differences between models, between TF or TA approach or over time. STS PROM discriminated numerically best in TF TAVR (c-index 0.66; range of c-indices 0.60 to 0.66); performance was very similar in TA TAVR (LogES I, ES II, FRANCE-2 and GAVS-II all with c-index 0.67). Regarding calibration, all risk scoring models—especially LogES I—overestimated mortality risk, especially in high-risk patients. Conclusions Surgical as well as TAVR-specific risk scoring models showed mediocre performance in prediction of 30-day mortality risk for TAVR in the German Rhine Transregio Aortic Diseases cohort. Development of new or updated risk models is necessary to improve risk stratification. Graphic abstract ![]()
Electronic supplementary material The online version of this article (10.1007/s00392-020-01731-9) contains supplementary material, which is available to authorized users.
Collapse
Affiliation(s)
- Georg Wolff
- Division of Cardiology, Pulmonology and Vascular Medicine, Department of Internal Medicine, Heinrich Heine University, Medical Faculty, Moorenstr. 5, 40225, Düsseldorf, Germany.
| | - Jasmin Shamekhi
- Department of Medicine II, Heart Center Bonn, University Hospital Bonn, Bonn, Germany
| | - Baravan Al-Kassou
- Department of Medicine II, Heart Center Bonn, University Hospital Bonn, Bonn, Germany
| | - Noriaki Tabata
- Department of Medicine II, Heart Center Bonn, University Hospital Bonn, Bonn, Germany
| | - Claudio Parco
- Division of Cardiology, Pulmonology and Vascular Medicine, Department of Internal Medicine, Heinrich Heine University, Medical Faculty, Moorenstr. 5, 40225, Düsseldorf, Germany
| | - Kathrin Klein
- Division of Cardiology, Pulmonology and Vascular Medicine, Department of Internal Medicine, Heinrich Heine University, Medical Faculty, Moorenstr. 5, 40225, Düsseldorf, Germany
| | - Oliver Maier
- Division of Cardiology, Pulmonology and Vascular Medicine, Department of Internal Medicine, Heinrich Heine University, Medical Faculty, Moorenstr. 5, 40225, Düsseldorf, Germany
| | - Alexander Sedaghat
- Department of Medicine II, Heart Center Bonn, University Hospital Bonn, Bonn, Germany
| | - Amin Polzin
- Division of Cardiology, Pulmonology and Vascular Medicine, Department of Internal Medicine, Heinrich Heine University, Medical Faculty, Moorenstr. 5, 40225, Düsseldorf, Germany
| | - Atsushi Sugiura
- Department of Medicine II, Heart Center Bonn, University Hospital Bonn, Bonn, Germany
| | - Christian Jung
- Division of Cardiology, Pulmonology and Vascular Medicine, Department of Internal Medicine, Heinrich Heine University, Medical Faculty, Moorenstr. 5, 40225, Düsseldorf, Germany
| | - Eberhard Grube
- Department of Medicine II, Heart Center Bonn, University Hospital Bonn, Bonn, Germany
| | - Ralf Westenfeld
- Division of Cardiology, Pulmonology and Vascular Medicine, Department of Internal Medicine, Heinrich Heine University, Medical Faculty, Moorenstr. 5, 40225, Düsseldorf, Germany
| | - Andrea Icks
- Institute for Health Services Research and Health Economics, Centre for Health and Society, Medical Faculty, Heinrich-Heine-University, Düsseldorf, Germany
| | - Tobias Zeus
- Division of Cardiology, Pulmonology and Vascular Medicine, Department of Internal Medicine, Heinrich Heine University, Medical Faculty, Moorenstr. 5, 40225, Düsseldorf, Germany
| | - Jan-Malte Sinning
- Department of Medicine II, Heart Center Bonn, University Hospital Bonn, Bonn, Germany.,Transregio 259: Aortic Diseases-Scientific Network of University Heart Centers in Düsseldorf/Bonn/Cologne, Düsseldorf/Bonn/Cologne, Germany
| | - Stephan Baldus
- Division of Cardiology, Pneumology, Angiology and Intensive Care, Department of Internal Medicine III, University of Cologne, Cologne, Germany.,Transregio 259: Aortic Diseases-Scientific Network of University Heart Centers in Düsseldorf/Bonn/Cologne, Düsseldorf/Bonn/Cologne, Germany
| | - Georg Nickenig
- Department of Medicine II, Heart Center Bonn, University Hospital Bonn, Bonn, Germany.,Transregio 259: Aortic Diseases-Scientific Network of University Heart Centers in Düsseldorf/Bonn/Cologne, Düsseldorf/Bonn/Cologne, Germany
| | - Malte Kelm
- Division of Cardiology, Pulmonology and Vascular Medicine, Department of Internal Medicine, Heinrich Heine University, Medical Faculty, Moorenstr. 5, 40225, Düsseldorf, Germany.,Transregio 259: Aortic Diseases-Scientific Network of University Heart Centers in Düsseldorf/Bonn/Cologne, Düsseldorf/Bonn/Cologne, Germany
| | - Verena Veulemans
- Division of Cardiology, Pulmonology and Vascular Medicine, Department of Internal Medicine, Heinrich Heine University, Medical Faculty, Moorenstr. 5, 40225, Düsseldorf, Germany
| |
Collapse
|
46
|
Mullan CW, Mori M, Pichert MD, Bin Mahmood SU, Yousef S, Geirsson A. United States national trends in comorbidity and outcomes of adult cardiac surgery patients. J Card Surg 2020; 35:2248-2253. [PMID: 33448476 DOI: 10.1111/jocs.14764] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
Abstract
BACKGROUND Comorbidity profiles of cardiac surgery patients are known to have changed over time, but modern national trends in these comorbidities and outcomes are not described. This study describes comorbidity trends over time for common adult cardiac surgery procedures. METHODS A retrospective, cross-sectional analysis of the National Inpatient Sample was conducted for years 2005-2014. Hospitalizations with coronary artery bypass grafting (CABG), aortic valve replacement (AVR), and mitral valve repair/replacement (MVRR), as well as combined CABG/valve operations, were identified by ICD-9 procedure codes. Comorbidities were defined based on ICD-9 codes to discriminate between comorbidities and complications. Surgical volume, patient age, in-hospital mortality, and length of stay trends over time were evaluated by linear regression. RESULTS Incidence increased for AVR, MVRR, and CABG + AVR and declined for CABG and CABG + MVRR (P < .001). The mean number of comorbidities across all surgeries increased from 1.4 to 1.9 (P < .001). Length of stay declined for AVR, CABG + AVR, and CABG + MVRR (P < .001) with an overall decline from 10.1 to 9.7 days (P = .003). In-hospital mortality decreased in all categories over time (P < .001). Overall, in-hospital mortality decreased from 2.9% to 2.3% (P < .001). CONCLUSIONS Despite increasing comorbidity in cardiac surgery, operations are being conducted with fewer in-hospital mortalities across all types of surgery and decreasing length of stay for most types of surgery, which should inform the frequency of risk model updates and raise questions of the applicability of earlier studies in cardiac surgery to the modern population.
Collapse
Affiliation(s)
- Clancy W Mullan
- Division of Cardiac Surgery, Department of Surgery, Yale School of Medicine, New Haven, Connecticut
| | - Makoto Mori
- Division of Cardiac Surgery, Department of Surgery, Yale School of Medicine, New Haven, Connecticut
| | - Matthew D Pichert
- Division of Cardiac Surgery, Department of Surgery, Yale School of Medicine, New Haven, Connecticut
| | - Syed U Bin Mahmood
- Division of Cardiac Surgery, Department of Surgery, Yale School of Medicine, New Haven, Connecticut
| | - Sameh Yousef
- Division of Cardiac Surgery, Department of Surgery, Yale School of Medicine, New Haven, Connecticut
| | - Arnar Geirsson
- Division of Cardiac Surgery, Department of Surgery, Yale School of Medicine, New Haven, Connecticut
| |
Collapse
|
47
|
A Novel Patient-Specific Model for Predicting Severe Oliguria; Development and Comparison With Kidney Disease: Improving Global Outcomes Acute Kidney Injury Classification. Crit Care Med 2020; 48:e18-e25. [PMID: 31663925 DOI: 10.1097/ccm.0000000000004074] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/20/2023]
Abstract
OBJECTIVES The Kidney Disease: Improving Global Outcomes urine output criteria for acute kidney injury lack specificity for identifying patients at risk of adverse renal outcomes. The objective was to develop a model that analyses hourly urine output values in real time to identify those at risk of developing severe oliguria. DESIGN This was a retrospective cohort study utilizing prospectively collected data. SETTING A cardiac ICU in the United Kingdom. PATIENTS Patients undergoing cardiac surgery between January 2013 and November 2017. INTERVENTIONS None. MEASUREMENT AND MAIN RESULTS Patients were randomly assigned to development (n = 981) and validation (n = 2,389) datasets. A patient-specific, dynamic Bayesian model was developed to predict future urine output on an hourly basis. Model discrimination and calibration for predicting severe oliguria (< 0.3 mL/kg/hr for 6 hr) occurring within the next 12 hours were tested in the validation dataset at multiple time points. Patients with a high risk of severe oliguria (p > 0.8) were identified and their outcomes were compared with those for low-risk patients and for patients who met the Kidney Disease: Improving Global Outcomes urine output criterion for acute kidney injury. Model discrimination was excellent at all time points (area under the curve > 0.9 for all). Calibration of the model's predictions was also excellent. After adjustment using multivariable logistic regression, patients in the high-risk group were more likely to require renal replacement therapy (odds ratio, 10.4; 95% CI, 5.9-18.1), suffer prolonged hospital stay (odds ratio, 4.4; 95% CI, 3.0-6.4), and die in hospital (odds ratio, 6.4; 95% CI, 2.8-14.0) (p < 0.001 for all). Outcomes for those identified as high risk by the model were significantly worse than for patients who met the Kidney Disease: Improving Global Outcomes urine output criterion. CONCLUSIONS This novel, patient-specific model identifies patients at increased risk of severe oliguria. Classification according to model predictions outperformed the Kidney Disease: Improving Global Outcomes urine output criterion. As the new model identifies patients at risk before severe oliguria develops it could potentially facilitate intervention to improve patient outcomes.
Collapse
|
48
|
Benedetto U, Dimagli A. Commentary: To underfit and to overfit the data. This is the dilemma. J Thorac Cardiovasc Surg 2020; 160:183. [PMID: 32014322 DOI: 10.1016/j.jtcvs.2019.12.079] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/19/2019] [Accepted: 12/19/2019] [Indexed: 11/18/2022]
Affiliation(s)
- Umberto Benedetto
- Bristol Heart Institute, University of Bristol, Bristol, United Kingdom.
| | - Arnaldo Dimagli
- Bristol Heart Institute, University of Bristol, Bristol, United Kingdom
| |
Collapse
|
49
|
Abbasciano RG, Roman MA, Rizzello A, Pathak S, Ramzi J, Lucarelli C, Layton G, Lai F, Kumar T, Wozniak MJ, Murphy G. Effects of interventions targeting the systemic inflammatory response to cardiac surgery on clinical outcomes in adults. Hippokratia 2020. [DOI: 10.1002/14651858.cd013584] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/07/2022]
Affiliation(s)
| | - Marius A Roman
- Department of Cardiothoracic Surgery / Cardiovascular Sciences; University Hospitals of Leicester / University of Leicester; Leicester UK
| | - Angelica Rizzello
- Department of Cardiovascular Sciences; University of Leicester; Leicester UK
| | - Suraj Pathak
- Department of Cardiothoracic Surgery; Glenfield Hospital; Leicester UK
| | - Joussi Ramzi
- Leicester Medical School; University of Leicester; Leicester UK
| | - Carla Lucarelli
- Department of Cardiac Surgery; University of Verona; Verona Italy
| | - Georgia Layton
- University Hospitals of Leicester NHS Trust; Leicester UK
| | - Florence Lai
- Leicester Clinical Trials Unit; University of Leicester, Glenfield Hospital; Leicester UK
| | - Tracy Kumar
- Department of Cardiovascular Sciences; University of Leicester; Leicester UK
| | - Marcin J Wozniak
- Department of Cardiovascular Sciences; University of Leicester; Leicester UK
| | - Gavin Murphy
- Department of Cardiovascular Sciences; University of Leicester; Leicester UK
| |
Collapse
|
50
|
Koola JD, Ho SB, Cao A, Chen G, Perkins AM, Davis SE, Matheny ME. Predicting 30-Day Hospital Readmission Risk in a National Cohort of Patients with Cirrhosis. Dig Dis Sci 2020; 65:1003-1031. [PMID: 31531817 PMCID: PMC7073276 DOI: 10.1007/s10620-019-05826-w] [Citation(s) in RCA: 9] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/25/2019] [Accepted: 09/04/2019] [Indexed: 12/17/2022]
Abstract
BACKGROUND Early hospital readmission for patients with cirrhosis continues to challenge the healthcare system. Risk stratification may help tailor resources, but existing models were designed using small, single-institution cohorts or had modest performance. AIMS We leveraged a large clinical database from the Department of Veterans Affairs (VA) to design a readmission risk model for patients hospitalized with cirrhosis. Additionally, we analyzed potentially modifiable or unexplored readmission risk factors. METHODS A national VA retrospective cohort of patients with a history of cirrhosis hospitalized for any reason from January 1, 2006, to November 30, 2013, was developed from 123 centers. Using 174 candidate variables within demographics, laboratory results, vital signs, medications, diagnoses and procedures, and healthcare utilization, we built a 47-variable penalized logistic regression model with the outcome of all-cause 30-day readmission. We excluded patients who left against medical advice, transferred to a non-VA facility, or if the hospital length of stay was greater than 30 days. We evaluated calibration and discrimination across variable volume and compared the performance to recalibrated preexisting risk models for readmission. RESULTS We analyzed 67,749 patients and 179,298 index hospitalizations. The 30-day readmission rate was 23%. Ascites was the most common cirrhosis-related cause of index hospitalization and readmission. The AUC of the model was 0.670 compared to existing models (0.649, 0.566, 0.577). The Brier score of 0.165 showed good calibration. CONCLUSION Our model achieved better discrimination and calibration compared to existing models, even after local recalibration. Assessment of calibration by variable parsimony revealed performance improvements for increasing variable inclusion well beyond those detectable for discrimination.
Collapse
Affiliation(s)
- Jejo D Koola
- Tennessee Valley Healthcare System (TVHS) VA Medical Center, Veterans Health Administration, Nashville, TN, USA.
- Division of Hospital Medicine, Department of Medicine, University of California, San Diego, CA, USA.
- Health System Department of Biomedical Informatics, University of California, San Diego, CA, USA.
| | - Sam B Ho
- VA San Diego Healthcare System, San Diego, CA, USA
- Division of Gastroenterology, Department of Medicine, University of California, San Diego, CA, USA
- Mohammed Bin Rashid University of Medicine and Health Sciences (MBRU), Dubai, UAE
| | - Aize Cao
- Tennessee Valley Healthcare System (TVHS) VA Medical Center, Veterans Health Administration, Nashville, TN, USA
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Guanhua Chen
- Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, Madison, WI, USA
| | - Amy M Perkins
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Sharon E Davis
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA
| | - Michael E Matheny
- Tennessee Valley Healthcare System (TVHS) VA Medical Center, Veterans Health Administration, Nashville, TN, USA
- Department of Biomedical Informatics, Vanderbilt University Medical Center, Nashville, TN, USA
- Department of Biostatistics, Vanderbilt University Medical Center, Nashville, TN, USA
- Department of Medicine, Vanderbilt University Medical Center, Nashville, TN, USA
| |
Collapse
|