Adverse Outcomes Prediction for Congenital Heart Surgery: A Machine Learning Approach

Congenital Heart Surgery Machine Learning-Derived In-Depth Benchmarking Tool

BACKGROUND

We previously showed that machine learning-based methodologies of optimal classification trees (OCTs) can accurately predict risk after congenital heart surgery and assess case-mix-adjusted performance after benchmark procedures. We extend this methodology to provide interpretable, easily accessible, and actionable hospital performance analysis across all procedures.

METHODS

The European Congenital Heart Surgeons Association Congenital Cardiac Database data subset of 172,888 congenital cardiac surgical procedures performed in European centers between 1989 and 2022 was analyzed. OCT models (decision trees) were built predicting hospital mortality (area under the curve [AUC], 0.866), prolonged postoperative mechanical ventilatory support time (AUC, 0.851), or hospital length of stay (AUC, 0.818), thereby establishing case-adjusted benchmarking standards reflecting the overall performance of all participating hospitals, designated as the "virtual hospital." OCT analysis of virtual hospital aggregate data yielded predicted expected outcomes (both aggregate and for risk-matched patient cohorts) for the individual hospital's own specific case-mix, readily available on-line.

RESULTS

Raw average rates were hospital mortality, 4.9%; mechanical ventilatory support time, 14.5%; and length of stay, 15.0%. Of 146 participating centers, compared with each hospital's overall case-adjusted predicted hospital mortality benchmark, 20.5% statistically (<90% CI) overperformed and 20.5% underperformed. An interactive tool based on the OCT analysis automatically reveals 14 hospital-specific patient cohorts, simultaneously assessing overperformance or underperformance, and enabling further analysis of cohort strata in any chosen time frame.

CONCLUSIONS

Machine learning-based OCT benchmarking analysis provides automatic assessment of hospital-specific case-adjusted performance after congenital heart surgery, not only overall but importantly, also by similar risk patient cohorts. This is a tool for hospital self-assessment, particularly facilitated by the user-accessible online-platform.

Center-Level Variation in Failure to Rescue After Pediatric Cardiac Surgery

BACKGROUND

Although failure to rescue (FTR) is increasingly recognized as a quality metric, studies in congenital cardiac surgery remain sparse. Within a national cohort of children undergoing cardiac operations, we characterized the presence of center-level variation in FTR and hypothesized a strong association with mortality but not complications.

METHODS

All children undergoing congenital cardiac operations were identified in the 2013 to 2019 Nationwide Readmissions Database. FTR was defined as in-hospital death after cardiac arrest, ventricular tachycardia/fibrillation, prolonged mechanical ventilation, pneumonia, stroke, venous thromboembolism, or sepsis, among other complications. Hierarchical models were used to generate hospital-specific, risk-adjusted rates of mortality, complications, and FTR. Centers in the highest decile of FTR were identified and compared with others.

RESULTS

Of an estimated 74,070 patients, 1.9% died before discharge, at least 1 perioperative complication developed in 43.0%, and 4.1% experienced FTR. After multilevel modeling, decreasing age, nonelective admission, and increasing operative complexity were associated with greater odds of FTR. Variations in overall mortality and FTR exhibited a strong, positive relationship (r = 0.97), whereas mortality and complications had a negligible association (r = -0.02). Compared with others, patients at centers with high rates of FTR had similar distributions of age, sex, chronic conditions, and operative complexity.

CONCLUSIONS

In the present study, center-level variations in mortality were more strongly explained by differences in FTR than complications. Our findings suggest the utility of FTR as a quality metric for congenital heart surgery, although further study is needed to develop a widely accepted definition and appropriate risk-adjustment models.

Predictors of in-ICU length of stay among congenital heart defect patients using artificial intelligence model: A pilot study

Objective

This study aims to develop a predictive model using artificial intelligence to estimate the ICU length of stay (LOS) for Congenital Heart Defects (CHD) patients after surgery, improving care planning and resource management.

Design

We analyze clinical data from 2240 CHD surgery patients to create and validate the predictive model. Twenty AI models are developed and evaluated for accuracy and reliability.

Setting

The study is conducted in a Brazilian hospital's Cardiovascular Surgery Department, focusing on transplants and cardiopulmonary surgeries.

Participants

Retrospective analysis is conducted on data from 2240 consecutive CHD patients undergoing surgery.

Interventions

Ninety-three pre and intraoperative variables are used as ICU LOS predictors.

Measurements and main results

Utilizing regression and clustering methodologies for ICU LOS (ICU Length of Stay) estimation, the Light Gradient Boosting Machine, using regression, achieved a Mean Squared Error (MSE) of 15.4, 11.8, and 15.2 days for training, testing, and unseen data. Key predictors included metrics such as "Mechanical Ventilation Duration", "Weight on Surgery Date", and "Vasoactive-Inotropic Score". Meanwhile, the clustering model, Cat Boost Classifier, attained an accuracy of 0.6917 and AUC of 0.8559 with similar key predictors.

Conclusions

Patients with higher ventilation times, vasoactive-inotropic scores, anoxia time, cardiopulmonary bypass time, and lower weight, height, BMI, age, hematocrit, and presurgical oxygen saturation have longer ICU stays, aligning with existing literature.

Technologies to Study Genetics and Molecular Pathways

Over the last few decades, the study of congenital heart disease (CHD) has benefited from various model systems and the development of molecular biological techniques enabling the analysis of single gene as well as global effects. In this chapter, we first describe different models including CHD patients and their families, animal models ranging from invertebrates to mammals, and various cell culture systems. Moreover, techniques to experimentally manipulate these models are discussed. Second, we introduce cardiac phenotyping technologies comprising the analysis of mouse and cell culture models, live imaging of cardiogenesis, and histological methods for fixed hearts. Finally, the most important and latest molecular biotechniques are described. These include genotyping technologies, different applications of next-generation sequencing, and the analysis of transcriptome, epigenome, proteome, and metabolome. In summary, the models and technologies presented in this chapter are essential to study the function and development of the heart and to understand the molecular pathways underlying CHD.

Uncovering the Genetic Basis of Congenital Heart Disease: Recent Advancements and Implications for Clinical Management

Congenital heart disease (CHD) is the most prevalent hereditary disorder, affecting approximately 1% of all live births. A reduction in morbidity and mortality has been achieved with advancements in surgical intervention, yet challenges in managing complications, extracardiac abnormalities, and comorbidities still exist. To address these, a more comprehensive understanding of the genetic basis underlying CHD is required to establish how certain variants are associated with the clinical outcomes. This will enable clinicians to provide personalized treatments by predicting the risk and prognosis, which might improve the therapeutic results and the patient's quality of life. We review how advancements in genome sequencing are changing our understanding of the genetic basis of CHD, discuss experimental approaches to determine the significance of novel variants, and identify barriers to use this knowledge in the clinics. Next-generation sequencing technologies are unravelling the role of oligogenic inheritance, epigenetic modification, genetic mosaicism, and noncoding variants in controlling the expression of candidate CHD-associated genes. However, clinical risk prediction based on these factors remains challenging. Therefore, studies involving human-induced pluripotent stem cells and single-cell sequencing help create preclinical frameworks for determining the significance of novel genetic variants. Clinicians should be aware of the benefits and implications of the responsible use of genomics. To facilitate and accelerate the clinical integration of these novel technologies, clinicians should actively engage in the latest scientific and technical developments to provide better, more personalized management plans for patients.

World Journal for Pediatric and Congenital Heart Surgery-The Official Journal of the Congenital Heart Surgeons' Society

The World Journal for Pediatric and Congenital Heart Surgery (WJPCHS) was established in 2009, as a means of advancing the educational and scholarship goals of the World Society for Pediatric and Congenital Heart Surgery. WJPCHS has grown steadily since the first issue was published in April 2010. In 2017, the Congenital Heart Surgeons' Society and the European Congenital Heart Surgeons Association both designated WJPCHS as the official journal of their respective organizations. The CHSS and ECHSA represent the face and the voice of congenital heart surgery in North America (United States and Canada) and in Europe, respectively. Each organization has advanced the science of surgical management of congenital heart disease through multicenter outcomes analyses, which have strongly and positively influenced the care of patients around the world.

Combining Congenital Heart Surgical and Interventional Cardiology Outcome Data in a Single Database: The Development of a Patient-Centered Collaboration of the European Congenital Heart Surgeons Association (ECHSA) and the Association for European Paediatric and Congenital Cardiology (AEPC)

The European Congenital Heart Surgeons Association (ECHSA) Congenital Database (CD) is the second largest clinical pediatric and congenital cardiac surgical database in the world and the largest in Europe, where various smaller national or regional databases exist. Despite the dramatic increase in interventional cardiology procedures over recent years, only scattered national or regional databases of such procedures exist in Europe. Most importantly, no congenital cardiac database exists in the world that seamlessly combines both surgical and interventional cardiology data on an international level; therefore, the outcomes of surgical and interventional procedures performed on the same or similar patients cannot easily be tracked, assessed, and analyzed. In order to fill this important gap in our capability to gather and analyze information on our common patients, ECHSA and The Association for European Paediatric and Congenital Cardiology (AEPC) have embarked on a collaborative effort to expand the ECHSA-CD with a new module designed to capture data about interventional cardiology procedures. The purpose of this manuscript is to describe the concept, the structure, and the function of the new AEPC Interventional Cardiology Part of the ECHSA-CD, as well as the potentially valuable synergies provided by the shared interventional and surgical analyses of outcomes of patients. The new AEPC Interventional Cardiology Part of the ECHSA-CD will allow centers to have access to robust surgical and transcatheter outcome data from their own center, as well as robust national and international aggregate outcome data for benchmarking. Each contributing center or department will have access to their own data, as well as aggregate data from the AEPC Interventional Cardiology Part of the ECHSA-CD. The new AEPC Interventional Cardiology Part of the ECHSA-CD will allow cardiology centers to have access to aggregate cardiology data, just as surgical centers already have access to aggregate surgical data. Comparison of surgical and catheter interventional outcomes could potentially strengthen decision processes. A study of the wealth of information collected in the database could potentially also contribute toward improved early and late survival, as well as enhanced quality of life of patients with pediatric and/or congenital heart disease treated with surgery and interventional cardiac catheterization across Europe and the world.

Changing epidemiology of congenital heart disease: effect on outcomes and quality of care in adults

The epidemiology of congenital heart disease (CHD) has changed in the past 50 years as a result of an increase in the prevalence and survival rate of CHD. In particular, mortality in patients with CHD has changed dramatically since the latter half of the twentieth century as a result of more timely diagnosis and the development of interventions for CHD that have prolonged life. As patients with CHD age, the disease burden shifts away from the heart and towards acquired cardiovascular and systemic complications. The societal costs of CHD are high, not just in terms of health-care utilization but also with regards to quality of life. Lifespan disease trajectories for populations with a high disease burden that is measured over prolonged time periods are becoming increasingly important to define long-term outcomes that can be improved. Quality improvement initiatives, including advanced physician training for adult CHD in the past 10 years, have begun to improve disease outcomes. As we seek to transform lifespan into healthspan, research efforts need to incorporate big data to allow high-value, patient-centred and artificial intelligence-enabled delivery of care. Such efforts will facilitate improved access to health care in remote areas and inform the horizontal integration of services needed to manage CHD for the prolonged duration of survival among adult patients.

Use of machine learning in pediatric surgical clinical prediction tools: A systematic review

PURPOSE

Clinical prediction tools (CPTs) are decision-making instruments utilizing patient data to predict specific clinical outcomes, risk-stratify patients, or suggest personalized diagnostic or therapeutic options. Recent advancements in artificial intelligence have resulted in a proliferation of CPTs created using machine learning (ML)-yet the clinical applicability of ML-based CPTs and their validation in clinical settings remain unclear. This systematic review aims to compare the validity and clinical efficacy of ML-based to traditional CPTs in pediatric surgery.

METHODS

Nine databases were searched from 2000 until July 9, 2021 to retrieve articles reporting on CPTs and ML for pediatric surgical conditions. PRISMA standards were followed, and screening was performed by two independent reviewers in Rayyan, with a third reviewer resolving conflicts. Risk of bias was assessed using the PROBAST.

RESULTS

Out of 8300 studies, 48 met the inclusion criteria. The most represented surgical specialties were pediatric general (14), neurosurgery (13) and cardiac surgery (12). Prognostic (26) CPTs were the most represented type of surgical pediatric CPTs followed by diagnostic (10), interventional (9), and risk stratifying (2). One study included a CPT for diagnostic, interventional and prognostic purposes. 81% of studies compared their CPT to ML-based CPTs, statistical CPTs, or the unaided clinician, but lacked external validation and/or evidence of clinical implementation.

CONCLUSIONS

While most studies claim significant potential improvements by incorporating ML-based CPTs in pediatric surgical decision-making, both external validation and clinical application remains limited. Further studies must focus on validating existing instruments or developing validated tools, and incorporating them in the clinical workflow.

TYPE OF STUDY

Systematic Review LEVEL OF EVIDENCE: Level III.

Artificial Intelligence in Congenital Heart Disease: Current State and Prospects

The current era of big data offers a wealth of new opportunities for clinicians to leverage artificial intelligence to optimize care for pediatric and adult patients with a congenital heart disease. At present, there is a significant underutilization of artificial intelligence in the clinical setting for the diagnosis, prognosis, and management of congenital heart disease patients. This document is a call to action and will describe the current state of artificial intelligence in congenital heart disease, review challenges, discuss opportunities, and focus on the top priorities of artificial intelligence-based deployment in congenital heart disease.

Artificial Intelligence in Pediatric Cardiology: A Scoping Review

The evolution of AI and data science has aided in mechanizing several aspects of medical care requiring critical thinking: diagnosis, risk stratification, and management, thus mitigating the burden of physicians and reducing the likelihood of human error. AI modalities have expanded feet to the specialty of pediatric cardiology as well. We conducted a scoping review searching the Scopus, Embase, and PubMed databases covering the recent literature between 2002-2022. We found that the use of neural networks and machine learning has significantly improved the diagnostic value of cardiac magnetic resonance imaging, echocardiograms, computer tomography scans, and electrocardiographs, thus augmenting the clinicians' diagnostic accuracy of pediatric heart diseases. The use of AI-based prediction algorithms in pediatric cardiac surgeries improves postoperative outcomes and prognosis to a great extent. Risk stratification and the prediction of treatment outcomes are feasible using the key clinical findings of each CHD with appropriate computational algorithms. Notably, AI can revolutionize prenatal prediction as well as the diagnosis of CHD using the EMR (electronic medical records) data on maternal risk factors. The use of AI in the diagnostics, risk stratification, and management of CHD in the near future is a promising possibility with current advancements in machine learning and neural networks. However, the challenges posed by the dearth of appropriate algorithms and their nascent nature, limited physician training, fear of over-mechanization, and apprehension of missing the 'human touch' limit the acceptability. Still, AI proposes to aid the clinician tomorrow with precision cardiology, paving a way for extremely efficient human-error-free health care.

Machine Learning-Based Systems for the Anticipation of Adverse Events After Pediatric Cardiac Surgery

Pediatric congenital heart disease (CHD) patients are at higher risk of postoperative complications and clinical deterioration either due to their underlying pathology or due to the cardiac surgery, contributing significantly to mortality, morbidity, hospital and family costs, and poor quality of life. In current clinical practice, clinical deterioration is detected, in most of the cases, when it has already occurred. Several early warning scores (EWS) have been proposed to assess children at risk of clinical deterioration using vital signs and risk indicators, in order to intervene in a timely manner to reduce the impact of deterioration and risk of death among children. However, EWS are based on measurements performed at a single time point without incorporating trends nor providing information about patient's risk trajectory. Moreover, some of these measurements rely on subjective assessment making them susceptible to different interpretations. All these limitations could explain why the implementation of EWS in high-resource settings failed to show a significant decrease in hospital mortality. By means of machine learning (ML) based algorithms we could integrate heterogeneous and complex data to predict patient's risk of deterioration. In this perspective article, we provide a brief overview of the potential of ML technologies to improve the identification of pediatric CHD patients at high-risk for clinical deterioration after cardiac surgery, and present the CORTEX traffic light, a ML-based predictive system that Sant Joan de Déu Barcelona Children's Hospital is implementing, as an illustration of the application of an ML-based risk stratification system in a relevant hospital setting.

Artificial intelligence in pediatric cardiology: taking baby steps in the big world of data

PURPOSE OF REVIEW

Artificial intelligence (AI) has changed virtually every aspect of modern life, and medicine is no exception. Pediatric cardiology is both a perceptual and a cognitive subspecialty that involves complex decision-making, so AI is a particularly attractive tool for this medical discipline. This review summarizes the foundational work and incremental progress made as AI applications have emerged in pediatric cardiology since 2020.

RECENT FINDINGS

AI-based algorithms can be useful for pediatric cardiology in many areas, including: (1) clinical examination and diagnosis, (2) image processing, (3) planning and management of cardiac interventions, (4) prognosis and risk stratification, (5) omics and precision medicine, and (6) fetal cardiology. Most AI initiatives showcased in medical journals seem to work well in silico, but progress toward implementation in actual clinical practice has been more limited. Several barriers to implementation are identified, some encountered throughout medicine generally, and others specific to pediatric cardiology.

SUMMARY

Despite barriers to acceptance in clinical practice, AI is already establishing a durable role in pediatric cardiology. Its potential remains great, but to fully realize its benefits, substantial investment to develop and refine AI for pediatric cardiology applications will be necessary to overcome the challenges of implementation.

Medicine-Based Evidence in Congenital Heart Disease: How Artificial Intelligence Can Guide Treatment Decisions for Individual Patients

Built on the foundation of the randomized controlled trial (RCT), Evidence Based Medicine (EBM) is at its best when optimizing outcomes for homogeneous cohorts of patients like those participating in an RCT. Its weakness is a failure to resolve a clinical quandary: patients appear for care individually, each may differ in important ways from an RCT cohort, and the physician will wonder each time if following EBM will provide best guidance for this unique patient. In an effort to overcome this weakness, and promote higher quality care through a more personalized approach, a new framework has been proposed: Medicine-Based Evidence (MBE). In this approach, big data and deep learning techniques are embraced to interrogate treatment responses among patients in real-world clinical practice. Such statistical models are then integrated with mechanistic disease models to construct a “digital twin,” which serves as the real-time digital counterpart of a patient. MBE is thereby capable of dynamically modeling the effects of various treatment decisions in the context of an individual's specific characteristics. In this article, we discuss how MBE could benefit patients with congenital heart disease, a field where RCTs are difficult to conduct and often fail to provide definitive solutions because of a small number of subjects, their clinical complexity, and heterogeneity. We will also highlight the challenges that must be addressed before MBE can be embraced in clinical practice and its full potential can be realized.

Benchmarking Cesarean Delivery Rates using Machine Learning-Derived Optimal Classification Trees

OBJECTIVE

To establish a case-adjusted hospital-specific performance evaluation tool using machine learning methodology for cesarean delivery.

DATA SOURCES

Secondary data were collected from patients between 1/1/2015-2/28/2018 using a hospital's "Electronic Data Warehouse" database from Illinois, USA.

STUDY DESIGN

The machine learning methodology of Optimal Classification Trees (OCT's) was used to predict cesarean delivery rate by physician group, thereby establishing the case-adjusted benchmarking standards in comparison to the overall hospital cesarean delivery rate. Outcomes of specific patient populations of each participating practice were predicted, as if each were treated in the overall hospital environment. The resulting OCTs estimate physician group expected cesarean delivery outcomes, both aggregate and in specific clinical situations.

DATA COLLECTION/EXTRACTION METHODS

12,841 singleton, vertex, term deliveries, cared for by practices with ≥50 births.

PRINCIPAL FINDINGS

The overall rate of cesarean delivery was 18.6% (n = 2384), with a range of 13.3%-33.7% amongst 22 physician practices. An optimal decision tree was used to create a prediction model for the overall hospital which defined 23 patient cohorts, divided by 46 nodes. The model's performance for prediction of cesarean delivery is as follows: area under the curve- 0.73, sensitivity- 98.4%, specificity- 16.1%, positive predictive value 83.7%, negative predictive value 70.6%. Comparisons with the overall hospital's specific-case adjusted benchmark groups revealed that several groups outperformed the overall hospital and some practice groups underperformed in comparison to the overall hospital.

CONCLUSIONS

OCT benchmarking can assess physician practice specific case-adjusted performance, both overall and clinical situation specific, and can serve as a valuable tool for hospital self-assessment and quality improvement. This article is protected by copyright. All rights reserved.

Benchmarking in Congenital Heart Surgery Using Machine Learning-Derived Optimal Classification Trees

Background: We have previously shown that the machine learning methodology of optimal classification trees (OCTs) can accurately predict risk after congenital heart surgery (CHS). We have now applied this methodology to define benchmarking standards after CHS, permitting case-adjusted hospital-specific performance evaluation. Methods: The European Congenital Heart Surgeons Association Congenital Database data subset (31 792 patients) who had undergone any of the 10 "benchmark procedure group" primary procedures were analyzed. OCT models were built predicting hospital mortality (HM), and prolonged postoperative mechanical ventilatory support time (MVST) or length of hospital stay (LOS), thereby establishing case-adjusted benchmarking standards reflecting the overall performance of all participating hospitals, designated as the "virtual hospital." These models were then used to predict individual hospitals' expected outcomes (both aggregate and, importantly, for risk-matched patient cohorts) for their own specific cases and case-mix, based on OCT analysis of aggregate data from the "virtual hospital." Results: The raw average rates were HM = 4.4%, MVST = 15.3%, and LOS = 15.5%. Of 64 participating centers, in comparison with each hospital's specific case-adjusted benchmark, 17.0% statistically (under 90% confidence intervals) overperformed and 26.4% underperformed with respect to the predicted outcomes for their own specific cases and case-mix. For MVST and LOS, overperformers were 34.0% and 26.4%, and underperformers were 28.3% and 43.4%, respectively. OCT analyses reveal hospital-specific patient cohorts of either overperformance or underperformance. Conclusions: OCT benchmarking analysis can assess hospital-specific case-adjusted performance after CHS, both overall and patient cohort-specific, serving as a tool for hospital self-assessment and quality improvement.

Does It Measure Up?

Measuring outcomes in pediatric cardiac care has been one of the more widespread, and at the same time controversial and often polarizing, quality improvement initiatives undertaken in the medical field. Risk models, such as the Society of Thoracic Surgeons Congenital Heart Surgery Risk Model, have been developed to account for comorbidities while predicting the expected mortality for a given surgical encounter. In this issue of the journal, Bertsimas and colleagues report on machine learning approaches to predict adverse outcomes in congenital heart surgery using the European Congenital Heart Surgeons Association's congenital database. A head-to-head comparison of machine learning models and the currently available risk models utilizing the same data set are required to better understand the strengths and weaknesses of each of these approaches. Such a focused analysis will shed light on future approaches for risk modeling, which will undoubtedly continue to benefit from the guidance provided by expert clinical intuition.