Poor performances of EuroSCORE and CARE score for prediction of perioperative mortality in octogenarians undergoing aortic valve replacement for aortic stenosis

Machine learning-based prediction of 1-year mortality in hypertensive patients undergoing coronary revascularization surgery

BACKGROUND

Machine learning (ML) has shown promising results in all fields of medicine, including preventive cardiology. Hypertensive patients are at higher risk of mortality after coronary artery bypass graft (CABG) surgery; thus, we aimed to design and evaluate five ML models to predict 1-year mortality among hypertensive patients who underwent CABG.

HYOTHESIS

ML algorithms can significantly improve mortality prediction after CABG.

METHODS

Tehran Heart Center's CABG data registry was used to extract several baseline and peri-procedural characteristics and mortality data. The best features were chosen using random forest (RF) feature selection algorithm. Five ML models were developed to predict 1-year mortality: logistic regression (LR), RF, artificial neural network (ANN), extreme gradient boosting (XGB), and naïve Bayes (NB). The area under the curve (AUC), sensitivity, and specificity were used to evaluate the models.

RESULTS

Among the 8,493 hypertensive patients who underwent CABG (mean age of 68.27 ± 9.27 years), 303 died in the first year. Eleven features were selected as the best predictors, among which total ventilation hours and ejection fraction were the leading ones. LR showed the best prediction ability with an AUC of 0.82, while the least AUC was for the NB model (0.79). Among the subgroups, the highest AUC for LR model was for two age range groups (50-59 and 80-89 years), overweight, diabetic, and smoker subgroups of hypertensive patients.

CONCLUSIONS

All ML models had excellent performance in predicting 1-year mortality among CABG hypertension patients, while LR was the best regarding AUC. These models can help clinicians assess the risk of mortality in specific subgroups at higher risk (such as hypertensive ones).

Cardiac surgery risk prediction using ensemble machine learning to incorporate legacy risk scores: A benchmarking study

Objective

The introduction of new clinical risk scores (e.g. European System for Cardiac Operative Risk Evaluation (EuroSCORE) II) superseding original scores (e.g. EuroSCORE I) with different variable sets typically result in disparate datasets due to high levels of missingness for new score variables prior to time of adoption. Little is known about the use of ensemble learning to incorporate disparate data from legacy scores. We tested the hypothesised that Homogenenous and Heterogeneous Machine Learning (ML) ensembles will have better performance than ensembles of Dynamic Model Averaging (DMA) for combining knowledge from EuroSCORE I legacy data with EuroSCORE II data to predict cardiac surgery risk.

Methods

Using the National Adult Cardiac Surgery Audit dataset, we trained 12 different base learner models, based on two different variable sets from either EuroSCORE I (LogES) or EuroScore II (ES II), partitioned by the time of score adoption (1996-2016 or 2012-2016) and evaluated on holdout set (2017-2019). These base learner models were ensembled using nine different combinations of six ML algorithms to produce homogeneous or heterogeneous ensembles. Performance was assessed using a consensus metric.

Results

Xgboost homogenous ensemble (HE) was the highest performing model (clinical effectiveness metric (CEM) 0.725) with area under the curve (AUC) (0.8327; 95% confidence interval (CI) 0.8323-0.8329) followed by Random Forest HE (CEM 0.723; AUC 0.8325; 95%CI 0.8320-0.8326). Across different heterogenous ensembles, significantly better performance was obtained by combining siloed datasets across time (CEM 0.720) than building ensembles of either 1996-2011 (t-test adjusted, p = 1.67×10-6) or 2012-2019 (t-test adjusted, p = 1.35×10-193) datasets alone.

Conclusions

Both homogenous and heterogenous ML ensembles performed significantly better than DMA ensemble of Bayesian Update models. Time-dependent ensemble combination of variables, having differing qualities according to time of score adoption, enabled previously siloed data to be combined, leading to increased power, clinical interpretability of variables and usage of data.

The performance of HEAR score for identification of low-risk chest pain: a systematic review and meta-analysis

Chest pain is one of the most common presentations to the emergency department (ED) and HEART score (history, ECG, age, risk factors, and cardiac troponin) is recommended for risk stratification. It has been proposed that the sum of four items with no troponin (HEAR score) below 2 can be used safely to lower testing and reduce length of stay. To assess the performance of the HEAR score in hospital and prehospital settings, we performed a systematic review and meta-analysis. English studies on the performance of the HEAR score in patients with acute chest pain were included. They were excluded if data are inaccessible. MEDLINE, Embase, Evidence-Based Medicine Reviews, Scopus, and web of science were searched from 1946 to July 2021. The quality of studies was assessed using Quality Assessment of Diagnostic Accuracy Studies version 2. Acute coronary syndrome or major adverse cardiac events prediction were outcomes of interest. The performance indices with 95% confidence intervals (CIs) were extracted. Inverse variance and the random-effects model were used to report the results. Of the 692 articles on the HEAR score, 10 studies were included in the analysis with 33 843 patients. Studies were at low to moderate risk of bias. Three studies were in prehospital and three were retrospective. The pooling of data on the HEAR score showed that the sensitivity at the HEAR<2, <3, and <4 cutoffs in the ED were 99.03% (95% CI, 98.29-99.77), 97.54% (95% CI, 94.50-100), and 91.80% (95% CI, 84.62-98.98), respectively. The negative predictive values (NPVs) for the above cutoffs were 99.84% (95% CI, 99.72-99.95), 99.75% (95% CI, 99.65-99.85), and 99.57% (95% CI, 99.11-100), respectively. Of note, for the HEAR<2, negative likelihood ratio was 0.07 (95% CI, 0.02-0.12). In the prehospital, at the HEAR<4 cutoff, the pooled sensitivity and NPV were 85.01% (95% CI, 80.56-89.47) and 91.48% (95% CI, 87.10-95.87), respectively. This study showed that in the ED, the HEAR score<2 can be used for an early discharge strategy. Currently, this score cannot be recommended in prehospital setting. Prospero (CRD42021273710).

Machine Learning Methods for Predicting Long-Term Mortality in Patients After Cardiac Surgery

Objective:

This study aims to construct and validate several machine learning (ML) algorithms to predict long-term mortality and identify risk factors in unselected patients post-cardiac surgery.

Methods

The Medical Information Mart for Intensive Care (MIMIC-III) database was used to perform a retrospective administrative database study. Candidate predictors consisted of the demographics, comorbidity, vital signs, laboratory test results, scoring systems, and treatment information on the first day of ICU admission. Four-year mortality was set as the study outcome. We used the ML methods of logistic regression (LR), artificial neural network (NNET), naïve bayes (NB), gradient boosting machine (GBM), adapting boosting (Ada), random forest (RF), bagged trees (BT), and eXtreme Gradient Boosting (XGB). The prognostic capacity and clinical utility of these ML models were compared using the area under the receiver operating characteristic curves (AUC), calibration curves, and decision curve analysis (DCA).

Results

Of 7,368 patients in MIMIC-III included in the final cohort, a total of 1,337 (18.15%) patients died during a 4-year follow-up. Among 65 variables extracted from the database, a total of 25 predictors were selected using recursive feature elimination and included in the subsequent analysis. The Ada model performed best among eight models in both discriminatory ability with the highest AUC of 0.801 and goodness of fit (visualized by calibration curve). Moreover, the DCA shows that the net benefit of the RF, Ada, and BT models surpassed that of other ML models for almost all threshold probability values. Additionally, through the Ada technique, we determined that red blood cell distribution width (RDW), blood urea nitrogen (BUN), SAPS II, anion gap (AG), age, urine output, chloride, creatinine, congestive heart failure, and SOFA were the Top 10 predictors in the feature importance rankings.

Conclusions

The Ada model performs best in predicting 4-year mortality after cardiac surgery among the eight ML models, which might have significant application in the development of early warning systems for patients following operations.

Development of machine learning models for mortality risk prediction after cardiac surgery

BACKGROUND

We developed machine learning models that combine preoperative and intraoperative risk factors to predict mortality after cardiac surgery.

METHODS

Machine learning involving random forest, neural network, support vector machine, and gradient boosting machine was developed and compared with the risk scores of EuroSCORE I and II, Society of Thoracic Surgeons (STS), as well as a logistic regression model. Clinical data were collected from patients undergoing adult cardiac surgery at the First Medical Centre of Chinese PLA General Hospital between December 2008 and December 2017. The primary outcome was post-operative mortality. Model performance was estimated using several metrics, including sensitivity, specificity, accuracy, and area under the receiver operating characteristic curve (AUC). The visualization algorithm was implemented using Shapley's additive explanations.

RESULTS

A total of 5,443 patients were enrolled during the study period. The mean EuroSCORE II score was 3.7%, and the actual in-hospital mortality rate was 2.7%. For predicting operative mortality after cardiac surgery, the AUC scores were 0.87, 0.79, 0.81, and 0.82 for random forest, neural network, support vector machine, and gradient boosting machine, compared with 0.70, 0.73, 0.71, and 0.74 for EuroSCORE I and II, STS, and logistic regression model. Shapley's additive explanations analysis of random forest yielded the top-20 predictors and individual-level explanations for each prediction.

CONCLUSIONS

Machine learning models based on available clinical data may be superior to clinical scoring tools in predicting postoperative mortality in patients following cardiac surgery. Explanatory models show the potential to provide personalized risk profiles for individuals by accounting for the contribution of influencing factors. Additional prospective multicenter studies are warranted to confirm the clinical benefit of these machine learning-driven models.

Impact of risk factors on in-hospital mortality for octogenarians undergoing cardiac surgery

BACKGROUND

Octogenarians undergoing cardiac surgery have higher mortality than their younger counterparts.

OBJECTIVES

To determine if various risk factors have the same effect on mortality in octogenarians as in younger patients.

METHODS

The National Inpatient Sample data set from 2004 to 2014 was queried to select patients aged 65 years and older who underwent either coronary artery bypass grafting (CABG), valvular heart surgery (VHS), or both (CABG + VHS) within 10 days of hospital admission. The patients were divided into two groups 65-79 years and 80 years and greater. Hospital mortality, patient demographics, comorbidities, and type of hospital admission was evaluated and compared using χ² and multivariable logistic regressions.

RESULTS

About 397,713 patients were identified including 86,345 (21.7%) aged 80 and above. Octogenarians had higher in-hospital mortality for all procedures: CABG (4.94% vs. 2.39%, p < .001), VHS (5.49% vs. 4.08%, p < .001), and CABG + VHS (7.59% vs. 5.95%, p < .001), and this relationship persisted when gender, race, comorbidities, and type of hospital admission were controlled for: CABG (odds ratio [OR] = 1.71; 95% confidence interval [CI] 1.62-1.81); VHS (OR = 1.18; 95% CI 1.11-1.27); and CABH + VHS (OR = 1.17; 95%CI 1.10-1.26). Female gender, renal, or heart failure, nonelective admission, and CABG + VHS were associated with increased risk of in-hospital mortality. Octogenarians had higher rates of these factors (p < .001). The effect size of renal and heart failure and type of surgery was smaller for octogenarians.

CONCLUSIONS

Octogenarians undergoing cardiac surgery have higher rates of nonelective admissions, renal and heart failure, and female gender, which are most strongly associated with in-hospital mortality. Differing effect sizes suggest that certain risk factors, such as renal and heart failure, contribute more to mortality in younger patients.

Is the EuroSCORE II reliable to estimate operative mortality among octogenarians?

Objectives

Concerns have been raised about the predictive performance (PP) of the EuroSCORE I (ES I) to estimate operative mortality (OM) of patients aged ≥80. The EuroSCORE II (ES II) has been described to have better PP of OM but external validations are scarce. Furthermore, the PP of ES II has not been investigated among the octogenarians. The goal of the study was to compare the PP of ES II and ES I among the overall population and patients ≥ 80.

Methods

The ES I and ES II were computed for 7161 consecutive patients who underwent major cardiac surgery in a 7-year period. Discrimination was assessed by using the c- index and calibration with the Hosmer-Lemeshow (HL) and calibration plot by comparing predicted and observed mortality.

Results

From the global cohort of 7161 patients, 832 (12%) were ≥80. The mean values of ES I and ES II were 7.4±9.4 and 5.2±9.1 respectively for the whole cohort, 6.3±8.6 and 4.7±8.5 for the patients <80, 15.1±11.8 and 8.5±11.0 for the patients ≥80. The mortality was 9.38% (≥80) versus 5.18% (<80). The discriminatory power was good for the two algorithms among the whole population and the <80 but less satisfying among the ≥80 (AUC 0.64 [0.58–0.71] for ES I and 0.67 [0.60–0.73] for the ES II without significant differences (p = 0.35) between the two scores. For the octogenarians, the ES II had a fair calibration until 10%-predicted values and over-predicted beyond.

Conclusions

The ES II has a better PP than the ES I among patients <80. Its discrimination and calibration are less satisfying in patients ≥80, showing an overestimation in the elderly at very high-surgical risk. Nevertheless, it shows an acceptable calibration until 10%- predicted mortality.

Postoperative Outcome of High-Risk Octogenarians Undergoing Cardiac Surgery: A Multicenter Observational Retrospective Study

PURPOSE

Cardiac surgery in octogenarians with severely deteriorated functional status is increasingly common, but outcome data are still limited. The aim of this study was to compare postoperative outcome, survival, and quality of life of low-, medium-, and high-risk octogenarians undergoing cardiac surgery.

METHODS

In all, 285 Czech octogenarians who underwent any cardiac surgical procedure between January 2011 and December 2012 were included in the study. Five out of all twelve national adult cardiac surgical centers participated in the study, representing almost half of all octogenarians operated in our country in that period. Patients' perioperative data were analyzed retrospectively. Follow-up was performed by interviewing patients by telephone.

RESULTS

There was higher 30-day mortality (20% vs. 6.4% vs. 5.2%, respectively, p <0.001), lower 2-year survival (60% vs. 84.0% vs. 85.4%, respectively, p <0.05), and lower Karnofsky score (44.4 vs. 70.1 vs. 70.6, respectively, p <0.001) in high-risk group compared with medium- and low-risk groups, respectively. Greater improvement in New York Heart Association (NYHA) status was noted in high- and medium-risk groups compared with low-risk group (51% vs. 45% vs. 24%, respectively, p <0.05).

CONCLUSION

High perioperative mortality, poor 2-year survival, and low postoperative quality of life have been observed in high-risk octogenarians undergoing cardiac surgery.

A Comparison of a Machine Learning Model with EuroSCORE II in Predicting Mortality after Elective Cardiac Surgery: A Decision Curve Analysis

Background

The benefits of cardiac surgery are sometimes difficult to predict and the decision to operate on a given individual is complex. Machine Learning and Decision Curve Analysis (DCA) are recent methods developed to create and evaluate prediction models.

Methods and finding

We conducted a retrospective cohort study using a prospective collected database from December 2005 to December 2012, from a cardiac surgical center at University Hospital. The different models of prediction of mortality in-hospital after elective cardiac surgery, including EuroSCORE II, a logistic regression model and a machine learning model, were compared by ROC and DCA. Of the 6,520 patients having elective cardiac surgery with cardiopulmonary bypass, 6.3% died. Mean age was 63.4 years old (standard deviation 14.4), and mean EuroSCORE II was 3.7 (4.8) %. The area under ROC curve (IC95%) for the machine learning model (0.795 (0.755–0.834)) was significantly higher than EuroSCORE II or the logistic regression model (respectively, 0.737 (0.691–0.783) and 0.742 (0.698–0.785), p < 0.0001). Decision Curve Analysis showed that the machine learning model, in this monocentric study, has a greater benefit whatever the probability threshold.

Conclusions

According to ROC and DCA, machine learning model is more accurate in predicting mortality after elective cardiac surgery than EuroSCORE II. These results confirm the use of machine learning methods in the field of medical prediction.

Prognosis of Prolonged Intensive Care Unit Stay After Aortic Valve Replacement for Severe Aortic Stenosis in Octogenarians

OBJECTIVES

Octogenarians considered for cardiac surgery encounter more complications than other patients. Postoperative complications raise the question of continuation of high-cost care for patients with limited life expectancy. Duration of hospitalization in intensive care after cardiac surgery may differ between octogenarians and other patients. The objectives were evaluating the mortality rate of octogenarians experiencing prolonged hospitalization in intensive care and defining the best cut-off for prolonged intensive care unit length of stay.

DESIGN

A single-center observational study.

SETTING

A postoperative surgical intensive care unit in a tertiary teaching hospital in Paris, France.

PARTICIPANTS

All consecutive patients older than 80 years considered for aortic valve replacement for aortic stenosis were included.

MEASUREMENTS AND MAIN RESULTS

Mortality rate was determined among patients experiencing prolonged stay in intensive care with organ failure and without organ failure. An ROC curve determined the optimal cut-off defining prolonged hospitalization in intensive care according to the occurrence of postoperative complications. Multivariate analysis determined risk factors for early death or prolonged intensive care stay. The optimal cut-off defining prolonged intensive care unit length of stay was 4 days. Low ventricular ejection fraction (odds ratio [OR] = 0.95; 95% confidence interval [CI] 0.96-0.83; p = 0.0016), coronary disease (OR = 2.34; 95% CI 1.19-4.85; p = 0.014), and need for catecholamine (OR = 2.79; 95% CI 1.33-5.88; p = 0.0068) were associated with eventful postoperative course. There was not a hospitalization duration beyond which the prognosis significantly worsened.

CONCLUSIONS

Prolonged length of stay in ICU without organ failure is not associated with increased mortality. No specific duration of hospitalization in intensive care was associated with increased mortality. Continuation of care should be discussed on an individual basis.

The role of age and comorbidities in postoperative outcome of mitral valve repair: A propensity-matched study

The average age of patients undergoing mitral valve repair is increasing each year. This retrospective study aimed to compare postoperative complications of mitral valve repair (known to be especially high-risk) between 2 age groups: under and over the age of 80.Patients who underwent mitral valve repair were divided into 2 groups: group 1 (<80 years old) and group 2 (≥80 years old). Baseline characteristics, pre- and postoperative hemodynamic data, surgical characteristics, and postoperative follow-up data until hospital discharge were collected.A total of 308 patients were included: 264 in group 1 (age 63 ± 13 years) and 44 in group 2 (age 83 ± 2 years). Older patients had more comorbidities (atrial fibrillation, history of cardiac decompensation, systemic hypertension, pulmonary hypertension, and chronic kidney disease) and they presented more postoperative complications (50.0% vs 33.7%; P = 0.043), with a longer hospital stay (8.9 ± 6.9 vs 6.6 ± 4.6 days; P = 0.005). To assess the burden of age, a propensity score was awarded to postoperative complications. Active smoking, chronic pulmonary disease, chronic kidney disease, associated ischemic heart disease, obesity, and cardio pulmonary by-pass duration were described as independent risk factors. When matched on this propensity score, there was no difference in morbidity or mortality between group 1 and group 2.Older patients suffered more postoperative complications, which were related to their comorbidities and not only to their age.

Long-term survival, autonomy, and quality of life of elderly patients undergoing aortic valve replacement

AIMS

We reviewed the long-term survival, autonomy, and quality of life (QoL) of elderly patients undergoing aortic valve replacement (AVR).

METHODS

Records of patients ≥75 years old that underwent AVR from 2002 to 2006 were retrospectively analyzed. Functional status was classified with Barthel Index (BI). QoL was presumed as the self-perception of well-being after AVR. Independent predictors of mortality were identified using the Cox proportional hazards model.

RESULTS

We included 114 patients, with a mean age of 78.5 ± 2.5 years. Seventy (59.8%) patients were females. Mean additive and logistic EuroSCORE were 7 ± 2 and 9 ± 7, respectively. Follow-up on vital status was achieved for 113 (99.1%) patients after a mean period of 47.2 ± 23.4 months. Twenty-seven (23.7%) patients died (including three operative deaths). Survival up to one, three, and five years of follow-up was 94.4%, 86.7%, and 76.1%, respectively. Multivariate analysis showed that pulmonary hypertension and diabetes were independent predictors of all-cause mortality. Information on BI score and QoL was obtained for 77 (89.5%) and patients. Among those, 69 (89.6%) were autonomous according to BI and 72 (93.5%) considered having had an improvement in QoL.

CONCLUSION

Patients ≥75 years old undergoing AVR presented good medium-term survival. Predictors of an adverse outcome were significant pulmonary hypertension and diabetes. At follow-up, most achieved improvement of QoL and remained autonomous. These results stress that excellent long-term outcomes with AVR can be achieved in appropriately selected elderly patients.