Risk prediction model for in-hospital mortality in women with ST-elevation myocardial infarction: A machine learning approach

Effective analysis of job satisfaction among medical staff in Chinese public hospitals: a random forest model

Objective

This study explored the factors and influence degree of job satisfaction among medical staff in Chinese public hospitals by constructing the optimal discriminant model.

Methods

The participant sample is based on the service volume of 12,405 officially appointed medical staff from different departments of 16 public hospitals for three consecutive years from 2017 to 2019. All medical staff (doctors, nurses, administrative personnel) invited to participate in the survey for the current year will no longer repeat their participation. The importance of all associated factors and the optimal evaluation model has been calculated.

Results

The overall job satisfaction of medical staff is 25.62%. The most important factors affecting medical staff satisfaction are: Value staff opinions (Q10), Get recognition for your work (Q11), Democracy (Q9), and Performance Evaluation Satisfaction (Q5). The random forest model is the best evaluation model for medical staff satisfaction, and its prediction accuracy is higher than other similar models.

Conclusion

The improvement of medical staff job satisfaction is significantly related to the improvement of democracy, recognition of work, and increased employee performance. It has shown that improving these five key variables can maximize the job satisfaction and motivation of medical staff. The random forest model can maximize the accuracy and effectiveness of similar research.

Enhanced Cardiovascular Disease Prediction Modelling using Machine Learning Techniques: A Focus on CardioVitalnet

Aiming at early detection and accurate prediction of cardiovascular disease (CVD) to reduce mortality rates, this study focuses on the development of an intelligent predictive system to identify individuals at risk of CVD. The primary objective of the proposed system is to combine deep learning models with advanced data mining techniques to facilitate informed decision-making and precise CVD prediction. This approach involves several essential steps, including the preprocessing of acquired data, optimized feature selection, and disease classification, all aimed at enhancing the effectiveness of the system. The chosen optimal features are fed as input to the disease classification models and into some Machine Learning (ML) algorithms for improved performance in CVD classification. The experiment was simulated in the Python platform and the evaluation metrics such as accuracy, sensitivity, and F1_score were employed to assess the models' performances. The ML models (Extra Trees (ET), Random Forest (RF), AdaBoost, and XG-Boost) classifiers achieved high accuracies of 94.35%, 97.87%, 96.44%, and 99.00%, respectively, on the test set, while the proposed CardioVitalNet (CVN) achieved 87.45% accuracy. These results offer valuable insights into the process of selecting models for medical data analysis, ultimately enhancing the ability to make more accurate diagnoses and predictions.

Analyzing predictors of in-hospital mortality in patients with acute ST-segment elevation myocardial infarction using an evolved machine learning approach

Acute ST-segment elevation myocardial infarction (STEMI) is a severe cardiac ailment characterized by the sudden complete blockage of a portion of the coronary artery, leading to the interruption of blood supply to the myocardium. This study examines the medical records of 3205 STEMI patients admitted to the coronary care unit of the First Affiliated Hospital of Wenzhou Medical University from January 2014 to December 2021. In this research, a novel predictive framework for STEMI is proposed, incorporating evolutionary computational methods and machine learning techniques. A variant algorithm, AGCOSCA, is introduced by integrating crossover operation and observation bee strategy into the original Sine Cosine Algorithm (SCA). The effectiveness of AGCOSCA is initially validated using IEEE CEC 2017 benchmark functions, demonstrating its ability to mitigate the deficiency in local mining after SCA random perturbation. Building upon this foundation, the AGCOSCA approach has been paired with Support Vector Machine (SVM) to forge the predictive framework referred to as AGCOSCA-SVM. Specifically, AGCOSCA is employed to refine the selection of predictors from a substantial feature set before SVM is utilized to forecast the occurrence of STEMI. In our analysis, we observed that SVM excels at managing nonlinear data relationships, a strength that becomes particularly prominent in smaller datasets of STEMI patients. To assess the effectiveness of AGCOSCA-SVM, diagnostic experiments were conducted based on the STEMI sample data. Results indicate that AGCOSCA-SVM outperforms traditional machine learning methods, achieving superior Accuracy, Sensitivity, and Specificity values of 97.83 %, 93.75 %, and 96.67 %, respectively. The selected features, such as acute kidney injury (AKI) stage, fibrinogen, mean platelet volume (MPV), free triiodothyronine (FT3), diuretics, and Killip class during hospitalization, are identified as crucial for predicting STEMI. In conclusion, AGCOSCA-SVM emerges as a promising model framework for supporting the diagnostic process of STEMI, showcasing potential applications in clinical settings.

Data analytics approach for short- and long-term mortality prediction following acute non-ST-elevation myocardial infarction (NSTEMI) and Unstable Angina (UA) in Asians

BACKGROUND

Traditional risk assessment tools often lack accuracy when predicting the short- and long-term mortality following a non-ST-segment elevation myocardial infarction (NSTEMI) or Unstable Angina (UA) in specific population.

OBJECTIVE

To employ machine learning (ML) and stacked ensemble learning (EL) methods in predicting short- and long-term mortality in Asian patients diagnosed with NSTEMI/UA and to identify the associated features, subsequently evaluating these findings against established risk scores.

METHODS

We analyzed data from the National Cardiovascular Disease Database for Malaysia (2006-2019), representing a diverse NSTEMI/UA Asian cohort. Algorithm development utilized in-hospital records of 9,518 patients, 30-day data from 7,133 patients, and 1-year data from 7,031 patients. This study utilized 39 features, including demographic, cardiovascular risk, medication, and clinical features. In the development of the stacked EL model, four base learner algorithms were employed: eXtreme Gradient Boosting (XGB), Support Vector Machine (SVM), Naive Bayes (NB), and Random Forest (RF), with the Generalized Linear Model (GLM) serving as the meta learner. Significant features were chosen and ranked using ML feature importance with backward elimination. The predictive performance of the algorithms was assessed using the area under the curve (AUC) as a metric. Validation of the algorithms was conducted against the TIMI for NSTEMI/UA using a separate validation dataset, and the net reclassification index (NRI) was subsequently determined.

RESULTS

Using both complete and reduced features, the algorithm performance achieved an AUC ranging from 0.73 to 0.89. The top-performing ML algorithm consistently surpassed the TIMI risk score for in-hospital, 30-day, and 1-year predictions (with AUC values of 0.88, 0.88, and 0.81, respectively, all p < 0.001), while the TIMI scores registered significantly lower at 0.55, 0.54, and 0.61. This suggests the TIMI score tends to underestimate patient mortality risk. The net reclassification index (NRI) of the best ML algorithm for NSTEMI/UA patients across these periods yielded an NRI between 40-60% (p < 0.001) relative to the TIMI NSTEMI/UA risk score. Key features identified for both short- and long-term mortality included age, Killip class, heart rate, and Low-Molecular-Weight Heparin (LMWH) administration.

CONCLUSIONS

In a broad multi-ethnic population, ML approaches outperformed conventional TIMI scoring in classifying patients with NSTEMI and UA. ML allows for the precise identification of unique characteristics within individual Asian populations, improving the accuracy of mortality predictions. Continuous development, testing, and validation of these ML algorithms holds the promise of enhanced risk stratification, thereby revolutionizing future management strategies and patient outcomes.

The predictive value of machine learning for mortality risk in patients with acute coronary syndromes: a systematic review and meta-analysis

BACKGROUND

Acute coronary syndromes (ACS) are the leading cause of global death. Optimizing mortality risk prediction and early identification of high-risk patients is essential for developing targeted prevention strategies. Many researchers have built machine learning (ML) models to predict the mortality risk in ACS patients. Our meta-analysis aimed to evaluate the predictive value of various ML models in predicting death in ACS patients at different times.

METHODS

PubMed, Embase, Web of Science, and Cochrane Library were searched systematically from database establishment to March 12, 2022 for studies developing or validating at least one ML predictive model for death in ACS patients. We used PROBAST to assess the risk of bias in the reported predictive models and a random-effects model to assess the pooled C-index and accuracy of these models.

RESULTS

Fifty papers were included, involving 216 ML prediction models, 119 of which were externally validated. The combined C-index of the ML models in the validation cohort predicting the in-hospital mortality, 30-day mortality, 3- or 6-month mortality, and 1 year or above mortality in ACS patients were 0.8633 (95% CI 0.8467-0.8802), 0.8296 (95% CI 0.8134-0.8462), 0.8205 (95% CI 0.7881-0.8541), and 0.8197 (95% CI 0.8042-0.8354), respectively, with the corresponding combined accuracy of 0.8569 (95% CI 0.8411-0.8715), 0.8282 (95% CI 0.7922-0.8591), 0.7303 (95% CI 0.7184-0.7418), and 0.7837 (95% CI 0.7455-0.8175), indicating that the ML models were relatively excellent in predicting ACS mortality at different times. Furthermore, common predictors of death in ML models included age, sex, systolic blood pressure, serum creatinine, Killip class, heart rate, diastolic blood pressure, blood glucose, and hemoglobin.

CONCLUSIONS

The ML models had excellent predictive power for mortality in ACS, and the methodologies may need to be addressed before they can be used in clinical practice.

A biomarker discovery of acute myocardial infarction using feature selection and machine learning

Acute myocardial infarction (AMI) or heart attack is a significant global health threat and one of the leading causes of death. The evolution of machine learning has greatly revamped the risk stratification and death prediction of AMI. In this study, an integrated feature selection and machine learning approach was used to identify potential biomarkers for early detection and treatment of AMI. First, feature selection was conducted and evaluated before all classification tasks with machine learning. Full classification models (using all 62 features) and reduced classification models (using various feature selection methods ranging from 5 to 30 features) were built and evaluated using six machine learning classification algorithms. The results showed that the reduced models performed generally better (mean AUPRC via random forest (RF) algorithm for recursive feature elimination (RFE) method ranges from 0.8048 to 0.8260, while for random forest importance (RFI) method, it ranges from 0.8301 to 0.8505) than the full models (mean AUPRC via RF: 0.8044). The most notable finding of this study was the identification of a five-feature model that included cardiac troponin I, HDL cholesterol, HbA1c, anion gap, and albumin, which had achieved comparable results (mean AUPRC via RF: 0.8462) as to the models that containing more features. These five features were proven by the previous studies as significant risk factors for AMI or cardiovascular disease and could be used as potential biomarkers to predict the prognosis of AMI patients. From the medical point of view, fewer features for diagnosis or prognosis could reduce the cost and time of a patient as lesser clinical and pathological tests are needed.

Explainable SHAP-XGBoost models for in-hospital mortality after myocardial infarction

Background

A lack of explainability in published machine learning (ML) models limits clinicians' understanding of how predictions are made, in turn undermining uptake of the models into clinical practice.

Objective

The purpose of this study was to develop explainable ML models to predict in-hospital mortality in patients hospitalized for myocardial infarction (MI).

Methods

Adult patients hospitalized for an MI were identified in the National Inpatient Sample between January 1, 2012, and September 30, 2015. The resulting cohort comprised 457,096 patients described by 64 predictor variables relating to demographic/comorbidity characteristics and in-hospital complications. The gradient boosting algorithm eXtreme Gradient Boosting (XGBoost) was used to develop explainable models for in-hospital mortality prediction in the overall cohort and patient subgroups based on MI type and/or sex.

Results

The resulting models exhibited an area under the receiver operating characteristic curve (AUC) ranging from 0.876 to 0.942, specificity 82% to 87%, and sensitivity 75% to 87%. All models exhibited high negative predictive value ≥0.974. The SHapley Additive exPlanation (SHAP) framework was applied to explain the models. The top predictor variables of increasing and decreasing mortality were age and undergoing percutaneous coronary intervention, respectively. Other notable findings included a decreased mortality risk associated with certain patient subpopulations with hyperlipidemia and a comparatively greater risk of death among women below age 55 years.

Conclusion

The literature lacks explainable ML models predicting in-hospital mortality after an MI. In a national registry, explainable ML models performed best in ruling out in-hospital death post-MI, and their explanation illustrated their potential for guiding hypothesis generation and future study design.

Machine learning prediction of mortality in Acute Myocardial Infarction

BACKGROUND

Acute Myocardial Infarction (AMI) is the leading cause of death in Portugal and globally. The present investigation created a model based on machine learning for predictive analysis of mortality in patients with AMI upon admission, using different variables to analyse their impact on predictive models.

METHODS

Three experiments were built for mortality in AMI in a Portuguese hospital between 2013 and 2015 using various machine learning techniques. The three experiments differed in the number and type of variables used. We used a discharged patients' episodes database, including administrative data, laboratory data, and cardiac and physiologic test results, whose primary diagnosis was AMI.

RESULTS

Results show that for Experiment 1, Stochastic Gradient Descent was more suitable than the other classification models, with a classification accuracy of 80%, a recall of 77%, and a discriminatory capacity with an AUC of 79%. Adding new variables to the models increased AUC in Experiment 2 to 81% for the Support Vector Machine method. In Experiment 3, we obtained an AUC, in Stochastic Gradient Descent, of 88% and a recall of 80%. These results were obtained when applying feature selection and the SMOTE technique to overcome imbalanced data.

CONCLUSIONS

Our results show that the introduction of new variables, namely laboratory data, impacts the performance of the methods, reinforcing the premise that no single approach is adapted to all situations regarding AMI mortality prediction. Instead, they must be selected, considering the context and the information available. Integrating Artificial Intelligence (AI) and machine learning with clinical decision-making can transform care, making clinical practice more efficient, faster, personalised, and effective. AI emerges as an alternative to traditional models since it has the potential to explore large amounts of information automatically and systematically.

A smarter perspective: Learning with and from AI-cases

Artificial intelligence (AI) has only partially (or not at all) been integrated into medical education, leading to growing concerns regarding how to train healthcare practitioners to handle the changes brought about by the introduction of AI. Programming lessons and other technical information into healthcare curricula has been proposed as a solution to support healthcare personnel in using AI or other future technology. However, integrating these core elements of computer science knowledge might not meet the observed need that students will benefit from gaining practical experience with AI in the direct application area. Therefore, this paper proposes a dynamic approach to case-based learning that utilizes the scenarios where AI is currently used in clinical practice as examples. This approach will support students' understanding of technical aspects. Case-based learning with AI as an example provides additional benefits: (1) it allows doctors to compare their thought processes to the AI suggestions and critically reflect on the assumptions and biases of AI and clinical practice; (2) it incentivizes doctors to discuss and address ethical issues inherent to technology and those already existing in current clinical practice; (3) it serves as a foundation for fostering interdisciplinary collaboration via discussion of different views between technologists, multidisciplinary experts, and healthcare professionals. The proposed knowledge shift from AI as a technical focus to AI as an example for case-based learning aims to encourage a different perspective on educational needs. Technical education does not need to compete with other essential clinical skills as it could serve as a basis for supporting them, which leads to better medical education and practice, ultimately benefiting patients.

Machine-learning Algorithms for Ischemic Heart Disease Prediction: A Systematic Review

PURPOSE

This review aims to summarize and evaluate the most accurate machinelearning algorithm used to predict ischemic heart disease.

METHODS

This systematic review was performed following PRISMA guidelines. A comprehensive search was carried out using multiple databases such as Science Direct, PubMed\ MEDLINE, CINAHL, and IEEE explore.

RESULTS

Thirteen articles published between 2017 to 2021 were eligible for inclusion. Three themes were extracted: the commonly used algorithm to predict ischemic heart disease, the accuracy of algorithms to predict ischemic heart disease, and the clinical outcomes to improve the quality of care. All methods have utilized supervised and unsupervised machine-learning.

CONCLUSION

Applying machine-learning is expected to assist clinicians in interpreting patients' data and implementing optimal algorithms for their datasets. Furthermore, machine-learning can build evidence-based that supports health care providers to manage individual situations who need invasive procedures such as catheterizations.

Robust prognostic prediction model developed with integrated biological markers for acute myocardial infarction

Commonly used prediction methods for acute myocardial infarction (AMI) were created before contemporary percutaneous coronary intervention was recognized as the primary therapy. Although several studies have used machine learning techniques for prognostic prediction of patients with AMI, its clinical application has not been achieved. Here, we developed an online application tool using a machine learning model to predict in-hospital mortality in patients with AMI. A total of 2,553 cases of ST-elevation AMI were assigned to 80% training subset for cross validation and 20% test subset for model performance evaluation. We implemented random forest classifier for the binary classification of in-hospital mortality. The selected best feature set consisted of ten clinical and biological markers including max creatine phosphokinase, hemoglobin, heart rate, creatinine, systolic blood pressure, blood sugar, age, Killip class, white blood cells, and c-reactive protein. Our model achieved high performance: the area under the curve of the receiver operating characteristic curve for the test subset, 0.95: sensitivity, 0.89: specificity, 0.91: precision, 0.43: accuracy, 0.91 respectively, which outperformed common scoring methods. The freely available application tool for prognostic prediction can contribute to risk triage and decision-making in patient-centered modern clinical practice for AMI.

An artificial intelligence-based risk prediction model of myocardial infarction

BACKGROUND

Myocardial infarction can lead to malignant arrhythmia, heart failure, and sudden death. Clinical studies have shown that early identification of and timely intervention for acute MI can significantly reduce mortality. The traditional MI risk assessment models are subjective, and the data that go into them are difficult to obtain. Generally, the assessment is only conducted among high-risk patient groups.

OBJECTIVE

To construct an artificial intelligence-based risk prediction model of myocardial infarction (MI) for continuous and active monitoring of inpatients, especially those in noncardiovascular departments, and early warning of MI.

METHODS

The imbalanced data contain 59 features, which were constructed into a specific dataset through proportional division, upsampling, downsampling, easy ensemble, and w-easy ensemble. Then, the dataset was traversed using supervised machine learning, with recursive feature elimination as the top-layer algorithm and random forest, gradient boosting decision tree (GBDT), logistic regression, and support vector machine as the bottom-layer algorithms, to select the best model out of many through a variety of evaluation indices.

RESULTS

GBDT was the best bottom-layer algorithm, and downsampling was the best dataset construction method. In the validation set, the F1 score and accuracy of the 24-feature downsampling GBDT model were both 0.84. In the test set, the F1 score and accuracy of the 24-feature downsampling GBDT model were both 0.83, and the area under the curve was 0.91.

CONCLUSION

Compared with traditional models, artificial intelligence-based machine learning models have better accuracy and real-time performance and can reduce the occurrence of in-hospital MI from a data-driven perspective, thereby increasing the cure rate of patients and improving their prognosis.

Predicting major adverse cardiovascular events in acute coronary syndrome: A scoping review of machine learning approaches

BACKGROUND

Acute coronary syndrome is the topmost cause of death worldwide; therefore, it is necessary to predict major adverse cardiovascular events and cardiovascular deaths in patients with acute coronary syndrome to make correct and timely clinical decisions.

OBJECTIVE

The current review aimed to highlight algorithms and important predictor variables through examining those studies which used machine learning algorithms for predicting major adverse cardiovascular events in patients with acute coronary syndrome.

METHODS

In order to predict major adverse cardiovascular events in patients with acute coronary syndrome, the preferred reporting items for scoping reviews guidelines were used. PubMed, Embase, Web of Science, Scopus, Springer, and IEEE Xplore databases were searched for articles published between 2005 and 2021. The findings of the studies are presented in the form of a narrative synthesis of evidence.

RESULTS

According to the results, 14 (63.64%) studies did not perform external validation and only used registry data. The algorithms used in this study comprised, inter alia, Regression Logistic, Random Forest, Boosting Ensemble, Non-Boosting Ensemble, Decision Trees, and Naive Bayes. Multiple studies (N=20) achieved a high Area under the ROC Curve between 0.8 to 0.99 in predicting mortality and major adverse cardiovascular events. The predictor variables used in these studies were divided into demographic, clinical, and therapeutic features. However, no study reported the integration of machine learning model into clinical practice.

CONCLUSION

Machine learning algorithms rendered acceptable results to predict major adverse cardiovascular events and mortality outcomes in patients with acute coronary syndrome. However, these approaches have never been integrated into clinical practice. Further research is required to develop feasible and effective machine learning prediction models to measure their potentially important implications for optimizing the quality of care in patients with acute coronary syndrome.

Prediction of Cardiovascular and All-Cause Mortality After Myocardial Infarction in US Veterans

Risk prediction models for cardiovascular disease (CVD) death developed from patients without vascular disease may not be suitable for myocardial infarction (MI) survivors. Prediction of mortality risk after MI may help to guide secondary prevention. Using national electronic record data from the Veterans Health Administration 2002 to 2012, we developed risk prediction models for CVD death and all-cause death based on 5-year follow-up data of 100,601 survivors of MI using Cox proportional hazards models. Model performance was evaluated using a cross-validation approach. During follow-up, there were 31,622 deaths and 12,901 CVD deaths. In men, older age, current smoking, atrial fibrillation, heart failure, peripheral artery disease, and lower body mass index were associated with greater risk of death from CVD or all-causes, and statin treatment, hypertension medication, estimated glomerular filtration rate level, and high body mass index were significantly associated with reduced risk of fatal outcomes. Similar associations and slightly different predictors were observed in women. The estimated Harrell's C-statistics of the final model versus the cross-validation estimates were 0.77 versus 0.77 in men and 0.81 versus 0.77 in women for CVD death. Similarly, the C-statistics were 0.75 versus 0.75 in men, 0.78 versus 0.75 in women for all-cause mortality. The predicted risk of death was well calibrated compared with the observed risk. In conclusion, we developed and internally validated risk prediction models of 5-year risk for CVD and all-cause death for outpatient survivors of MI. Traditional risk factors, co-morbidities, and lack of blood pressure or lipid treatment were all associated with greater risk of CVD and all-cause mortality.

Artificial Intelligence: A Shifting Paradigm in Cardio-Cerebrovascular Medicine

The future of healthcare is an organic blend of technology, innovation, and human connection. As artificial intelligence (AI) is gradually becoming a go-to technology in healthcare to improve efficiency and outcomes, we must understand our limitations. We should realize that our goal is not only to provide faster and more efficient care, but also to deliver an integrated solution to ensure that the care is fair and not biased to a group of sub-population. In this context, the field of cardio-cerebrovascular diseases, which encompasses a wide range of conditions-from heart failure to stroke-has made some advances to provide assistive tools to care providers. This article aimed to provide an overall thematic review of recent development focusing on various AI applications in cardio-cerebrovascular diseases to identify gaps and potential areas of improvement. If well designed, technological engines have the potential to improve healthcare access and equitability while reducing overall costs, diagnostic errors, and disparity in a system that affects patients and providers and strives for efficiency.

A review of risk prediction models in cardiovascular disease: conventional approach vs. artificial intelligent approach

Cardiovascular disease (CVD) is the leading cause of death worldwide and is a global health issue. Traditionally, statistical models are used commonly in the risk prediction and assessment of CVD. However, the adoption of artificial intelligent (AI) approach is rapidly taking hold in the current era of technology to evaluate patient risks and predict the outcome of CVD. In this review, we outline various conventional risk scores and prediction models and do a comparison with the AI approach. The strengths and limitations of both conventional and AI approaches are discussed. Besides that, biomarker discovery related to CVD are also elucidated as the biomarkers can be used in the risk stratification as well as early detection of the disease. Moreover, problems and challenges involved in current CVD studies are explored. Lastly, future prospects of CVD risk prediction and assessment in the multi-modality of big data integrative approaches are proposed.

Machine learning enhances the performance of short and long-term mortality prediction model in non-ST-segment elevation myocardial infarction

Machine learning (ML) has been suggested to improve the performance of prediction models. Nevertheless, research on predicting the risk in patients with acute myocardial infarction (AMI) has been limited and showed inconsistency in the performance of ML models versus traditional models (TMs). This study developed ML-based models (logistic regression with regularization, random forest, support vector machine, and extreme gradient boosting) and compared their performance in predicting the short- and long-term mortality of patients with AMI with those of TMs with comparable predictors. The endpoints were the in-hospital mortality of 14,183 participants and the three- and 12-month mortality in patients who survived at discharge. The performance of the ML models in predicting the mortality of patients with an ST-segment elevation myocardial infarction (STEMI) was comparable to the TMs. In contrast, the areas under the curves (AUC) of the ML models for non-STEMI (NSTEMI) in predicting the in-hospital, 3-month, and 12-month mortality were 0.889, 0.849, and 0.860, respectively, which were superior to the TMs, which had corresponding AUCs of 0.873, 0.795, and 0.808. Overall, the performance of the predictive model could be improved, particularly for long-term mortality in NSTEMI, from the ML algorithm rather than using more clinical predictors.

Use of Machine Learning Models to Predict Death After Acute Myocardial Infarction

Importance

Accurate prediction of adverse outcomes after acute myocardial infarction (AMI) can guide the triage of care services and shared decision-making, and novel methods hold promise for using existing data to generate additional insights.

Objective

To evaluate whether contemporary machine learning methods can facilitate risk prediction by including a larger number of variables and identifying complex relationships between predictors and outcomes.

Design, Setting, and Participants

This cohort study used the American College of Cardiology Chest Pain-MI Registry to identify all AMI hospitalizations between January 1, 2011, and December 31, 2016. Data analysis was performed from February 1, 2018, to October 22, 2020.

Main Outcomes and Measures

Three machine learning models were developed and validated to predict in-hospital mortality based on patient comorbidities, medical history, presentation characteristics, and initial laboratory values. Models were developed based on extreme gradient descent boosting (XGBoost, an interpretable model), a neural network, and a meta-classifier model. Their accuracy was compared against the current standard developed using a logistic regression model in a validation sample.

Results

A total of 755 402 patients (mean [SD] age, 65 [13] years; 495 202 [65.5%] male) were identified during the study period. In independent validation, 2 machine learning models, gradient descent boosting and meta-classifier (combination including inputs from gradient descent boosting and a neural network), marginally improved discrimination compared with logistic regression (C statistic, 0.90 for best performing machine learning model vs 0.89 for logistic regression). Nearly perfect calibration in independent validation data was found in the XGBoost (slope of predicted to observed events, 1.01; 95% CI, 0.99-1.04) and the meta-classifier model (slope of predicted-to-observed events, 1.01; 95% CI, 0.99-1.02), with more precise classification across the risk spectrum. The XGBoost model reclassified 32 393 of 121 839 individuals (27%) and the meta-classifier model reclassified 30 836 of 121 839 individuals (25%) deemed at moderate to high risk for death in logistic regression as low risk, which were more consistent with the observed event rates.

Conclusions and Relevance

In this cohort study using a large national registry, none of the tested machine learning models were associated with substantive improvement in the discrimination of in-hospital mortality after AMI, limiting their clinical utility. However, compared with logistic regression, XGBoost and meta-classifier models, but not the neural network, offered improved resolution of risk for high-risk individuals.

Treatment of individual predictors with neural network algorithms improves Global Registry of Acute Coronary Events score discrimination

Objective:

The aim of this study was to develop, train, and test different neural network (NN) algorithm-based models to improve the Global Registry of Acute Coronary Events (GRACE) score performance to predict in-hospital mortality after an acute coronary syndrome.

Methods:

We analyzed a prospective database, including 40 admission variables of 1255 patients admitted with the acute coronary syndrome in a community hospital. Individual predictors included in GRACE score were used to train and test three NN algorithm-based models (guided models), namely: one- and two-hidden layer multilayer perceptron and a radial basis function network. Three extra NNs were built using the 40 admission variables of the entire database (unguided models). Expected mortality according to GRACE score was calculated using the logistic regression equation.

Results:

In terms of receiver operating characteristic area and negative predictive value (NPV), almost all NN algorithms outperformed logistic regression. Only radial basis function models obtained a better accuracy level based on NPV improvement, at the expense of positive predictive value (PPV) reduction. The independent normalized importance of variables for the best unguided NN was: creatinine 100%, Killip class 61%, ejection fraction 52%, age 44%, maximum creatine-kinase level 41%, glycemia 40%, left bundle branch block 35%, and weight 33%, among the top 8 predictors.

Conclusions:

Treatment of individual predictors of GRACE score with NN algorithms improved accuracy and discrimination power in all models with respect to the traditional logistic regression approach; nevertheless, PPV was only marginally enhanced. Unguided variable selection would be able to achieve better results in PPV terms.

Intersections of machine learning and epidemiological methods for health services research

The field of health services research is broad and seeks to answer questions about the health care system. It is inherently interdisciplinary, and epidemiologists have made crucial contributions. Parametric regression techniques remain standard practice in health services research with machine learning techniques currently having low penetrance in comparison. However, studies in several prominent areas, including health care spending, outcomes and quality, have begun deploying machine learning tools for these applications. Nevertheless, major advances in epidemiological methods are also as yet underleveraged in health services research. This article summarizes the current state of machine learning in key areas of health services research, and discusses important future directions at the intersection of machine learning and epidemiological methods for health services research.

Machine learning versus traditional risk stratification methods in acute coronary syndrome: a pooled randomized clinical trial analysis

Traditional statistical models allow population based inferences and comparisons. Machine learning (ML) explores datasets to develop algorithms that do not assume linear relationships between variables and outcomes and that may account for higher order interactions to make individualized outcome predictions. To evaluate the performance of machine learning models compared to traditional risk stratification methods for the prediction of major adverse cardiovascular events (MACE) and bleeding in patients with acute coronary syndrome (ACS) that are treated with antithrombotic therapy. Data on 24,178 ACS patients were pooled from four randomized controlled trials. The super learner ensemble algorithm selected weights for 23 machine learning models and was compared to traditional models. The efficacy endpoint was a composite of cardiovascular death, myocardial infarction, or stroke. The safety endpoint was a composite of TIMI major and minor bleeding or bleeding requiring medical attention. For the MACE outcome, the super learner model produced a higher c-statistic (0.734) than logistic regression (0.714), the TIMI risk score (0.489), and a new cardiovascular risk score developed in the dataset (0.644). For the bleeding outcome, the super learner demonstrated a similar c-statistic as the logistic regression model (0.670 vs. 0.671). The machine learning risk estimates were highly calibrated with observed efficacy and bleeding outcomes (Hosmer-Lemeshow p value = 0.692 and 0.970, respectively). The super learner algorithm was highly calibrated on both efficacy and safety outcomes and produced the highest c-statistic for prediction of MACE compared to traditional risk stratification methods. This analysis demonstrates a contemporary application of machine learning to guide patient-level antithrombotic therapy treatment decisions.Clinical Trial Registration ATLAS ACS-2 TIMI 46: https://clinicaltrials.gov/ct2/show/NCT00402597. Unique Identifier: NCT00402597. ATLAS ACS-2 TIMI 51: https://clinicaltrials.gov/ct2/show/NCT00809965. Unique Identifier: NCT00809965. GEMINI ACS-1: https://clinicaltrials.gov/ct2/show/NCT02293395. Unique Identifier: NCT02293395. PIONEER-AF PCI: https://clinicaltrials.gov/ct2/show/NCT01830543. Unique Identifier: NCT01830543.

Prediction of incident myocardial infarction using machine learning applied to harmonized electronic health record data

Background

With cardiovascular disease increasing, substantial research has focused on the development of prediction tools. We compare deep learning and machine learning models to a baseline logistic regression using only ‘known’ risk factors in predicting incident myocardial infarction (MI) from harmonized EHR data.

Methods

Large-scale case-control study with outcome of 6-month incident MI, conducted using the top 800, from an initial 52 k procedures, diagnoses, and medications within the UCHealth system, harmonized to the Observational Medical Outcomes Partnership common data model, performed on 2.27 million patients. We compared several over- and under- sampling techniques to address the imbalance in the dataset. We compared regularized logistics regression, random forest, boosted gradient machines, and shallow and deep neural networks. A baseline model for comparison was a logistic regression using a limited set of ‘known’ risk factors for MI. Hyper-parameters were identified using 10-fold cross-validation.

Results

Twenty thousand Five hundred and ninety-one patients were diagnosed with MI compared with 2.25 million who did not. A deep neural network with random undersampling provided superior classification compared with other methods. However, the benefit of the deep neural network was only moderate, showing an F1 Score of 0.092 and AUC of 0.835, compared to a logistic regression model using only ‘known’ risk factors. Calibration for all models was poor despite adequate discrimination, due to overfitting from low frequency of the event of interest.

Conclusions

Our study suggests that DNN may not offer substantial benefit when trained on harmonized data, compared to traditional methods using established risk factors for MI.

Predicting in-hospital mortality of patients with febrile neutropenia using machine learning models

BACKGROUND

Febrile neutropenia (FN) has been associated with high mortality among adults with cancer. Current systems for early detection of inpatient FN mortality are based on scoring indexes that require intensive physicians' subjective evaluation.

OBJECTIVE

In this study, we leveraged machine learning techniques to build a FN mortality risk evaluation tool focused on FN admissions without physicians' subjective evaluation.

METHODS

We used the National Inpatient Sample and Nationwide Inpatient Sample (NIS) that included mortality data among adult inpatients who were diagnosed with FN during a hospital admission. Machine learning techniques that we compared included linear models (ridge logistic regression and linear support vector machine) and non-linear models (gradient boosting tree and neural network). The primary outcome for this study was death among individuals with a recorded FN admission. Model comparison was evaluated based on areas under the receiver operating characteristic curve (AUROC) and model performance was estimated using 30 % test set created via stratified split.

RESULTS

Our analysis detected 126,013 adult admissions within the NIS data that were diagnosed with FN, among which 5,856 were declared as deceased (4.6 %). Our machine learning results demonstrate linear models and non-linear models achieved areas under the receiver operating characteristic (AUROC) around 92 % in survival prediction.

CONCLUSIONS

We developed machine learning models that do not require physicians' subjective evaluation for FN mortality risk prediction.

Assessment of a Machine Learning Model Applied to Harmonized Electronic Health Record Data for the Prediction of Incident Atrial Fibrillation

IMPORTANCE

Atrial fibrillation (AF) is the most common sustained cardiac arrhythmia, and its early detection could lead to significant improvements in outcomes through the appropriate prescription of anticoagulation medication. Although a variety of methods exist for screening for AF, a targeted approach, which requires an efficient method for identifying patients at risk, would be preferred.

OBJECTIVE

To examine machine learning approaches applied to electronic health record data that have been harmonized to the Observational Medical Outcomes Partnership Common Data Model for identifying risk of AF.

DESIGN, SETTING, AND PARTICIPANTS

This diagnostic study used data from 2 252 219 individuals cared for in the UCHealth hospital system, which comprises 3 large hospitals in Colorado, from January 1, 2011, to October 1, 2018. Initial analysis was performed in December 2018; follow-up analysis was performed in July 2019.

EXPOSURES

All Observational Medical Outcomes Partnership Common Data Model-harmonized electronic health record features, including diagnoses, procedures, medications, age, and sex.

MAIN OUTCOMES AND MEASURES

Classification of incident AF in designated 6-month intervals, adjudicated retrospectively, based on area under the receiver operating characteristic curve and F1 statistic.

RESULTS

Of 2 252 219 individuals (1 225 533 [54.4%] women; mean [SD] age, 42.9 [22.3] years), 28 036 (1.2%) developed incident AF during a designated 6-month interval. The machine learning model that used the 200 most common electronic health record features, including age and sex, and random oversampling with a single-layer, fully connected neural network provided the optimal prediction of 6-month incident AF, with an area under the receiver operating characteristic curve of 0.800 and an F1 score of 0.110. This model performed only slightly better than a more basic logistic regression model composed of known clinical risk factors for AF, which had an area under the receiver operating characteristic curve of 0.794 and an F1 score of 0.079.

CONCLUSIONS AND RELEVANCE

Machine learning approaches to electronic health record data offer a promising method for improving risk prediction for incident AF, but more work is needed to show improvement beyond standard risk factors.

Comparing different supervised machine learning algorithms for disease prediction

Background

Supervised machine learning algorithms have been a dominant method in the data mining field. Disease prediction using health data has recently shown a potential application area for these methods. This study ai7ms to identify the key trends among different types of supervised machine learning algorithms, and their performance and usage for disease risk prediction.

Methods

In this study, extensive research efforts were made to identify those studies that applied more than one supervised machine learning algorithm on single disease prediction. Two databases (i.e., Scopus and PubMed) were searched for different types of search items. Thus, we selected 48 articles in total for the comparison among variants supervised machine learning algorithms for disease prediction.

Results

We found that the Support Vector Machine (SVM) algorithm is applied most frequently (in 29 studies) followed by the Naïve Bayes algorithm (in 23 studies). However, the Random Forest (RF) algorithm showed superior accuracy comparatively. Of the 17 studies where it was applied, RF showed the highest accuracy in 9 of them, i.e., 53%. This was followed by SVM which topped in 41% of the studies it was considered.

Conclusion

This study provides a wide overview of the relative performance of different variants of supervised machine learning algorithms for disease prediction. This important information of relative performance can be used to aid researchers in the selection of an appropriate supervised machine learning algorithm for their studies.