Developing Machine Learning Algorithms for the Prediction of Early Death in Elderly Cancer Patients: Usability Study

Thirty-day mortality after palliative radiotherapy in advanced cancer patients: Optimizing end-of-life care in Asia

INTRODUCTION

Evidence-based guidelines recommend hypofractionated palliative radiotherapy (PRT); nonetheless, many patients receive prolonged course of PRT. To identify patients with limited benefits from PRT in end-of-life care, we evaluated the pattern of PRT at an Asian institution and factors associated with 30-day mortality after PRT (30dM).

METHODS

We retrospectively reviewed 228 patients who died after PRT in Yonsei Wonju Severance Christian hospital between October 2014 and March 2022. The associations between clinical factors and survival were assessed using the Cox proportional hazards method. Survival was analysed using the existing models to evaluate their performance in our cohort.

RESULTS

The median PRT duration was 13 (IQR, 7-15) days. Only 11.4% of the patients were treated with hypofractionated radiotherapy. One-third of the patients (32.9%) could not complete PRT and 39 (17.1%) died during PRT. The 30dM was 31.6%. The median time from PRT to death was 17 (IQR, 11-23) days for the patients who died within 30 days. The number of involved organs (≤2 vs. >2; P < 0.001), albumin level (<3.3 vs. ≥3.3; P = 0.016), admission during PRT (P < 0.001), admission 3 months before PRT (P = 0.036) and ICU care during PRT (P < 0.001) were prognostic factors. A comparison of survival based on the existing models yielded unsatisfactory results in our cohort.

CONCLUSION

Almost one-third of the patients received PRT in the last 30 days of life. The use of hypofractionation for PRT was low in this Asian population. Further research is necessary to develop a predictive model of early mortality, allowing tailored end-of-life care for Asian patients.

A model for predicting physical function upon discharge of hospitalized older adults in Taiwan-a machine learning approach based on both electronic health records and comprehensive geriatric assessment

Background

Predicting physical function upon discharge among hospitalized older adults is important. This study has aimed to develop a prediction model of physical function upon discharge through use of a machine learning algorithm using electronic health records (EHRs) and comprehensive geriatrics assessments (CGAs) among hospitalized older adults in Taiwan.

Methods

Data was retrieved from the clinical database of a tertiary medical center in central Taiwan. Older adults admitted to the acute geriatric unit during the period from January 2012 to December 2018 were included for analysis, while those with missing data were excluded. From data of the EHRs and CGAs, a total of 52 clinical features were input for model building. We used 3 different machine learning algorithms, XGBoost, random forest and logistic regression.

Results

In total, 1,755 older adults were included in final analysis, with a mean age of 80.68 years. For linear models on physical function upon discharge, the accuracy of prediction was 87% for XGBoost, 85% for random forest, and 32% for logistic regression. For classification models on physical function upon discharge, the accuracy for random forest, logistic regression and XGBoost were 94, 92 and 92%, respectively. The auROC reached 98% for XGBoost and random forest, while logistic regression had an auROC of 97%. The top 3 features of importance were activity of daily living (ADL) at baseline, ADL during admission, and mini nutritional status (MNA) during admission.

Conclusion

The results showed that physical function upon discharge among hospitalized older adults can be predicted accurately during admission through use of a machine learning model with data taken from EHRs and CGAs.

Machine Learning-Based Short-Term Mortality Prediction Models for Patients With Cancer Using Electronic Health Record Data: Systematic Review and Critical Appraisal

BACKGROUND

In the United States, national guidelines suggest that aggressive cancer care should be avoided in the final months of life. However, guideline compliance currently requires clinicians to make judgments based on their experience as to when a patient is nearing the end of their life. Machine learning (ML) algorithms may facilitate improved end-of-life care provision for patients with cancer by identifying patients at risk of short-term mortality.

OBJECTIVE

This study aims to summarize the evidence for applying ML in ≤1-year cancer mortality prediction to assist with the transition to end-of-life care for patients with cancer.

METHODS

We searched MEDLINE, Embase, Scopus, Web of Science, and IEEE to identify relevant articles. We included studies describing ML algorithms predicting ≤1-year mortality in patients of oncology. We used the prediction model risk of bias assessment tool to assess the quality of the included studies.

RESULTS

We included 15 articles involving 110,058 patients in the final synthesis. Of the 15 studies, 12 (80%) had a high or unclear risk of bias. The model performance was good: the area under the receiver operating characteristic curve ranged from 0.72 to 0.92. We identified common issues leading to biased models, including using a single performance metric, incomplete reporting of or inappropriate modeling practice, and small sample size.

CONCLUSIONS

We found encouraging signs of ML performance in predicting short-term cancer mortality. Nevertheless, no included ML algorithms are suitable for clinical practice at the current stage because of the high risk of bias and uncertainty regarding real-world performance. Further research is needed to develop ML models using the modern standards of algorithm development and reporting.

Cause-specific mortality prediction in older residents of São Paulo, Brazil: a machine learning approach

BACKGROUND

Populational ageing has been increasing in a remarkable rate in developing countries. In this scenario, preventive strategies could help to decrease the burden of higher demands for healthcare services. Machine learning algorithms have been increasingly applied for identifying priority candidates for preventive actions, presenting a better predictive performance than traditional parsimonious models.

METHODS

Data were collected from the Health, Well Being and Aging (SABE) Study, a representative sample of older residents of São Paulo, Brazil. Machine learning algorithms were applied to predict death by diseases of respiratory system (DRS), diseases of circulatory system (DCS), neoplasms and other specific causes within 5 years, using socioeconomic, demographic and health features. The algorithms were trained in a random sample of 70% of subjects, and then tested in the other 30% unseen data.

RESULTS

The outcome with highest predictive performance was death by DRS (AUC-ROC = 0.89), followed by the other specific causes (AUC-ROC = 0.87), DCS (AUC-ROC = 0.67) and neoplasms (AUC-ROC = 0.52). Among only the 25% of individuals with the highest predicted risk of mortality from DRS were included 100% of the actual cases. The machine learning algorithms with the highest predictive performance were light gradient boosted machine and extreme gradient boosting.

CONCLUSION

The algorithms had a high predictive performance for DRS, but lower for DCS and neoplasms. Mortality prediction with machine learning can improve clinical decisions especially regarding targeted preventive measures for older individuals.

A review of AI and Data Science support for cancer management

INTRODUCTION

Thanks to improvement of care, cancer has become a chronic condition. But due to the toxicity of treatment, the importance of supporting the quality of life (QoL) of cancer patients increases. Monitoring and managing QoL relies on data collected by the patient in his/her home environment, its integration, and its analysis, which supports personalization of cancer management recommendations. We review the state-of-the-art of computerized systems that employ AI and Data Science methods to monitor the health status and provide support to cancer patients managed at home.

OBJECTIVE

Our main objective is to analyze the literature to identify open research challenges that a novel decision support system for cancer patients and clinicians will need to address, point to potential solutions, and provide a list of established best-practices to adopt.

METHODS

We designed a review study, in compliance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines, analyzing studies retrieved from PubMed related to monitoring cancer patients in their home environments via sensors and self-reporting: what data is collected, what are the techniques used to collect data, semantically integrate it, infer the patient's state from it and deliver coaching/behavior change interventions.

RESULTS

Starting from an initial corpus of 819 unique articles, a total of 180 papers were considered in the full-text analysis and 109 were finally included in the review. Our findings are organized and presented in four main sub-topics consisting of data collection, data integration, predictive modeling and patient coaching.

CONCLUSION

Development of modern decision support systems for cancer needs to utilize best practices like the use of validated electronic questionnaires for quality-of-life assessment, adoption of appropriate information modeling standards supplemented by terminologies/ontologies, adherence to FAIR data principles, external validation, stratification of patients in subgroups for better predictive modeling, and adoption of formal behavior change theories. Open research challenges include supporting emotional and social dimensions of well-being, including PROs in predictive modeling, and providing better customization of behavioral interventions for the specific population of cancer patients.

Clinical Analytics Prediction Engine (CAPE): Development, electronic health record integration and prospective validation of hospital mortality, 180-day mortality and 30-day readmission risk prediction models

BACKGROUND

Numerous predictive models in the literature stratify patients by risk of mortality and readmission. Few prediction models have been developed to optimize impact while sustaining sufficient performance.

OBJECTIVE

We aimed to derive models for hospital mortality, 180-day mortality and 30-day readmission, implement these models within our electronic health record and prospectively validate these models for use across an entire health system.

MATERIALS & METHODS

We developed, integrated into our electronic health record and prospectively validated three predictive models using logistic regression from data collected from patients 18 to 99 years old who had an inpatient or observation admission at NorthShore University HealthSystem, a four-hospital integrated system in the United States, from January 2012 to September 2018. We analyzed the area under the receiver operating characteristic curve (AUC) for model performance.

RESULTS

Models were derived and validated at three time points: retrospective, prospective at discharge, and prospective at 4 hours after presentation. AUCs of hospital mortality were 0.91, 0.89 and 0.77, respectively. AUCs for 30-day readmission were 0.71, 0.71 and 0.69, respectively. 180-day mortality models were only retrospectively validated with an AUC of 0.85.

DISCUSSION

We were able to retain good model performance while optimizing potential model impact by also valuing model derivation efficiency, usability, sensitivity, generalizability and ability to prescribe timely interventions to reduce underlying risk. Measuring model impact by tying prediction models to interventions that are then rapidly tested will establish a path for meaningful clinical improvement and implementation.

Nutritional status according to the mini nutritional assessment (MNA)® as potential prognostic factor for health and treatment outcomes in patients with cancer - a systematic review

Background

Patients with cancer have an increased risk of malnutrition which is associated with poor outcome. The Mini Nutritional Assessment (MNA®) is often used in older patients with cancer but its relation to outcome is not known.

Methods

Four databases were systematically searched for studies relating MNA-results with any reported outcome. Two reviewers screened titles/abstracts and full-texts, extracted data and rated the risk of bias (RoB) independently.

Results

We included 56 studies which varied widely in patient and study characteristics. In multivariable analyses, (risk of) malnutrition assessed by MNA significantly predicts a higher chance for mortality/poor overall survival (22/27 studies), shorter progression-free survival/time to progression (3/5 studies), treatment maintenance (5/8 studies) and (health-related) quality of life (2/2 studies), but not treatment toxicity/complications (1/7 studies) or functional status/decline in (1/3 studies). For other outcomes – length of hospital stay (2 studies), falls, fatigue and unplanned (hospital) admissions (1 study each) – no adjusted results were reported. RoB was rated as moderate to high.

Conclusions

MNA®-result predicts mortality/survival, cancer progression, treatment maintenance and (health-related) quality of life and did not predict adverse treatment outcomes and functional status/ decline in patients with cancer. For other outcomes results are less clear. The moderate to high RoB calls for studies with better control of potential confounders.

Predictive Modeling for Frailty Conditions in Elderly People: Machine Learning Approaches

Background

Frailty is one of the most critical age-related conditions in older adults. It is often recognized as a syndrome of physiological decline in late life, characterized by a marked vulnerability to adverse health outcomes. A clear operational definition of frailty, however, has not been agreed so far. There is a wide range of studies on the detection of frailty and their association with mortality. Several of these studies have focused on the possible risk factors associated with frailty in the elderly population while predicting who will be at increased risk of frailty is still overlooked in clinical settings.

Objective

The objective of our study was to develop predictive models for frailty conditions in older people using different machine learning methods based on a database of clinical characteristics and socioeconomic factors.

Methods

An administrative health database containing 1,095,612 elderly people aged 65 or older with 58 input variables and 6 output variables was used. We first identify and define six problems/outputs as surrogates of frailty. We then resolve the imbalanced nature of the data through resampling process and a comparative study between the different machine learning (ML) algorithms – Artificial neural network (ANN), Genetic programming (GP), Support vector machines (SVM), Random Forest (RF), Logistic regression (LR) and Decision tree (DT) – was carried out. The performance of each model was evaluated using a separate unseen dataset.

Results

Predicting mortality outcome has shown higher performance with ANN (TPR 0.81, TNR 0.76, accuracy 0.78, F1-score 0.79) and SVM (TPR 0.77, TNR 0.80, accuracy 0.79, F1-score 0.78) than predicting the other outcomes. On average, over the six problems, the DT classifier has shown the lowest accuracy, while other models (GP, LR, RF, ANN, and SVM) performed better. All models have shown lower accuracy in predicting an event of an emergency admission with red code than predicting fracture and disability. In predicting urgent hospitalization, only SVM achieved better performance (TPR 0.75, TNR 0.77, accuracy 0.73, F1-score 0.76) with the 10-fold cross validation compared with other models in all evaluation metrics.

Conclusions

We developed machine learning models for predicting frailty conditions (mortality, urgent hospitalization, disability, fracture, and emergency admission). The results show that the prediction performance of machine learning models significantly varies from problem to problem in terms of different evaluation metrics. Through further improvement, the model that performs better can be used as a base for developing decision-support tools to improve early identification and prediction of frail older adults.

A multicenter random forest model for effective prognosis prediction in collaborative clinical research network

BACKGROUND

The accuracy of a prognostic prediction model has become an essential aspect of the quality and reliability of the health-related decisions made by clinicians in modern medicine. Unfortunately, individual institutions often lack sufficient samples, which might not provide sufficient statistical power for models. One mitigation is to expand data collection from a single institution to multiple centers to collectively increase the sample size. However, sharing sensitive biomedical data for research involves complicated issues. Machine learning models such as random forests (RF), though they are commonly used and achieve good performances for prognostic prediction, usually suffer worse performance under multicenter privacy-preserving data mining scenarios compared to a centrally trained version.

METHODS AND MATERIALS

In this study, a multicenter random forest prognosis prediction model is proposed that enables federated clinical data mining from horizontally partitioned datasets. By using a novel data enhancement approach based on a differentially private generative adversarial network customized to clinical prognosis data, the proposed model is able to provide a multicenter RF model with performances on par with-or even better than-centrally trained RF but without the need to aggregate the raw data. Moreover, our model also incorporates an importance ranking step designed for feature selection without sharing patient-level information.

RESULT

The proposed model was evaluated on colorectal cancer datasets from the US and China. Two groups of datasets with different levels of heterogeneity within the collaborative research network were selected. First, we compare the performance of the distributed random forest model under different privacy parameters with different percentages of enhancement datasets and validate the effectiveness and plausibility of our approach. Then, we compare the discrimination and calibration ability of the proposed multicenter random forest with a centrally trained random forest model and other tree-based classifiers as well as some commonly used machine learning methods. The results show that the proposed model can provide better prediction performance in terms of discrimination and calibration ability than the centrally trained RF model or the other candidate models while following the privacy-preserving rules in both groups. Additionally, good discrimination and calibration ability are shown on the simplified model based on the feature importance ranking in the proposed approach.

CONCLUSION

The proposed random forest model exhibits ideal prediction capability using multicenter clinical data and overcomes the performance limitation arising from privacy guarantees. It can also provide feature importance ranking across institutions without pooling the data at a central site. This study offers a practical solution for building a prognosis prediction model in the collaborative clinical research network and solves practical issues in real-world applications of medical artificial intelligence.