Development of Imminent Mortality Predictor for Advanced Cancer (IMPAC), a Tool to Predict Short-Term Mortality in Hospitalized Patients With Advanced Cancer

A review of the utility of prognostic tools in predicting 6-month mortality in cancer patients, conducted in the context of voluntary assisted dying

BACKGROUND

Eligibility to access the Victorian voluntary assisted dying (VAD) legislation requires that people have a prognosis of 6 months or less (or 12 months or less in the setting of a neurodegenerative diagnosis). Yet prognostic determination is frequently inaccurate and prompts clinician discomfort. Based on functional capacity and clinical and biochemical markers, prognostic tools have been developed to increase the accuracy of life expectancy predictions.

AIMS

This review of prognostic tools explores their accuracy to determine 6-month mortality in adults when treated under palliative care with a primary diagnosis of cancer (the diagnosis of a large proportion of people who are requesting VAD).

METHODS

A systematic search of the literature was performed on electronic databases Medline, Embase and Cinahl.

RESULTS

Limitations of prognostication identified include the following: (i) prognostic tools still provide uncertain prognoses; (ii) prognostic tools have greater accuracy predicting shorter prognoses, such as weeks to months, rather than 6 months; and (iii) functionality was often weighted significantly when calculating prognoses. Challenges of prognostication identified include the following: (i) the area under the curve (a value that represents how well a model can distinguish between two outcomes) cannot be directly interpreted clinically and (ii) difficulties exist related to determining appropriate thresholds of accuracy in this context.

CONCLUSIONS

Prognostication is a significant aspect of VAD, and the utility of the currently available prognostic tools appears limited but may prompt discussions about prognosis and alternative means (other than prognostic estimates) to identify those eligible for VAD.

Dynamic Risk Prediction of 30-Day Mortality in Patients With Advanced Lung Cancer: Comparing Five Machine Learning Approaches

PURPOSE

Administering systemic anticancer treatment (SACT) to patients near death can negatively affect their health-related quality of life. Late SACT administrations should be avoided in these cases. Machine learning techniques could be used to build decision support tools leveraging registry data for clinicians to limit late SACT administration.

MATERIALS AND METHODS

Patients with advanced lung cancer who were treated at the Department of Oncology, Aalborg University Hospital and died between 2010 and 2019 were included (N = 2,368). Diagnoses, treatments, biochemical data, and histopathologic results were used to train predictive models of 30-day mortality using logistic regression with elastic net penalty, random forest, gradient tree boosting, multilayer perceptron, and long short-term memory network. The importance of the variables and the clinical utility of the models were evaluated.

RESULTS

The random forest and gradient tree boosting models outperformed other models, whereas the artificial neural network–based models underperformed. Adding summary variables had a modest effect on performance with an increase in average precision from 0.500 to 0.505 and from 0.498 to 0.509 for the gradient tree boosting and random forest models, respectively. Biochemical results alone contained most of the information with a limited degradation of the performances when fitting models with only these variables. The utility analysis showed that by applying a simple threshold to the predicted risk of 30-day mortality, 40% of late SACT administrations could have been prevented at the cost of 2% of patients stopping their treatment 90 days before death.

CONCLUSION

This study demonstrates the potential of a decision support tool to limit late SACT administration in patients with cancer. Further work is warranted to refine the model, build an easy-to-use prototype, and conduct a prospective validation study.

Evaluating High-Dimensional Machine Learning Models to Predict Hospital Mortality Among Older Patients With Cancer

PURPOSE

Older hospitalized cancer patients face high risks of hospital mortality. Improved risk stratification could help identify high-risk patients who may benefit from future interventions, although we lack validated tools to predict in-hospital mortality for patients with cancer. We evaluated the ability of a high-dimensional machine learning prediction model to predict inpatient mortality and compared the performance of this model to existing prediction indices.

METHODS

We identified patients with cancer older than 75 years from the National Emergency Department Sample between 2016 and 2018. We constructed a high-dimensional predictive model called Cancer Frailty Assessment Tool (cFAST), which used an extreme gradient boosting algorithm to predict in-hospital mortality. cFAST model inputs included patient demographic, hospital variables, and diagnosis codes. Model performance was assessed with an area under the curve (AUC) from receiver operating characteristic curves, with an AUC of 1.0 indicating perfect prediction. We compared model performance to existing indices including the Modified 5-Item Frailty Index, Charlson comorbidity index, and Hospital Frailty Risk Score.

RESULTS

We identified 2,723,330 weighted emergency department visits among older patients with cancer, of whom 144,653 (5.3%) died in the hospital. Our cFAST model included 240 features and demonstrated an AUC of 0.92. Comparator models including the Modified 5-Item Frailty Index, Charlson comorbidity index, and Hospital Frailty Risk Score achieved AUCs of 0.58, 0.62, and 0.71, respectively. Predictive features of the cFAST model included acute conditions (respiratory failure and shock), chronic conditions (lipidemia and hypertension), patient demographics (age and sex), and cancer and treatment characteristics (metastasis and palliative care).

CONCLUSION

High-dimensional machine learning models enabled accurate prediction of in-hospital mortality among older patients with cancer, outperforming existing prediction indices. These models show promise in identifying patients at risk of severe adverse outcomes, although additional validation and research studying clinical implementation of these tools is needed.

Development of a Machine Learning Model Using Limited Features to Predict 6-Month Mortality at Treatment Decision Points for Patients With Advanced Solid Tumors

Patients with advanced solid tumors may receive intensive treatments near the end of life. This study aimed to create a machine learning (ML) model using limited features to predict 6-month mortality at treatment decision points (TDPs).

Predicting 6-month mortality at treatment decisions for patients with advanced solid tumors.

A prognostic model to identify short survival expectancy of medical oncology patients at the time of hospital discharge

Background

Hospitalization of cancer patients is associated with poor overall survival, but prognostic misclassification may lead to suboptimal therapeutic decisions and transitions of care. No model is currently available for stratifying the heterogeneous population of oncological patients after a hospital admission to a general Medical Oncology ward. We developed a multivariable prognostic model based on readily available and objective clinical data to estimate survival in oncological patients after hospital discharge.

Methods

A multivariable model and nomogram for overall survival after hospital discharge was developed in a retrospective training cohort and prospectively validated in an independent set of adult patients with solid tumors and a first admission to a unit of medical oncology. Performance of the model was assessed by C-index and Kaplan–Meier survival curves stratified by risk categories.

Results

From a population of 1089 patients with a first hospitalization, 757 patients were included in the training group [median survival, 43 weeks; 95% confidence interval (CI), 37-51 weeks] and 200 patients in the validation cohort (median survival, 44 weeks; 95% CI, 34 weeks-not reached). An accelerated failure time log-normal model was built, including five variables (primary tumor, stage, cause of admission, active treatment, and age). The C-index was 0.71 (95% CI, 0.69-0.73), with a good calibration, and adequate validation in the prospective cohort (C-index: 0.69; 95% CI, 0.65-0.74). Median survival in three predefined model-based risk groups was 10.7 weeks (high), 27.0 weeks (intermediate), and 3 years (low) in the training cohort, with comparable values in the validation cohort.

Conclusions

In oncological patients, individualized predictions of survival after hospitalization were provided by a simple and validated model. Further evaluation of the model might determine whether its use improves shared decision making at discharge.

•

Hospitalization of poor prognosis oncology patients is a frequently missed opportunity for transition to palliative care.

•

We developed and validated a prognostic index for cancer patients at hospital discharge based on five objective variables.

•

Adequate prognostic stratification at discharge may facilitate transitions of care and shared decision making.

Impact of Augmented Intelligence on Utilization of Palliative Care Services in a Real-World Oncology Setting

PURPOSE

For patients with advanced cancer, timely referral to palliative care (PC) services can ensure that end-of-life care aligns with their preferences and goals. Overestimation of life expectancy may result in underutilization of PC services, counterproductive treatment measures, and reduced quality of life for patients. We assessed the impact of a commercially available augmented intelligence (AI) tool to predict 30-day mortality risk on PC service utilization in a real-world setting.

METHODS

Patients within a large hematology-oncology practice were scored weekly between June 2018 and October 2019 with an AI tool to generate insights into short-term mortality risk. Patients identified by the tool as being at high or medium risk were assessed for a supportive care visit and further referred as appropriate. Average monthly rates of PC and hospice referrals were calculated 5 months predeployment and 17 months postdeployment of the tool in the practice.

RESULTS

The mean rate of PC consults increased from 17.3 to 29.1 per 1,000 patients per month (PPM) pre- and postdeployment, whereas the mean rate of hospice referrals increased from 0.2 to 1.6 per 1,000 PPM. Eliminating the first 6 months following deployment to account for user learning curve, the mean rate of PC consults nearly doubled over baseline to 33.0 and hospice referrals increased 12-fold to 2.4 PPM.

CONCLUSION

Deployment of an AI tool at a hematology-oncology practice was found to be feasible for identifying patients at high or medium risk for short-term mortality. Insights generated by the tool drove clinical practice changes, resulting in significant increases in PC and hospice referrals.

BOOSTED NONPARAMETRIC HAZARDS WITH TIME-DEPENDENT COVARIATES

Given functional data from a survival process with time-dependent covariates, we derive a smooth convex representation for its nonparametric log-likelihood functional and obtain its functional gradient. From this we devise a generic gradient boosting procedure for estimating the hazard function nonparametrically. An illustrative implementation of the procedure using regression trees is described to show how to recover the unknown hazard. The generic estimator is consistent if the model is correctly specified; alternatively an oracle inequality can be demonstrated for tree-based models. To avoid overfitting, boosting employs several regularization devices. One of them is step-size restriction, but the rationale for this is somewhat mysterious from the viewpoint of consistency. Our work brings some clarity to this issue by revealing that step-size restriction is a mechanism for preventing the curvature of the risk from derailing convergence.

Expected clinical utility of automatable prediction models for improving palliative and end-of-life care outcomes: Toward routine decision analysis before implementation

Objective

The study sought to evaluate the expected clinical utility of automatable prediction models for increasing goals-of-care discussions (GOCDs) among hospitalized patients at the end of life (EOL).

Materials and Methods

We built a decision model from the perspective of clinicians who aim to increase GOCDs at the EOL using an automated alert system. The alternative strategies were 4 prediction models—3 random forest models and the Modified Hospital One-year Mortality Risk model—to generate alerts for patients at a high risk of 1-year mortality. They were trained on admissions from 2011 to 2016 (70 788 patients) and tested with admissions from 2017-2018 (16 490 patients). GOCDs occurring in usual care were measured with code status orders. We calculated the expected risk difference (beneficial outcomes with alerts minus beneficial outcomes without alerts among those at the EOL), the number needed to benefit (number of alerts needed to increase benefit over usual care by 1 outcome), and the net benefit (benefit minus cost) of each strategy.

Results

Models had a C-statistic between 0.79 and 0.86. A code status order occurred during 2599 of 3773 (69%) hospitalizations at the EOL. At a risk threshold corresponding to an alert prevalence of 10%, the expected risk difference ranged from 5.4% to 10.7% and the number needed to benefit ranged from 5.4 to 10.9 alerts. Using revealed preferences, only 2 models improved net benefit over usual care. A random forest model with diagnostic predictors had the highest expected value, including in sensitivity analyses.

Discussion

Prediction models with acceptable predictive validity differed meaningfully in their ability to improve over usual decision making.

Conclusions

An evaluation of clinical utility, such as by using decision curve analysis, is recommended after validating a prediction model because metrics of model predictiveness, such as the C-statistic, are not informative of clinical value.

Development and Validation of a 30-Day In-hospital Mortality Model Among Seriously Ill Transferred Patients: a Retrospective Cohort Study

BACKGROUND

Predicting the risk of in-hospital mortality on admission is challenging but essential for risk stratification of patient outcomes and designing an appropriate plan-of-care, especially among transferred patients.

OBJECTIVE

Develop a model that uses administrative and clinical data within 24 h of transfer to predict 30-day in-hospital mortality at an Academic Health Center (AHC).

DESIGN

Retrospective cohort study. We used 30 putative variables in a multiple logistic regression model in the full data set (n = 10,389) to identify 20 candidate variables obtained from the electronic medical record (EMR) within 24 h of admission that were associated with 30-day in-hospital mortality (p < 0.05). These 20 variables were tested using multiple logistic regression and area under the curve (AUC)-receiver operating characteristics (ROC) analysis to identify an optimal risk threshold score in a randomly split derivation sample (n = 5194) which was then examined in the validation sample (n = 5195).

PARTICIPANTS

Ten thousand three hundred eighty-nine patients greater than 18 years transferred to the Indiana University (IU)-Adult Academic Health Center (AHC) between 1/1/2016 and 12/31/2017.

MAIN MEASURES

Sensitivity, specificity, positive predictive value, C-statistic, and risk threshold score of the model.

KEY RESULTS

The final model was strongly discriminative (C-statistic = 0.90) and had a good fit (Hosmer-Lemeshow goodness-of-fit test [X² (8) =6.26, p = 0.62]). The positive predictive value for 30-day in-hospital death was 68%; AUC-ROC was 0.90 (95% confidence interval 0.89-0.92, p < 0.0001). We identified a risk threshold score of -2.19 that had a maximum sensitivity (79.87%) and specificity (85.24%) in the derivation and validation sample (sensitivity: 75.00%, specificity: 85.71%). In the validation sample, 34.40% (354/1029) of the patients above this threshold died compared to only 2.83% (118/4166) deaths below this threshold.

CONCLUSION

This model can use EMR and administrative data within 24 h of transfer to predict the risk of 30-day in-hospital mortality with reasonable accuracy among seriously ill transferred patients.

External validation of life expectancy prognostic models in patients evaluated for palliative radiotherapy at the end-of-life

BACKGROUND

The TEACHH and Chow models were developed to predict life expectancy (LE) in patients evaluated for palliative radiotherapy (PRT). We sought to validate the TEACHH and Chow models in patients who died within 90 days of PRT consultation.

METHODS

A retrospective review was conducted on patients evaluated for PRT from 2017 to 2019 who died within 90 days of consultation. Data were collected for the TEACHH and Chow models; one point was assigned for each adverse factor. TEACHH model included: primary site of disease, ECOG performance status, age, prior palliative chemotherapy courses, hospitalization within the last 3 months, and presence of hepatic metastases; patients with 0-1, 2-4, and 5-6 adverse factors were categorized into groups (A, B, and C). The Chow model included non-breast primary, site of metastases other than bone only, and KPS; patients with 0-1, 2, or 3 adverse factors were categorized into groups (I, II, and III).

RESULTS

A total of 505 patients with a median overall survival of 2.1 months (IQR: 0.7-2.6) were identified. Based on the TEACHH model, 10 (2.0%), 387 (76.6%), and 108 (21.4%) patients were predicted to live >1 year, >3 months to ≤1 year, and ≤3 months, respectively. Utilizing the Chow model, 108 (21.4%), 250 (49.5%), and 147 (29.1%) patients were expected to live 15.0, 6.5, and 2.3 months, respectively.

CONCLUSION

Neither the TEACHH nor Chow model correctly predict prognosis in a patient population with a survival <3 months. A better predictive tool is required to identify patients with short LE.

Applied Informatics Decision Support Tool for Mortality Predictions in Patients With Cancer

PURPOSE

With rapidly evolving treatment options in cancer, the complexity in the clinical decision-making process for oncologists represents a growing challenge magnified by oncologists' disposition of intuition-based assessment of treatment risks and overall mortality. Given the unmet need for accurate prognostication with meaningful clinical rationale, we developed a highly interpretable prediction tool to identify patients with high mortality risk before the start of treatment regimens.

METHODS

We obtained electronic health record data between 2004 and 2014 from a large national cancer center and extracted 401 predictors, including demographics, diagnosis, gene mutations, treatment history, comorbidities, resource utilization, vital signs, and laboratory test results. We built an actionable tool using novel developments in modern machine learning to predict 60-, 90- and 180-day mortality from the start of an anticancer regimen. The model was validated in unseen data against benchmark models.

RESULTS

We identified 23,983 patients who initiated 46,646 anticancer treatment lines, with a median survival of 514 days. Our proposed prediction models achieved significantly higher estimation quality in unseen data (area under the curve, 0.83 to 0.86) compared with benchmark models. We identified key predictors of mortality, such as change in weight and albumin levels. The results are presented in an interactive and interpretable tool ( www.oncomortality.com ).

CONCLUSION

Our fully transparent prediction model was able to distinguish with high precision between highest- and lowest-risk patients. Given the rich data available in electronic health records and advances in machine learning methods, this tool can have significant implications for value-based shared decision making at the point of care and personalized goals-of-care management to catalyze practice reforms.

PALLIA-10, a screening tool to identify patients needing palliative care referral in comprehensive cancer centers: A prospective multicentric study (PREPA-10)

Purpose

The identification and referral of patients in need of palliative care should be improved. The French society for palliative support and care recommended to use the PALLIA‐10 questionnaire and its score greater than 3 to refer patients to palliative care. We explored the use of the PALLIA‐10 questionnaire and its related score in a population of advanced cancer patients.

Methods

This prospective multicentric study is to be conducted in authorized French comprehensive cancer centers on hospitalized patients on a given day. We aimed to use the PALLIA‐10 score to determine the proportion of palliative patients with a score >3. Main secondary endpoints were to determine the proportion of patients already managed by palliative care teams at the study date or referred to palliative care in six following months, the prevalence of patients with a score greater than 5, and the overall survival using the predefined thresholds of 3 and 5.

Results

In 2015, eighteen French cancer centers enrolled 840 patients, including 687 (82%) palliative patients. 479 (69.5%) patients had a score >3, 230 (33.5%) had a score >5, 216 (31.4%) patients were already followed‐up by a palliative care team, 152 patients were finally referred to PC in the six subsequent months.

The PALLIA‐10 score appeared as a reliable predictive (adjusted OR_Ref≤3: 1.9 [1.17‐3.16] and 3.59 [2.18‐5.91]) and prognostic (adjusted HR_Ref≤3 = 1.58 [95%CI 1.20‐2.08] and 2.18 [95%CI 1.63‐2.92]) factor for patients scored 4‐5 and >5, respectively.

Conclusion

The PALLIA‐10 questionnaire is an easy‐to‐use tool to refer cancer inpatients to palliative care in current practice. However a score greater than 5 using the PALLIA‐10 questionnaire would be more appropriate for advanced cancer patients hospitalized in comprehensive cancer center.