Improving palliative care with deep learning

Towards proactive palliative care in oncology: developing an explainable EHR-based machine learning model for mortality risk prediction

BACKGROUND

Ex-ante identification of the last year in life facilitates a proactive palliative approach. Machine learning models trained on electronic health records (EHR) demonstrate promising performance in cancer prognostication. However, gaps in literature include incomplete reporting of model performance, inadequate alignment of model formulation with implementation use-case, and insufficient explainability hindering trust and adoption in clinical settings. Hence, we aim to develop an explainable machine learning EHR-based model that prompts palliative care processes by predicting for 365-day mortality risk among patients with advanced cancer within an outpatient setting.

METHODS

Our cohort consisted of 5,926 adults diagnosed with Stage 3 or 4 solid organ cancer between July 1, 2017, and June 30, 2020 and receiving ambulatory cancer care within a tertiary center. The classification problem was modelled using Extreme Gradient Boosting (XGBoost) and aligned to our envisioned use-case: "Given a prediction point that corresponds to an outpatient cancer encounter, predict for mortality within 365-days from prediction point, using EHR data up to 365-days prior." The model was trained with 75% of the dataset (n = 39,416 outpatient encounters) and validated on a 25% hold-out dataset (n = 13,122 outpatient encounters). To explain model outputs, we used Shapley Additive Explanations (SHAP) values. Clinical characteristics, laboratory tests and treatment data were used to train the model. Performance was evaluated using area under the receiver operating characteristic curve (AUROC) and area under the precision-recall curve (AUPRC), while model calibration was assessed using the Brier score.

RESULTS

In total, 17,149 of the 52,538 prediction points (32.6%) had a mortality event within the 365-day prediction window. The model demonstrated an AUROC of 0.861 (95% CI 0.856-0.867) and AUPRC of 0.771. The Brier score was 0.147, indicating slight overestimations of mortality risk. Explanatory diagrams utilizing SHAP values allowed visualization of feature impacts on predictions at both the global and individual levels.

CONCLUSION

Our machine learning model demonstrated good discrimination and precision-recall in predicting 365-day mortality risk among individuals with advanced cancer. It has the potential to provide personalized mortality predictions and facilitate earlier integration of palliative care.

Consumer satisfaction, palliative care and artificial intelligence (AI)

The scope of artificial intelligence (AI) in healthcare is promising, and AI has the potential to revolutionise the field of palliative care services also. Consumer satisfaction in palliative care is a critical aspect of providing high-quality end-of-life support. It encompasses various elements that contribute to a positive experience for both patients and their families. AI-based tools and technologies can help in early identification of the beneficiaries, reduce the cost, improve the quality of care and satisfaction to the patients with chronic life-limiting illnesses. However, it is essential to ensure that AI is used ethically and in a way that complements, rather than replaces, the human touch and compassionate care, which are the core components of palliative care. This article tries to analyse the scope and challenges of improving consumer satisfaction through AI-based technology in palliative care services.

Prediction Models for Glaucoma in a Multicenter Electronic Health Records Consortium: The Sight Outcomes Research Collaborative

Purpose

Advances in artificial intelligence have enabled the development of predictive models for glaucoma. However, most work is single-center and uncertainty exists regarding the generalizability of such models. The purpose of this study was to build and evaluate machine learning (ML) approaches to predict glaucoma progression requiring surgery using data from a large multicenter consortium of electronic health records (EHR).

Design

Cohort study.

Participants

Thirty-six thousand five hundred forty-eight patients with glaucoma, as identified by International Classification of Diseases (ICD) codes from 6 academic eye centers participating in the Sight OUtcomes Research Collaborative (SOURCE).

Methods

We developed ML models to predict whether patients with glaucoma would progress to glaucoma surgery in the coming year (identified by Current Procedural Terminology codes) using the following modeling approaches: (1) penalized logistic regression (lasso, ridge, and elastic net); (2) tree-based models (random forest, gradient boosted machines, and XGBoost), and (3) deep learning models. Model input features included demographics, diagnosis codes, medications, and clinical information (intraocular pressure, visual acuity, refractive status, and central corneal thickness) available from structured EHR data. One site was reserved as an "external site" test set (N = 1550); of the patients from the remaining sites, 10% each were randomly selected to be in development and test sets, with the remaining 27 999 reserved for model training.

Main Outcome Measures

Evaluation metrics included area under the receiver operating characteristic curve (AUROC) on the test set and the external site.

Results

Six thousand nineteen (16.5%) of 36 548 patients underwent glaucoma surgery. Overall, the AUROC ranged from 0.735 to 0.771 on the random test set and from 0.706 to 0.754 on the external test site, with the XGBoost and random forest model performing best, respectively. There was greatest performance decrease from the random test set to the external test site for the penalized regression models.

Conclusions

Machine learning models developed using structured EHR data can reasonably predict whether glaucoma patients will need surgery, with reasonable generalizability to an external site. Additional research is needed to investigate the impact of protected class characteristics such as race or gender on model performance and fairness.

Financial Disclosures

Proprietary or commercial disclosure may be found in the Footnotes and Disclosures at the end of this article.

Applying Machine Learning Techniques to Implementation Science

Machine learning (ML) approaches could expand the usefulness and application of implementation science methods in clinical medicine and public health settings. The aim of this viewpoint is to introduce a roadmap for applying ML techniques to address implementation science questions, such as predicting what will work best, for whom, under what circumstances, and with what predicted level of support, and what and when adaptation or deimplementation are needed. We describe how ML approaches could be used and discuss challenges that implementation scientists and methodologists will need to consider when using ML throughout the stages of implementation.

Predictive Models for Palliative Care Needs of Advanced Cancer Patients Receiving Chemotherapy

CONTEXT

Early palliative care is recommended within eight-week of diagnosing advanced cancer. Although guidelines suggest routine screening to identify cancer patients who could benefit from palliative care, implementing screening can be challenging due to understaffing and time constraints.

OBJECTIVES

To develop and evaluate machine learning models for predicting specialist palliative care needs in advanced cancer patients undergoing chemotherapy, and to investigate if predictive models could substitute screening tools.

METHODS

We conducted a retrospective cohort study using supervised machine learning. The study included patients aged 18 or older, diagnosed with metastatic or stage IV cancer, who underwent chemotherapy and distress screening at a designated cancer hospital in Japan from April 1, 2018, to March 31, 2023. Specialist palliative care needs were assessed based on distress screening scores and expert evaluations. Data sources were hospital's cancer registry, health claims database, and nursing admission records. The predictive model was developed using XGBoost, a machine learning algorithm.

RESULTS

Out of the 1878 included patients, 561 were analyzed. Among them, 114 (20.3%) exhibited needs for specialist palliative care. After under-sampling to address data imbalance, the models achieved an Area Under the Curve (AUC) of 0.89 with 95.8% sensitivity and a specificity of 71.9%. After feature selection, the model retained five variables, including the patient-reported pain score, and showcased an 0.82 AUC.

CONCLUSION

Our models could forecast specialist palliative care needs for advanced cancer patients on chemotherapy. Using five variables as predictors could replace screening tools and has the potential to contribute to earlier palliative care.

Longitudinal machine learning uncouples healthy aging factors from chronic disease risks

To understand human longevity, inherent aging processes must be distinguished from known etiologies leading to age-related chronic diseases. Such deconvolution is difficult to achieve because it requires tracking patients throughout their entire lives. Here, we used machine learning to infer health trajectories over the entire adulthood age range using extrapolation from electronic medical records with partial longitudinal coverage. Using this approach, our model tracked the state of patients who were healthy and free from known chronic disease risk and distinguished individuals with higher or lower longevity potential using a multivariate score. We showed that the model and the markers it uses performed consistently on data from Israeli, British and US populations. For example, mildly low neutrophil counts and alkaline phosphatase levels serve as early indicators of healthy aging that are independent of risk for major chronic diseases. We characterize the heritability and genetic associations of our longevity score and demonstrate at least 1 year of extended lifespan for parents of high-scoring patients compared to matched controls. Longitudinal modeling of healthy individuals is thereby established as a tool for understanding healthy aging and longevity.

Lessons Learned from a Multi-Site, Team-Based Serious Illness Care Program Implementation at an Academic Medical Center

Background: Patients with serious illness benefit from conversations to share prognosis and explore goals and values. To address this, we implemented Ariadne Labs' Serious Illness Care Program (SICP) at Stanford Health Care. Objective: Improve quantity, timing, and quality of serious illness conversations. Methods: Initial implementation followed Ariadne Labs' SICP framework. We later incorporated a team-based approach that included nonphysician care team members. Outcomes included number of patients with documented conversations according to clinician role and practice location. Machine learning algorithms were used in some settings to identify eligible patients. Results: Ambulatory oncology and hospital medicine were our largest implementation sites, engaging 4707 and 642 unique patients in conversations, respectively. Clinicians across eight disciplines engaged in these conversations. Identified barriers that included leadership engagement, complex workflows, and patient identification. Conclusion: Several factors contributed to successful SICP implementation across clinical sites: innovative clinical workflows, machine learning based predictive algorithms, and nonphysician care team member engagement.

Deep Learning for Epidemiologists: An Introduction to Neural Networks

Deep learning methods are increasingly being applied to problems in medicine and health care. However, few epidemiologists have received formal training in these methods. To bridge this gap, this article introduces the fundamentals of deep learning from an epidemiologic perspective. Specifically, this article reviews core concepts in machine learning (e.g., overfitting, regularization, and hyperparameters); explains several fundamental deep learning architectures (convolutional neural networks, recurrent neural networks); and summarizes training, evaluation, and deployment of models. Conceptual understanding of supervised learning algorithms is the focus of the article; instructions on the training of deep learning models and applications of deep learning to causal learning are out of this article's scope. We aim to provide an accessible first step towards enabling the reader to read and assess research on the medical applications of deep learning and to familiarize readers with deep learning terminology and concepts to facilitate communication with computer scientists and machine learning engineers.

"Just the Facts Ma'am": Moral and Ethical Considerations for Artificial Intelligence in Medicine and its Potential to Impact Patient Autonomy and Hope

Applying machine-based learning and synthetic cognition, commonly referred to as artificial intelligence (AI), to medicine intimates prescient knowledge. The ability of these algorithms to potentially unlock secrets held within vast data sets makes them invaluable to healthcare. Complex computer algorithms are routinely used to enhance diagnoses in fields like oncology, cardiology, and neurology. These algorithms have found utility in making healthcare decisions that are often complicated by seemingly endless relationships between exogenous and endogenous variables. They have also found utility in the allocation of limited healthcare resources and the management of end-of-life issues. With the increase in computing power and the ability to test a virtually unlimited number of relationships, scientists and engineers have the unprecedented ability to increase the prognostic confidence that comes from complex data analysis. While these systems present exciting opportunities for the democratization and precision of healthcare, their use raises important moral and ethical considerations around Christian concepts of autonomy and hope. The purpose of this essay is to explore some of the practical limitations associated with AI in medicine and discuss some of the potential theological implications that machine-generated diagnoses may present. Specifically, this article examines how these systems may disrupt the patient and healthcare provider relationship emblematic of Christ's healing mission. Finally, this article seeks to offer insights that might help in the development of a more robust ethical framework for the application of these systems in the future.

Exploring the Potential of Chatbots in Critical Care Nephrology

The exponential growth of artificial intelligence (AI) has allowed for its integration into multiple sectors, including, notably, healthcare. Chatbots have emerged as a pivotal resource for improving patient outcomes and assisting healthcare practitioners through various AI-based technologies. In critical care, kidney-related conditions play a significant role in determining patient outcomes. This article examines the potential for integrating chatbots into the workflows of critical care nephrology to optimize patient care. We detail their specific applications in critical care nephrology, such as managing acute kidney injury, alert systems, and continuous renal replacement therapy (CRRT); facilitating discussions around palliative care; and bolstering collaboration within a multidisciplinary team. Chatbots have the potential to augment real-time data availability, evaluate renal health, identify potential risk factors, build predictive models, and monitor patient progress. Moreover, they provide a platform for enhancing communication and education for both patients and healthcare providers, paving the way for enriched knowledge and honed professional skills. However, it is vital to recognize the inherent challenges and limitations when using chatbots in this domain. Here, we provide an in-depth exploration of the concerns tied to chatbots' accuracy, dependability, data protection and security, transparency, potential algorithmic biases, and ethical implications in critical care nephrology. While human discernment and intervention are indispensable, especially in complex medical scenarios or intricate situations, the sustained advancements in AI signal that the integration of precision-engineered chatbot algorithms within critical care nephrology has considerable potential to elevate patient care and pivotal outcome metrics in the future.

Evaluating equity in performance of an electronic health record-based 6-month mortality risk model to trigger palliative care consultation: a retrospective model validation analysis

OBJECTIVE

Evaluate predictive performance of an electronic health record (EHR)-based, inpatient 6-month mortality risk model developed to trigger palliative care consultation among patient groups stratified by age, race, ethnicity, insurance and socioeconomic status (SES), which may vary due to social forces (eg, racism) that shape health, healthcare and health data.

DESIGN

Retrospective evaluation of prediction model.

SETTING

Three urban hospitals within a single health system.

PARTICIPANTS

All patients ≥18 years admitted between 1 January and 31 December 2017, excluding observation, obstetric, rehabilitation and hospice (n=58 464 encounters, 41 327 patients).

MAIN OUTCOME MEASURES

General performance metrics (c-statistic, integrated calibration index (ICI), Brier Score) and additional measures relevant to health equity (accuracy, false positive rate (FPR), false negative rate (FNR)).

RESULTS

For black versus non-Hispanic white patients, the model's accuracy was higher (0.051, 95% CI 0.044 to 0.059), FPR lower (-0.060, 95% CI -0.067 to -0.052) and FNR higher (0.049, 95% CI 0.023 to 0.078). A similar pattern was observed among patients who were Hispanic, younger, with Medicaid/missing insurance, or living in low SES zip codes. No consistent differences emerged in c-statistic, ICI or Brier Score. Younger age had the second-largest effect size in the mortality prediction model, and there were large standardised group differences in age (eg, 0.32 for non-Hispanic white versus black patients), suggesting age may contribute to systematic differences in the predicted probabilities between groups.

CONCLUSIONS

An EHR-based mortality risk model was less likely to identify some marginalised patients as potentially benefiting from palliative care, with younger age pinpointed as a possible mechanism. Evaluating predictive performance is a critical preliminary step in addressing algorithmic inequities in healthcare, which must also include evaluating clinical impact, and governance and regulatory structures for oversight, monitoring and accountability.

Evaluating the Potential of Machine Learning and Wearable Devices in End-of-Life Care in Predicting 7-Day Death Events Among Patients With Terminal Cancer: Cohort Study

BACKGROUND

An accurate prediction of mortality in end-of-life care is crucial but presents challenges. Existing prognostic tools demonstrate moderate performance in predicting survival across various time frames, primarily in in-hospital settings and single-time evaluations. However, these tools may fail to capture the individualized and diverse trajectories of patients. Limited evidence exists regarding the use of artificial intelligence (AI) and wearable devices, specifically among patients with cancer at the end of life.

OBJECTIVE

This study aimed to investigate the potential of using wearable devices and AI to predict death events among patients with cancer at the end of life. Our hypothesis was that continuous monitoring through smartwatches can offer valuable insights into the progression of patients at the end of life and enable the prediction of changes in their condition, which could ultimately enhance personalized care, particularly in outpatient or home care settings.

METHODS

This prospective study was conducted at the National Taiwan University Hospital. Patients diagnosed with cancer and receiving end-of-life care were invited to enroll in wards, outpatient clinics, and home-based care settings. Each participant was given a smartwatch to collect physiological data, including steps taken, heart rate, sleep time, and blood oxygen saturation. Clinical assessments were conducted weekly. The participants were followed until the end of life or up to 52 weeks. With these input features, we evaluated the prediction performance of several machine learning-based classifiers and a deep neural network in 7-day death events. We used area under the receiver operating characteristic curve (AUROC), F1-score, accuracy, and specificity as evaluation metrics. A Shapley additive explanations value analysis was performed to further explore the models with good performance.

RESULTS

From September 2021 to August 2022, overall, 1657 data points were collected from 40 patients with a median survival time of 34 days, with the detection of 28 death events. Among the proposed models, extreme gradient boost (XGBoost) yielded the best result, with an AUROC of 96%, F1-score of 78.5%, accuracy of 93%, and specificity of 97% on the testing set. The Shapley additive explanations value analysis identified the average heart rate as the most important feature. Other important features included steps taken, appetite, urination status, and clinical care phase.

CONCLUSIONS

We demonstrated the successful prediction of patient deaths within the next 7 days using a combination of wearable devices and AI. Our findings highlight the potential of integrating AI and wearable technology into clinical end-of-life care, offering valuable insights and supporting clinical decision-making for personalized patient care. It is important to acknowledge that our study was conducted in a relatively small cohort; thus, further research is needed to validate our approach and assess its impact on clinical care.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05054907; https://classic.clinicaltrials.gov/ct2/show/NCT05054907.

Automated machine learning (AutoML) can predict 90-day mortality after gastrectomy for cancer

Early postoperative mortality risk prediction is crucial for clinical management of gastric cancer. This study aims to predict 90-day mortality in gastric cancer patients undergoing gastrectomy using automated machine learning (AutoML), optimize models for preoperative prediction, and identify factors influential in prediction. National Cancer Database was used to identify stage I-III gastric cancer patients undergoing gastrectomy between 2004 and 2016. 26 features were used to train predictive models using H2O.ai AutoML. Performance on validation cohort was measured. In 39,108 patients, 90-day mortality rate was 8.8%. The highest performing model was an ensemble (AUC = 0.77); older age, nodal ratio, and length of inpatient stay (LOS) following surgery were most influential for prediction. Removing the latter two parameters decreased model performance (AUC 0.71). For optimizing models for preoperative use, models were developed to first predict node ratio or LOS, and these predicted values were inputted for 90-day mortality prediction (AUC of 0.73-0.74). AutoML performed well in predicting 90-day mortality in a larger cohort of gastric cancer patients that underwent gastrectomy. These models can be implemented preoperatively to inform prognostication and patient selection for surgery. Our study supports broader evaluation and application of AutoML to guide surgical oncologic care.

The use of artificial intelligence for delivery of essential health services across WHO regions: a scoping review

Background

Artificial intelligence (AI) is a broad outlet of computer science aimed at constructing machines capable of simulating and performing tasks usually done by human beings. The aim of this scoping review is to map existing evidence on the use of AI in the delivery of medical care.

Methods

We searched PubMed and Scopus in March 2022, screened identified records for eligibility, assessed full texts of potentially eligible publications, and extracted data from included studies in duplicate, resolving differences through discussion, arbitration, and consensus. We then conducted a narrative synthesis of extracted data.

Results

Several AI methods have been used to detect, diagnose, classify, manage, treat, and monitor the prognosis of various health issues. These AI models have been used in various health conditions, including communicable diseases, non-communicable diseases, and mental health.

Conclusions

Presently available evidence shows that AI models, predominantly deep learning, and machine learning, can significantly advance medical care delivery regarding the detection, diagnosis, management, and monitoring the prognosis of different illnesses.

Perception of Nurses' Knowledge about Palliative Care in West Bank/ Palestine: Levels and Influencing Factors

Objective: Healthcare professionals particularly nurses should be professionally prepared with knowledge about the standards of palliative care and their roles in providing palliative care. Nurses' knowledge about palliative care and influencing factors has not been examined adequately in Arab countries including Palestine. Thus, this study aimed to assess the adequacy of knowledge level and influencing factors (socio-demographic) about palliative care among nurses in West Bank/ Palestine. Methods: A descriptive-correlational design was utilized. A cluster random sampling method was applied to select 12 hospitals from the three regions in West Bank. Then, four hospitals were selected from each region using a simple random method. All registered nurses working in critical care units and medical and surgical wards in the selected hospitals were recruited. The sample consists of 424 registered nurses and data were collected using Palliative Care Quiz for Nursing (PCQN). Results: The Findings revealed that nurses' level of knowledge about palliative care was low/inadequate (M = 7.75, SD = 2.96). Knowledge about palliative care was influenced by age (B = -.106; p < 0.05), gender (B = -.223; p < 0.001), and hospital ward (B = -.597; p < 0.001), in which younger nurses, females, and those who work in critical care units reported higher levels of knowledge about palliative care. Conclusions: Findings of this study emphasized the need for developing educational and training courses, seminars, and workshops on palliative care to increase nurses' knowledge in order to enhance the quality of patient care. Also, policymakers should develop national strategic plans and policies regarding palliative care and apply these plans in all hospitals in West Bank/ Palestine.

Avoiding Biased Clinical Machine Learning Model Performance Estimates in the Presence of Label Selection

When evaluating the performance of clinical machine learning models, one must consider the deployment population. When the population of patients with observed labels is only a subset of the deployment population (label selection), standard model performance estimates on the observed population may be misleading. In this study we describe three classes of label selection and simulate five causally distinct scenarios to assess how particular selection mechanisms bias a suite of commonly reported binary machine learning model performance metrics. Simulations reveal that when selection is affected by observed features, naive estimates of model discrimination may be misleading. When selection is affected by labels, naive estimates of calibration fail to reflect reality. We borrow traditional weighting estimators from causal inference literature and find that when selection probabilities are properly specified, they recover full population estimates. We then tackle the real-world task of monitoring the performance of deployed machine learning models whose interactions with clinicians feed-back and affect the selection mechanism of the labels. We train three machine learning models to flag low-yield laboratory diagnostics, and simulate their intended consequence of reducing wasteful laboratory utilization. We find that naive estimates of AUROC on the observed population undershoot actual performance by up to 20%. Such a disparity could be large enough to lead to the wrongful termination of a successful clinical decision support tool. We propose an altered deployment procedure, one that combines injected randomization with traditional weighted estimates, and find it recovers true model performance.

Measuring Pediatric Palliative Care Quality: Challenges and Opportunities

Pediatric palliative care (PPC) programs vary widely in structure, staffing, funding, and patient census, resulting in inconsistency in service provision. Improving the quality of palliative care for children living with serious illness and their families requires measuring care quality, ensuring that quality measurement is embedded into day-to-day clinical practice, and aligning quality measurement with healthcare policy priorities. Yet, numerous challenges exist in measuring PPC quality. This paper provides an overview of PPC quality measurement, including challenges, current initiatives, and future opportunities. While important strides toward addressing quality measurement challenges in PPC have been made, including ongoing quality measurement initiatives like the Cambia Metrics Project, the PPC What Matters Most study, and collaborative learning networks, more work remains. Providing high-quality PPC to all children and families will require a multi-pronged approach. In this paper, we suggest several strategies for advancing high-quality PPC, which includes 1) considering how and by whom success is defined, 2) evaluating, adapting, and developing PPC measures, including those that address care disparities within PPC for historically marginalized and excluded communities, 3) improving the infrastructure with which to routinely and prospectively measure, monitor, and report clinical and administrative quality measures, 4) increasing endorsement of PPC quality measures by prominent quality organizations to facilitate accountability and possible reimbursement, and 5) integrating PPC-specific quality measures into the administrative, funding, and policy landscape of pediatric healthcare.

A framework to identify ethical concerns with ML-guided care workflows: a case study of mortality prediction to guide advance care planning

OBJECTIVE

Identifying ethical concerns with ML applications to healthcare (ML-HCA) before problems arise is now a stated goal of ML design oversight groups and regulatory agencies. Lack of accepted standard methodology for ethical analysis, however, presents challenges. In this case study, we evaluate use of a stakeholder "values-collision" approach to identify consequential ethical challenges associated with an ML-HCA for advanced care planning (ACP). Identification of ethical challenges could guide revision and improvement of the ML-HCA.

MATERIALS AND METHODS

We conducted semistructured interviews of the designers, clinician-users, affiliated administrators, and patients, and inductive qualitative analysis of transcribed interviews using modified grounded theory.

RESULTS

Seventeen stakeholders were interviewed. Five "values-collisions"-where stakeholders disagreed about decisions with ethical implications-were identified: (1) end-of-life workflow and how model output is introduced; (2) which stakeholders receive predictions; (3) benefit-harm trade-offs; (4) whether the ML design team has a fiduciary relationship to patients and clinicians; and, (5) how and if to protect early deployment research from external pressures, like news scrutiny, before research is completed.

DISCUSSION

From these findings, the ML design team prioritized: (1) alternative workflow implementation strategies; (2) clarification that prediction was only evaluated for ACP need, not other mortality-related ends; and (3) shielding research from scrutiny until endpoint driven studies were completed.

CONCLUSION

In this case study, our ethical analysis of this ML-HCA for ACP was able to identify multiple sites of intrastakeholder disagreement that mark areas of ethical and value tension. These findings provided a useful initial ethical screening.

Predicting near-term glaucoma progression: An artificial intelligence approach using clinical free-text notes and data from electronic health records

Purpose

The purpose of this study was to develop a model to predict whether or not glaucoma will progress to the point of requiring surgery within the following year, using data from electronic health records (EHRs), including both structured data and free-text progress notes.

Methods

A cohort of adult glaucoma patients was identified from the EHR at Stanford University between 2008 and 2020, with data including free-text clinical notes, demographics, diagnosis codes, prior surgeries, and clinical information, including intraocular pressure, visual acuity, and central corneal thickness. Words from patients' notes were mapped to ophthalmology domain-specific neural word embeddings. Word embeddings and structured clinical data were combined as inputs to deep learning models to predict whether a patient would undergo glaucoma surgery in the following 12 months using the previous 4-12 months of clinical data. We also evaluated models using only structured data inputs (regression-, tree-, and deep-learning-based models) and models using only text inputs.

Results

Of the 3,469 glaucoma patients included in our cohort, 26% underwent surgery. The baseline penalized logistic regression model achieved an area under the receiver operating curve (AUC) of 0.873 and F1 score of 0.750, compared with the best tree-based model (random forest, AUC 0.876; F1 0.746), the deep learning structured features model (AUC 0.885; F1 0.757), the deep learning clinical free-text features model (AUC 0.767; F1 0.536), and the deep learning model with both the structured clinical features and free-text features (AUC 0.899; F1 0.745).

Discussion

Fusion models combining text and EHR structured data successfully and accurately predicted glaucoma progression to surgery. Future research incorporating imaging data could further optimize this predictive approach and be translated into clinical decision support tools.

Generating Older Adult Multimorbidity Trajectories Using Various Comorbidity Indices and Calculation Methods

Background and Objectives

Older adult multimorbidity trajectories are helpful for understanding the current and future health patterns of aging populations. The construction of multimorbidity trajectories from comorbidity index scores will help inform public health and clinical interventions targeting those individuals that are on unhealthy trajectories. Investigators have used many different techniques when creating multimorbidity trajectories in prior literature, and no standard way has emerged. This study compares and contrasts multimorbidity trajectories constructed from various methods.

Research Design and Methods

We describe the difference between aging trajectories constructed with the Charlson Comorbidity Index (CCI) and Elixhauser Comorbidity Index (ECI). We also explore the differences between acute (single-year) and chronic (cumulative) derivations of CCI and ECI scores. Social determinants of health can affect disease burden over time; thus, our models include income, race/ethnicity, and sex differences.

Results

We use group-based trajectory modeling (GBTM) to estimate multimorbidity trajectories for 86,909 individuals aged 66-75 in 1992 using Medicare claims data collected over the following 21 years. We identify low-chronic disease and high-chronic disease trajectories in all 8 generated trajectory models. Additionally, all 8 models satisfied prior established statistical diagnostic criteria for well-performing GBTM models.

Discussion and Implications

Clinicians may use these trajectories to identify patients on an unhealthy path and prompt a possible intervention that may shift the patient to a healthier trajectory.

Advanced Care Planning for Hospitalized Patients Following Clinician Notification of Patient Mortality by a Machine Learning Algorithm

Importance

Goal-concordant care is an ongoing challenge in hospital settings. Identification of high mortality risk within 30 days may call attention to the need to have serious illness conversations, including the documentation of patient goals of care.

Objective

To examine goals of care discussions (GOCDs) in a community hospital setting with patients identified as having a high risk of mortality by a machine learning mortality prediction algorithm.

Design, Setting, and Participants

This cohort study took place at community hospitals within 1 health care system. Participants included adult patients with a high risk of 30-day mortality who were admitted to 1 of 4 hospitals between January 2 and July 15, 2021. Patient encounters of inpatients in the intervention hospital where physicians were notified of the computed high risk mortality score were compared with patient encounters of inpatients in 3 community hospitals without the intervention (ie, matched control).

Intervention

Physicians of patients with a high risk of mortality within 30 days received notification and were encouraged to arrange for GOCDs.

Main Outcomes and Measures

The primary outcome was the percentage change of documented GOCDs prior to discharge. Propensity-score matching was completed on a preintervention and postintervention period using age, sex, race, COVID-19 status, and machine learning-predicted mortality risk scores. A difference-in-difference analysis validated the results.

Results

Overall, 537 patients were included in this study with 201 in the preintervention period (94 in the intervention group; 104 in the control group) and 336 patients in the postintervention period. The intervention and control groups included 168 patients per group and were well-balanced in age (mean [SD], 79.3 [9.60] vs 79.6 [9.21] years; standardized mean difference [SMD], 0.03), sex (female, 85 [51%] vs 85 [51%]; SMD, 0), race (White patients, 145 [86%] vs 144 [86%]; SMD 0.006), and Charlson comorbidities (median [range], 8.00 [2.00-15.0] vs 9.00 [2.00 to 19.0]; SMD, 0.34). Patients in the intervention group from preintervention to postintervention period were associated with being 5 times more likely to have documented GOCDs (OR, 5.11 [95% CI, 1.93 to 13.42]; P = .001) by discharge compared with matched controls, and GOCD occurred significantly earlier in the hospitalization in the intervention patients as compared with matched controls (median, 4 [95% CI, 3 to 6] days vs 16 [95% CI, 15 to not applicable] days; P < .001). Similar findings were observed for Black patient and White patient subgroups.

Conclusions and Relevance

In this cohort study, patients whose physicians had knowledge of high-risk predictions from machine learning mortality algorithms were associated with being 5 times more likely to have documented GOCDs than matched controls. Additional external validation is needed to determine if similar interventions would be helpful at other institutions.

Machine Learning Analysis of RNA-seq Data for Diagnostic and Prognostic Prediction of Colon Cancer

Data from omics studies have been used for prediction and classification of various diseases in biomedical and bioinformatics research. In recent years, Machine Learning (ML) algorithms have been used in many different fields related to healthcare systems, especially for disease prediction and classification tasks. Integration of molecular omics data with ML algorithms has offered a great opportunity to evaluate clinical data. RNA sequence (RNA-seq) analysis has been emerged as the gold standard for transcriptomics analysis. Currently, it is being used widely in clinical research. In our present work, RNA-seq data of extracellular vesicles (EV) from healthy and colon cancer patients are analyzed. Our aim is to develop models for prediction and classification of colon cancer stages. Five different canonical ML and Deep Learning (DL) classifiers are used to predict colon cancer of an individual with processed RNA-seq data. The classes of data are formed on the basis of both colon cancer stages and cancer presence (healthy or cancer). The canonical ML classifiers, which are k-Nearest Neighbor (kNN), Logistic Model Tree (LMT), Random Tree (RT), Random Committee (RC), and Random Forest (RF), are tested with both forms of the data. In addition, to compare the performance with canonical ML models, One-Dimensional Convolutional Neural Network (1-D CNN), Long Short-Term Memory (LSTM), and Bidirectional LSTM (BiLSTM) DL models are utilized. Hyper-parameter optimizations of DL models are constructed by using genetic meta-heuristic optimization algorithm (GA). The best accuracy in cancer prediction is obtained with RC, LMT, and RF canonical ML algorithms as 97.33%. However, RT and kNN show 95.33% performance. The best accuracy in cancer stage classification is achieved with RF as 97.33%. This result is followed by LMT, RC, kNN, and RT with 96.33%, 96%, 94.66%, and 94%, respectively. According to the results of the experiments with DL algorithms, the best accuracy in cancer prediction is obtained with 1-D CNN as 97.67%. BiLSTM and LSTM show 94.33% and 93.67% performance, respectively. In classification of the cancer stages, the best accuracy is achieved with BiLSTM as 98%. 1-D CNN and LSTM show 97% and 94.33% performance, respectively. The results reveal that both canonical ML and DL models may outperform each other for different numbers of features.

Applications of Machine Learning in Palliative Care: A Systematic Review

Objective: To summarize the available literature on using machine learning (ML) for palliative care practice as well as research and to assess the adherence of the published studies to the most important ML best practices. Methods: The MEDLINE database was searched for the use of ML in palliative care practice or research, and the records were screened according to PRISMA guidelines. Results: In total, 22 publications using machine learning for mortality prediction (n = 15), data annotation (n = 5), predicting morbidity under palliative therapy (n = 1), and predicting response to palliative therapy (n = 1) were included. Publications used a variety of supervised or unsupervised models, but mostly tree-based classifiers and neural networks. Two publications had code uploaded to a public repository, and one publication uploaded the dataset. Conclusions: Machine learning in palliative care is mainly used to predict mortality. Similarly to other applications of ML, external test sets and prospective validations are the exception.

Predicting Patient Mortality for Earlier Palliative Care Identification in Medicare Advantage Plans: Features of a Machine Learning Model

BACKGROUND

Machine learning (ML) can offer greater precision and sensitivity in predicting Medicare patient end of life and potential need for palliative services compared to provider recommendations alone. However, earlier ML research on older community dwelling Medicare beneficiaries has provided insufficient exploration of key model feature impacts and the role of the social determinants of health.

OBJECTIVE

This study describes the development of a binary classification ML model predicting 1-year mortality among Medicare Advantage plan members aged ≥65 years (N=318,774) and further examines the top features of the predictive model.

METHODS

A light gradient-boosted trees model configuration was selected based on 5-fold cross-validation. The model was trained with 80% of cases (n=255,020) using randomized feature generation periods, with 20% (n=63,754) reserved as a holdout for validation. The final algorithm used 907 feature inputs extracted primarily from claims and administrative data capturing patient diagnoses, service utilization, demographics, and census tract-based social determinants index measures.

RESULTS

The total sample had an actual mortality prevalence of 3.9% in the 2018 outcome period. The final model correctly predicted 44.2% of patient expirations among the top 1% of highest risk members (AUC=0.84; 95% CI 0.83-0.85) versus 24.0% predicted by the model iteration using only age, gender, and select high-risk utilization features (AUC=0.74; 95% CI 0.73-0.74). The most important algorithm features included patient demographics, diagnoses, pharmacy utilization, mean costs, and certain social determinants of health.

CONCLUSIONS

The final ML model better predicts Medicare Advantage member end of life using a variety of routinely collected data and supports earlier patient identification for palliative care.

Deep Learning Provides a New Magnetic Resonance Imaging-Based Prognostic Biomarker for Recurrence Prediction in High-Grade Serous Ovarian Cancer

This study aims to use a deep learning method to develop a signature extract from preoperative magnetic resonance imaging (MRI) and to evaluate its ability as a non-invasive recurrence risk prognostic marker in patients with advanced high-grade serous ovarian cancer (HGSOC). Our study comprises a total of 185 patients with pathologically confirmed HGSOC. A total of 185 patients were randomly assigned in a 5:3:2 ratio to a training cohort (n = 92), validation cohort 1 (n = 56), and validation cohort 2 (n = 37). We built a new deep learning network from 3839 preoperative MRI images (T2-weighted images and diffusion-weighted images) to extract HGSOC prognostic indicators. Following that, a fusion model including clinical and deep learning features is developed to predict patients' individual recurrence risk and 3-year recurrence likelihood. In the two validation cohorts, the consistency index of the fusion model was higher than both the deep learning model and the clinical feature model (0.752, 0.813 vs. 0.625, 0.600 vs. 0.505, 0.501). Among the three models, the fusion model had a higher AUC than either the deep learning model or the clinical model in validation cohorts 1 or 2 (AUC = was 0.986, 0.961 vs. 0.706, 0.676/0.506, 0.506). Using the DeLong method, the difference between them was statistically significant (p < 0.05). The Kaplan-Meier analysis distinguished two patient groups with high and low recurrence risk (p = 0.0008 and 0.0035, respectively). Deep learning may be a low-cost, non-invasive method for predicting risk for advanced HGSOC recurrence. Deep learning based on multi-sequence MRI serves as a prognostic biomarker for advanced HGSOC, which provides a preoperative model for predicting recurrence in HGSOC. Additionally, using the fusion model as a new prognostic analysis means that can use MRI data can be used without the need to follow-up the prognostic biomarker.

Huchaiah M. A Deep Learning Approaches for Modeling and Predicting of HIV Test Results Using EDHS Dataset

At present, HIV/AIDS has steadily been listed in the top position as a major cause of death. However, HIV is largely preventable and can be avoided by making strategies to increase HIV early prediction. So, there is a need for a predictive tool that can help the domain experts with early prediction of the disease and hence can recommend strategies to stop the prognosis of the diseases. Using deep learning models, we investigated whether demographic and health survey dataset might be utilized to predict HIV test status. The contribution of this work is to improve the accuracy of a model for predicting an individual’s HIV test status. We employed deep learning models to predict HIV status using Ethiopian demography and health survey (EDHS) datasets. Furthermore, we discovered that predictive models based on these dataset may be used to forecast individuals’ HIV test status, which might assist domain experts prioritize strategies and policies to safeguard the pandemic. The outcome of the study confirms that a DL model provides the best results with the most promising extracted features. The accuracy of the all DL models can further be enhanced by including the big dataset for predicting the prognosis of the disease.

The Virtues of Interpretable Medical AI

Artificial intelligence (AI) systems have demonstrated impressive performance across a variety of clinical tasks. However, notoriously, sometimes these systems are "black boxes." The initial response in the literature was a demand for "explainable AI." However, recently, several authors have suggested that making AI more explainable or "interpretable" is likely to be at the cost of the accuracy of these systems and that prioritizing interpretability in medical AI may constitute a "lethal prejudice." In this paper, we defend the value of interpretability in the context of the use of AI in medicine. Clinicians may prefer interpretable systems over more accurate black boxes, which in turn is sufficient to give designers of AI reason to prefer more interpretable systems in order to ensure that AI is adopted and its benefits realized. Moreover, clinicians may be justified in this preference. Achieving the downstream benefits from AI is critically dependent on how the outputs of these systems are interpreted by physicians and patients. A preference for the use of highly accurate black box AI systems, over less accurate but more interpretable systems, may itself constitute a form of lethal prejudice that may diminish the benefits of AI to-and perhaps even harm-patients.

The Virtues of Interpretable Medical Artificial Intelligence

Artificial intelligence (AI) systems have demonstrated impressive performance across a variety of clinical tasks. However, notoriously, sometimes these systems are "black boxes." The initial response in the literature was a demand for "explainable AI." However, recently, several authors have suggested that making AI more explainable or "interpretable" is likely to be at the cost of the accuracy of these systems and that prioritizing interpretability in medical AI may constitute a "lethal prejudice." In this article, we defend the value of interpretability in the context of the use of AI in medicine. Clinicians may prefer interpretable systems over more accurate black boxes, which in turn is sufficient to give designers of AI reason to prefer more interpretable systems in order to ensure that AI is adopted and its benefits realized. Moreover, clinicians may be justified in this preference. Achieving the downstream benefits from AI is critically dependent on how the outputs of these systems are interpreted by physicians and patients. A preference for the use of highly accurate black box AI systems, over less accurate but more interpretable systems, may itself constitute a form of lethal prejudice that may diminish the benefits of AI to-and perhaps even harm-patients.

Towards cataloguing and characterising advance care planning and end-of-life care resources

BACKGROUND

Resources for healthcare professionals, patients and those important to them relating to planning and coordinating treatment and care at the end of life are abundant, and can be difficult to navigate. However, they have not been systematically collated or catalogued in terms of their purpose, scope or intended audience.

AIM

To collate, categorise and characterise advance care planning and end-of-life treatment and care (EoLT + C) resources directed towards healthcare professionals, patients and their families.

METHODS

Rapid review and thematic synthesis of resources available in the United Kingdom. Google searches and reviews of websites belonging to selected organisations that develop and publish materials relating to EoLT + C, and advance care planning were used. Materials were included if they were intended for those over 18 living in the UK and pertained to five domains of EoLT + C: identifying those approaching end of life; accessing EoLT + C services; conducting important conversations about EoLT + C and preferences; advance care planning, including recording of preferences and plans; and ensuring that plans and preferences are accessed and used by health and social care services.

RESULTS

246 resources directed at healthcare professionals, patients and their families were identified, collated, catalogued and made internationally available for clinicians, researchers, patients and the public. 61 were classified as interactive, providing decision support in EoLT + C that went beyond simply providing information. Of these, there was notable content overlap among tools for identifying patients in their last year of life. There was variation in the development of tools across all domains of end-of-life care by geography and patient group. Few interactive resources integrated seamlessly with a digital interface or healthcare provider workflows. Incentives for the adoption of best-practice appeared rare.

CONCLUSIONS

We present a repeatable and scalable approach to the cataloguing and characterisation of palliative care resources. The identified resources will be of benefit not only to those in the UK but to those in other countries, developing or evaluating their own resources for aiding professionals and patients to plan and deliver excellent treatment and care at the end of life.

Breaking bad news in the era of artificial intelligence and algorithmic medicine: an exploration of disclosure and its ethical justification using the hedonic calculus

An appropriate ethical framework around the use of Artificial Intelligence (AI) in healthcare has become a key desirable with the increasingly widespread deployment of this technology. Advances in AI hold the promise of improving the precision of outcome prediction at the level of the individual. However, the addition of these technologies to patient-clinician interactions, as with any complex human interaction, has potential pitfalls. While physicians have always had to carefully consider the ethical background and implications of their actions, detailed deliberations around fast-moving technological progress may not have kept up. We use a common but key challenge in healthcare interactions, the disclosure of bad news (likely imminent death), to illustrate how the philosophical framework of the 'Felicific Calculus' developed in the eighteenth century by Jeremy Bentham, may have a timely quasi-quantitative application in the age of AI. We show how this ethical algorithm can be used to assess, across seven mutually exclusive and exhaustive domains, whether an AI-supported action can be morally justified.

Integrated imaging and molecular analysis to decipher tumor microenvironment in the era of immunotherapy

Radiological imaging is an integral component of cancer care, including diagnosis, staging, and treatment response monitoring. It contains rich information about tumor phenotypes that are governed not only by cancer cellintrinsic biological processes but also by the tumor microenvironment, such as the composition and function of tumor-infiltrating immune cells. By analyzing the radiological scans using a quantitative radiomics approach, robust relations between specific imaging and molecular phenotypes can be established. Indeed, a number of studies have demonstrated the feasibility of radiogenomics for predicting intrinsic molecular subtypes and gene expression signatures in breast cancer based on MRI. In parallel, promising results have been shown for inferring the amount of tumor-infiltrating lymphocytes, a key factor for the efficacy of cancer immunotherapy, from standard-of-care radiological images. Compared with the biopsy-based approach, radiogenomics offers a unique avenue to profile the molecular makeup of the tumor and immune microenvironment as well as its evolution in a noninvasive and holistic manner through longitudinal imaging scans. Here, we provide a systematic review of the state of the art radiogenomics studies in the era of immunotherapy and discuss emerging paradigms and opportunities in AI and deep learning approaches. These technical advances are expected to transform the radiogenomics field, leading to the discovery of reliable imaging biomarkers. This will pave the way for their clinical translation to guide precision cancer therapy.

Contribution of Synthetic Data Generation towards an Improved Patient Stratification in Palliative Care

AI model development for synthetic data generation to improve Machine Learning (ML) methodologies is an integral part of research in Computer Science and is currently being transferred to related medical fields, such as Systems Medicine and Medical Informatics. In general, the idea of personalized decision-making support based on patient data has driven the motivation of researchers in the medical domain for more than a decade, but the overall sparsity and scarcity of data are still major limitations. This is in contrast to currently applied technology that allows us to generate and analyze patient data in diverse forms, such as tabular data on health records, medical images, genomics data, or even audio and video. One solution arising to overcome these data limitations in relation to medical records is the synthetic generation of tabular data based on real world data. Consequently, ML-assisted decision-support can be interpreted more conveniently, using more relevant patient data at hand. At a methodological level, several state-of-the-art ML algorithms generate and derive decisions from such data. However, there remain key issues that hinder a broad practical implementation in real-life clinical settings. In this review, we will give for the first time insights towards current perspectives and potential impacts of using synthetic data generation in palliative care screening because it is a challenging prime example of highly individualized, sparsely available patient information. Taken together, the reader will obtain initial starting points and suitable solutions relevant for generating and using synthetic data for ML-based screenings in palliative care and beyond.

Predicting mortality in The Irish Longitudinal Study on Ageing (TILDA): development of a four-year index and comparison with international measures

Objectives

We aimed to replicate existing international (US and UK) mortality indices using Irish data. We developed and validated a four-year mortality index for adults aged 50 + in Ireland and compared performance with these international indices. We then extended this model by including additional predictors (self-report and healthcare utilization) and compared its performance to our replication model.

Methods

Eight thousand one hundred seventy-four participants in The Irish Longitudinal Study on Ageing were split for development (n = 4,121) and validation (n = 4,053). Six baseline predictor categories were examined (67 variables total): demographics; cardiovascular-related illness; non-cardiovascular illness; health and lifestyle variables; functional variables; self-report (wellbeing and social connectedness) and healthcare utilization. We identified variables independently associated with four-year mortality in the development cohort and attached these variables a weight according to strength of association. We summed the weights to calculate a single index score for each participant and evaluated predicted accuracy in the validation cohort.

Results

Our final 14-predictor (extended) model assigned risk points for: male (1pt); age (65–69: 2pts; 70–74: 4 pts; 75–79: 4pts; 80–84: 6pts; 85 + : 7pts); heart attack (1pt); cancer (3pts); smoked past age 30 (2pts); difficulty walking 100 m (2pts); difficulty using the toilet (3pts); difficulty lifting 10lbs (1pts); poor self-reported health (1pt); and hospital admission in previous year (1pt). Index discrimination was strong (ROC area = 0.78).

Discussion

Our index is predictive of four-year mortality in community-dwelling older Irish adults. Comparisons with the international indices show that our 12-predictor (replication) model performed well and suggests that generalisability is high. Our 14-predictor (extended) model showed modest improvements compared to the 12-predictor model.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12877-022-03196-z.

Random Noise vs. State-of-the-Art Probabilistic Forecasting Methods: A Case Study on CRPS-Sum Discrimination Ability

The recent developments in the machine-learning domain have enabled the development of complex multivariate probabilistic forecasting models. To evaluate the predictive power of these complex methods, it is pivotal to have a precise evaluation method to gauge the performance and predictability power of these complex methods. To do so, several evaluation metrics have been proposed in the past (such as the energy score, Dawid–Sebastiani score, and variogram score); however, these cannot reliably measure the performance of a probabilistic forecaster. Recently, CRPS-Sum has gained a lot of prominence as a reliable metric for multivariate probabilistic forecasting. This paper presents a systematic evaluation of CRPS-Sum to understand its discrimination ability. We show that the statistical properties of target data affect the discrimination ability of CRPS-Sum. Furthermore, we highlight that CRPS-Sum calculation overlooks the performance of the model on each dimension. These flaws can lead us to an incorrect assessment of model performance. Finally, with experiments on real-world datasets, we demonstrate that the shortcomings of CRPS-Sum provide a misleading indication of the probabilistic forecasting performance method. We illustrate that it is easily possible to have a better CRPS-Sum for a dummy model, which looks like random noise, in comparison to the state-of-the-art method.

Prospective Comparison of Medical Oncologists and a Machine Learning Model to Predict 3-Month Mortality in Patients With Metastatic Solid Tumors

Importance

To date, oncologist and model prognostic performance have been assessed independently and mostly retrospectively; however, how model prognostic performance compares with oncologist prognostic performance prospectively remains unknown.

Objective

To compare oncologist performance with a model in predicting 3-month mortality for patients with metastatic solid tumors in an outpatient setting.

Design, Setting, and Participants

This prognostic study evaluated prospective predictions for a cohort of patients with metastatic solid tumors seen in outpatient oncology clinics at a National Cancer Institute-designated cancer center and associated satellites between December 6, 2019, and August 6, 2021. Oncologists (57 physicians and 17 advanced practice clinicians) answered a 3-month surprise question (3MSQ) within clinical pathways. A model was trained with electronic health record data from January 1, 2013, to April 24, 2019, to identify patients at high risk of 3-month mortality and deployed silently in October 2019. Analysis was limited to oncologist prognostications with a model prediction within the preceding 30 days.

Exposures

Three-month surprise question and gradient-boosting binary classifier.

Main Outcomes and Measures

The primary outcome was performance comparison between oncologists and the model to predict 3-month mortality. The primary performance metric was the positive predictive value (PPV) at the sensitivity achieved by the medical oncologists with their 3MSQ answers.

Results

A total of 74 oncologists answered 3099 3MSQs for 2041 patients with advanced cancer (median age, 62.6 [range, 18-96] years; 1271 women [62.3%]). In this cohort with a 15% prevalence of 3-month mortality and 30% sensitivity for both oncologists and the model, the PPV of oncologists was 34.8% (95% CI, 30.1%-39.5%) and the PPV of the model was 60.0% (95% CI, 53.6%-66.3%). Area under the receiver operating characteristic curve for the model was 81.2% (95% CI, 79.1%-83.3%). The model significantly outperformed the oncologists in short-term mortality.

Conclusions and Relevance

In this prognostic study, the model outperformed oncologists overall and within the breast and gastrointestinal cancer cohorts in predicting 3-month mortality for patients with advanced cancer. These findings suggest that further studies may be useful to examine how model predictions could improve oncologists' prognostic confidence and patient-centered goal-concordant care at the end of life.

Intelligent Palliative Care Based on Patient-Reported Outcome Measures

CONTEXT

The growth of patient reported outcome measures data in palliative care provides an opportunity for machine learning to identify patterns in patient responses signifying different phases of illness.

OBJECTIVES

The study will explore if machine learning and network analysis can identify phases in patient palliative status through symptoms reported on the Integrated Palliative Care Outcome Scale (IPOS).

METHODS

A partly cross-sectional and partially longitudinal observational study was undertaken using the Australasian Karnofsky Performance Scale (AKPS); Integrated Palliative Care Outcome Scale (IPOS); Phase of Illness (POI). Patient palliative records (n = 1507, 65% stable, 20% unstable, 9% deteriorating, 2% terminal) from 804 adult patients enrolled in a New Zealand palliative care service were analysed using a combination of statistical, machine learning and network analysis techniques.

RESULTS

Data from IPOS showed considerable variation with phase. Also, network analysis showed clear associations between items by phase. Six machine learning techniques identified the most important variables for predicting possible transition between phases of illness. Network analysis for all patients showed that Poor Appetite and Loss of Energy were central IPOS items, with Loss of Energy linked to Drowsiness, Shortness of Breath and Lack of Mobility on the one hand, and Poor Appetite linked to Nausea, Vomiting, Constipation and Sore and Dry Mouth on the other.

CONCLUSION

These preliminary results, when coupled with the latest technological developments in mobile apps and wearable technology, could point the way to increased use of digital therapeutics in continuous palliative care monitoring.

Development and Structure of an Accurate Machine Learning Algorithm to Predict Inpatient Mortality and Hospice Outcomes in the Coronavirus Disease 2019 Era

BACKGROUND

The coronavirus disease 2019 (COVID-19) pandemic has challenged the accuracy and racial biases present in traditional mortality scores. An accurate prognostic model that can be applied to hospitalized patients irrespective of race or COVID-19 status may benefit patient care.

RESEARCH DESIGN

This cohort study utilized historical and ongoing electronic health record features to develop and validate a deep-learning model applied on the second day of admission predicting a composite outcome of in-hospital mortality, discharge to hospice, or death within 30 days of admission. Model features included patient demographics, diagnoses, procedures, inpatient medications, laboratory values, vital signs, and substance use history. Conventional performance metrics were assessed, and subgroup analysis was performed based on race, COVID-19 status, and intensive care unit admission.

SUBJECTS

A total of 35,521 patients hospitalized between April 2020 and October 2020 at a single health care system including a tertiary academic referral center and 9 community hospitals.

RESULTS

Of 35,521 patients, including 9831 non-White patients and 2020 COVID-19 patients, 2838 (8.0%) met the composite outcome. Patients who experienced the composite outcome were older (73 vs. 61 y old) with similar sex and race distributions between groups. The model achieved an area under the receiver operating characteristic curve of 0.89 (95% confidence interval: 0.88, 0.91) and an average positive predictive value of 0.46 (0.40, 0.52). Model performance did not differ significantly in White (0.89) and non-White (0.90) subgroups or when grouping by COVID-19 status and intensive care unit admission.

CONCLUSION

A deep-learning model using large-volume, structured electronic health record data can effectively predict short-term mortality or hospice outcomes on the second day of admission in the general inpatient population without significant racial bias.

Complementary frailty and mortality prediction models on older patients as a tool for assessing palliative care needs

Palliative care (PC) has demonstrated benefits for life-limiting illnesses. Bad survival prognosis and patients' decline are working criteria to guide PC decision-making for older patients. Still, there is not a clear consensus on when to initiate early PC. This work aims to propose machine learning approaches to predict frailty and mortality in older patients in supporting PC decision-making. Predictive models based on Gradient Boosting Machines (GBM) and Deep Neural Networks (DNN) were implemented for binary 1-year mortality classification, survival estimation and 1-year frailty classification. Besides, we tested the similarity between mortality and frailty distributions. The 1-year mortality classifier achieved an Area Under the Curve Receiver Operating Characteristic (AUC ROC) of 0.87 [0.86, 0.87], whereas the mortality regression model achieved an mean absolute error (MAE) of 333.13 [323.10, 342.49] days. Moreover, the 1-year frailty classifier obtained an AUC ROC of 0.89 [0.88, 0.90]. Mortality and frailty criteria were weakly correlated and had different distributions, which can be interpreted as these assessment measurements are complementary for PC decision-making. This study provides new models that can be part of decision-making systems for PC services in older patients after their external validation.

A systematic review of the prediction of hospital length of stay: Towards a unified framework

Hospital length of stay of patients is a crucial factor for the effective planning and management of hospital resources. There is considerable interest in predicting the LoS of patients in order to improve patient care, control hospital costs and increase service efficiency. This paper presents an extensive review of the literature, examining the approaches employed for the prediction of LoS in terms of their merits and shortcomings. In order to address some of these problems, a unified framework is proposed to better generalise the approaches that are being used to predict length of stay. This includes the investigation of the types of routinely collected data used in the problem as well as recommendations to ensure robust and meaningful knowledge modelling. This unified common framework enables the direct comparison of results between length of stay prediction approaches and will ensure that such approaches can be used across several hospital environments. A literature search was conducted in PubMed, Google Scholar and Web of Science from 1970 until 2019 to identify LoS surveys which review the literature. 32 Surveys were identified, from these 32 surveys, 220 papers were manually identified to be relevant to LoS prediction. After removing duplicates, and exploring the reference list of studies included for review, 93 studies remained. Despite the continuing efforts to predict and reduce the LoS of patients, current research in this domain remains ad-hoc; as such, the model tuning and data preprocessing steps are too specific and result in a large proportion of the current prediction mechanisms being restricted to the hospital that they were employed in. Adopting a unified framework for the prediction of LoS could yield a more reliable estimate of the LoS as a unified framework enables the direct comparison of length of stay methods. Additional research is also required to explore novel methods such as fuzzy systems which could build upon the success of current models as well as further exploration of black-box approaches and model interpretability.

Identification of Digital Health Priorities for Palliative Care Research: Modified Delphi Study

BACKGROUND

Developments in digital health have the potential to transform the delivery of health and social care to help citizens manage their health. Currently, there is a lack of consensus about digital health research priorities in palliative care and a lack of theories about how these technologies might improve care outcomes. Therefore, it is important for health care leaders to identify innovations to ensure that an increasingly frail population has appropriate access to palliative care services. Consequently, it is important to articulate research priorities as the first step in determining how finite resources should be allocated to a field saturated with rapidly developing innovation.

OBJECTIVE

The aim of this study is to identify research priority areas for digital health in palliative care.

METHODS

We selected digital health trends, most relevant to palliative care, from a list of emerging trends reported by a leading institute of quantitative futurists. We conducted 2 rounds of the Delphi questionnaire, followed by a consensus meeting and public engagement workshop to establish a final consensus on research priorities for digital technology in palliative care. We used the views of public representatives to gain their perspectives on the agreed priorities.

RESULTS

A total of 103 experts (representing 11 countries) participated in the first Delphi round. Of the 103 experts, 55 (53.3%) participated in the second round. The final consensus meetings were attended by 10.7% (11/103) of the experts. We identified 16 priority areas, which involved many applications of technologies, including care for patients and caregivers, self-management and reporting of diseases, education and training, communication, care coordination, and research methodology. We summarized the priority areas into eight topics: big data, mobile devices, telehealth and telemedicine, virtual reality, artificial intelligence, smart home, biotechnology, and digital legacy.

CONCLUSIONS

The priorities identified in this study represent a wide range of important emerging areas in the fields of digital health, personalized medicine, and data science. Human-centered design and robust governance systems should be considered in future research. It is important that the risks of using these technologies in palliative care are properly addressed to ensure that these tools are used meaningfully, wisely, and safely and do not cause unintentional harm.

A Comparison of Models Predicting One-Year Mortality at Time of Admission

CONTEXT

Hospitalization provides an opportunity to address end-of-life care (EoLC) preferences if patients at risk of death can be accurately identified while in the hospital. The modified Hospital One-Year Mortality Risk (mHOMR) uses demographic and admission data in a logistic regression algorithm to identify patients at risk of death one year from admission.

OBJECTIVES

This project sought to validate mHOMR and identify superior models.

METHODS

The mHOMR model was validated using historical data from an academic health system. Alternative logistic regression and random forest (RF) models were developed using the same variables. Receiver operating characteristic (ROC) and precision recall curves were developed, and sensitivity, specificity, and positive and negative predictive values were compared over a range of model thresholds.

RESULTS

The RF model demonstrated higher area under the ROC curve (0.950, 95% CI 0.947 - 0.954) as compared to the logistic regression models (0.818 [95% CI 0.812 - 0.825] and 0.841 [95% CI 0.836 - 0.847]). Area under the precision recall curve was higher with the random forest model compared to the logistic regression models (0.863 vs. 0.458 and 0.494, respectively). Across a range of thresholds, the RF model demonstrated superior sensitivity, equivalent specificity, and higher positive and negative predictive values.

CONCLUSION

A machine learning RF model, using common demographic and utilization data available on hospital admission, identified inpatients at risk of death more effectively than logistic regression models using the same variables. Machine learning models have promise for identifying admitted patients with elevated one-year mortality risk, increasing opportunities to prompt discussion of EoLC preferences.

Utilisation of goals of care discussions and palliative care prior to image-guided procedures near the end of life

AIM

To characterise image-guided procedures performed near the end of life and the use of goals of care discussions (GOC) and palliative care consultation (PCC) prior to these procedures.

MATERIALS AND METHODS

Retrospective chart review of 3,714 consecutive inpatient procedures performed for 2,351 patients and 8,206 outpatient procedures performed for 5,225 patients within a suburban medical system. Data were collected on demographics, procedures performed, mortality, and use of GOC or PCC prior to the procedures. Procedures near the end of life were classified as emergent, elective, or palliative. Logistic regression was used to assess for demographic disparities in care.

RESULTS

Nine percent of inpatients died within 30 days of their procedure, 57% of which were within the same hospitalisation. Of these patients, 59% had a documented GOC and 35% had a PCC. Similarly, 7% of outpatients died within 6 months of their procedure. A minority of these patients had a documented GOC (37%) or PCC (13%). There were few statistically significant demographic disparities in this care and the associated odds ratios were small.

CONCLUSION

A wide array of image-guided procedures is performed near the end of life. GOC and PCC are underutilised prior to these procedures. Few demographic disparities exist in this care.

The path from big data analytics capabilities to value in hospitals: a scoping review

BACKGROUND

As the uptake of health information technologies increased, most healthcare organizations have become producers of big data. A growing number of hospitals are investing in the development of big data analytics (BDA) capabilities. If the promises associated with these capabilities are high, how hospitals create value from it remains unclear. The present study undertakes a scoping review of existing research on BDA use in hospitals to describe the path from BDA capabilities (BDAC) to value and its associated challenges.

METHODS

This scoping review was conducted following Arksey and O'Malley's 5 stages framework. A systematic search strategy was adopted to identify relevant articles in Scopus and Web of Science. Data charting and extraction were performed following an analytical framework that builds on the resource-based view of the firm to describe the path from BDA capabilities to value in hospitals.

RESULTS

Of 1,478 articles identified, 94 were included. Most of them are experimental research (n=69) published in medical (n=66) or computer science journals (n=28). The main value targets associated with the use of BDA are improving the quality of decision-making (n=56) and driving innovation (n=52) which apply mainly to care (n=67) and administrative (n=48) activities. To reach these targets, hospitals need to adequately combine BDA capabilities and value creation mechanisms (VCM) to enable knowledge generation and drive its assimilation. Benefits are endpoints of the value creation process. They are expected in all articles but realized in a few instances only (n=19).

CONCLUSIONS

This review confirms the value creation potential of BDA solutions in hospitals. It also shows the organizational challenges that prevent hospitals from generating actual benefits from BDAC-building efforts. The configuring of strategies, technologies and organizational capabilities underlying the development of value-creating BDA solutions should become a priority area for research, with focus on the mechanisms that can drive the alignment of BDA and organizational strategies, and the development of organizational capabilities to support knowledge generation and assimilation.

Evaluation and Mitigation of Racial Bias in Clinical Machine Learning Models: A Scoping Review (Preprint)

Background

Racial bias is a key concern regarding the development, validation, and implementation of machine learning (ML) models in clinical settings. Despite the potential of bias to propagate health disparities, racial bias in clinical ML has yet to be thoroughly examined and best practices for bias mitigation remain unclear.

Objective

Our objective was to perform a scoping review to characterize the methods by which the racial bias of ML has been assessed and describe strategies that may be used to enhance algorithmic fairness in clinical ML.

Methods

A scoping review was conducted in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) Extension for Scoping Reviews. A literature search using PubMed, Scopus, and Embase databases, as well as Google Scholar, identified 635 records, of which 12 studies were included.

Results

Applications of ML were varied and involved diagnosis, outcome prediction, and clinical score prediction performed on data sets including images, diagnostic studies, clinical text, and clinical variables. Of the 12 studies, 1 (8%) described a model in routine clinical use, 2 (17%) examined prospectively validated clinical models, and the remaining 9 (75%) described internally validated models. In addition, 8 (67%) studies concluded that racial bias was present, 2 (17%) concluded that it was not, and 2 (17%) assessed the implementation of bias mitigation strategies without comparison to a baseline model. Fairness metrics used to assess algorithmic racial bias were inconsistent. The most commonly observed metrics were equal opportunity difference (5/12, 42%), accuracy (4/12, 25%), and disparate impact (2/12, 17%). All 8 (67%) studies that implemented methods for mitigation of racial bias successfully increased fairness, as measured by the authors’ chosen metrics. Preprocessing methods of bias mitigation were most commonly used across all studies that implemented them.

Conclusions

The broad scope of medical ML applications and potential patient harms demand an increased emphasis on evaluation and mitigation of racial bias in clinical ML. However, the adoption of algorithmic fairness principles in medicine remains inconsistent and is limited by poor data availability and ML model reporting. We recommend that researchers and journal editors emphasize standardized reporting and data availability in medical ML studies to improve transparency and facilitate evaluation for racial bias.