Benchmarking Deep Learning Architectures for Predicting Readmission to the ICU and Describing Patients-at-Risk

Prediction of emergency department revisits among child and youth mental health outpatients using deep learning techniques

BACKGROUND

The proportion of Canadian youth seeking mental health support from an emergency department (ED) has risen in recent years. As EDs typically address urgent mental health crises, revisiting an ED may represent unmet mental health needs. Accurate ED revisit prediction could aid early intervention and ensure efficient healthcare resource allocation. We examine the potential increased accuracy and performance of graph neural network (GNN) machine learning models compared to recurrent neural network (RNN), and baseline conventional machine learning and regression models for predicting ED revisit in electronic health record (EHR) data.

METHODS

This study used EHR data for children and youth aged 4-17 seeking services at McMaster Children's Hospital's Child and Youth Mental Health Program outpatient service to develop and evaluate GNN and RNN models to predict whether a child/youth with an ED visit had an ED revisit within 30 days. GNN and RNN models were developed and compared against conventional baseline models. Model performance for GNN, RNN, XGBoost, decision tree and logistic regression models was evaluated using F1 scores.

RESULTS

The GNN model outperformed the RNN model by an F1-score increase of 0.0511 and the best performing conventional machine learning model by an F1-score increase of 0.0470. Precision, recall, receiver operating characteristic (ROC) curves, and positive and negative predictive values showed that the GNN model performed the best, and the RNN model performed similarly to the XGBoost model. Performance increases were most noticeable for recall and negative predictive value than for precision and positive predictive value.

CONCLUSIONS

This study demonstrates the improved accuracy and potential utility of GNN models in predicting ED revisits among children and youth, although model performance may not be sufficient for clinical implementation. Given the improvements in recall and negative predictive value, GNN models should be further explored to develop algorithms that can inform clinical decision-making in ways that facilitate targeted interventions, optimize resource allocation, and improve outcomes for children and youth.

Predicting unplanned readmissions in the intensive care unit: a multimodality evaluation

A hospital readmission is when a patient who was discharged from the hospital is admitted again for the same or related care within a certain period. Hospital readmissions are a significant problem in the healthcare domain, as they lead to increased hospitalization costs, decreased patient satisfaction, and increased risk of adverse outcomes such as infections, medication errors, and even death. The problem of hospital readmissions is particularly acute in intensive care units (ICUs), due to the severity of the patients' conditions, and the substantial risk of complications. Predicting Unplanned Readmissions in ICUs is a challenging task, as it involves analyzing different data modalities, such as static data, unstructured free text, sequences of diagnoses and procedures, and multivariate time-series. Here, we investigate the effectiveness of each data modality separately, then alongside with others, using state-of-the-art machine learning approaches in time-series analysis and natural language processing. Using our evaluation process, we are able to determine the contribution of each data modality, and for the first time in the context of readmission, establish a hierarchy of their predictive value. Additionally, we demonstrate the impact of Temporal Abstractions in enhancing the performance of time-series approaches to readmission prediction. Due to conflicting definitions in the literature, we also provide a clear definition of the term Unplanned Readmission to enhance reproducibility and consistency of future research and to prevent any potential misunderstandings that could result from diverse interpretations of the term. Our experimental results on a large benchmark clinical data set show that Discharge Notes written by physicians, have better capabilities for readmission prediction than all other modalities.

Assessment of Prediction Tasks and Time Window Selection in Temporal Modeling of Electronic Health Record Data: a Systematic Review

Temporal electronic health record (EHR) data are often preferred for clinical prediction tasks because they offer more complete representations of a patient's pathophysiology than static data. A challenge when working with temporal EHR data is problem formulation, which includes defining the time windows of interest and the prediction task. Our objective was to conduct a systematic review that assessed the definition and reporting of concepts relevant to temporal clinical prediction tasks. We searched PubMed® and IEEE Xplore® databases for studies from January 1, 2010 applying machine learning models to EHR data for patient outcome prediction. Publications applying time-series methods were selected for further review. We identified 92 studies and summarized them by clinical context and definition and reporting of the prediction problem. For the time windows of interest, 12 studies did not discuss window lengths, 57 used a single set of window lengths, and 23 evaluated the relationship between window length and model performance. We also found that 72 studies had appropriate reporting of the prediction task. However, evaluation of prediction problem formulation for temporal EHR data was complicated by heterogeneity in assessing and reporting of these concepts. Even among studies modeling similar clinical outcomes, there were variations in terminology used to describe the prediction problem, rationale for window lengths, and determination of the outcome of interest. As temporal modeling using EHR data expands, minimal reporting standards should include time-series specific concerns to promote rigor and reproducibility in future studies and facilitate model implementation in clinical settings.

Supplementary Information

The online version contains supplementary material available at 10.1007/s41666-023-00143-4.

Representation of time-varying and time-invariant EMR data and its application in modeling outcome prediction for heart failure patients

OBJECTIVE

To represent a patient record with both time-invariant and time-varying features as a single vector using an end-to-end deep learning model, and further to predict the kidney failure (KF) status and mortality of heart failure (HF) patients.

MATERIALS AND METHODS

The time-invariant EMR data included demographic information and comorbidities, and the time-varying EMR data were lab tests. We used a Transformer encoder module to represent the time-invariant data, and refined a long short-term memory (LSTM) with a Transformer encoder attached to the top to represent the time-varying data, taking the original measured values and their corresponding embedding vectors, masking vectors, and two types of time intervals as inputs. The proposed representations of patients with time-invariant and time-varying data were used to predict KF status (949 out of 5268 HF patients diagnosed with KF) and mortality (463 in-hospital deaths) for HF patients. Comparative experiments were conducted between the proposed model and some representative machine learning models. Ablation experiments were also performed around the time-varying data representation, including replacing the refined LSTM with the standard LSTM, GRU-D and T-LSTM, respectively, and removing the Transformer encoder and the time-varying data representation module, respectively. The visualization of the attention weights of the time-invariant and time-varying features was used to clinically interpret the predictive performance. We used the area under the receiver operating characteristic curve (AUROC), the area under the precision-recall curve (AUPRC), and the F1-score to evaluate the predictive performance of the models.

RESULTS

The proposed model achieved superior performance, with average AUROCs, AUPRCs and F1-scores of 0.960, 0.610 and 0.759 for KF prediction and 0.937, 0.353 and 0.537 for mortality prediction, respectively. Predictive performance improved with the addition of time-varying data from longer time periods. The proposed model outperformed the comparison and ablation references in both prediction tasks.

CONCLUSIONS

Both time-invariant and time-varying EMR data of patients could be efficiently represented by the proposed unified deep learning model, which shows higher performance in clinical prediction tasks. The way to use time-varying data in the current study is hopeful to be used in other kinds of time-varying data and other clinical tasks.

LGTRL-DE: Local and Global Temporal Representation Learning with Demographic Embedding for in-hospital mortality prediction

Predicting the patient's in-hospital mortality from the historical Electronic Medical Records (EMRs) can assist physicians to make clinical decisions and assign medical resources. In recent years, researchers proposed many deep learning methods to predict in-hospital mortality by learning patient representations. However, most of these methods fail to comprehensively learn the temporal representations and do not sufficiently mine the contextual knowledge of demographic information. We propose a novel end-to-end approach based on Local and Global Temporal Representation Learning with Demographic Embedding (LGTRL-DE) to address the current issues for in-hospital mortality prediction. LGTRL-DE is enabled by (1) a local temporal representation learning module that captures the temporal information and analyzes the health status from a local perspective through a recurrent neural network with the demographic initialization and the local attention mechanism; (2) a Transformer-based global temporal representation learning module that extracts the interaction dependencies among clinical events; (3) a multi-view representation fusion module that fuses temporal and static information and generates the final patient's health representations. We evaluate our proposed LGTRL-DE on two public real-world clinical datasets (MIMIC-III and e-ICU). Experimental results show that LGTRL-DE achieves an area under receiver operating characteristic curve of 0.8685 and 0.8733 on the MIMIC-III and e-ICU datasets, respectively, outperforming state-of-the-art approaches.

Automated screening of potential organ donors using a temporal machine learning model

Organ donation is not meeting demand, and yet 30-60% of potential donors are potentially not identified. Current systems rely on manual identification and referral to an Organ Donation Organization (ODO). We hypothesized that developing an automated screening system based on machine learning could reduce the proportion of missed potentially eligible organ donors. Using routine clinical data and laboratory time-series, we retrospectively developed and tested a neural network model to automatically identify potential organ donors. We first trained a convolutive autoencoder that learned from the longitudinal changes of over 100 types of laboratory results. We then added a deep neural network classifier. This model was compared to a simpler logistic regression model. We observed an AUROC of 0.966 (CI 0.949-0.981) for the neural network and 0.940 (0.908-0.969) for the logistic regression model. At a prespecified cutoff, sensitivity and specificity were similar between both models at 84% and 93%. Accuracy of the neural network model was robust across donor subgroups and remained stable in a prospective simulation, while the logistic regression model performance declined when applied to rarer subgroups and in the prospective simulation. Our findings support using machine learning models to help with the identification of potential organ donors using routinely collected clinical and laboratory data.

All-Cause Readmission or Potentially Avoidable Readmission: Which Is More Predictable Using Frailty, Comorbidities, and ADL?

Background and Objectives

Readmission-related health care reforms have shifted their focus from all-cause readmissions (ACR) to potentially avoidable readmissions (PAR). However, little is known about the utility of analytic tools from administrative data in predicting PAR. This study determined whether 30-day ACR or 30-day PAR is more predictable using tools that assess frailty, comorbidities, and activities of daily living (ADL) from administrative data.

Research Design and Methods

This retrospective cohort study was conducted at a large general acute care hospital in Tokyo, Japan. We analyzed patients aged ≥70 years who had been admitted to and discharged from the subject hospital between July 2016 and February 2021. Using administrative data, we assessed each patient's Hospital Frailty Risk Score, Charlson Comorbidity Index, and Barthel Index on admission. To determine the influence of each tool on readmission predictions, we constructed logistic regression models with different combinations of independent variables for predicting unplanned ACR and PAR within 30 days of discharge.

Results

Among 16 313 study patients, 4.1% experienced 30-day ACR and 1.8% experienced 30-day PAR. The full model (including sex, age, annual household income, frailty, comorbidities, and ADL as independent variables) for 30-day PAR showed better discrimination (C-statistic: 0.79, 95% confidence interval: 0.77-0.82) than the full model for 30-day ACR (0.73, 0.71-0.75). The other prediction models for 30-day PAR also had consistently better discrimination than their corresponding models for 30-day ACR.

Discussion and Implications

PAR is more predictable than ACR when using tools that assess frailty, comorbidities, and ADL from administrative data. Our PAR prediction model may contribute to the accurate identification of at-risk patients in clinical settings who would benefit from transitional care interventions.

Improving Intensive Care Unit Early Readmission Prediction Using Optimized and Explainable Machine Learning

It is of great interest to develop and introduce new techniques to automatically and efficiently analyze the enormous amount of data generated in today's hospitals, using state-of-the-art artificial intelligence methods. Patients readmitted to the ICU in the same hospital stay have a higher risk of mortality, morbidity, longer length of stay, and increased cost. The methodology proposed to predict ICU readmission could improve the patients' care. The objective of this work is to explore and evaluate the potential improvement of existing models for predicting early ICU patient readmission by using optimized artificial intelligence algorithms and explainability techniques. In this work, XGBoost is used as a predictor model, combined with Bayesian techniques to optimize it. The results obtained predicted early ICU readmission (AUROC of 0.92 ± 0.03) improves state-of-the-art consulted works (whose AUROC oscillate between 0.66 and 0.78). Moreover, we explain the internal functioning of the model by using Shapley Additive Explanation-based techniques, allowing us to understand the model internal performance and to obtain useful information, as patient-specific information, the thresholds from which a feature begins to be critical for a certain group of patients, and the feature importance ranking.

The risk assessment tool for intensive care unit readmission: A systematic review and meta-analysis

OBJECTIVE

To review and evaluate existing risk assessment tools for intensive care unitreadmission.

METHODS

Nine electronic databases (Medline, CINAHL, Web of Science, Cochrane Library, Embase, Sino Med, CNKI, VIP, and Wan fang) were systematically searched from their inception to September 2022. Two authors independently extracted data from the literature included. Meta-analysis was performed under the bivariate modeling and summary receiver operating characteristic curve method.

RESULTS

A total of 29 studies were included in this review, among which 11 were quantitatively Meta-analyzed. The results showed Stability and Workload Index for Transfer: Sensitivity = 0.55, Specificity = 0.65, Area under curve = 0.63. And Early warning score: Sensitivity = 0.78, Specificity = 0.83, Area under curve = 0.88. The remaining tools included scores, nomograms, machine learning models, and deep learning models. These studies, with varying reports on thresholds, case selection, data preprocessing, and model performance, have a high risk of bias.

CONCLUSION

We cannot identify a tool that can be used directly in intensive care unit readmission risk assessment. Scores based on early warning score are moderately accurate in predicting readmission, but there is heterogeneity and publication bias that requires model adjustment for local factors such as resources, demographics, and case mix. Machine learning models present a promising modeling technique but have a high methodological bias and require further validation.

IMPLICATIONS FOR CLINICAL PRACTICE

Using reliable risk assessment tools is essential for the early identification of unplanned intensive care unit readmission risk in critically ill patients.  A reliable risk assessment tool must be developed, which is the focus of further research.

Predictive Modeling for Readmission to Intensive Care: A Systematic Review

To evaluate the methodologic rigor and predictive performance of models predicting ICU readmission; to understand the characteristics of ideal prediction models; and to elucidate relationships between appropriate triage decisions and patient outcomes.

DATA SOURCES

PubMed, Web of Science, Cochrane, and Embase.

STUDY SELECTION

Primary literature that reported the development or validation of ICU readmission prediction models within from 2010 to 2021.

DATA EXTRACTION

Relevant study information was extracted independently by two authors using the Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies checklist. Bias was evaluated using the Prediction model Risk Of Bias ASsessment Tool. Data sources, modeling methodology, definition of outcomes, performance, and risk of bias were critically evaluated to elucidate relevant relationships.

DATA SYNTHESIS

Thirty-three articles describing models were included. Six studies had a high overall risk of bias due to improper inclusion criteria or omission of critical analysis details. Four other studies had an unclear overall risk of bias due to lack of detail describing the analysis. Overall, the most common (50% of studies) source of bias was the filtering of candidate predictors via univariate analysis. The poorest performing models used existing clinical risk or acuity scores such as Acute Physiologic Assessment and Chronic Health Evaluation II, Sequential Organ Failure Assessment, or Stability and Workload Index for Transfer as the sole predictor. The higher-performing ICU readmission prediction models used homogenous patient populations, specifically defined outcomes, and routinely collected predictors that were analyzed over time.

CONCLUSIONS

Models predicting ICU readmission can achieve performance advantages by using longitudinal time series modeling, homogenous patient populations, and predictor variables tailored to those populations.

An AI based digital-twin for prioritising pneumonia patient treatment

A digital-twin based three-tiered system is proposed to prioritise patients for urgent intensive care and ventilator support. The deep learning methods are used to build patient-specific digital-twins to identify and prioritise critical cases amongst severe pneumonia patients. The three-tiered strategy is proposed to generate severity indices to: (1) identify urgent cases, (2) assign critical care and mechanical ventilation, and (3) discontinue mechanical ventilation and critical care at the optimal time. The severity indices calculated in the present study are the probability of death and the probability of requiring mechanical ventilation. These enable the generation of patient prioritisation lists and facilitates the smooth flow of patients in and out of Intensive Therapy Units (ITUs). The proposed digital-twin is built on pre-trained deep learning models using data from more than 1895 pneumonia patients. The severity indices calculated in the present study are assessed using the standard benchmark of Area Under Receiving Operating Characteristic Curve (AUROC). The results indicate that the ITU and mechanical ventilation can be prioritised correctly to an AUROC value as high as 0.89. This model may be employed in its current form to COVID-19 patients, but transfer learning with COVID-19 patient data will improve the predictions. The digital-twin model developed and tested is available via accompanying Supplemental material.

Development and validation of an interpretable 3 day intensive care unit readmission prediction model using explainable boosting machines

Background

Intensive care unit (ICU) readmissions are associated with mortality and poor outcomes. To improve discharge decisions, machine learning (ML) could help to identify patients at risk of ICU readmission. However, as many models are black boxes, dangerous properties may remain unnoticed. Widely used post hoc explanation methods also have inherent limitations. Few studies are evaluating inherently interpretable ML models for health care and involve clinicians in inspecting the trained model.

Methods

An inherently interpretable model for the prediction of 3 day ICU readmission was developed. We used explainable boosting machines that learn modular risk functions and which have already been shown to be suitable for the health care domain. We created a retrospective cohort of 15,589 ICU stays and 169 variables collected between 2006 and 2019 from the University Hospital Münster. A team of physicians inspected the model, checked the plausibility of each risk function, and removed problematic ones. We collected qualitative feedback during this process and analyzed the reasons for removing risk functions. The performance of the final explainable boosting machine was compared with a validated clinical score and three commonly used ML models. External validation was performed on the widely used Medical Information Mart for Intensive Care version IV database.

Results

The developed explainable boosting machine used 67 features and showed an area under the precision-recall curve of 0.119 ± 0.020 and an area under the receiver operating characteristic curve of 0.680 ± 0.025. It performed on par with state-of-the-art gradient boosting machines (0.123 ± 0.016, 0.665 ± 0.036) and outperformed the Simplified Acute Physiology Score II (0.084 ± 0.025, 0.607 ± 0.019), logistic regression (0.092 ± 0.026, 0.587 ± 0.016), and recurrent neural networks (0.095 ± 0.008, 0.594 ± 0.027). External validation confirmed that explainable boosting machines (0.221 ± 0.023, 0.760 ± 0.010) performed similarly to gradient boosting machines (0.232 ± 0.029, 0.772 ± 0.018). Evaluation of the model inspection showed that explainable boosting machines can be useful to detect and remove problematic risk functions.

Conclusions

We developed an inherently interpretable ML model for 3 day ICU readmission prediction that reached the state-of-the-art performance of black box models. Our results suggest that for low- to medium-dimensional datasets that are common in health care, it is feasible to develop ML models that allow a high level of human control without sacrificing performance.

A Machine Learning Approach for Predicting Readmission Charges Billed by Hospitals

Background

The Centers for Medicare and Medicaid Services projects that health care costs will continue to grow over the next few years. Rising readmission costs contribute significantly to increasing health care costs. Multiple areas of health care, including readmissions, have benefited from the application of various machine learning algorithms in several ways.

Objective

We aimed to identify suitable models for predicting readmission charges billed by hospitals. Our literature review revealed that this application of machine learning is underexplored. We used various predictive methods, ranging from glass-box models (such as regularization techniques) to black-box models (such as deep learning–based models).

Methods

We defined readmissions as readmission with the same major diagnostic category (RSDC) and all-cause readmission category (RADC). For these readmission categories, 576,701 and 1,091,580 individuals, respectively, were identified from the Nationwide Readmission Database of the Healthcare Cost and Utilization Project by the Agency for Healthcare Research and Quality for 2013. Linear regression, lasso regression, elastic net, ridge regression, eXtreme gradient boosting (XGBoost), and a deep learning model based on multilayer perceptron (MLP) were the 6 machine learning algorithms we tested for RSDC and RADC through 10-fold cross-validation.

Results

Our preliminary analysis using a data-driven approach revealed that within RADC, the subsequent readmission charge billed per patient was higher than the previous charge for 541,090 individuals, and this number was 319,233 for RSDC. The top 3 major diagnostic categories (MDCs) for such instances were the same for RADC and RSDC. The average readmission charge billed was higher than the previous charge for 21 of the MDCs in the case of RSDC, whereas it was only for 13 of the MDCs in RADC. We recommend XGBoost and the deep learning model based on MLP for predicting readmission charges. The following performance metrics were obtained for XGBoost: (1) RADC (mean absolute percentage error [MAPE]=3.121%; root mean squared error [RMSE]=0.414; mean absolute error [MAE]=0.317; root relative squared error [RRSE]=0.410; relative absolute error [RAE]=0.399; normalized RMSE [NRMSE]=0.040; mean absolute deviation [MAD]=0.031) and (2) RSDC (MAPE=3.171%; RMSE=0.421; MAE=0.321; RRSE=0.407; RAE=0.393; NRMSE=0.041; MAD=0.031). The performance obtained for MLP-based deep neural networks are as follows: (1) RADC (MAPE=3.103%; RMSE=0.413; MAE=0.316; RRSE=0.410; RAE=0.397; NRMSE=0.040; MAD=0.031) and (2) RSDC (MAPE=3.202%; RMSE=0.427; MAE=0.326; RRSE=0.413; RAE=0.399; NRMSE=0.041; MAD=0.032). Repeated measures ANOVA revealed that the mean RMSE differed significantly across models with P<.001. Post hoc tests using the Bonferroni correction method indicated that the mean RMSE of the deep learning/XGBoost models was statistically significantly (P<.001) lower than that of all other models, namely linear regression/elastic net/lasso/ridge regression.

Conclusions

Models built using XGBoost and MLP are suitable for predicting readmission charges billed by hospitals. The MDCs allow models to accurately predict hospital readmission charges.

Individualizing deep dynamic models for psychological resilience data

Deep learning approaches can uncover complex patterns in data. In particular, variational autoencoders achieve this by a non-linear mapping of data into a low-dimensional latent space. Motivated by an application to psychological resilience in the Mainz Resilience Project, which features intermittent longitudinal measurements of stressors and mental health, we propose an approach for individualized, dynamic modeling in this latent space. Specifically, we utilize ordinary differential equations (ODEs) and develop a novel technique for obtaining person-specific ODE parameters even in settings with a rather small number of individuals and observations, incomplete data, and a differing number of observations per individual. This technique allows us to subsequently investigate individual reactions to stimuli, such as the mental health impact of stressors. A potentially large number of baseline characteristics can then be linked to this individual response by regularized regression, e.g., for identifying resilience factors. Thus, our new method provides a way of connecting different kinds of complex longitudinal and baseline measures via individualized, dynamic models. The promising results obtained in the exemplary resilience application indicate that our proposal for dynamic deep learning might also be more generally useful for other application domains.

Improving Performance of Outcome Prediction for In-patients with Acute Myocardial Infarction Based on Embedding Representation Learned from Electronic Medical Records: Development and Validation Study (Preprint)

Background

The widespread secondary use of electronic medical records (EMRs) promotes health care quality improvement. Representation learning that can automatically extract hidden information from EMR data has gained increasing attention.

Objective

We aimed to propose a patient representation with more feature associations and task-specific feature importance to improve the outcome prediction performance for inpatients with acute myocardial infarction (AMI).

Methods

Medical concepts, including patients’ age, gender, disease diagnoses, laboratory tests, structured radiological features, procedures, and medications, were first embedded into real-value vectors using the improved skip-gram algorithm, where concepts in the context windows were selected by feature association strengths measured by association rule confidence. Then, each patient was represented as the sum of the feature embeddings weighted by the task-specific feature importance, which was applied to facilitate predictive model prediction from global and local perspectives. We finally applied the proposed patient representation into mortality risk prediction for 3010 and 1671 AMI inpatients from a public data set and a private data set, respectively, and compared it with several reference representation methods in terms of the area under the receiver operating characteristic curve (AUROC), area under the precision-recall curve (AUPRC), and F1-score.

Results

Compared with the reference methods, the proposed embedding-based representation showed consistently superior predictive performance on the 2 data sets, achieving mean AUROCs of 0.878 and 0.973, AUPRCs of 0.220 and 0.505, and F1-scores of 0.376 and 0.674 for the public and private data sets, respectively, while the greatest AUROCs, AUPRCs, and F1-scores among the reference methods were 0.847 and 0.939, 0.196 and 0.283, and 0.344 and 0.361 for the public and private data sets, respectively. Feature importance integrated in patient representation reflected features that were also critical in prediction tasks and clinical practice.

Conclusions

The introduction of feature associations and feature importance facilitated an effective patient representation and contributed to prediction performance improvement and model interpretation.

Data Science Trends Relevant to Nursing Practice: A Rapid Review of the 2020 Literature

BACKGROUND

The term "data science" encompasses several methods, many of which are considered cutting edge and are being used to influence care processes across the world. Nursing is an applied science and a key discipline in health care systems in both clinical and administrative areas, making the profession increasingly influenced by the latest advances in data science. The greater informatics community should be aware of current trends regarding the intersection of nursing and data science, as developments in nursing practice have cross-professional implications.

OBJECTIVES

This study aimed to summarize the latest (calendar year 2020) research and applications of nursing-relevant patient outcomes and clinical processes in the data science literature.

METHODS

We conducted a rapid review of the literature to identify relevant research published during the year 2020. We explored the following 16 topics: (1) artificial intelligence/machine learning credibility and acceptance, (2) burnout, (3) complex care (outpatient), (4) emergency department visits, (5) falls, (6) health care-acquired infections, (7) health care utilization and costs, (8) hospitalization, (9) in-hospital mortality, (10) length of stay, (11) pain, (12) patient safety, (13) pressure injuries, (14) readmissions, (15) staffing, and (16) unit culture.

RESULTS

Of 16,589 articles, 244 were included in the review. All topics were represented by literature published in 2020, ranging from 1 article to 59 articles. Numerous contemporary data science methods were represented in the literature including the use of machine learning, neural networks, and natural language processing.

CONCLUSION

This review provides an overview of the data science trends that were relevant to nursing practice in 2020. Examinations of such literature are important to monitor the status of data science's influence in nursing practice.

Fusion of Sequential Visits and Medical Ontology for Mortality Prediction

The goal of mortality prediction task is to predict the future death risk of patients according to their previous Electronic Healthcare Records (EHR). The main challenge of mortality prediction is how to design an accurate and robust predictive model with sequential, multivariate, sparse and irregular EHR data. In addition, the performance of model may be affected by lack of sufficient information of some patients with rare diseases in EHRs. To address these challenges, we propose a model to fuse Sequential visits and Medical Ontology to predict patients' death risk. SeMO not only learns reasonable embeddings for medical concepts from sequential and irregular visits, but also exploits medical ontology to improve the prediction performance. With integration of multivariate features, SeMO learns robust representations of medical codes, mitigating data insufficiency and insightful sequential dependencies among patient's visits. Experimental results on real world datasets prove that the proposed SeMO improves the prediction performance compared with the baseline approaches. Our model achieves an precision of up to 0.975. Compared with RNN, the precision has been improved up to 2.204%.

Plague risk in the western United States over seven decades of environmental change

After several pandemics over the last two millennia, the wildlife reservoirs of plague (Yersinia pestis) now persist around the world, including in the western United States. Routine surveillance in this region has generated comprehensive records of human cases and animal seroprevalence, creating a unique opportunity to test how plague reservoirs are responding to environmental change. Here, we test whether animal and human data suggest that plague reservoirs and spillover risk have shifted since 1950. To do so, we develop a new method for detecting the impact of climate change on infectious disease distributions, capable of disentangling long-term trends (signal) and interannual variation in both weather and sampling (noise). We find that plague foci are associated with high-elevation rodent communities, and soil biochemistry may play a key role in the geography of long-term persistence. In addition, we find that human cases are concentrated only in a small subset of endemic areas, and that spillover events are driven by higher rodent species richness (the amplification hypothesis) and climatic anomalies (the trophic cascade hypothesis). Using our detection model, we find that due to the changing climate, rodent communities at high elevations have become more conducive to the establishment of plague reservoirs-with suitability increasing up to 40% in some places-and that spillover risk to humans at mid-elevations has increased as well, although more gradually. These results highlight opportunities for deeper investigation of plague ecology, the value of integrative surveillance for infectious disease geography, and the need for further research into ongoing climate change impacts.

Predicting Unplanned 7-day Intensive Care Unit Readmissions with Machine Learning Models for Improved Discharge Risk Assessment

Unplanned readmission to the intensive care unit (ICU) confers excess morbidity and mortality. We explore whether machine learning models can outperform the current standard, the Stability and Workload Index for Transfer (SWIFT) score, in assessing 7-day ICU readmission risk at discharge. Logistic regression, random forest, support vector machine, and gradient boosting models were trained and validated on Stanford Hospital data (2009-2019), externally validated on Beth Israel Deaconess Medical Center (BIDMC) data (2008-2019) and benchmarked against SWIFT. The best performing model was gradient boosting, with AUROC of 0.85 and 0.60 and F1-score of 0.43 and 0.14 on internal and external validation, respectively. SWIFT had an AUROC of 0.67 and 0.51 and F1-score of 0.33 and 0.10 on Stanford and BIDMC data, respectively. Machine learning models predicting 7-day ICU readmission risk can improve current ICU discharge risk assessment standards, but performance may be limited without local training.

Deep learning for temporal data representation in electronic health records: A systematic review of challenges and methodologies

OBJECTIVE

Temporal electronic health records (EHRs) contain a wealth of information for secondary uses, such as clinical events prediction and chronic disease management. However, challenges exist for temporal data representation. We therefore sought to identify these challenges and evaluate novel methodologies for addressing them through a systematic examination of deep learning solutions.

METHODS

We searched five databases (PubMed, Embase, the Institute of Electrical and Electronics Engineers [IEEE] Xplore Digital Library, the Association for Computing Machinery [ACM] Digital Library, and Web of Science) complemented with hand-searching in several prestigious computer science conference proceedings. We sought articles that reported deep learning methodologies on temporal data representation in structured EHR data from January 1, 2010, to August 30, 2020. We summarized and analyzed the selected articles from three perspectives: nature of time series, methodology, and model implementation.

RESULTS

We included 98 articles related to temporal data representation using deep learning. Four major challenges were identified, including data irregularity, heterogeneity, sparsity, and model opacity. We then studied how deep learning techniques were applied to address these challenges. Finally, we discuss some open challenges arising from deep learning.

CONCLUSION

Temporal EHR data present several major challenges for clinical prediction modeling and data utilization. To some extent, current deep learning solutions can address these challenges. Future studies may consider designing comprehensive and integrated solutions. Moreover, researchers should incorporate clinical domain knowledge into study designs and enhance model interpretability to facilitate clinical implementation.

Plague reservoir species throughout the world

Plague has been known since ancient times as a re-emerging infectious disease, causing considerable socioeconomic burden in regional hotspots. To better understand the epidemiological cycle of the causative agent of the plague, its potential occurrence, and possible future dispersion, one must carefully consider the taxonomy, distribution, and ecological requirements of reservoir-species in relation either to natural or human-driven changes (e.g. climate change or urbanization). In recent years, the depth of knowledge on species taxonomy and species composition in different landscapes has undergone a dramatic expansion, driven by modern taxonomic methods such as synthetic surveys that take into consideration morphology, genetics, and the ecological setting of captured animals to establish their species identities. Here, we consider the recent taxonomic changes of the rodent species in known plague reservoirs and detail their distribution across the world, with a particular focus on those rodents considered to be keystone host species. A complete checklist of all known plague-infectable vertebrates living in plague foci is provided as a Supporting Information table.

Comparison of Back-Propagation Neural Network, LACE Index and HOSPITAL Score in Predicting All-Cause Risk of 30-Day Readmission

Background

The main purpose of this study is to predict the all-cause risk of 30-day readmission by employing the back-propagation neural network (BPNN) in comparison with traditional risk assessment tools of LACE index and HOSPITAL scores.

Methods

This was a retrospective cohort study from January 1st, 2018 to December 31st, 2019. A total of 55,688 hospitalizations from a medical center in Taiwan were examined. The LACE index (length of stay, acute admission, Charlson comorbidity index score, emergency department visits in previous 6 months) and HOSPITAL score (hemoglobin level at discharge, discharge from an Oncology service, sodium level at discharge, procedure during hospital stay, Index admission type, number of hospital admissions during the previous year, length of stay) are calculated. We employed variables from LACE index and HOSPITAL score as the input vector of BPNN for comparison purposes.

Results

The BPNN constructed in the current study has a considerably better ability with a C statistics achieved 0.74 (95% CI 0.73 to 0.75), which is statistically significant larger than that of the other two models using DeLong’s test. Also, it was possible to achieve higher sensitivity (70.32%) without penalizing the specificity (71.76%) and accuracy (71.68%) at its optimal threshold, which is at the 20% of patients with the highest predicted risk. Moreover, it is much more informative than the other two methods because of a considerably higher LR+ and a lower LR-.

Conclusion

Our findings suggest that more attention should be paid to methods based on non-linear classification systems, as they lead to substantial differences in risk-scores.

Uncertainty-Gated Stochastic Sequential Model for EHR Mortality Prediction

Electronic health records (EHRs) are characterized as nonstationary, heterogeneous, noisy, and sparse data; therefore, it is challenging to learn the regularities or patterns inherent within them. In particular, sparseness caused mostly by many missing values has attracted the attention of researchers who have attempted to find a better use of all available samples for determining the solution of a primary target task through defining a secondary imputation problem. Methodologically, existing methods, either deterministic or stochastic, have applied different assumptions to impute missing values. However, once the missing values are imputed, most existing methods do not consider the fidelity or confidence of the imputed values in the modeling of downstream tasks. Undoubtedly, an erroneous or improper imputation of missing variables can cause difficulties in the modeling as well as a degraded performance. In this study, we present a novel variational recurrent network that: 1) estimates the distribution of missing variables (e.g., the mean and variance) allowing to represent uncertainty in the imputed values; 2) updates hidden states by explicitly applying fidelity based on a variance of the imputed values during a recurrence (i.e., uncertainty propagation over time); and 3) predicts the possibility of in-hospital mortality. It is noteworthy that our model can conduct these procedures in a single stream and learn all network parameters jointly in an end-to-end manner. We validated the effectiveness of our method using the public data sets of MIMIC-III and PhysioNet challenge 2012 by comparing with and outperforming other state-of-the-art methods for mortality prediction considered in our experiments. In addition, we identified the behavior of the model that well represented the uncertainties for the imputed estimates, which showed a high correlation between the uncertainties and mean absolute error (MAE) scores for imputation.

The Physiological Deep Learner: First application of multitask deep learning to predict hypotension in critically ill patients

Critical care clinicians are trained to analyze simultaneously multiple physiological parameters to predict critical conditions such as hemodynamic instability. We developed the Multi-task Learning Physiological Deep Learner (MTL-PDL), a deep learning algorithm that predicts simultaneously the mean arterial pressure (MAP) and the heart rate (HR). In an external validation dataset, our model exhibited very good calibration: R² of 0.747 (95% confidence interval, 0.692 to 0.794) and 0.850 (0.815 to 0.879) for respectively, MAP and HR prediction 60-minutes ahead of time. For acute hypotensive episodes defined as a MAP below 65 mmHg for 5 min, our MTL-PDL reached a predictive value of 90% for patients at very high risk (predicted MAP ≤ 60 mmHg) and 2‰ for patients at low risk (predicted MAP >70 mmHg). Based on its excellent prediction performance, the Physiological Deep Learner has the potential to help the clinician proactively adjust the treatment in order to avoid hypotensive episodes and end-organ hypoperfusion.

Current Trends in Readmission Prediction: An Overview of Approaches

Hospital readmission shortly after discharge threatens the quality of patient care and leads to increased medical care costs. In the United States, hospitals with high readmission rates are subject to federal financial penalties. This concern calls for incentives for healthcare facilities to reduce their readmission rates by predicting patients who are at high risk of readmission. Conventional practices involve the use of rule-based assessment scores and traditional statistical methods, such as logistic regression, in developing risk prediction models. The recent advancements in machine learning driven by improved computing power and sophisticated algorithms have the potential to produce highly accurate predictions. However, the value of such models could be overrated. Meanwhile, the use of other flexible models that leverage simple algorithms offer great transparency in terms of feature interpretation, which is beneficial in clinical settings. This work presents an overview of the current trends in risk prediction models developed in the field of readmission. The various techniques adopted by researchers in recent years are described, and the topic of whether complex models outperform simple ones in readmission risk stratification is investigated.

Patient Representation From Structured Electronic Medical Records Based on Embedding Technique: Development and Validation Study

Background

The secondary use of structured electronic medical record (sEMR) data has become a challenge due to the diversity, sparsity, and high dimensionality of the data representation. Constructing an effective representation for sEMR data is becoming more and more crucial for subsequent data applications.

Objective

We aimed to apply the embedding technique used in the natural language processing domain for the sEMR data representation and to explore the feasibility and superiority of the embedding-based feature and patient representations in clinical application.

Methods

The entire training corpus consisted of records of 104,752 hospitalized patients with 13,757 medical concepts of disease diagnoses, physical examinations and procedures, laboratory tests, medications, etc. Each medical concept was embedded into a 200-dimensional real number vector using the Skip-gram algorithm with some adaptive changes from shuffling the medical concepts in a record 20 times. The average of vectors for all medical concepts in a patient record represented the patient. For embedding-based feature representation evaluation, we used the cosine similarities among the medical concept vectors to capture the latent clinical associations among the medical concepts. We further conducted a clustering analysis on stroke patients to evaluate and compare the embedding-based patient representations. The Hopkins statistic, Silhouette index (SI), and Davies-Bouldin index were used for the unsupervised evaluation, and the precision, recall, and F1 score were used for the supervised evaluation.

Results

The dimension of patient representation was reduced from 13,757 to 200 using the embedding-based representation. The average cosine similarity of the selected disease (subarachnoid hemorrhage) and its 15 clinically relevant medical concepts was 0.973. Stroke patients were clustered into two clusters with the highest SI (0.852). Clustering analyses conducted on patients with the embedding representations showed higher applicability (Hopkins statistic 0.931), higher aggregation (SI 0.862), and lower dispersion (Davies-Bouldin index 0.551) than those conducted on patients with reference representation methods. The clustering solutions for patients with the embedding-based representation achieved the highest F1 scores of 0.944 and 0.717 for two clusters.

Conclusions

The feature-level embedding-based representations can reflect the potential clinical associations among medical concepts effectively. The patient-level embedding-based representation is easy to use as continuous input to standard machine learning algorithms and can bring performance improvements. It is expected that the embedding-based representation will be helpful in a wide range of secondary uses of sEMR data.

The Relationship Between Prognostic Nutritional Index and All-Cause Mortality in Critically Ill Patients: A Retrospective Study

Purpose

The effectiveness and prognostic value of the prognostic nutritional index (PNI) in critically ill patients are unknown. Hence, this study aimed to analyze the relationship between the PNI and all-cause mortality in critically ill patients.

Patients and Methods

Patient data were obtained from the Multiparameter Intelligent Monitoring in Intensive Care III database. The relationship between the PNI and in-hospital mortality was analyzed using receiver operating characteristic curve analysis and a logistic regression model. Propensity score matching (PSM) was used to eliminate the bias caused by confounding factors. The Kaplan-Meier curve and Cox regression model were used to test the effect of the PNI on 30-, 90-, 180-, and 365-day mortality.

Results

A low PNI score is an independent risk factor for in-hospital mortality in critically ill patients. A total of 3644 cases were successfully matched using PSM. The PSM group with balanced covariates obtained similar results in the three models, which were statistically significant. The Kaplan-Meier curve and Cox regression model showed that the PNI was negatively correlated with 30-, 90-, 180-, and 365-day all-cause mortality.

Conclusion

The PNI score is an independent risk factor for all-cause mortality in critically ill patients, where a low PNI score is associated with increased mortality.

Predictors of Early and Late Unplanned Intensive Care Unit Readmission: A Retrospective Cohort Study

PURPOSE

Intensive care unit (ICU) readmission is considered one of the major quality indicators of critical care. Reducing ICU readmission can improve patients' outcomes and optimize health resources, but there are limited data on the predictors of unplanned ICU readmission. This study aimed to identify the risk factors associated with unplanned ICU readmission within 48 hr (early) and after 48 hr (late) from ICU discharge.

DESIGN

Retrospective cohort study.

METHODS

Data were collected from patients' electronic medical records in a 24-bed medical ICU at a tertiary academic medical center in Busan, South Korea. Among all the patients admitted to the medical ICU (n = 1,033) between January 2015 and December 2017, 739 eligible patients were analyzed. A multivariable multinomial logistic regression model was conducted to identify predictors of ICU readmission.

FINDINGS

Out of the 739 patients analyzed, 66 (8.9%) were readmitted to the medical ICU: 13 (1.8%) as early readmission and 53 (7.1%) as late readmission. Two significant predictors were identified for early readmission: ICU admission from the ward (odds ratio [OR] = 4.14; 95% confidence interval [CI] 1.25, 13.67) and mechanical ventilation support >14 days (OR = 13.25; 95% CI 1.78, 98.89). For late ICU admission, there were four risk factors: ICU admission from the ward (OR = 2.69; 95% CI 1.44, 5.05), tracheostomy placement (OR = 3.58; 95% CI 1.49, 8.59), mechanical ventilation support >14 days (OR = 4.77; 95% CI 1.67, 13.63), and continuous renal replacement therapy (OR = 4.57; 95% CI 2.42, 8.63).

CONCLUSIONS

To prevent unplanned ICU readmission in patients at high risk, it is necessary to investigate further the role of clinical judgment and communication within the ICU clinical team and institutional-level support regarding ICU readmission events.

CLINICAL RELEVANCE

Both ICU nurses and nurses in post-ICU settings should be aware of the potential risk factors associated with early and late ICU readmission. Predictors and readmission strategies may be different for early and late readmissions. Prospective multicenter studies are needed to examine how these factors influence post-ICU outcomes.

Generalized and transferable patient language representation for phenotyping with limited data

The paradigm of representation learning through transfer learning has the potential to greatly enhance clinical natural language processing. In this work, we propose a multi-task pre-training and fine-tuning approach for learning generalized and transferable patient representations from medical language. The model is first pre-trained with different but related high-prevalence phenotypes and further fine-tuned on downstream target tasks. Our main contribution focuses on the impact this technique can have on low-prevalence phenotypes, a challenging task due to the dearth of data. We validate the representation from pre-training, and fine-tune the multi-task pre-trained models on low-prevalence phenotypes including 38 circulatory diseases, 23 respiratory diseases, and 17 genitourinary diseases. We find multi-task pre-training increases learning efficiency and achieves consistently high performance across the majority of phenotypes. Most important, the multi-task pre-training is almost always either the best-performing model or performs tolerably close to the best-performing model, a property we refer to as robust. All these results lead us to conclude that this multi-task transfer learning architecture is a robust approach for developing generalized and transferable patient language representations for numerous phenotypes.

Deep representation learning of patient data from Electronic Health Records (EHR): A systematic review

OBJECTIVES

Patient representation learning refers to learning a dense mathematical representation of a patient that encodes meaningful information from Electronic Health Records (EHRs). This is generally performed using advanced deep learning methods. This study presents a systematic review of this field and provides both qualitative and quantitative analyses from a methodological perspective.

METHODS

We identified studies developing patient representations from EHRs with deep learning methods from MEDLINE, EMBASE, Scopus, the Association for Computing Machinery (ACM) Digital Library, and the Institute of Electrical and Electronics Engineers (IEEE) Xplore Digital Library. After screening 363 articles, 49 papers were included for a comprehensive data collection.

RESULTS

Publications developing patient representations almost doubled each year from 2015 until 2019. We noticed a typical workflow starting with feeding raw data, applying deep learning models, and ending with clinical outcome predictions as evaluations of the learned representations. Specifically, learning representations from structured EHR data was dominant (37 out of 49 studies). Recurrent Neural Networks were widely applied as the deep learning architecture (Long short-term memory: 13 studies, Gated recurrent unit: 11 studies). Learning was mainly performed in a supervised manner (30 studies) optimized with cross-entropy loss. Disease prediction was the most common application and evaluation (31 studies). Benchmark datasets were mostly unavailable (28 studies) due to privacy concerns of EHR data, and code availability was assured in 20 studies.

DISCUSSION & CONCLUSION

The existing predictive models mainly focus on the prediction of single diseases, rather than considering the complex mechanisms of patients from a holistic review. We show the importance and feasibility of learning comprehensive representations of patient EHR data through a systematic review. Advances in patient representation learning techniques will be essential for powering patient-level EHR analyses. Future work will still be devoted to leveraging the richness and potential of available EHR data. Reproducibility and transparency of reported results will hopefully improve. Knowledge distillation and advanced learning techniques will be exploited to assist the capability of learning patient representation further.

Diagnostic Prediction with Sequence-of-sets Representation Learning for Clinical Events

Electronic health records (EHRs) contain both ordered and unordered chronologies of clinical events that occur during a patient encounter. However, during data preprocessing steps, many predictive models impose a predefined order on unordered clinical events sets (e.g., alphabetical, natural order from the chart, etc.), which is potentially incompatible with the temporal nature of the sequence and predictive task. To address this issue, we propose DPSS, which seeks to capture each patient's clinical event records as sequences of event sets. For each clinical event set, we assume that the predictive model should be invariant to the order of concurrent events and thus employ a novel permutation sampling mechanism. This paper evaluates the use of this permuted sampling method given different data-driven models for predicting a heart failure (HF) diagnosis in subsequent patient visits. Experimental results using the MIMIC-III dataset show that the permutation sampling mechanism offers improved discriminative power based on the area under the receiver operating curve (AUROC) and precision-recall curve (pr-AUC) metrics as HF diagnosis prediction becomes more robust to different data ordering schemes.

Untangling the complexity of multimorbidity with machine learning

The prevalence of multimorbidity has been increasing in recent years, posing a major burden for health care delivery and service. Understanding its determinants and impact is proving to be a challenge yet it offers new opportunities for research to go beyond the study of diseases in isolation. In this paper, we review how the field of machine learning provides many tools for addressing research challenges in multimorbidity. We highlight recent advances in promising methods such as matrix factorisation, deep learning, and topological data analysis and how these can take multimorbidity research beyond cross-sectional, expert-driven or confirmatory approaches to gain a better understanding of evolving patterns of multimorbidity. We discuss the challenges and opportunities of machine learning to identify likely causal links between previously poorly understood disease associations while giving an estimate of the uncertainty on such associations. We finally summarise some of the challenges for wider clinical adoption of machine learning research tools and propose some solutions.

Diagnostic Prediction with Sequence-of-sets Representation Learning for Clinical Events

Electronic health records (EHRs) contain both ordered and unordered chronologies of clinical events that occur during a patient encounter. However, during data preprocessing steps, many predictive models impose a predefined order on unordered clinical events sets (e.g., alphabetical, natural order from the chart, etc.), which is potentially incompatible with the temporal nature of the sequence and predictive task. To address this issue, we propose DPSS, which seeks to capture each patient's clinical event records as sequences of event sets. For each clinical event set, we assume that the predictive model should be invariant to the order of concurrent events and thus employ a novel permutation sampling mechanism. This paper evaluates the use of this permuted sampling method given different data-driven models for predicting a heart failure (HF) diagnosis in subsequent patient visits. Experimental results using the MIMIC-III dataset show that the permutation sampling mechanism offers improved discriminative power based on the area under the receiver operating curve (AUROC) and precision-recall curve (pr-AUC) metrics as HF diagnosis prediction becomes more robust to different data ordering schemes.