Real-Time Automated Sampling of Electronic Medical Records Predicts Hospital Mortality

EHR-QC: A streamlined pipeline for automated electronic health records standardisation and preprocessing to predict clinical outcomes

The adoption of electronic health records (EHRs) has created opportunities to analyse historical data for predicting clinical outcomes and improving patient care. However, non-standardised data representations and anomalies pose major challenges to the use of EHRs in digital health research. To address these challenges, we have developed EHR-QC, a tool comprising two modules: the data standardisation module and the preprocessing module. The data standardisation module migrates source EHR data to a standard format using advanced concept mapping techniques, surpassing expert curation in benchmarking analysis. The preprocessing module includes several functions designed specifically to handle healthcare data subtleties. We provide automated detection of data anomalies and solutions to handle those anomalies. We believe that the development and adoption of tools like EHR-QC is critical for advancing digital health. Our ultimate goal is to accelerate clinical research by enabling rapid experimentation with data-driven observational research to generate robust, generalisable biomedical knowledge.

Improving Unadjusted and Adjusted Mortality With an Early Warning Sepsis System in the Emergency Department and Inpatient Wards

Background

Mortality reduction has been a major focus of improvement for health care systems. Although several studies have noted improved sepsis-related mortality with the use of electronic health record (EHR) systems, there are no known published early warning sepsis systems using the Veterans Health Administration (VHA) EHR system.

Methods

The Malcom Randall Veterans Affairs Medical Center (MRVAMC), a large academic 1a VHA facility within the North Florida/South Georgia Veterans Health System (NF/SGVHS), was identified as having opportunities for improvement related to inpatient mortality outcomes. Sepsis was discovered as the primary contributor to inpatient mortality for MRVAMC's acute level of care (LOC). Education along with implementation of an early warning sepsis system (EWSS) was subsequently integrated in the VHA EHR known as the Veterans Information Systems and Technology Architecture/ Computerized Patient Record System (VistA/CPRS) at NF/SGVHS, which applied a combination of informatics solutions within a Lean Six Sigma quality improvement framework.

Results

At MRVAMC, there was an observed decrease in the number of inpatient deaths for the acute LOC from a high of 48 in fiscal year (FY) 2017, quarter 3 to a low of 27 in FY 2019, quarter 4. This resulted in as large of an improvement as a 44% reduction in unadjusted mortality with education and implementation of an EWSS from FYs 2017 to 2019. Additionally, the MRVAMC acute LOC risk-adjusted mortality (standardized mortality ratio) improved from > 1.0 to < 1.0, demonstrating fewer inpatient mortalities than predicted from FYs 2017 to 2019.

Conclusions

Education along with the possible implementation of an EWSS within the VHA EHR was associated with improvement in unadjusted and adjusted inpatient mortality at MRVAMC. This may be an effective approach for patients with sepsis.

Stratifying individuals into non-alcoholic fatty liver disease risk levels using time series machine learning models

Non-alcoholic fatty liver disease (NAFLD) affects 25% of the population worldwide, and its prevalence is anticipated to increase globally. While most NAFLD patients are asymptomatic, NAFLD may progress to fibrosis, cirrhosis, cardiovascular disease, and diabetes. Research reports, with daunting results, show the challenge that NAFLD's burden causes to global population health. The current process for identifying fibrosis risk levels is inefficient, expensive, does not cover all potential populations, and does not identify the risk in time. Instead of invasive liver biopsies, we implemented a non-invasive fibrosis assessment process calculated from clinical data (accessed via EMRs/EHRs). We stratified patients' risks for fibrosis from 2007 to 2017 by modeling the risk in 5579 individuals. The process involved time-series machine learning models (Hidden Markov Models and Group-Based Trajectory Models) profiled fibrosis risk by modeling patients' latent medical status resulted in three groups. The high-risk group had abnormal lab test values and a higher prevalence of chronic conditions. This study can help overcome the inefficient, traditional process of detecting fibrosis via biopsies (that are also medically unfeasible due to their invasive nature, the medical resources involved, and costs) at early stages. Thus longitudinal risk assessment may be used to make population-specific medical recommendations targeting early detection of high risk patients, to avoid the development of fibrosis disease and its complications as well as decrease healthcare costs.

Emergency Department Visits and Subsequent Hospital Admission Trends for Patients with Chest Pain and a History of Coronary Artery Disease

INTRODUCTION

Hospitalization is the largest component of health care spending in the United States. Most hospitalized patients first visit the emergency department (ED), where hospitalization decisions are made. Optimal utilization of hospital resources is critical for all stakeholders.

METHODS

We performed a population-based, cross-sectional study evaluating ED visits and subsequent inpatient admissions for patients with coronary artery disease (CAD) and chest pain (CP) suggestive of CAD from 2006 to 2013 using the Nationwide Emergency Department Sample database weighted for national estimates. We analyzed trends using a generalized linear regression model with a Poisson distribution and Wald test.

RESULTS

From 2006 to 2013, there was a 15% decrease in ED visits for CAD (p < 0.01), while ED visit rates for CP increased 31% (p < 0.01). Subsequent inpatient admission rates decreased 18% for CAD (p < 0.01) and 33% for CP (p < 0.01). Trends were not modified by patient and hospital strata.

CONCLUSION

ED visits and subsequent inpatient admissions resulting from CAD decreased from 2006 to 2013. Patients with CP had a substantially higher number of ED visits, with a significant decline in inpatient admissions.

A Machine Learning Algorithm to Predict Severe Sepsis and Septic Shock: Development, Implementation, and Impact on Clinical Practice

OBJECTIVES

Develop and implement a machine learning algorithm to predict severe sepsis and septic shock and evaluate the impact on clinical practice and patient outcomes.

DESIGN

Retrospective cohort for algorithm derivation and validation, pre-post impact evaluation.

SETTING

Tertiary teaching hospital system in Philadelphia, PA.

PATIENTS

All non-ICU admissions; algorithm derivation July 2011 to June 2014 (n = 162,212); algorithm validation October to December 2015 (n = 10,448); silent versus alert comparison January 2016 to February 2017 (silent n = 22,280; alert n = 32,184).

INTERVENTIONS

A random-forest classifier, derived and validated using electronic health record data, was deployed both silently and later with an alert to notify clinical teams of sepsis prediction.

MEASUREMENT AND MAIN RESULT

Patients identified for training the algorithm were required to have International Classification of Diseases, 9th Edition codes for severe sepsis or septic shock and a positive blood culture during their hospital encounter with either a lactate greater than 2.2 mmol/L or a systolic blood pressure less than 90 mm Hg. The algorithm demonstrated a sensitivity of 26% and specificity of 98%, with a positive predictive value of 29% and positive likelihood ratio of 13. The alert resulted in a small statistically significant increase in lactate testing and IV fluid administration. There was no significant difference in mortality, discharge disposition, or transfer to ICU, although there was a reduction in time-to-ICU transfer.

CONCLUSIONS

Our machine learning algorithm can predict, with low sensitivity but high specificity, the impending occurrence of severe sepsis and septic shock. Algorithm-generated predictive alerts modestly impacted clinical measures. Next steps include describing clinical perception of this tool and optimizing algorithm design and delivery.

Prospective and External Evaluation of a Machine Learning Model to Predict In-Hospital Mortality of Adults at Time of Admission

IMPORTANCE

The ability to accurately predict in-hospital mortality for patients at the time of admission could improve clinical and operational decision-making and outcomes. Few of the machine learning models that have been developed to predict in-hospital death are both broadly applicable to all adult patients across a health system and readily implementable. Similarly, few have been implemented, and none have been evaluated prospectively and externally validated.

OBJECTIVES

To prospectively and externally validate a machine learning model that predicts in-hospital mortality for all adult patients at the time of hospital admission and to design the model using commonly available electronic health record data and accessible computational methods.

DESIGN, SETTING, AND PARTICIPANTS

In this prognostic study, electronic health record data from a total of 43 180 hospitalizations representing 31 003 unique adult patients admitted to a quaternary academic hospital (hospital A) from October 1, 2014, to December 31, 2015, formed a training and validation cohort. The model was further validated in additional cohorts spanning from March 1, 2018, to August 31, 2018, using 16 122 hospitalizations representing 13 094 unique adult patients admitted to hospital A, 6586 hospitalizations representing 5613 unique adult patients admitted to hospital B, and 4086 hospitalizations representing 3428 unique adult patients admitted to hospital C. The model was integrated into the production electronic health record system and prospectively validated on a cohort of 5273 hospitalizations representing 4525 unique adult patients admitted to hospital A between February 14, 2019, and April 15, 2019.

MAIN OUTCOMES AND MEASURES

The main outcome was in-hospital mortality. Model performance was quantified using the area under the receiver operating characteristic curve and area under the precision recall curve.

RESULTS

A total of 75 247 hospital admissions (median [interquartile range] patient age, 59.5 [29.0] years; 45.9% involving male patients) were included in the study. The in-hospital mortality rates for the training validation; retrospective validations at hospitals A, B, and C; and prospective validation cohorts were 3.0%, 2.7%, 1.8%, 2.1%, and 1.6%, respectively. The area under the receiver operating characteristic curves were 0.87 (95% CI, 0.83-0.89), 0.85 (95% CI, 0.83-0.87), 0.89 (95% CI, 0.86-0.92), 0.84 (95% CI, 0.80-0.89), and 0.86 (95% CI, 0.83-0.90), respectively. The area under the precision recall curves were 0.29 (95% CI, 0.25-0.37), 0.17 (95% CI, 0.13-0.22), 0.22 (95% CI, 0.14-0.31), 0.13 (95% CI, 0.08-0.21), and 0.14 (95% CI, 0.09-0.21), respectively.

CONCLUSIONS AND RELEVANCE

Prospective and multisite retrospective evaluations of a machine learning model demonstrated good discrimination of in-hospital mortality for adult patients at the time of admission. The data elements, methods, and patient selection make the model implementable at a system level.

Sepsis surveillance: an examination of parameter sensitivity and alert reliability

Objective

To examine performance of a sepsis surveillance system in a simulated environment where modifications to parameters and settings for identification of at-risk patients can be explored in-depth.

Materials and Methods

This was a multiple center observational cohort study. The study population comprised 14 917 adults hospitalized in 2016. An expert-driven rules algorithm was applied against 15.1 million data points to simulate a system with binary notification of sepsis events. Three system scenarios were examined: a scenario as derived from the second version of the Consensus Definitions for Sepsis and Septic Shock (SEP-2), the same scenario but without systolic blood pressure (SBP) decrease criteria (near SEP-2), and a conservative scenario with limited parameters. Patients identified by scenarios as being at-risk for sepsis were assessed for suspected infection. Multivariate binary logistic regression models estimated mortality risk among patients with suspected infection.

Results

First, the SEP-2-based scenario had a hyperactive, unreliable parameter SBP decrease >40 mm Hg from baseline. Second, the near SEP-2 scenario demonstrated adequate reliability and sensitivity. Third, the conservative scenario had modestly higher reliability, but sensitivity degraded quickly. Parameters differed in predicting mortality risk and represented a substitution effect between scenarios.

Discussion

Configuration of parameters and alert criteria have implications for patient identification and predicted outcomes.

Conclusion

Performance of scenarios was associated with scenario design. A single hyperactive, unreliable parameter may negatively influence adoption of the system. A trade-off between modest improvements in alert reliability corresponded to a steep decline in condition sensitivity in scenarios explored.

Dynamic readmission prediction using routine postoperative laboratory results after radical cystectomy

OBJECTIVE

To determine if the addition of electronic health record data enables better risk stratification and readmission prediction after radical cystectomy. Despite efforts to reduce their frequency and severity, complications and readmissions following radical cystectomy remain common. Leveraging readily available, dynamic information such as laboratory results may allow for improved prediction and targeted interventions for patients at risk of readmission.

METHODS

We used an institutional electronic medical records database to obtain demographic, clinical, and laboratory data for patients undergoing radical cystectomy. We characterized the trajectory of common postoperative laboratory values during the index hospital stay using support vector machine learning techniques. We compared models with and without laboratory results to assess predictive ability for readmission.

RESULTS

Among 996 patients who underwent radical cystectomy, 259 patients (26%) experienced a readmission within 30 days. During the first week after surgery, median daily values for white blood cell count, urea nitrogen, bicarbonate, and creatinine differentiated readmitted and nonreadmitted patients. Inclusion of laboratory results greatly increased the ability of models to predict 30-day readmissions after cystectomy.

CONCLUSIONS

Common postoperative laboratory values may have discriminatory power to help identify patients at higher risk of readmission after radical cystectomy. Dynamic sources of physiological data such as laboratory values could enable more accurate identification and targeting of patients at greatest readmission risk after cystectomy. This is a proof of concept study that suggests further exploration of these techniques is warranted.

Artificial intelligence with multi-functional machine learning platform development for better healthcare and precision medicine

Precision medicine is one of the recent and powerful developments in medical care, which has the potential to improve the traditional symptom-driven practice of medicine, allowing earlier interventions using advanced diagnostics and tailoring better and economically personalized treatments. Identifying the best pathway to personalized and population medicine involves the ability to analyze comprehensive patient information together with broader aspects to monitor and distinguish between sick and relatively healthy people, which will lead to a better understanding of biological indicators that can signal shifts in health. While the complexities of disease at the individual level have made it difficult to utilize healthcare information in clinical decision-making, some of the existing constraints have been greatly minimized by technological advancements. To implement effective precision medicine with enhanced ability to positively impact patient outcomes and provide real-time decision support, it is important to harness the power of electronic health records by integrating disparate data sources and discovering patient-specific patterns of disease progression. Useful analytic tools, technologies, databases, and approaches are required to augment networking and interoperability of clinical, laboratory and public health systems, as well as addressing ethical and social issues related to the privacy and protection of healthcare data with effective balance. Developing multifunctional machine learning platforms for clinical data extraction, aggregation, management and analysis can support clinicians by efficiently stratifying subjects to understand specific scenarios and optimize decision-making. Implementation of artificial intelligence in healthcare is a compelling vision that has the potential in leading to the significant improvements for achieving the goals of providing real-time, better personalized and population medicine at lower costs. In this study, we focused on analyzing and discussing various published artificial intelligence and machine learning solutions, approaches and perspectives, aiming to advance academic solutions in paving the way for a new data-centric era of discovery in healthcare.

Implementation of an automated early warning scoring system in a surgical ward: Practical use and effects on patient outcomes

Introduction

Early warning scores (EWS) are being increasingly embedded in hospitals over the world due to their promise to reduce adverse events and improve the outcomes of clinical patients.

The aim of this study was to evaluate the clinical use of an automated modified EWS (MEWS) for patients after surgery.

Methods

This study conducted retrospective before-and-after comparative analysis of non-automated and automated MEWS for patients admitted to the surgical high-dependency unit in a tertiary hospital. Operational outcomes included number of recorded assessments of the individual MEWS elements, number of complete MEWS assessments, as well as adherence rate to related protocols. Clinical outcomes included hospital length of stay, in-hospital and 28-day mortality, and ICU readmission rate.

Results

Recordings in the electronic medical record from the control period contained 7929 assessments of MEWS elements and were performed in 320 patients. Recordings from the intervention period contained 8781 assessments of MEWS elements in 273 patients, of which 3418 were performed with the automated EWS system. During the control period, 199 (2.5%) complete MEWS were recorded versus 3991 (45.5%) during intervention period. With the automated MEWS systems, the percentage of missing assessments and the time until the next assessment for patients with a MEWS of ≥2 decreased significantly. The protocol adherence improved from 1.1% during the control period to 25.4% when the automated MEWS system was involved. There were no significant differences in clinical outcomes.

Conclusion

Implementation of an automated EWS system on a surgical high dependency unit improves the number of complete MEWS assessments, registered vital signs, and adherence to the EWS hospital protocol. However, this positive effect did not translate into a significant decrease in mortality, hospital length of stay, or ICU readmissions. Future research and development on automated EWS systems should focus on data management and technology interoperability.

An investigation of sepsis surveillance and emergency treatment on patient mortality outcomes: An observational cohort study

Objective

To determine the prevalence of initiating the sepsis 3-h bundle of care and estimate effects of bundle completion on risk-adjusted mortality among emergency department (ED) patients screened-in by electronic surveillance.

Materials and Methods

This was a multiple center observational cohort study conducted in 2016. The study population was comprised of patients screened-in by St. John Sepsis Surveillance Agent within 4 h of ED arrival, had a sepsis bundle initiated, and admitted to hospital. We built multivariable logistic regression models to estimate impact of a 3-h bundle completed within 3 h of arrival on mortality outcomes.

Results

Approximately 3% ED patients were screened-in by electronic surveillance within 4 h of arrival and admitted to hospital. Nearly 7 in 10 (69%) patients had a bundle initiated, with most bundles completed within 3 h of arrival. The fully-adjusted risk model achieved good discrimination on mortality outcomes [area under the receiver operating characteristic 0.82, 95% confidence interval (CI) 0.79-0.85] and estimated 34% reduced mortality risk among patients with a bundle completed within 3 h of arrival compared to non-completers.

Discussion

The sepsis bundle is an effective intervention for many vulnerable patients, and likely to be completed within 3 h after arrival when electronic surveillance with reliable alert notifications are integrated into clinical workflow. Beginning at triage, the platform and sepsis program enables identification and management of patients with greater precision, and increases the odds of good outcomes.

Conclusion

Sepsis surveillance and clinical decision support accelerate accurate recognition and stratification of patients, and facilitate timely delivery of health care.

Scalable and accurate deep learning with electronic health records

Predictive modeling with electronic health record (EHR) data is anticipated to drive personalized medicine and improve healthcare quality. Constructing predictive statistical models typically requires extraction of curated predictor variables from normalized EHR data, a labor-intensive process that discards the vast majority of information in each patient's record. We propose a representation of patients' entire raw EHR records based on the Fast Healthcare Interoperability Resources (FHIR) format. We demonstrate that deep learning methods using this representation are capable of accurately predicting multiple medical events from multiple centers without site-specific data harmonization. We validated our approach using de-identified EHR data from two US academic medical centers with 216,221 adult patients hospitalized for at least 24 h. In the sequential format we propose, this volume of EHR data unrolled into a total of 46,864,534,945 data points, including clinical notes. Deep learning models achieved high accuracy for tasks such as predicting: in-hospital mortality (area under the receiver operator curve [AUROC] across sites 0.93-0.94), 30-day unplanned readmission (AUROC 0.75-0.76), prolonged length of stay (AUROC 0.85-0.86), and all of a patient's final discharge diagnoses (frequency-weighted AUROC 0.90). These models outperformed traditional, clinically-used predictive models in all cases. We believe that this approach can be used to create accurate and scalable predictions for a variety of clinical scenarios. In a case study of a particular prediction, we demonstrate that neural networks can be used to identify relevant information from the patient's chart.

Using System Inflammatory Response Syndrome as an Easy-to-Implement, Sustainable, and Automated Tool for All-Cause Deterioration Among Medical Inpatients

OBJECTIVES

Clinical deterioration detection among adult inpatients is known to be suboptimal, and many electronic health record tools have been developed to help identify these patients. Many of these tools are focused on sepsis spectrum disorders, but the evolution of the definition of sepsis is moving toward increased specificity, which may make automated detection of clinical deterioration from nonsepsis-related conditions less likely. The objectives of this study were to develop and to examine the use of a low-cost, highly sustainable deterioration detection tool based on systemic inflammatory response syndrome (SIRS) criteria.

METHODS

Using existing resources, a SIRS-based electronic health record monitoring and intervention tool was developed with a focus on ease of implementation and high sustainability. This tool was used to monitor 15,739 adult inpatients in real time during their admission.

RESULTS

The SIRS-based tool, created with focus on ease of implementation and high sustainability, identified patients with higher risk of clinical deterioration. The project was rapidly deployed for a 4-month period at a 900-bed campus of an academic medical center with minimal additional resources required.

CONCLUSIONS

Whereas the definition of sepsis moves away from SIRS, SIRS-based criteria may still have clinical benefit as an easy-to-automate detection tool for all-cause clinical deterioration among medical inpatients.

Comparison of the Between the Flags calling criteria to the MEWS, NEWS and the electronic Cardiac Arrest Risk Triage (eCART) score for the identification of deteriorating ward patients

INTRODUCTION

Traditionally, paper based observation charts have been used to identify deteriorating patients, with emerging recent electronic medical records allowing electronic algorithms to risk stratify and help direct the response to deterioration.

OBJECTIVE(S)

We sought to compare the Between the Flags (BTF) calling criteria to the Modified Early Warning Score (MEWS), National Early Warning Score (NEWS) and electronic Cardiac Arrest Risk Triage (eCART) score.

DESIGN AND PARTICIPANTS

Multicenter retrospective analysis of electronic health record data from all patients admitted to five US hospitals from November 2008-August 2013.

MAIN OUTCOME MEASURES

Cardiac arrest, ICU transfer or death within 24h of a score RESULTS: Overall accuracy was highest for eCART, with an AUC of 0.801 (95% CI 0.799-0.802), followed by NEWS, MEWS and BTF respectively (0.718 [0.716-0.720]; 0.698 [0.696-0.700]; 0.663 [0.661-0.664]). BTF criteria had a high risk (Red Zone) specificity of 95.0% and a moderate risk (Yellow Zone) specificity of 27.5%, which corresponded to MEWS thresholds of >=4 and >=2, NEWS thresholds of >=5 and >=2, and eCART thresholds of >=12 and >=4, respectively. At those thresholds, eCART caught 22 more adverse events per 10,000 patients than BTF using the moderate risk criteria and 13 more using high risk criteria, while MEWS and NEWS identified the same or fewer.

CONCLUSION(S)

An electronically generated eCART score was more accurate than commonly used paper based observation tools for predicting the composite outcome of in-hospital cardiac arrest, ICU transfer and death within 24h of observation. The outcomes of this analysis lend weight for a move towards an algorithm based electronic risk identification tool for deteriorating patients to ensure earlier detection and prevent adverse events in the hospital.

A quality improvement project to improve the Medicare and Medicaid Services (CMS) sepsis bundle compliance rate in a large healthcare system

Sepsis is a leading cause of mortality and morbidity in hospitalised patients. The Centers for Medicare and Medicaid Services (CMS) mandated that US hospitals report sepsis bundle compliance rate as a quality process measure in October 2015. The specific aim of our study was to improve the CMS sepsis bundle compliance rate from 30% to 40% across 20 acute care hospitals in our healthcare system within 1 year. The study included all adult inpatients with sepsis sampled according to CMS specifications from October 2015 to September 2016. The CMS sepsis bundle compliance rate was tracked monthly using statistical process control charting. A baseline rate of 28.5% with 99% control limits was established. We implemented multiple interventions including computerised decision support systems (CDSSs) to increase compliance with the most commonly missing bundle elements. Compliance reached 42% (99% statistical process control limits 18.4%-38.6%) as CDSS was implemented system-wide, but this improvement was not sustained after CMS changed specifications of the outcome measure. Difficulties encountered elucidate shortcomings of our study methodology and of the CMS sepsis bundle compliance rate as a quality process measure.

Quick Sequential [Sepsis-Related] Organ Failure Assessment (qSOFA) and St. John Sepsis Surveillance Agent to Detect Patients at Risk of Sepsis: An Observational Cohort Study

The 2016 Sepsis-3 guidelines included the Quick Sequential [Sepsis-related] Organ Failure Assessment (qSOFA) tool to identify patients at risk of sepsis. The objective was to compare the utility of qSOFA to the St. John Sepsis Surveillance Agent among patients with suspected infection. The primary outcomes were in-hospital mortality or admission to the intensive care unit. A multiple center observational cohort study design was used. The study population comprised 17 044 hospitalized patients between January and March 2016. For the primary analysis, receiver operator characteristic curves were constructed for patient outcomes using qSOFA and the St. John Sepsis Surveillance Agent, and the areas under the curve were compared against a baseline risk model. Time-to-event clinical process modeling also was applied. The St. John Sepsis Surveillance Agent, when compared to qSOFA, activated earlier and was more accurate in predicting patient outcomes; in this regard, qSOFA fell far behind on both objectives.