Scalable and accurate deep learning with electronic health records

Artificial Intelligence in Cardiovascular Medicine

PURPOSE OF REVIEW

The ripples of artificial intelligence are being felt in various sectors of human life. Machine learning, a subset of artificial intelligence, extracts information from large databases of information and is gaining traction in various fields of cardiology. In this review, we highlight noteworthy examples of machine learning utilization in echocardiography, nuclear cardiology, computed tomography, and magnetic resonance imaging over the past year.

RECENT FINDINGS

In the past year, machine learning (ML) has expanded its boundaries in cardiology with several positive results. Some studies have integrated clinical and imaging information to further augment the accuracy of these ML algorithms. All the studies mentioned in this review have clearly demonstrated superior results of ML in relation to conventional approaches for identifying obstructions or predicting major adverse events in reference to conventional approaches. As the influx of data arriving from gradually evolving technologies in health care and wearable devices continues to be more complex, ML may serve as the bridge to transcend the gap between health care and patients in the future. In order to facilitate a seamless transition between both, a few issues must be resolved for a successful implementation of ML in health care.

VetTag: improving automated veterinary diagnosis coding via large-scale language modeling

Unlike human medical records, most of the veterinary records are free text without standard diagnosis coding. The lack of systematic coding is a major barrier to the growing interest in leveraging veterinary records for public health and translational research. Recent machine learning effort is limited to predicting 42 top-level diagnosis categories from veterinary notes. Here we develop a large-scale algorithm to automatically predict all 4577 standard veterinary diagnosis codes from free text. We train our algorithm on a curated dataset of over 100 K expert labeled veterinary notes and over one million unlabeled notes. Our algorithm is based on the adapted Transformer architecture and we demonstrate that large-scale language modeling on the unlabeled notes via pretraining and as an auxiliary objective during supervised learning greatly improves performance. We systematically evaluate the performance of the model and several baselines in challenging settings where algorithms trained on one hospital are evaluated in a different hospital with substantial domain shift. In addition, we show that hierarchical training can address severe data imbalances for fine-grained diagnosis with a few training cases, and we provide interpretation for what is learned by the deep network. Our algorithm addresses an important challenge in veterinary medicine, and our model and experiments add insights into the power of unsupervised learning for clinical natural language processing.

An overview of GeoAI applications in health and healthcare

The moulding together of artificial intelligence (AI) and the geographic/geographic information systems (GIS) dimension creates GeoAI. There is an emerging role for GeoAI in health and healthcare, as location is an integral part of both population and individual health. This article provides an overview of GeoAI technologies (methods, tools and software), and their current and potential applications in several disciplines within public health, precision medicine, and Internet of Things-powered smart healthy cities. The potential challenges currently facing GeoAI research and applications in health and healthcare are also briefly discussed.

Semantic integration of clinical laboratory tests from electronic health records for deep phenotyping and biomarker discovery

Electronic Health Record (EHR) systems typically define laboratory test results using the Laboratory Observation Identifier Names and Codes (LOINC) and can transmit them using Fast Healthcare Interoperability Resource (FHIR) standards. LOINC has not yet been semantically integrated with computational resources for phenotype analysis. Here, we provide a method for mapping LOINC-encoded laboratory test results transmitted in FHIR standards to Human Phenotype Ontology (HPO) terms. We annotated the medical implications of 2923 commonly used laboratory tests with HPO terms. Using these annotations, our software assesses laboratory test results and converts each result into an HPO term. We validated our approach with EHR data from 15,681 patients with respiratory complaints and identified known biomarkers for asthma. Finally, we provide a freely available SMART on FHIR application that can be used within EHR systems. Our approach allows readily available laboratory tests in EHR to be reused for deep phenotyping and exploits the hierarchical structure of HPO to integrate distinct tests that have comparable medical interpretations for association studies.

Exploiting machine learning for end-to-end drug discovery and development

A variety of machine learning methods such as naive Bayesian, support vector machines and more recently deep neural networks are demonstrating their utility for drug discovery and development. These leverage the generally bigger datasets created from high-throughput screening data and allow prediction of bioactivities for targets and molecular properties with increased levels of accuracy. We have only just begun to exploit the potential of these techniques but they may already be fundamentally changing the research process for identifying new molecules and/or repurposing old drugs. The integrated application of such machine learning models for end-to-end (E2E) application is broadly relevant and has considerable implications for developing future therapies and their targeting.

Real-World Evidence, Causal Inference, and Machine Learning

The current focus on real world evidence (RWE) is occurring at a time when at least two major trends are converging. First, is the progress made in observational research design and methods over the past decade. Second, the development of numerous large observational healthcare databases around the world is creating repositories of improved data assets to support observational research. OBJECTIVE: This paper examines the implications of the improvements in observational methods and research design, as well as the growing availability of real world data for the quality of RWE. These developments have been very positive. On the other hand, unstructured data, such as medical notes, and the sparcity of data created by merging multiple data assets are not easily handled by traditional health services research statistical methods. In response, machine learning methods are gaining increased traction as potential tools for analyzing massive, complex datasets. CONCLUSIONS: Machine learning methods have traditionally been used for classification and prediction, rather than causal inference. The prediction capabilities of machine learning are valuable by themselves. However, using machine learning for causal inference is still evolving. Machine learning can be used for hypothesis generation, followed by the application of traditional causal methods. But relatively recent developments, such as targeted maximum likelihood methods, are directly integrating machine learning with causal inference.

An algorithm strategy for precise patient monitoring in a connected healthcare enterprise

This perspective paper describes the building elements for realizing a precise patient monitoring algorithm to fundamentally address the alarm fatigue problem. Alarm fatigue is well recognized but no solution has been widely successful. Physiologic patient monitors are responsible for the lion's share of alarms at the bedside, most of which are either false or non-actionable. Algorithms on patient monitors lack precision because they fail to leverage multivariate relationship among variables monitored, to integrate rich patient clinical information from electronic health record system, and to utilize temporal patterns in data streams. Therefore, a solution to patient monitor alarm fatigue is to open the black-box of patient monitors to integrate physiologic data with clinical data from EHR under a four-element algorithm strategy to be described in this paper. This strategy will be presented in this paper in the context of its current status as described in our prior publications.

Deep learning in bioinformatics: Introduction, application, and perspective in the big data era

Deep learning, which is especially formidable in handling big data, has achieved great success in various fields, including bioinformatics. With the advances of the big data era in biology, it is foreseeable that deep learning will become increasingly important in the field and will be incorporated in vast majorities of analysis pipelines. In this review, we provide both the exoteric introduction of deep learning, and concrete examples and implementations of its representative applications in bioinformatics. We start from the recent achievements of deep learning in the bioinformatics field, pointing out the problems which are suitable to use deep learning. After that, we introduce deep learning in an easy-to-understand fashion, from shallow neural networks to legendary convolutional neural networks, legendary recurrent neural networks, graph neural networks, generative adversarial networks, variational autoencoder, and the most recent state-of-the-art architectures. After that, we provide eight examples, covering five bioinformatics research directions and all the four kinds of data type, with the implementation written in Tensorflow and Keras. Finally, we discuss the common issues, such as overfitting and interpretability, that users will encounter when adopting deep learning methods and provide corresponding suggestions. The implementations are freely available at https://github.com/lykaust15/Deep_learning_examples.

The Revival of the Notes Field: Leveraging the Unstructured Content in Electronic Health Records

Problem: Clinical practice requires the production of a time- and resource-consuming great amount of notes. They contain relevant information, but their secondary use is almost impossible, due to their unstructured nature. Researchers are trying to address this problems, with traditional and promising novel techniques. Application in real hospital settings seems not to be possible yet, though, both because of relatively small and dirty dataset, and for the lack of language-specific pre-trained models. Aim: Our aim is to demonstrate the potential of the above techniques, but also raise awareness of the still open challenges that the scientific communities of IT and medical practitioners must jointly address to realize the full potential of unstructured content that is daily produced and digitized in hospital settings, both to improve its data quality and leverage the insights from data-driven predictive models. Methods: To this extent, we present a narrative literature review of the most recent and relevant contributions to leverage the application of Natural Language Processing techniques to the free-text content electronic patient records. In particular, we focused on four selected application domains, namely: data quality, information extraction, sentiment analysis and predictive models, and automated patient cohort selection. Then, we will present a few empirical studies that we undertook at a major teaching hospital specializing in musculoskeletal diseases. Results: We provide the reader with some simple and affordable pipelines, which demonstrate the feasibility of reaching literature performance levels with a single institution non-English dataset. In such a way, we bridged literature and real world needs, performing a step further toward the revival of notes fields.

Using the Shapes of Clinical Data Trajectories to Predict Mortality in ICUs

Supplemental Digital Content is available in the text.

1) To show how to exploit the information contained in the trajectories of time-varying patient clinical data for dynamic predictions of mortality in the ICU; and 2) to demonstrate the additional predictive value that can be achieved by incorporating this trajectory information.

Developing neural network models for early detection of cardiac arrest in emergency department

BACKGROUND

Automated surveillance for cardiac arrests would be useful in overcrowded emergency departments. The purpose of this study is to develop and test artificial neural network (ANN) classifiers for early detection of patients at risk of cardiac arrest in emergency departments.

METHODS

This is a single-center electronic health record (EHR)-based study. The primary outcome was the development of cardiac arrest within 24 h after prediction. Three ANN models were trained: multilayer perceptron (MLP), long-short-term memory (LSTM), and hybrid. These were compared to other classifiers including the modified early warning score (MEWS), logistic regression, and random forest. We used AUROC, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) for the comparison.

RESULTS

During the study period, there were a total of 374,605 ED visits and 2,910,321 patient status updates. The ANN models (MLP, LSTM, and hybrid) achieved higher AUROC (AUROC: 0.929, 0.933, and 0.936; 95% confidential interval: 0.926-0.932, 0.930-0.936, and 0.933-0.939, respectively) compared to the non-ANN models, and the hybrid model exhibited the best performance. The ANN classifiers displayed higher performance in most of the test characteristics when the threshold levels of the classifiers were fixed to display the same positive result as those at the three MEWS thresholds (score ≥ 3, ≥4, and ≥5), and when compared with each other.

CONCLUSIONS

The ANN improves upon MEWS and conventional machine learning algorithms for the prediction of cardiac arrests in emergency departments. The hybrid ANN model utilizing both baseline and sequence information achieved the best performance.

Big data analytics for personalized medicine

Big Data are radically changing biomedical research. The unprecedented advances in automated collection of large-scale molecular and clinical data pose major challenges to data analysis and interpretation, calling for the development of new computational approaches. The creation of powerful systems for the effective use of biomedical Big Data in Personalized Medicine (a.k.a. Precision Medicine) will require significant scientific and technical developments, including infrastructure, engineering, project and financial management. We review here how the evolution of data-driven methods offers the possibility to address many of these problems, guiding the formulation of hypotheses on systems functioning and the generation of mechanistic models, and facilitating the design of clinical procedures in Personalized Medicine.

Automated early detection of obstetric complications: theoretic and methodologic considerations

Compared with adults who are admitted to general medical-surgical wards, women who are admitted to labor and delivery services are at much lower risk of experiencing unexpected critical illness. Nonetheless, critical illness and other complications that put either the mother or fetus at risk do occur. One potential approach to prevention is to use automated early warning systems, such as those used for nonpregnant adults. Predictive models that use data extracted in real time from electronic records constitute the cornerstone of such systems. This article addresses several issues that are involved in the development of such predictive models: specification of temporal characteristics, choice of denominator, selection of outcomes for model calibration, potential uses of existing adult severity of illness scores, approaches to data processing, statistical considerations, validation, and options for instantiation. These have not been addressed explicitly in the obstetrics literature, which has focused on the use of manually assigned scores. In addition, this article provides some results from work in progress to develop 2 obstetric predictive models with the use of data from 262,071 women who were admitted to a labor and delivery service at 15 Kaiser Permanente Northern California hospitals between 2010 and 2017.

Predicting need for advanced illness or palliative care in a primary care population using electronic health record data

Timely outreach to individuals in an advanced stage of illness offers opportunities to exercise decision control over health care. Predictive models built using Electronic health record (EHR) data are being explored as a way to anticipate such need with enough lead time for patient engagement. Prior studies have focused on hospitalized patients, who typically have more data available for predicting care needs. It is unclear if prediction driven outreach is feasible in the primary care setting. In this study, we apply predictive modeling to the primary care population of a large, regional health system and systematically examine the impact of technical choices, such as requiring a minimum number of health care encounters (data density requirements) and aggregating diagnosis codes using Clinical Classifications Software (CCS) groupings to reduce dimensionality, on model performance in terms of discrimination and positive predictive value. We assembled a cohort of 349,667 primary care patients between 65 and 90 years of age who sought care from Sutter Health between July 1, 2011 and June 30, 2014, of whom 2.1% died during the study period. EHR data comprising demographics, encounters, orders, and diagnoses for each patient from a 12 month observation window prior to the point when a prediction is made were extracted. L1 regularized logistic regression and gradient boosted tree models were fit to training data and tuned by cross validation. Model performance in predicting one year mortality was assessed using held-out test patients. Our experiments systematically varied three factors: model type, diagnosis coding, and data density requirements. We found substantial, consistent benefit from using gradient boosting vs logistic regression (mean AUROC over all other technical choices of 84.8% vs 80.7% respectively). There was no benefit from aggregation of ICD codes into CCS code groups (mean AUROC over all other technical choices of 82.9% vs 82.6% respectively). Likewise increasing data density requirements did not affect discrimination (mean AUROC over other technical choices ranged from 82.5% to 83%). We also examine model performance as a function of lead time, which is the interval between death and when a prediction was made. In subgroup analysis by lead time, mean AUROC over all other choices ranged from 87.9% for patients who died within 0 to 3 months to 83.6% for those who died 9 to 12 months after prediction time.

ROMOP: a light-weight R package for interfacing with OMOP-formatted electronic health record data

OBJECTIVES

Electronic health record (EHR) data are increasingly used for biomedical discoveries. The nature of the data, however, requires expertise in both data science and EHR structure. The Observational Medical Out-comes Partnership (OMOP) common data model (CDM) standardizes the language and structure of EHR data to promote interoperability of EHR data for research. While the OMOP CDM is valuable and more attuned to research purposes, it still requires extensive domain knowledge to utilize effectively, potentially limiting more widespread adoption of EHR data for research and quality improvement.

MATERIALS AND METHODS

We have created ROMOP: an R package for direct interfacing with EHR data in the OMOP CDM format.

RESULTS

ROMOP streamlines typical EHR-related data processes. Its functions include exploration of data types, extraction and summarization of patient clinical and demographic data, and patient searches using any CDM vocabulary concept.

CONCLUSION

ROMOP is freely available under the Massachusetts Institute of Technology (MIT) license and can be obtained from GitHub (http://github.com/BenGlicksberg/ROMOP). We detail instructions for setup and use in the Supplementary Materials. Additionally, we provide a public sandbox server containing synthesized clinical data for users to explore OMOP data and ROMOP (http://romop.ucsf.edu).

Machine learning for patient risk stratification for acute respiratory distress syndrome

BACKGROUND

Existing prediction models for acute respiratory distress syndrome (ARDS) require manual chart abstraction and have only fair performance-limiting their suitability for driving clinical interventions. We sought to develop a machine learning approach for the prediction of ARDS that (a) leverages electronic health record (EHR) data, (b) is fully automated, and (c) can be applied at clinically relevant time points throughout a patient's stay.

METHODS AND FINDINGS

We trained a risk stratification model for ARDS using a cohort of 1,621 patients with moderate hypoxia from a single center in 2016, of which 51 patients developed ARDS. We tested the model in a temporally distinct cohort of 1,122 patients from 2017, of which 27 patients developed ARDS. Gold standard diagnosis of ARDS was made by intensive care trained physicians during retrospective chart review. We considered both linear and non-linear approaches to learning the model. The best model used L2-logistic regression with 984 features extracted from the EHR. For patients observed in the hospital at least six hours who then developed moderate hypoxia, the model achieved an area under the receiver operating characteristics curve (AUROC) of 0.81 (95% CI: 0.73-0.88). Selecting a threshold based on the 85th percentile of risk, the model had a sensitivity of 56% (95% CI: 35%, 74%), specificity of 86% (95% CI: 85%, 87%) and positive predictive value of 9% (95% CI: 5%, 14%), identifying a population at four times higher risk for ARDS than other patients with moderate hypoxia and 17 times the risk of hospitalized adults.

CONCLUSIONS

We developed an ARDS prediction model based on EHR data with good discriminative performance. Our results demonstrate the feasibility of a machine learning approach to risk stratifying patients for ARDS solely from data extracted automatically from the EHR.

A roadmap for semi-automatically extracting predictive and clinically meaningful temporal features from medical data for predictive modeling

Predictive modeling based on machine learning with medical data has great potential to improve healthcare and reduce costs. However, two hurdles, among others, impede its widespread adoption in hdealthcare. First, medical data are by nature longitudinal. Pre-processing them, particularly for feature engineering, is labor intensive and often takes 50-80% of the model building effort. Predictive temporal features are the basis of building accurate models, but are difficult to identify. This is problematic. Healthcare systems have limited resources for model building, while inaccurate models produce sub-optimal outcomes and are often useless. Second, most machine learning models provide no explanation of their prediction results. However, offering such explanations is essential for a model to be used in usual clinical practice. To address these two hurdles, this paper outlines: 1) a data-driven method for semi-automatically extracting predictive and clinically meaningful temporal features from medical data for predictive modeling; and 2) a method of using these features to automatically explain machine learning prediction results and suggest tailored interventions. This provides a roadmap for future research.

Artificial Intelligence in Cardiovascular Imaging: JACC State-of-the-Art Review

Data science is likely to lead to major changes in cardiovascular imaging. Problems with timing, efficiency, and missed diagnoses occur at all stages of the imaging chain. The application of artificial intelligence (AI) is dependent on robust data; the application of appropriate computational approaches and tools; and validation of its clinical application to image segmentation, automated measurements, and eventually, automated diagnosis. AI may reduce cost and improve value at the stages of image acquisition, interpretation, and decision-making. Moreover, the precision now possible with cardiovascular imaging, combined with "big data" from the electronic health record and pathology, is likely to better characterize disease and personalize therapy. This review summarizes recent promising applications of AI in cardiology and cardiac imaging, which potentially add value to patient care.

A Human(e) Factor in Clinical Decision Support Systems

The overwhelming amount, production speed, multidimensionality, and potential value of data currently available—often simplified and referred to as big data —exceed the limits of understanding of the human brain. At the same time, developments in data analytics and computational power provide the opportunity to obtain new insights and transfer data-provided added value to clinical practice in real time. What is the role of the health care professional in collaboration with the data scientist in the changing landscape of modern care? We discuss how health care professionals should provide expert knowledge in each of the stages of clinical decision support design: data level, algorithm level, and decision support level. Including various ethical considerations, we advocate for health care professionals to responsibly initiate and guide interprofessional teams, including patients, and embrace novel analytic technologies to translate big data into patient benefit driven by human(e) values.

[German Society for General and Visceral Surgery (DGAV) risk calculator of interventions for colorectal cancer : Presentation of a surgical algorithm on patient participation and quality assurance]

Algorithms are increasingly being developed on the basis of large data sets, also in the field of health, whether for predicting treatment outcomes or life-expectancy. In surgery it is also becoming increasingly more important to analyze complications at an early stage and to subsequently reduce them. The aim is to improve the quality of treatment and quality of life and thus to improve patient well-being. The German Society for General and Visceral Surgery (DGAV) has developed 12 StuDoQ registers in which pseudonymized data from a total of 150,000 patients are recorded. Risk models were developed and validated at the Institute for Medical Information Processing, Biometry and Epidemiology (IBE) of the Ludwig Maximilian University in Munich using the collected data from the StuDoQ|colon cancer and StuDoQ|rectal cancer registers. Based on the collected patient data, the risk calculator determines the statistical probability of the individual complication profile of the patient who is to undergo surgery. The aim is to support surgeons and patients in the decision making process for the individual procedure. The surgeon with his individual experience ultimately remains responsible for the patient.

Development of an intelligent decision support system for ischemic stroke risk assessment in a population-based electronic health record database

BACKGROUND

Intelligent decision support systems (IDSS) have been applied to tasks of disease management. Deep neural networks (DNNs) are artificial intelligent techniques to achieve high modeling power. The application of DNNs to large-scale data for estimating stroke risk needs to be assessed and validated. This study aims to apply a DNN for deriving a stroke predictive model using a big electronic health record database.

METHODS AND RESULTS

The Taiwan National Health Insurance Research Database was used to conduct a retrospective population-based study. The database was divided into one development dataset for model training (~70% of total patients for training and ~10% for parameter tuning) and two testing datasets (each ~10%). A total of 11,192,916 claim records from 840,487 patients were used. The primary outcome was defined as any ischemic stroke in inpatient records within 3 years after study enrollment. The DNN was evaluated using the area under the receiver operating characteristic curve (AUC or c-statistic). The development dataset included 672,214 patients (a total of 8,952,000 records) of whom 2,060 patients had stroke events. The mean age of the population was 35.5±20.2 years, with 48.5% men. The model achieved AUC values of 0.920 (95% confidence interval [CI], 0.908-0.932) in testing dataset 1 and 0.925 (95% CI, 0.914-0.937) in testing dataset 2. Under a high sensitivity operating point, the sensitivity and specificity were 92.5% and 79.8% for testing dataset 1; 91.8% and 79.9% for testing dataset 2. Under a high specificity operating point, the sensitivity and specificity were 80.3% and 87.5% for testing dataset 1; 83.7% and 87.5% for testing dataset 2. The DNN model maintained high predictability 5 years after being developed. The model achieved similar performance to other clinical risk assessment scores.

CONCLUSIONS

Using a DNN algorithm on this large electronic health record database is capable of obtaining a high performing model for assessment of ischemic stroke risk. Further research is needed to determine whether such a DNN-based IDSS could lead to an improvement in clinical practice.

Exploring the use of machine learning for risk adjustment: A comparison of standard and penalized linear regression models in predicting health care costs in older adults

BACKGROUND

Payers and providers still primarily use ordinary least squares (OLS) to estimate expected economic and clinical outcomes for risk adjustment purposes. Penalized linear regression represents a practical and incremental step forward that provides transparency and interpretability within the familiar regression framework. This study conducted an in-depth comparison of prediction performance of standard and penalized linear regression in predicting future health care costs in older adults.

METHODS AND FINDINGS

This retrospective cohort study included 81,106 Medicare Advantage patients with 5 years of continuous medical and pharmacy insurance from 2009 to 2013. Total health care costs in 2013 were predicted with comorbidity indicators from 2009 to 2012. Using 2012 predictors only, OLS performed poorly (e.g., R2 = 16.3%) compared to penalized linear regression models (R2 ranging from 16.8 to 16.9%); using 2009-2012 predictors, the gap in prediction performance increased (R2:15.0% versus 18.0-18.2%). OLS with a reduced set of predictors selected by lasso showed improved performance (R2 = 16.6% with 2012 predictors, 17.4% with 2009-2012 predictors) relative to OLS without variable selection but still lagged behind the prediction performance of penalized regression. Lasso regression consistently generated prediction ratios closer to 1 across different levels of predicted risk compared to other models.

CONCLUSIONS

This study demonstrated the advantages of using transparent and easy-to-interpret penalized linear regression for predicting future health care costs in older adults relative to standard linear regression. Penalized regression showed better performance than OLS in predicting health care costs. Applying penalized regression to longitudinal data increased prediction accuracy. Lasso regression in particular showed superior prediction ratios across low and high levels of predicted risk. Health care insurers, providers and policy makers may benefit from adopting penalized regression such as lasso regression for cost prediction to improve risk adjustment and population health management and thus better address the underlying needs and risk of the populations they serve.

Assessment of a Deep Learning Model Based on Electronic Health Record Data to Forecast Clinical Outcomes in Patients With Rheumatoid Arthritis

Importance

Knowing the future condition of a patient would enable a physician to customize current therapeutic options to prevent disease worsening, but predicting that future condition requires sophisticated modeling and information. If artificial intelligence models were capable of forecasting future patient outcomes, they could be used to aid practitioners and patients in prognosticating outcomes or simulating potential outcomes under different treatment scenarios.

Objective

To assess the ability of an artificial intelligence system to prognosticate the state of disease activity of patients with rheumatoid arthritis (RA) at their next clinical visit.

Design, Setting, and Participants

This prognostic study included 820 patients with RA from rheumatology clinics at 2 distinct health care systems with different electronic health record platforms: a university hospital (UH) and a public safety-net hospital (SNH). The UH and SNH had substantially different patient populations and treatment patterns. The UH has records on approximately 1 million total patients starting in January 2012. The UH data for this study were accessed on July 1, 2017. The SNH has records on 65 000 unique individuals starting in January 2013. The SNH data for the study were collected on February 27, 2018.

Exposures

Structured data were extracted from the electronic health record, including exposures (medications), patient demographics, laboratories, and prior measures of disease activity. A longitudinal deep learning model was used to predict disease activity for patients with RA at their next rheumatology clinic visit and to evaluate interhospital performance and model interoperability strategies.

Main Outcomes and Measures

Model performance was quantified using the area under the receiver operating characteristic curve (AUROC). Disease activity in RA was measured using a composite index score.

Results

A total of 578 UH patients (mean [SD] age, 57 [15] years; 477 [82.5%] female; 296 [51.2%] white) and 242 SNH patients (mean [SD] age, 60 [15] years; 195 [80.6%] female; 30 [12.4%] white) were included in the study. Patients at the UH compared with those at the SNH were seen more frequently (median time between visits, 100 vs 180 days) and were more frequently prescribed higher-class medications (biologics) (364 [63.0%] vs 70 [28.9%]). At the UH, the model reached an AUROC of 0.91 (95% CI, 0.86-0.96) in a test cohort of 116 patients. The UH-trained model had an AUROC of 0.74 (95% CI, 0.65-0.83) in the SNH test cohort (n = 117) despite marked differences in the patient populations. In both settings, baseline prediction using each patients' most recent disease activity score had statistically random performance.

Conclusions and Relevance

The findings suggest that building accurate models to forecast complex disease outcomes using electronic health record data is possible and these models can be shared across hospitals with diverse patient populations.

Models and Machines: How Deep Learning Will Take Clinical Pharmacology to the Next Level

Recent advances in machine learning (ML) have led to enthusiasm about its use throughout the biopharmaceutical industry. The ML methods can be applied to a wide range of problems and have the potential to revolutionize aspects of drug development. The incorporation of ML in modeling and simulation (M&S) has been eagerly anticipated, and in this perspective, we highlight examples in which ML and M&S approaches can be integrated as complementary parts of a clinical pharmacology workflow.

Assessment of Machine Learning vs Standard Prediction Rules for Predicting Hospital Readmissions

Importance

Hospital readmissions are associated with patient harm and expense. Ways to prevent hospital readmissions have focused on identifying patients at greatest risk using prediction scores.

Objective

To identify the type of score that best predicts hospital readmissions.

Design, Setting, and Participants

This prognostic study included 14 062 consecutive adult hospital patients with 16 649 discharges from a tertiary care center, suburban community hospital, and urban critical access hospital in Maryland from September 1, 2016, through December 31, 2016. Patients not included as eligible discharges by the Centers for Medicare & Medicaid Services or the Chesapeake Regional Information System for Our Patients were excluded. A machine learning rank score, the Baltimore score (B score) developed using a machine learning technique, for each individual hospital using data from the 2 years before September 1, 2016, was compared with standard readmission risk assessment scores to predict 30-day unplanned readmissions.

Main Outcomes and Measures

The 30-day readmission rate evaluated using various readmission scores: B score, HOSPITAL score, modified LACE score, and Maxim/RightCare score.

Results

Of the 10 732 patients (5605 [52.2%] male; mean [SD] age, 54.56 [22.42] years) deemed to be eligible for the study, 1422 were readmitted. The area under the receiver operating characteristic curve (AUROC) for individual rules was 0.63 (95% CI, 0.61-0.65) for the HOSPITAL score, which was significantly lower than the 0.66 for modified LACE score (95% CI, 0.64-0.68; P < .001). The B score machine learning score was significantly better than all other scores; 48 hours after admission, the AUROC of the B score was 0.72 (95% CI, 0.70-0.73), which increased to 0.78 (95% CI, 0.77-0.79) at discharge (all P < .001). At the hospital using Maxim/RightCare score, the AUROC was 0.63 (95% CI, 0.59-0.69) for HOSPITAL, 0.64 (95% CI, 0.61-0.68) for Maxim/RightCare, and 0.66 (95% CI, 0.62-0.69) for modified LACE score. The B score was 0.72 (95% CI, 0.69-0.75) 48 hours after admission and 0.81 (95% CI, 0.79-0.84) at discharge. In directly comparing the B score with the sensitivity at cutoff values for modified LACE, HOSPITAL, and Maxim/RightCare scores, the B score was able to identify the same number of readmitted patients while flagging 25.5% to 54.9% fewer patients.

Conclusions and Relevance

Among 3 hospitals in different settings, an automated machine learning score better predicted readmissions than commonly used readmission scores. More efficiently targeting patients at higher risk of readmission may be the first step toward potentially preventing readmissions.

Predicting life expectancy with a long short-term memory recurrent neural network using electronic medical records

BACKGROUND

Life expectancy is one of the most important factors in end-of-life decision making. Good prognostication for example helps to determine the course of treatment and helps to anticipate the procurement of health care services and facilities, or more broadly: facilitates Advance Care Planning. Advance Care Planning improves the quality of the final phase of life by stimulating doctors to explore the preferences for end-of-life care with their patients, and people close to the patients. Physicians, however, tend to overestimate life expectancy, and miss the window of opportunity to initiate Advance Care Planning. This research tests the potential of using machine learning and natural language processing techniques for predicting life expectancy from electronic medical records.

METHODS

We approached the task of predicting life expectancy as a supervised machine learning task. We trained and tested a long short-term memory recurrent neural network on the medical records of deceased patients. We developed the model with a ten-fold cross-validation procedure, and evaluated its performance on a held-out set of test data. We compared the performance of a model which does not use text features (baseline model) to the performance of a model which uses features extracted from the free texts of the medical records (keyword model), and to doctors' performance on a similar task as described in scientific literature.

RESULTS

Both doctors and the baseline model were correct in 20% of the cases, taking a margin of 33% around the actual life expectancy as the target. The keyword model, in comparison, attained an accuracy of 29% with its prognoses. While doctors overestimated life expectancy in 63% of the incorrect prognoses, which harms anticipation to appropriate end-of-life care, the keyword model overestimated life expectancy in only 31% of the incorrect prognoses.

CONCLUSIONS

Prognostication of life expectancy is difficult for humans. Our research shows that machine learning and natural language processing techniques offer a feasible and promising approach to predicting life expectancy. The research has potential for real-life applications, such as supporting timely recognition of the right moment to start Advance Care Planning.

Exploration, Inference, and Prediction in Neuroscience and Biomedicine

Recent decades have seen dramatic progress in brain research. These advances were often buttressed by probing single variables to make circumscribed discoveries, typically through null hypothesis significance testing. New ways for generating massive data fueled tension between the traditional methodology that is used to infer statistically relevant effects in carefully chosen variables, and pattern-learning algorithms that are used to identify predictive signatures by searching through abundant information. In this article we detail the antagonistic philosophies behind two quantitative approaches: certifying robust effects in understandable variables, and evaluating how accurately a built model can forecast future outcomes. We discourage choosing analytical tools via categories such as 'statistics' or 'machine learning'. Instead, to establish reproducible knowledge about the brain, we advocate prioritizing tools in view of the core motivation of each quantitative analysis: aiming towards mechanistic insight or optimizing predictive accuracy.

An attention based deep learning model of clinical events in the intensive care unit

This study trained long short-term memory (LSTM) recurrent neural networks (RNNs) incorporating an attention mechanism to predict daily sepsis, myocardial infarction (MI), and vancomycin antibiotic administration over two week patient ICU courses in the MIMIC-III dataset. These models achieved next-day predictive AUC of 0.876 for sepsis, 0.823 for MI, and 0.833 for vancomycin administration. Attention maps built from these models highlighted those times when input variables most influenced predictions and could provide a degree of interpretability to clinicians. These models appeared to attend to variables that were proxies for clinician decision-making, demonstrating a challenge of using flexible deep learning approaches trained with EHR data to build clinical decision support. While continued development and refinement is needed, we believe that such models could one day prove useful in reducing information overload for ICU physicians by providing needed clinical decision support for a variety of clinically important tasks.

An argument for reporting data standardization procedures in multi-site predictive modeling: case study on the impact of LOINC standardization on model performance

Objectives

We aimed to gain a better understanding of how standardization of laboratory data can impact predictive model performance in multi-site datasets. We hypothesized that standardizing local laboratory codes to logical observation identifiers names and codes (LOINC) would produce predictive models that significantly outperform those learned utilizing local laboratory codes.

Materials and Methods

We predicted 30-day hospital readmission for a set of heart failure-specific visits to 13 hospitals from 2008 to 2012. Laboratory test results were extracted and then manually cleaned and mapped to LOINC. We extracted features to summarize laboratory data for each patient and used a training dataset (2008–2011) to learn models using a variety of feature selection techniques and classifiers. We evaluated our hypothesis by comparing model performance on an independent test dataset (2012).

Results

Models that utilized LOINC performed significantly better than models that utilized local laboratory test codes, regardless of the feature selection technique and classifier approach used.

Discussion and Conclusion

We quantitatively demonstrated the positive impact of standardizing multi-site laboratory data to LOINC prior to use in predictive models. We used our findings to argue for the need for detailed reporting of data standardization procedures in predictive modeling, especially in studies leveraging multi-site datasets extracted from electronic health records.

Intelligent diagnosis with Chinese electronic medical records based on convolutional neural networks

BACKGROUND

Benefiting from big data, powerful computation and new algorithmic techniques, we have been witnessing the renaissance of deep learning, particularly the combination of natural language processing (NLP) and deep neural networks. The advent of electronic medical records (EMRs) has not only changed the format of medical records but also helped users to obtain information faster. However, there are many challenges regarding researching directly using Chinese EMRs, such as low quality, huge quantity, imbalance, semi-structure and non-structure, particularly the high density of the Chinese language compared with English. Therefore, effective word segmentation, word representation and model architecture are the core technologies in the literature on Chinese EMRs.

RESULTS

In this paper, we propose a deep learning framework to study intelligent diagnosis using Chinese EMR data, which incorporates a convolutional neural network (CNN) into an EMR classification application. The novelty of this paper is reflected in the following: (1) We construct a pediatric medical dictionary based on Chinese EMRs. (2) Word2vec adopted in word embedding is used to achieve the semantic description of the content of Chinese EMRs. (3) A fine-tuning CNN model is constructed to feed the pediatric diagnosis with Chinese EMR data. Our results on real-world pediatric Chinese EMRs demonstrate that the average accuracy and F1-score of the CNN models are up to 81%, which indicates the effectiveness of the CNN model for the classification of EMRs. Particularly, a fine-tuning one-layer CNN performs best among all CNNs, recurrent neural network (RNN) (long short-term memory, gated recurrent unit) and CNN-RNN models, and the average accuracy and F1-score are both up to 83%.

CONCLUSION

The CNN framework that includes word segmentation, word embedding and model training can serve as an intelligent auxiliary diagnosis tool for pediatricians. Particularly, a fine-tuning one-layer CNN performs well, which indicates that word order does not appear to have a useful effect on our Chinese EMRs.

Predicting opioid dependence from electronic health records with machine learning

BACKGROUND

The opioid epidemic in the United States is averaging over 100 deaths per day due to overdose. The effectiveness of opioids as pain treatments, and the drug-seeking behavior of opioid addicts, leads physicians in the United States to issue over 200 million opioid prescriptions every year. To better understand the biomedical profile of opioid-dependent patients, we analyzed information from electronic health records (EHR) including lab tests, vital signs, medical procedures, prescriptions, and other data from millions of patients to predict opioid substance dependence.

RESULTS

We trained a machine learning model to classify patients by likelihood of having a diagnosis of substance dependence using EHR data from patients diagnosed with substance dependence, along with control patients with no history of substance-related conditions, matched by age, gender, and status of HIV, hepatitis C, and sickle cell disease. The top machine learning classifier using all features achieved a mean area under the receiver operating characteristic (AUROC) curve of ~ 92%, and analysis of the model uncovered associations between basic clinical factors and substance dependence. Additionally, diagnoses, prescriptions, and procedures prior to the diagnoses of substance dependence were analyzed to elucidate the clinical profile of substance-dependent patients, relative to controls.

CONCLUSIONS

The predictive model may hold utility for identifying patients at risk of developing dependence, risk of overdose, and opioid-seeking patients that report other symptoms in their visits to the emergency room.

A scalable discrete-time survival model for neural networks

There is currently great interest in applying neural networks to prediction tasks in medicine. It is important for predictive models to be able to use survival data, where each patient has a known follow-up time and event/censoring indicator. This avoids information loss when training the model and enables generation of predicted survival curves. In this paper, we describe a discrete-time survival model that is designed to be used with neural networks, which we refer to as Nnet-survival. The model is trained with the maximum likelihood method using mini-batch stochastic gradient descent (SGD). The use of SGD enables rapid convergence and application to large datasets that do not fit in memory. The model is flexible, so that the baseline hazard rate and the effect of the input data on hazard probability can vary with follow-up time. It has been implemented in the Keras deep learning framework, and source code for the model and several examples is available online. We demonstrate the performance of the model on both simulated and real data and compare it to existing models Cox-nnet and Deepsurv.

Machine learning as a supportive tool to recognize cardiac arrest in emergency calls

BACKGROUND

Emergency medical dispatchers fail to identify approximately 25% of cases of out of hospital cardiac arrest, thus lose the opportunity to provide the caller instructions in cardiopulmonary resuscitation. We examined whether a machine learning framework could recognize out-of-hospital cardiac arrest from audio files of calls to the emergency medical dispatch center.

METHODS

For all incidents responded to by Emergency Medical Dispatch Center Copenhagen in 2014, the associated call was retrieved. A machine learning framework was trained to recognize cardiac arrest from the recorded calls. Sensitivity, specificity, and positive predictive value for recognizing out-of-hospital cardiac arrest were calculated. The performance of the machine learning framework was compared to the actual recognition and time-to-recognition of cardiac arrest by medical dispatchers.

RESULTS

We examined 108,607 emergency calls, of which 918 (0.8%) were out-of-hospital cardiac arrest calls eligible for analysis. Compared with medical dispatchers, the machine learning framework had a significantly higher sensitivity (72.5% vs. 84.1%, p < 0.001) with lower specificity (98.8% vs. 97.3%, p < 0.001). The machine learning framework had a lower positive predictive value than dispatchers (20.9% vs. 33.0%, p < 0.001). Time-to-recognition was significantly shorter for the machine learning framework compared to the dispatchers (median 44 seconds vs. 54 s, p < 0.001).

CONCLUSIONS

A machine learning framework performed better than emergency medical dispatchers for identifying out-of-hospital cardiac arrest in emergency phone calls. Machine learning may play an important role as a decision support tool for emergency medical dispatchers.

A guide to deep learning in healthcare

Here we present deep-learning techniques for healthcare, centering our discussion on deep learning in computer vision, natural language processing, reinforcement learning, and generalized methods. We describe how these computational techniques can impact a few key areas of medicine and explore how to build end-to-end systems. Our discussion of computer vision focuses largely on medical imaging, and we describe the application of natural language processing to domains such as electronic health record data. Similarly, reinforcement learning is discussed in the context of robotic-assisted surgery, and generalized deep-learning methods for genomics are reviewed.

Predicting peptide presentation by major histocompatibility complex class I: an improved machine learning approach to the immunopeptidome

BACKGROUND

To further our understanding of immunopeptidomics, improved tools are needed to identify peptides presented by major histocompatibility complex class I (MHC-I). Many existing tools are limited by their reliance upon chemical affinity data, which is less biologically relevant than sampling by mass spectrometry, and other tools are limited by incomplete exploration of machine learning approaches. Herein, we assemble publicly available data describing human peptides discovered by sampling the MHC-I immunopeptidome with mass spectrometry and use this database to train random forest classifiers (ForestMHC) to predict presentation by MHC-I.

RESULTS

As measured by precision in the top 1% of predictions, our method outperforms NetMHC and NetMHCpan on test sets, and it outperforms both these methods and MixMHCpred on new data from an ovarian carcinoma cell line. We also find that random forest scores correlate monotonically, but not linearly, with known chemical binding affinities, and an information-based analysis of classifier features shows the importance of anchor positions for our classification. The random-forest approach also outperforms a deep neural network and a convolutional neural network trained on identical data. Finally, we use our large database to confirm that gene expression partially determines peptide presentation.

CONCLUSIONS

ForestMHC is a promising method to identify peptides bound by MHC-I. We have demonstrated the utility of random forest-based approaches in predicting peptide presentation by MHC-I, assembled the largest known database of MS binding data, and mined this database to show the effect of gene expression on peptide presentation. ForestMHC has potential applicability to basic immunology, rational vaccine design, and neoantigen binding prediction for cancer immunotherapy. This method is publicly available for applications and further validation.

The Effectiveness of Multitask Learning for Phenotyping with Electronic Health Records Data

Electronic phenotyping is the task of ascertaining whether an individual has a medical condition of interest by analyzing their medical record and is foundational in clinical informatics. Increasingly, electronic phenotyping is performed via supervised learning. We investigate the effectiveness of multitask learning for phenotyping using electronic health records (EHR) data. Multitask learning aims to improve model performance on a target task by jointly learning additional auxiliary tasks and has been used in disparate areas of machine learning. However, its utility when applied to EHR data has not been established, and prior work suggests that its benefits are inconsistent. We present experiments that elucidate when multitask learning with neural nets improves performance for phenotyping using EHR data relative to neural nets trained for a single phenotype and to well-tuned baselines. We find that multitask neural nets consistently outperform single-task neural nets for rare phenotypes but underperform for relatively more common phenotypes. The effect size increases as more auxiliary tasks are added. Moreover, multitask learning reduces the sensitivity of neural nets to hyperparameter settings for rare phenotypes. Last, we quantify phenotype complexity and find that neural nets trained with or without multitask learning do not improve on simple baselines unless the phenotypes are sufficiently complex.

Artificial Intelligence and Personalized Medicine

The development of high-throughput, data-intensive biomedical research assays and technologies has created a need for researchers to develop strategies for analyzing, integrating, and interpreting the massive amounts of data they generate. Although a wide variety of statistical methods have been designed to accommodate 'big data,'  experiences with the use of artificial intelligence (AI) techniques suggest that they might be particularly appropriate. In addition,  the results of the application of these assays reveal a great heterogeneity in the pathophysiologic factors and processes that contribute to disease, suggesting that there is a need to tailor, or 'personalize,' medicines to the nuanced and often unique features possessed by individual patients. Given how important data-intensive assays are to revealing appropriate intervention targets and strategies for  treating an individual with a disease, AI can play an important role in the development of personalized medicines. We describe many areas where AI can play such a role and argue that AI's ability to advance personalized medicine will depend critically on not only the refinement of relevant assays, but also on ways of storing, aggregating, accessing, and ultimately integrating, the data they produce. We also point out the limitations of many AI techniques in developing personalized medicines as well as consider areas for further research.

Artificial Intelligence in Pathology

As in other domains, artificial intelligence is becoming increasingly important in medicine. In particular,deep learning-based pattern recognition methods can advance the field of pathology byincorporating clinical, radiologic, and genomic data to accurately diagnose diseases and predictpatient prognoses. In this review, we present an overview of artificial intelligence, the brief historyof artificial intelligence in the medical domain, recent advances in artificial intelligence applied topathology, and future prospects of pathology driven by artificial intelligence.

Big data hurdles in precision medicine and precision public health

BACKGROUND

Nowadays, trendy research in biomedical sciences juxtaposes the term 'precision' to medicine and public health with companion words like big data, data science, and deep learning. Technological advancements permit the collection and merging of large heterogeneous datasets from different sources, from genome sequences to social media posts or from electronic health records to wearables. Additionally, complex algorithms supported by high-performance computing allow one to transform these large datasets into knowledge. Despite such progress, many barriers still exist against achieving precision medicine and precision public health interventions for the benefit of the individual and the population.

MAIN BODY

The present work focuses on analyzing both the technical and societal hurdles related to the development of prediction models of health risks, diagnoses and outcomes from integrated biomedical databases. Methodological challenges that need to be addressed include improving semantics of study designs: medical record data are inherently biased, and even the most advanced deep learning's denoising autoencoders cannot overcome the bias if not handled a priori by design. Societal challenges to face include evaluation of ethically actionable risk factors at the individual and population level; for instance, usage of gender, race, or ethnicity as risk modifiers, not as biological variables, could be replaced by modifiable environmental proxies such as lifestyle and dietary habits, household income, or access to educational resources.

CONCLUSIONS

Data science for precision medicine and public health warrants an informatics-oriented formalization of the study design and interoperability throughout all levels of the knowledge inference process, from the research semantics, to model development, and ultimately to implementation.

Artificial Intelligence and Integrated Genotype⁻Phenotype Identification

The integration of phenotypes and genotypes is at an unprecedented level and offers new opportunities to establish deep phenotypes. There are a number of challenges to overcome, specifically, accelerated growth of data, data silos, incompleteness, inaccuracies, and heterogeneity within and across data sources. This perspective report discusses artificial intelligence (AI) approaches that hold promise in addressing these challenges by automating computable phenotypes and integrating them with genotypes. Collaborations between biomedical and AI researchers will be highlighted in order to describe initial successes with an eye toward the future.