Predictability Bounds of Electronic Health Records

Contribution of information about acute and geriatric characteristics to decisions about life-sustaining treatment for old patients in intensive care

BACKGROUND

Life-sustaining treatment (LST) in the intensive care unit (ICU) is withheld or withdrawn when there is no reasonable expectation of beneficial outcome. This is especially relevant in old patients where further functional decline might be detrimental for the self-perceived quality of life. However, there still is substantial uncertainty involved in decisions about LST. We used the framework of information theory to assess that uncertainty by measuring information processed during decision-making.

METHODS

Datasets from two multicentre studies (VIP1, VIP2) with a total of 7488 ICU patients aged 80 years or older were analysed concerning the contribution of information about the acute illness, age, gender, frailty and other geriatric characteristics to decisions about LST. The role of these characteristics in the decision-making process was quantified by the entropy of likelihood distributions and the Kullback-Leibler divergence with regard to withholding or withdrawing decisions.

RESULTS

Decisions to withhold or withdraw LST were made in 2186 and 1110 patients, respectively. Both in VIP1 and VIP2, information about the acute illness had the lowest entropy and largest Kullback-Leibler divergence with respect to decisions about withdrawing LST. Age, gender and geriatric characteristics contributed to that decision only to a smaller degree.

CONCLUSIONS

Information about the severity of the acute illness and, thereby, short-term prognosis dominated decisions about LST in old ICU patients. The smaller contribution of geriatric features suggests persistent uncertainty about the importance of functional outcome. There still remains a gap to fully explain decision-making about LST and further research involving contextual information is required.

TRIAL REGISTRATION

VIP1 study: NCT03134807 (1 May 2017), VIP2 study: NCT03370692 (12 December 2017).

Transitive Sequencing Medical Records for Mining Predictive and Interpretable Temporal Representations

Electronic health records (EHRs) contain important temporal information about the progression of disease and treatment outcomes. This paper proposes a transitive sequencing approach for constructing temporal representations from EHR observations for downstream machine learning. Using clinical data from a cohort of patients with congestive heart failure, we mined temporal representations by transitive sequencing of EHR medication and diagnosis records for classification and prediction tasks. We compared the classification and prediction performances of the transitive sequential representations (bag-of-sequences approach) with the conventional approach of using aggregated vectors of EHR data (aggregated vector representation) across different classifiers. We found that the transitive sequential representations are better phenotype "differentiators" and predictors than the "atemporal" EHR records. Our results also demonstrated that data representations obtained from transitive sequencing of EHR observations can present novel insights about the progression of the disease that are difficult to discern when clinical data are treated independently of the patient's history.

Wavelet entropy-based evaluation of intrinsic predictability of time series

Intrinsic predictability is imperative to quantify inherent information contained in a time series and assists in evaluating the performance of different forecasting methods to get the best possible prediction. Model forecasting performance is the measure of the probability of success. Nevertheless, model performance or the model does not provide understanding for improvement in prediction. Intuitively, intrinsic predictability delivers the highest level of predictability for a time series and informative in unfolding whether the system is unpredictable or the chosen model is a poor choice. We introduce a novel measure, the Wavelet Entropy Energy Measure (WEEM), based on wavelet transformation and information entropy for quantification of intrinsic predictability of time series. To investigate the efficiency and reliability of the proposed measure, model forecast performance was evaluated via a wavelet networks approach. The proposed measure uses the wavelet energy distribution of a time series at different scales and compares it with the wavelet energy distribution of white noise to quantify a time series as deterministic or random. We test the WEEM using a wide variety of time series ranging from deterministic, non-stationary, and ones contaminated with white noise with different noise-signal ratios. Furthermore, a relationship is developed between the WEEM and Nash-Sutcliffe Efficiency, one of the widely known measures of forecast performance. The reliability of WEEM is demonstrated by exploring the relationship to logistic map and real-world data.

Linking glycemic dysregulation in diabetes to symptoms, comorbidities, and genetics through EHR data mining

Diabetes is a diverse and complex disease, with considerable variation in phenotypic manifestation and severity. This variation hampers the study of etiological differences and reduces the statistical power of analyses of associations to genetics, treatment outcomes, and complications. We address these issues through deep, fine-grained phenotypic stratification of a diabetes cohort. Text mining the electronic health records of 14,017 patients, we matched two controlled vocabularies (ICD-10 and a custom vocabulary developed at the clinical center Steno Diabetes Center Copenhagen) to clinical narratives spanning a 19 year period. The two matched vocabularies comprise over 20,000 medical terms describing symptoms, other diagnoses, and lifestyle factors. The cohort is genetically homogeneous (Caucasian diabetes patients from Denmark) so the resulting stratification is not driven by ethnic differences, but rather by inherently dissimilar progression patterns and lifestyle related risk factors. Using unsupervised Markov clustering, we defined 71 clusters of at least 50 individuals within the diabetes spectrum. The clusters display both distinct and shared longitudinal glycemic dysregulation patterns, temporal co-occurrences of comorbidities, and associations to single nucleotide polymorphisms in or near genes relevant for diabetes comorbidities.

Relevant Word Order Vectorization for Improved Natural Language Processing in Electronic Health Records

Electronic health records (EHR) represent a rich resource for conducting observational studies, supporting clinical trials, and more. However, much of the data contains unstructured text, presenting an obstacle to automated extraction. Natural language processing (NLP) can structure and learn from text, but NLP algorithms were not designed for the unique characteristics of EHR. Here, we propose Relevant Word Order Vectorization (RWOV) to aid with structuring. RWOV is based on finding the positional relationship between the most relevant words to predicting the class of a text. This facilitates machine learning algorithms to use the interaction of not just keywords but positional dependencies (e.g. a relevant word occurs 5 relevant words before some term of interest). As a proof-of-concept, we attempted to classify the hormone receptor status of breast cancer patients treated at the University of Kansas Medical Center, comparing RWOV to other methods using the F1 score and AUC. RWOV performed as well as, or better than other methods in all but one case. For F1 score, RWOV had a clear edge on most tasks. AUC tended to be closer, but for HER2, RWOV was significantly better for most comparisons. These results suggest RWOV should be further developed for EHR-related NLP.

Biases in electronic health record data due to processes within the healthcare system: retrospective observational study

OBJECTIVE

To evaluate on a large scale, across 272 common types of laboratory tests, the impact of healthcare processes on the predictive value of electronic health record (EHR) data.

DESIGN

Retrospective observational study.

SETTING

Two large hospitals in Boston, Massachusetts, with inpatient, emergency, and ambulatory care.

PARTICIPANTS

All 669 452 patients treated at the two hospitals over one year between 2005 and 2006.

MAIN OUTCOME MEASURES

The relative predictive accuracy of each laboratory test for three year survival, using the time of the day, day of the week, and ordering frequency of the test, compared to the value of the test result.

RESULTS

The presence of a laboratory test order, regardless of any other information about the test result, has a significant association (P<0.001) with the odds of survival in 233 of 272 (86%) tests. Data about the timing of when laboratory tests were ordered were more accurate than the test results in predicting survival in 118 of 174 tests (68%).

CONCLUSIONS

Healthcare processes must be addressed and accounted for in analysis of observational health data. Without careful consideration to context, EHR data are unsuitable for many research questions. However, if explicitly modeled, the same processes that make EHR data complex can be leveraged to gain insight into patients' state of health.

A Systems Approach to Refine Disease Taxonomy by Integrating Phenotypic and Molecular Networks

The International Classification of Diseases (ICD) relies on clinical features and lags behind the current understanding of the molecular specificity of disease pathobiology, necessitating approaches that incorporate growing biomedical data for classifying diseases to meet the needs of precision medicine. Our analysis revealed that the heterogeneous molecular diversity of disease chapters and the blurred boundary between disease categories in ICD should be further investigated. Here, we propose a new classification of diseases (NCD) by developing an algorithm that predicts the additional categories of a disease by integrating multiple networks consisting of disease phenotypes and their molecular profiles. With statistical validations from phenotype-genotype associations and interactome networks, we demonstrate that NCD improves disease specificity owing to its overlapping categories and polyhierarchical structure. Furthermore, NCD captures the molecular diversity of diseases and defines clearer boundaries in terms of both phenotypic similarity and molecular associations, establishing a rational strategy to reform disease taxonomy.

•

The International Classification of Diseases (ICD) lags behind the current molecular characteristics of disease.

•

We quantified the limitations (specificity and blurred boundary) of ICD with integrated phenotypic and molecular profiles.

•

An integrative disease network integrating phenotypic and genotypic profiles proposes a refined disease category framework.

Disease taxonomy is one of the foundations of medical science and healthcare solutions. The most widely used disease taxonomy in clinical settings is the International Classification of Diseases (ICD), a system established >100 years ago and maintained by the World Health Organization to track disease incidence. It is well recognized that ICD, which is based on clinical observations, largely lags behind the molecular achievements of this medical big data era. We quantified the limitations of ICD using integrated phenotypic and molecular profiles and proposed a refined disease taxonomy with possible applications for precision medicine.

Yield and bias in defining a cohort study baseline from electronic health record data

AIMS

Despite growing interest in using electronic health records (EHR) to create longitudinal cohort studies, the distribution and missingness of EHR data might introduce selection bias and information bias to such analyses. We aimed to examine the yield and potential for these healthcare process biases in defining a study baseline using EHR data, using the example of cholesterol and blood pressure (BP) measurements.

METHODS

We created a virtual cohort study of cardiovascular disease (CVD) from patients with eligible cholesterol profiles in the New England (NE) and Southeast (SE) networks of the Veterans Health Administration in the United States. Using clinical data from the EHR, we plotted the yield of patients with BP measurements within an expanding timeframe around an index date of cholesterol testing. We compared three groups: (1) patients with BP from the exact index date; (2) patients with BP not on the index date but within the network-specific 90th percentile around the index date; and (3) patients with no BP within the network-specific 90th percentile.

RESULTS

Among 589,361 total patients in the two networks, 146,636 (61.0%) of 240,479 patients from NE and 289,906 (83.1%) of 348,882 patients from SE had BP measurements on the index date. Ninety percent had BP measured within 11 days of the index date in NE and within 5 days of the index date in SE. Group 3 in both networks had fewer available race data, fewer comorbidities and CVD medications, and fewer health system encounters.

CONCLUSIONS

Requiring same-day risk factor measurement in the creation of a virtual CVD cohort study from EHR data might exclude 40% of eligible patients, but including patients with infrequent visits might introduce bias. Data visualization can inform study-specific strategies to address these challenges for the research use of EHR data.

Estimating summary statistics for electronic health record laboratory data for use in high-throughput phenotyping algorithms

We study the question of how to represent or summarize raw laboratory data taken from an electronic health record (EHR) using parametric model selection to reduce or cope with biases induced through clinical care. It has been previously demonstrated that the health care process (Hripcsak and Albers, 2012, 2013), as defined by measurement context (Hripcsak and Albers, 2013; Albers et al., 2012) and measurement patterns (Albers and Hripcsak, 2010, 2012), can influence how EHR data are distributed statistically (Kohane and Weber, 2013; Pivovarov et al., 2014). We construct an algorithm, PopKLD, which is based on information criterion model selection (Burnham and Anderson, 2002; Claeskens and Hjort, 2008), is intended to reduce and cope with health care process biases and to produce an intuitively understandable continuous summary. The PopKLD algorithm can be automated and is designed to be applicable in high-throughput settings; for example, the output of the PopKLD algorithm can be used as input for phenotyping algorithms. Moreover, we develop the PopKLD-CAT algorithm that transforms the continuous PopKLD summary into a categorical summary useful for applications that require categorical data such as topic modeling. We evaluate our methodology in two ways. First, we apply the method to laboratory data collected in two different health care contexts, primary versus intensive care. We show that the PopKLD preserves known physiologic features in the data that are lost when summarizing the data using more common laboratory data summaries such as mean and standard deviation. Second, for three disease-laboratory measurement pairs, we perform a phenotyping task: we use the PopKLD and PopKLD-CAT algorithms to define high and low values of the laboratory variable that are used for defining a disease state. We then compare the relationship between the PopKLD-CAT summary disease predictions and the same predictions using empirically estimated mean and standard deviation to a gold standard generated by clinical review of patient records. We find that the PopKLD laboratory data summary is substantially better at predicting disease state. The PopKLD or PopKLD-CAT algorithms are not meant to be used as phenotyping algorithms, but we use the phenotyping task to show what information can be gained when using a more informative laboratory data summary. In the process of evaluation our method we show that the different clinical contexts and laboratory measurements necessitate different statistical summaries. Similarly, leveraging the principle of maximum entropy we argue that while some laboratory data only have sufficient information to estimate a mean and standard deviation, other laboratory data captured in an EHR contain substantially more information than can be captured in higher-parameter models.

Deep Patient: An Unsupervised Representation to Predict the Future of Patients from the Electronic Health Records

Secondary use of electronic health records (EHRs) promises to advance clinical research and better inform clinical decision making. Challenges in summarizing and representing patient data prevent widespread practice of predictive modeling using EHRs. Here we present a novel unsupervised deep feature learning method to derive a general-purpose patient representation from EHR data that facilitates clinical predictive modeling. In particular, a three-layer stack of denoising autoencoders was used to capture hierarchical regularities and dependencies in the aggregated EHRs of about 700,000 patients from the Mount Sinai data warehouse. The result is a representation we name “deep patient”. We evaluated this representation as broadly predictive of health states by assessing the probability of patients to develop various diseases. We performed evaluation using 76,214 test patients comprising 78 diseases from diverse clinical domains and temporal windows. Our results significantly outperformed those achieved using representations based on raw EHR data and alternative feature learning strategies. Prediction performance for severe diabetes, schizophrenia, and various cancers were among the top performing. These findings indicate that deep learning applied to EHRs can derive patient representations that offer improved clinical predictions, and could provide a machine learning framework for augmenting clinical decision systems.

Predicting quantitative traits from genome and phenome with near perfect accuracy

In spite of decades of linkage and association studies and its potential impact on human health, reliable prediction of an individual's risk for heritable disease remains difficult. Large numbers of mapped loci do not explain substantial fractions of heritable variation, leaving an open question of whether accurate complex trait predictions can be achieved in practice. Here, we use a genome sequenced population of ∼7,000 yeast strains of high but varying relatedness, and predict growth traits from family information, effects of segregating genetic variants and growth in other environments with an average coefficient of determination R(2) of 0.91. This accuracy exceeds narrow-sense heritability, approaches limits imposed by measurement repeatability and is higher than achieved with a single assay in the laboratory. Our results prove that very accurate prediction of complex traits is possible, and suggest that additional data from families rather than reference cohorts may be more useful for this purpose.