Temporal relationship of computed and structured diagnoses in electronic health record data

Automated Identification of Heart Failure With Reduced Ejection Fraction Using Deep Learning-Based Natural Language Processing

BACKGROUND

The lack of automated tools for measuring care quality limits the implementation of a national program to assess guideline-directed care in heart failure with reduced ejection fraction (HFrEF).

OBJECTIVES

The authors aimed to automate the identification of patients with HFrEF at hospital discharge, an opportunity to evaluate and improve the quality of care.

METHODS

The authors developed a novel deep-learning language model for identifying patients with HFrEF from discharge summaries of hospitalizations with heart failure at Yale New Haven Hospital during 2015 to 2019. HFrEF was defined by left ventricular ejection fraction <40% on antecedent echocardiography. The authors externally validated the model at Northwestern Medicine, community hospitals of Yale, and the MIMIC-III (Medical Information Mart for Intensive Care III) database.

RESULTS

A total of 13,251 notes from 5,392 unique individuals (age 73 ± 14 years, 48% women), including 2,487 patients with HFrEF (46.1%), were used for model development (train/held-out: 70%/30%). The model achieved an area under receiver-operating characteristic curve (AUROC) of 0.97 and area under precision recall curve (AUPRC) of 0.97 in detecting HFrEF on the held-out set. The model had high performance in identifying HFrEF with AUROC = 0.94 and AUPRC = 0.91 on 19,242 notes from Northwestern Medicine, AUROC = 0.95 and AUPRC = 0.96 on 139 manually abstracted notes from Yale community hospitals, and AUROC = 0.91 and AUPRC = 0.92 on 146 manually reviewed notes from MIMIC-III. Model-based predictions of HFrEF corresponded to a net reclassification improvement of 60.2 ± 1.9% compared with diagnosis codes (P < 0.001).

CONCLUSIONS

The authors developed a language model that identifies HFrEF from clinical notes with high precision and accuracy, representing a key element in automating quality assessment for individuals with HFrEF.

Rising to the challenge of defining and operationalising multimorbidity in a UK hospital setting: the ADMISSION research collaborative

PURPOSE

Greater transparency and consistency when defining multimorbidity in different settings is needed. We aimed to: (1) adapt published principles that can guide the selection of long-term conditions for inclusion in research studies of multimorbidity in hospitals; (2) apply these principles and identify a list of long-term conditions; (3) operationalise this list by mapping it to International Classification of Diseases 10th revision (ICD-10) codes.

METHODS

Review by independent assessors and ratification by an interdisciplinary programme management group.

RESULTS

Agreement was reached that when defining multimorbidity in hospitals for research purposes all conditions must meet the following four criteria: (1) medical diagnosis; (2) typically present for ≥ 12 months; (3) at least one of currently active; permanent in effect; requiring current treatment, care or therapy; requiring surveillance; remitting-relapsing and requiring ongoing treatment or care, and; (4) lead to at least one of: significantly increased risk of death; significantly reduced quality of life; frailty or physical disability; significantly worsened mental health; significantly increased treatment burden (indicated by an increased risk of hospital admission or increased length of hospital stay). Application of these principles to two existing lists of conditions led to the selection of 60 conditions that can be used when defining multimorbidity for research focused on hospitalised patients. ICD-10 codes were identified for each of these conditions to ensure consistency in their operationalisation.

CONCLUSIONS

This work contributes to achieving the goal of greater transparency and consistency in the approach to the study of multimorbidity, with a specific focus on the UK hospital setting.

How far back do we need to look to capture diagnoses in electronic health records? A retrospective observational study of hospital electronic health record data

OBJECTIVES

Analysis of routinely collected electronic health data is a key tool for long-term condition research and practice for hospitalised patients. This requires accurate and complete ascertainment of a broad range of diagnoses, something not always recorded on an admission document at a single point in time. This study aimed to ascertain how far back in time electronic hospital records need to be interrogated to capture long-term condition diagnoses.

DESIGN

Retrospective observational study of routinely collected hospital electronic health record data.

SETTING

Queen Elizabeth Hospital Birmingham (UK)-linked data held by the PIONEER acute care data hub.

PARTICIPANTS

Patients whose first recorded admission for chronic obstructive pulmonary disease (COPD) exacerbation (n=560) or acute stroke (n=2142) was between January and December 2018 and who had a minimum of 10 years of data prior to the index date.

OUTCOME MEASURES

We identified the most common International Classification of Diseases version 10-coded diagnoses received by patients with COPD and acute stroke separately. For each diagnosis, we derived the number of patients with the diagnosis recorded at least once over the full 10-year lookback period, and then compared this with shorter lookback periods from 1 year to 9 years prior to the index admission.

RESULTS

Seven of the top 10 most common diagnoses in the COPD dataset reached >90% completeness by 6 years of lookback. Atrial fibrillation and diabetes were >90% coded with 2-3 years of lookback, but hypertension and asthma completeness continued to rise all the way out to 10 years of lookback. For stroke, 4 of the top 10 reached 90% completeness by 5 years of lookback; angina pectoris was >90% coded at 7 years and previous transient ischaemic attack completeness continued to rise out to 10 years of lookback.

CONCLUSION

A 7-year lookback captures most, but not all, common diagnoses. Lookback duration should be tailored to the conditions being studied.

Exploring the impact of missingness on racial disparities in predictive performance of a machine learning model for emergency department triage

Objective

To investigate how missing data in the patient problem list may impact racial disparities in the predictive performance of a machine learning (ML) model for emergency department (ED) triage.

Materials and Methods

Racial disparities may exist in the missingness of EHR data (eg, systematic differences in access, testing, and/or treatment) that can impact model predictions across racialized patient groups. We use an ML model that predicts patients' risk for adverse events to produce triage-level recommendations, patterned after a clinical decision support tool deployed at multiple EDs. We compared the model's predictive performance on sets of observed (problem list data at the point of triage) versus manipulated (updated to the more complete problem list at the end of the encounter) test data. These differences were compared between Black and non-Hispanic White patient groups using multiple performance measures relevant to health equity.

Results

There were modest, but significant, changes in predictive performance comparing the observed to manipulated models across both Black and non-Hispanic White patient groups; c-statistic improvement ranged between 0.027 and 0.058. The manipulation produced no between-group differences in c-statistic by race. However, there were small between-group differences in other performance measures, with greater change for non-Hispanic White patients.

Discussion

Problem list missingness impacted model performance for both patient groups, with marginal differences detected by race.

Conclusion

Further exploration is needed to examine how missingness may contribute to racial disparities in clinical model predictions across settings. The novel manipulation method demonstrated may aid future research.

Leveraging Electronic Health Records to Construct a Phenotype for Hypertension Surveillance in the United States

BACKGROUND

Hypertension is an important risk factor for cardiovascular diseases. Electronic health records (EHRs) may augment chronic disease surveillance. We aimed to develop an electronic phenotype (e-phenotype) for hypertension surveillance.

METHODS

We included 11,031,368 eligible adults from the 2019 IQVIA Ambulatory Electronic Medical Records-US (AEMR-US) dataset. We identified hypertension using three criteria, alone or in combination: diagnosis codes, blood pressure (BP) measurements, and antihypertensive medications. We compared AEMR-US estimates of hypertension prevalence and control against those from the National Health and Nutrition Examination Survey (NHANES) 2017-18, which defined hypertension as BP ≥130/80 mm Hg or ≥1 antihypertensive medication.

RESULTS

The study population had a mean (SD) age of 52.3 (6.7) years, and 56.7% were women. The selected three-criteria e-phenotype (≥1 diagnosis code, ≥2 BP measurements of ≥130/80 mm Hg, or ≥1 antihypertensive medication) yielded similar trends in hypertension prevalence as NHANES: 42.2% (AEMR-US) vs. 44.9% (NHANES) overall, 39.0% vs. 38.7% among women, and 46.5% vs. 50.9% among men. The pattern of age-related increase in hypertension prevalence was similar between AEMR-US and NHANES. The prevalence of hypertension control in AEMR-US was 31.5% using the three-criteria e-phenotype, which was higher than NHANES (14.5%).

CONCLUSIONS

Using an EHR dataset of 11 million adults, we constructed a hypertension e-phenotype using three criteria, which can be used for surveillance of hypertension prevalence and control.

Automated Identification of Heart Failure with Reduced Ejection Fraction using Deep Learning-based Natural Language Processing

Background

The lack of automated tools for measuring care quality has limited the implementation of a national program to assess and improve guideline-directed care in heart failure with reduced ejection fraction (HFrEF). A key challenge for constructing such a tool has been an accurate, accessible approach for identifying patients with HFrEF at hospital discharge, an opportunity to evaluate and improve the quality of care.

Methods

We developed a novel deep learning-based language model for identifying patients with HFrEF from discharge summaries using a semi-supervised learning framework. For this purpose, hospitalizations with heart failure at Yale New Haven Hospital (YNHH) between 2015 to 2019 were labeled as HFrEF if the left ventricular ejection fraction was under 40% on antecedent echocardiography. The model was internally validated with model-based net reclassification improvement (NRI) assessed against chart-based diagnosis codes. We externally validated the model on discharge summaries from hospitalizations with heart failure at Northwestern Medicine, community hospitals of Yale New Haven Health in Connecticut and Rhode Island, and the publicly accessible MIMIC-III database, confirmed with chart abstraction.

Results

A total of 13,251 notes from 5,392 unique individuals (mean age 73 ± 14 years, 48% female), including 2,487 patients with HFrEF (46.1%), were used for model development (train/held-out test: 70/30%). The deep learning model achieved an area under receiving operating characteristic (AUROC) of 0.97 and an area under precision-recall curve (AUPRC) of 0.97 in detecting HFrEF on the held-out set. In external validation, the model had high performance in identifying HFrEF from discharge summaries with AUROC 0.94 and AUPRC 0.91 on 19,242 notes from Northwestern Medicine, AUROC 0.95 and AUPRC 0.96 on 139 manually abstracted notes from Yale community hospitals, and AUROC 0.91 and AUPRC 0.92 on 146 manually reviewed notes at MIMIC-III. Model-based prediction of HFrEF corresponded to an overall NRI of 60.2 ± 1.9% compared with the chart diagnosis codes (p-value < 0.001) and an increase in AUROC from 0.61 [95% CI: 060-0.63] to 0.91 [95% CI 0.90-0.92].

Conclusions

We developed and externally validated a deep learning language model that automatically identifies HFrEF from clinical notes with high precision and accuracy, representing a key element in automating quality assessment and improvement for individuals with HFrEF.

Leveraging electronic health record data for endometriosis research

Endometriosis is a chronic, complex disease for which there are vast disparities in diagnosis and treatment between sociodemographic groups. Clinical presentation of endometriosis can vary from asymptomatic disease-often identified during (in)fertility consultations-to dysmenorrhea and debilitating pelvic pain. Because of this complexity, delayed diagnosis (mean time to diagnosis is 1.7-3.6 years) and misdiagnosis is common. Early and accurate diagnosis of endometriosis remains a research priority for patient advocates and healthcare providers. Electronic health records (EHRs) have been widely adopted as a data source in biomedical research. However, they remain a largely untapped source of data for endometriosis research. EHRs capture diverse, real-world patient populations and care trajectories and can be used to learn patterns of underlying risk factors for endometriosis which, in turn, can be used to inform screening guidelines to help clinicians efficiently and effectively recognize and diagnose the disease in all patient populations reducing inequities in care. Here, we provide an overview of the advantages and limitations of using EHR data to study endometriosis. We describe the prevalence of endometriosis observed in diverse populations from multiple healthcare institutions, examples of variables that can be extracted from EHRs to enhance the accuracy of endometriosis prediction, and opportunities to leverage longitudinal EHR data to improve our understanding of long-term health consequences for all patients.

A multicenter evaluation of computable phenotyping approaches for SARS-CoV-2 infection and COVID-19 hospitalizations

Diagnosis codes are used to study SARS-CoV2 infections and COVID-19 hospitalizations in administrative and electronic health record (EHR) data. Using EHR data (April 2020-March 2021) at the Yale-New Haven Health System and the three hospital systems of the Mayo Clinic, computable phenotype definitions based on ICD-10 diagnosis of COVID-19 (U07.1) were evaluated against positive SARS-CoV-2 PCR or antigen tests. We included 69,423 patients at Yale and 75,748 at Mayo Clinic with either a diagnosis code or a positive SARS-CoV-2 test. The precision and recall of a COVID-19 diagnosis for a positive test were 68.8% and 83.3%, respectively, at Yale, with higher precision (95%) and lower recall (63.5%) at Mayo Clinic, varying between 59.2% in Rochester to 97.3% in Arizona. For hospitalizations with a principal COVID-19 diagnosis, 94.8% at Yale and 80.5% at Mayo Clinic had an associated positive laboratory test, with secondary diagnosis of COVID-19 identifying additional patients. These patients had a twofold higher inhospital mortality than based on principal diagnosis. Standardization of coding practices is needed before the use of diagnosis codes in clinical research and epidemiological surveillance of COVID-19.

Comparing ascertainment of chronic condition status with problem lists versus encounter diagnoses from electronic health records

OBJECTIVE

To assess and compare electronic health record (EHR) documentation of chronic disease in problem lists and encounter diagnosis records among Community Health Center (CHC) patients.

MATERIALS AND METHODS

We assessed patient EHR data in a large clinical research network during 2012-2019. We included CHCs who provided outpatient, older adult primary care to patients age ≥45 years, with ≥2 office visits during the study. Our study sample included 1 180 290 patients from 545 CHCs across 22 states. We used diagnosis codes from 39 Chronic Condition Warehouse algorithms to identify chronic conditions from encounter diagnoses only and compared against problem list records. We measured correspondence including agreement, kappa, prevalence index, bias index, and prevalence-adjusted bias-adjusted kappa.

RESULTS

Overlap of encounter diagnosis and problem list ascertainment was 59.4% among chronic conditions identified, with 12.2% of conditions identified only in encounters and 28.4% identified only in problem lists. Rates of coidentification varied by condition from 7.1% to 84.4%. Greatest agreement was found in diabetes (84.4%), HIV (78.1%), and hypertension (74.7%). Sixteen conditions had <50% agreement, including cancers and substance use disorders. Overlap for mental health conditions ranged from 47.4% for anxiety to 59.8% for depression.

DISCUSSION

Agreement between the 2 sources varied substantially. Conditions requiring regular management in primary care settings may have a higher agreement than those diagnosed and treated in specialty care.

CONCLUSION

Relying on EHR encounter data to identify chronic conditions without reference to patient problem lists may under-capture conditions among CHC patients in the United States.

Comparing reliability of ICD-10-based COVID-19 comorbidity data to manual chart review, a retrospective cross-sectional study

International Statistical Classification of Disease and Related Health Problems, 10th Revision codes (ICD‐10) are used to characterize cohort comorbidities. Recent literature does not demonstrate standardized extraction methods. Objective: Compare COVID‐19 cohort manual‐chart‐review and ICD‐10‐based comorbidity data; characterize the accuracy of different methods of extracting ICD‐10‐code‐based comorbidity, including the temporal accuracy with respect to critical time points such as day of admission. Design: Retrospective cross‐sectional study. Measurements: ICD‐10‐based‐data performance characteristics relative to manual‐chart‐review. Results: Discharge billing diagnoses had a sensitivity of 0.82 (95% confidence interval [CI]: 0.79–0.85; comorbidity range: 0.35–0.96). The past medical history table had a sensitivity of 0.72 (95% CI: 0.69–0.76; range: 0.44–0.87). The active problem list had a sensitivity of 0.67 (95% CI: 0.63–0.71; range: 0.47–0.71). On day of admission, the active problem list had a sensitivity of 0.58 (95% CI: 0.54–0.63; range: 0.30–0.68)and past medical history table had a sensitivity of 0.48 (95% CI: 0.43–0.53; range: 0.30–0.56). Conclusions and Relevance: ICD‐10‐based comorbidity data performance varies depending on comorbidity, data source, and time of retrieval; there are notable opportunities for improvement. Future researchers should clearly outline comorbidity data source and validate against manual‐chart‐review.

Accuracy of Computable Phenotyping Approaches for SARS-CoV-2 Infection and COVID-19 Hospitalizations from the Electronic Health Record

Objective:

Real-world data have been critical for rapid-knowledge generation throughout the COVID-19 pandemic. To ensure high-quality results are delivered to guide clinical decision making and the public health response, as well as characterize the response to interventions, it is essential to establish the accuracy of COVID-19 case definitions derived from administrative data to identify infections and hospitalizations.

Methods:

Electronic Health Record (EHR) data were obtained from the clinical data warehouse of the Yale New Haven Health System (Yale, primary site) and 3 hospital systems of the Mayo Clinic (validation site). Detailed characteristics on demographics, diagnoses, and laboratory results were obtained for all patients with either a positive SARS-CoV-2 PCR or antigen test or ICD-10 diagnosis of COVID-19 (U07.1) between April 1, 2020 and March 1, 2021. Various computable phenotype definitions were evaluated for their accuracy to identify SARS-CoV-2 infection and COVID-19 hospitalizations.

Results:

Of the 69,423 individuals with either a diagnosis code or a laboratory diagnosis of a SARS-CoV-2 infection at Yale, 61,023 had a principal or a secondary diagnosis code for COVID-19 and 50,355 had a positive SARS-CoV-2 test. Among those with a positive laboratory test, 38,506 (76.5%) and 3449 (6.8%) had a principal and secondary diagnosis code of COVID-19, respectively, while 8400 (16.7%) had no COVID-19 diagnosis. Moreover, of the 61,023 patients with a COVID-19 diagnosis code, 19,068 (31.2%) did not have a positive laboratory test for SARS-CoV-2 in the EHR. Of the 20 cases randomly sampled from this latter group for manual review, all had a COVID-19 diagnosis code related to asymptomatic testing with negative subsequent test results. The positive predictive value (precision) and sensitivity (recall) of a COVID-19 diagnosis in the medical record for a documented positive SARS-CoV-2 test were 68.8% and 83.3%, respectively. Among 5,109 patients who were hospitalized with a principal diagnosis of COVID-19, 4843 (94.8%) had a positive SARS-CoV-2 test within the 2 weeks preceding hospital admission or during hospitalization. In addition, 789 hospitalizations had a secondary diagnosis of COVID-19, of which 446 (56.5%) had a principal diagnosis consistent with severe clinical manifestation of COVID-19 (e.g., sepsis or respiratory failure). Compared with the cohort that had a principal diagnosis of COVID-19, those with a secondary diagnosis had a more than 2-fold higher in-hospital mortality rate (13.2% vs 28.0%, P<0.001). In the validation sample at Mayo Clinic, diagnosis codes more consistently identified SARS-CoV-2 infection (precision of 95%) but had lower recall (63.5%) with substantial variation across the 3 Mayo Clinic sites. Similar to Yale, diagnosis codes consistently identified COVID-19 hospitalizations at Mayo, with hospitalizations defined by secondary diagnosis code with 2-fold higher in-hospital mortality compared to those with a primary diagnosis of COVID-19.

Conclusions:

COVID-19 diagnosis codes misclassified the SARS-CoV-2 infection status of many people, with implications for clinical research and epidemiological surveillance. Moreover, the codes had different performance across two academic health systems and identified groups with different risks of mortality. Real-world data from the EHR can be used to in conjunction with diagnosis codes to improve the identification of people infected with SARS-CoV-2.

Downstream Cascades of Care Following High-Sensitivity Troponin Test Implementation

BACKGROUND

Patients with chest pain are often evaluated for acute myocardial infarction through troponin testing, which may prompt downstream services (cascades) of uncertain value.

OBJECTIVES

This study sought to determine the association of high-sensitivity cardiac troponin (hs-cTn) assay implementation with cascade events.

METHODS

Using electronic health record and billing data, this study examined patient-visits to 5 emergency departments from April 1, 2017, to April 1, 2019. Difference-in-differences analysis compared patient-visits for chest pain (n = 7,564) to patient-visits for other symptoms (n = 100,415) (irrespective of troponin testing) before and after hs-cTn assay implementation. Outcomes included presence of any cascade event potentially associated with an initial hs-cTn test (primary), individual cascade events, length of stay, and spending on cardiac services.

RESULTS

Following hs-cTn implementation, patients with chest pain had a 2.8% (95% confidence interval [CI]: 0.72% to 4.9%) net increase in experiencing any cascade event. They were more likely to have multiple troponin tests (10.5%; 95% CI: 9.0% to 12.0%) and electrocardiograms (7.1 per 100 patient-visits; 95% CI: 1.8 to 12.4). However, they received net fewer computed tomography scans (-1.5 per 100 patient-visits; 95% CI: -1.8 to -1.1), stress tests (-5.9 per 100 patient-visits; 95% CI: -6.5 to -5.3), and percutaneous coronary intervention (PCI) (-0.65 per 100 patient-visits; 95% CI: -1.01 to -0.30) and were less likely to receive cardiac medications, undergo cardiology evaluation (-3.5%; 95% CI: -4.5% to 2.6%), or be hospitalized (-5.8%; 95% CI: -7.7% to -3.8%). Patients with chest pain had lower net mean length of stay (-0.24 days; 95% CI: -0.32 to -0.16) but no net change in spending.

CONCLUSIONS

Hs-cTn assay implementation was associated with more net upfront tests yet fewer net stress tests, PCI, cardiology evaluations, and hospital admissions in patients with chest pain relative to patients with other symptoms.