Comparing ascertainment of chronic condition status with problem lists versus encounter diagnoses from electronic health records

DEVELOPMENT AND INTERNAL-EXTERNAL VALIDATION OF THE ACCI-SOFA MODEL FOR PREDICTING IN-HOSPITAL MORTALITY OF PATIENTS WITH SEPSIS-3 IN THE ICU: A MULTICENTER RETROSPECTIVE COHORT STUDY

ABSTRACT

Objective: To achieve a better prediction of in-hospital mortality, the Sequential Organ Failure Assessment (SOFA) score needs to be adjusted and combined with comorbidities. This study aims to enhance the prediction of SOFA score for in-hospital mortality in patients with Sepsis-3. Methods: This study adjusted the maximum SOFA score within the first 3 days (Max Day3 SOFA) in relation to in-hospital mortality using logistic regression and incorporated the age-adjusted Charlson Comorbidity Index (aCCI) as a continuous variable to build the age-adjusted Charlson Comorbidity Index-Sequential Organ Failure Assessment (aCCI-SOFA) model. The outcome was in-hospital mortality. We developed, internally validated, and externally validated the aCCI-SOFA model using cohorts of Sepsis-3 patients from the MIMIC-IV, MIMIC-III (CareVue), and the FAHWMU cohort. The predictive performance of the model was assessed through discrimination and calibration, which was assessed using the area under the receiver operating characteristic and calibration curves, respectively. The overall predictive effect was evaluated using the Brier score. Measurements and main results: Compared with the Max Day3 SOFA, the aCCI-SOFA model showed significant improvement in area under the receiver operating characteristic with all cohorts: development cohort (0.81 vs 0.75, P < 0.001), internal validation cohort (0.81 vs 0.76, P < 0.001), MIMIC-III (CareVue) cohort (0.75 vs 0.68, P < 0.001), and FAHWMU cohort (0.72 vs 0.67, P = 0.001). In sensitivity analysis, it was suggested that the application of aCCI-SOFA in early nonseptic shock patients had greater clinical value, with significant differences compared with the original SOFA scores in all cohorts ( P < 0.05). Conclusion: For septic patients in intensive care unit, the aCCI-SOFA model exhibited superior predictive performance. The application of aCCI-SOFA in early nonseptic shock patients had greater clinical value.

Correctly structured problem lists lead to better and faster clinical decision-making in electronic health records compared to non-curated problem lists: A single-blinded crossover randomized controlled trial

BACKGROUND

Correctly structured problem lists in electronic health records (EHRs) offer major benefits to patient care. Without structured lists, diagnosis information is often scatteredly documented in free text, which may contribute to errors and inefficient information retrieval. This study aims to assess whether EHRs with correctly structured problem lists result in better and faster clinical decision-making compared to non-curated problem lists.

METHODS

Two versions of two patient records (A and B) were created in an EHR training environment: one version included diagnosis information structured and coded on the problem list ("correctly structured problem list"), the other version had missing problem list diagnoses and diagnosis information partly documented in free text ("non-curated problem list"). In this single-blinded crossover randomized controlled trial, healthcare providers, who can prescribe medications, from two Dutch university medical center locations first evaluated a randomized version of patient A, then B. Participants were asked to motivate their answer to two medication prescription questions. One (test) question required information similarly presented in both record versions. The second (comparison) question required information documented on problem lists and/or in notes. The primary outcome measure was the correctness of the motivated answer to the comparison question. Secondary outcome measure was the time to answer and motivate both questions correctly.

RESULTS

As planned, 160 participants enrolled. Two were excluded for not meeting inclusion criteria. Correctly structured problem lists increased providers' ability to answer the comparison question correctly (56.3 % versus 33.5 %, McNemar odds ratio 2.80 (1.65-4.93) 95 %-CI). Median time to answer both questions correctly was significantly lower for EHRs with correctly structured problem lists (Wilcoxon-signed-rank test p = 0.00002, with incorrect answers coded equally at slowest time).

CONCLUSIONS

Correctly structured problem lists lead to better and faster clinical decision-making. Increased structured problem lists usage may be warranted for which implementation policies should be developed.

Impact of Health Insurance Patterns on Chronic Health Conditions Among Older Patients

BACKGROUND

Patients have varying levels of chronic conditions and health insurance patterns as they become Medicare age-eligible. Understanding these dynamics will inform policies and reforms that direct capacity and resources for primary care clinics to care for these aging patients. This study 1) determined changes in chronic condition rates following Medicare age eligibility among patients with different insurance patterns and 2) estimated the number of chronically ill patients who remain inadequately insured post-Medicare eligibility among patients receiving care in community health centers.

METHOD

We used retrospective electronic health record data from 45,527 patients aged 62 to 68 from 990 community health centers in 25 states in 2014 to 2019. Insurance patterns (continuously insured, continuously uninsured, uninsured/discontinuously insured who gained insurance after age 65, lost insurance after age 65, discontinuously insured) and diagnosis of chronic conditions were defined at each visit pre- and post-Medicare eligibility. Difference-in-differences Poisson GEE models estimated changes of chronic condition rates by insurance groups pre- to post-Medicare age eligibility.

RESULTS

Post-Medicare eligibility, 72% patients were continuously insured, 14% gained insurance; and 14% were uninsured or discontinuously insured. The prevalence of multimorbidity (≥2 chronic conditions) was 77%. Those who gained insurance had a significantly larger increase in the rate of documented chronic conditions from pre- to post-Medicare (DID: 1.06, 95%CI:1.05-1.07) compared with the continuously insured group.

CONCLUSIONS

Post-Medicare age eligibility, a significant proportion of patients were diagnosed with new conditions leading to high burden of disease. One in 4 older adults continue to have inadequate health care coverage in their older age.

Impact of data source choice on multimorbidity measurement: a comparison study of 2.3 million individuals in the Welsh National Health Service

BACKGROUND

Measurement of multimorbidity in research is variable, including the choice of the data source used to ascertain conditions. We compared the estimated prevalence of multimorbidity and associations with mortality using different data sources.

METHODS

A cross-sectional study of SAIL Databank data including 2,340,027 individuals of all ages living in Wales on 01 January 2019. Comparison of prevalence of multimorbidity and constituent 47 conditions using data from primary care (PC), hospital inpatient (HI), and linked PC-HI data sources and examination of associations between condition count and 12-month mortality.

RESULTS

Using linked PC-HI compared with only HI data, multimorbidity was more prevalent (32.2% versus 16.5%), and the population of people identified as having multimorbidity was younger (mean age 62.5 versus 66.8 years) and included more women (54.2% versus 52.6%). Individuals with multimorbidity in both PC and HI data had stronger associations with mortality than those with multimorbidity only in HI data (adjusted odds ratio 8.34 [95% CI 8.02-8.68] versus 6.95 (95%CI 6.79-7.12] in people with ≥ 4 conditions). The prevalence of conditions identified using only PC versus only HI data was significantly higher for 37/47 and significantly lower for 10/47: the highest PC/HI ratio was for depression (14.2 [95% CI 14.1-14.4]) and the lowest for aneurysm (0.51 [95% CI 0.5-0.5]). Agreement in ascertainment of conditions between the two data sources varied considerably, being slight for five (kappa < 0.20), fair for 12 (kappa 0.21-0.40), moderate for 16 (kappa 0.41-0.60), and substantial for 12 (kappa 0.61-0.80) conditions, and by body system was lowest for mental and behavioural disorders. The percentage agreement, individuals with a condition identified in both PC and HI data, was lowest in anxiety (4.6%) and highest in coronary artery disease (62.9%).

CONCLUSIONS

The use of single data sources may underestimate prevalence when measuring multimorbidity and many important conditions (especially mental and behavioural disorders). Caution should be used when interpreting findings of research examining individual and multiple long-term conditions using single data sources. Where available, researchers using electronic health data should link primary care and hospital inpatient data to generate more robust evidence to support evidence-based healthcare planning decisions for people with multimorbidity.

Diagnosing major depressive disorder and substance use disorder using the electronic health record: A preliminary validation study

Background

One mechanism to examine if major depressive disorder (MDD) is related to the development of substance use disorder (SUD) is by leveraging naturalistic data available in the electronic health record (EHR). Rules for data extraction and variable construction linked to psychometrics validating their use are needed to extract data accurately.

Objective

We propose and validate a methodologic framework for using EHR variables to identify patients with MDD and non-nicotine SUD.

Methods

Proxy diagnoses and index dates of MDD and/or SUD were established using billing codes, problem lists, patient-reported outcome measures, and prescriptions. Manual chart reviews were conducted for the 1-year period surrounding each index date to determine (1) if proxy diagnoses were supported by chart notes and (2) if the index dates accurately captured disorder onset.

Results

The results demonstrated 100% positive predictive value for proxy diagnoses of MDD. The proxy diagnoses for SUD exhibited strong agreement (Cohen's kappa of 0.84) compared to manual chart review and 92% sensitivity, specificity, positive predictive value, and negative predictive value. Sixteen percent of patients showed inaccurate SUD index dates generated by EHR extraction with discrepancies of over 6 months compared to SUD onset identified through chart review.

Conclusions

Our methodology was very effective in identifying patients with MDD with or without SUD and moderately effective in identifying SUD onset date. These findings support the use of EHR data to make proxy diagnoses of MDD with or without SUD.

Clinical encounter heterogeneity and methods for resolving in networked EHR data: a study from N3C and RECOVER programs

OBJECTIVE

Clinical encounter data are heterogeneous and vary greatly from institution to institution. These problems of variance affect interpretability and usability of clinical encounter data for analysis. These problems are magnified when multisite electronic health record (EHR) data are networked together. This article presents a novel, generalizable method for resolving encounter heterogeneity for analysis by combining related atomic encounters into composite "macrovisits."

MATERIALS AND METHODS

Encounters were composed of data from 75 partner sites harmonized to a common data model as part of the NIH Researching COVID to Enhance Recovery Initiative, a project of the National Covid Cohort Collaborative. Summary statistics were computed for overall and site-level data to assess issues and identify modifications. Two algorithms were developed to refine atomic encounters into cleaner, analyzable longitudinal clinical visits.

RESULTS

Atomic inpatient encounters data were found to be widely disparate between sites in terms of length-of-stay (LOS) and numbers of OMOP CDM measurements per encounter. After aggregating encounters to macrovisits, LOS and measurement variance decreased. A subsequent algorithm to identify hospitalized macrovisits further reduced data variability.

DISCUSSION

Encounters are a complex and heterogeneous component of EHR data and native data issues are not addressed by existing methods. These types of complex and poorly studied issues contribute to the difficulty of deriving value from EHR data, and these types of foundational, large-scale explorations, and developments are necessary to realize the full potential of modern real-world data.

CONCLUSION

This article presents method developments to manipulate and resolve EHR encounter data issues in a generalizable way as a foundation for future research and analysis.

Telehealth Use, Care Continuity, and Quality: Diabetes and Hypertension Care in Community Health Centers Before and During the COVID-19 Pandemic

BACKGROUND

Community health centers (CHCs) pivoted to using telehealth to deliver chronic care during the coronavirus COVID-19 pandemic. While care continuity can improve care quality and patients' experiences, it is unclear whether telehealth supported this relationship.

OBJECTIVE

We examine the association of care continuity with diabetes and hypertension care quality in CHCs before and during COVID-19 and the mediating effect of telehealth.

RESEARCH DESIGN

This was a cohort study.

PARTICIPANTS

Electronic health record data from 166 CHCs with n=20,792 patients with diabetes and/or hypertension with ≥2 encounters/year during 2019 and 2020.

METHODS

Multivariable logistic regression models estimated the association of care continuity (Modified Modified Continuity Index; MMCI) with telehealth use and care processes. Generalized linear regression models estimated the association of MMCI and intermediate outcomes. Formal mediation analyses assessed whether telehealth mediated the association of MMCI with A1c testing during 2020.

RESULTS

MMCI [2019: odds ratio (OR)=1.98, marginal effect=0.69, z=165.50, P<0.001; 2020: OR=1.50, marginal effect=0.63, z=147.73, P<0.001] and telehealth use (2019: OR=1.50, marginal effect=0.85, z=122.87, P<0.001; 2020: OR=10.00, marginal effect=0.90, z=155.57, P<0.001) were associated with higher odds of A1c testing. MMCI was associated with lower systolic (β=-2.90, P<0.001) and diastolic blood pressure (β=-1.44, P<0.001) in 2020, and lower A1c values (2019: β=-0.57, P=0.007; 2020: β=-0.45, P=0.008) in both years. In 2020, telehealth use mediated 38.7% of the relationship between MMCI and A1c testing.

CONCLUSIONS

Higher care continuity is associated with telehealth use and A1c testing, and lower A1c and blood pressure. Telehealth use mediates the association of care continuity and A1c testing. Care continuity may facilitate telehealth use and resilient performance on process measures.

Accessing and utilizing clinical and genomic data from an electronic health record data warehouse

Electronic health records (EHRs) and linked biobanks have tremendous potential to advance biomedical research and ultimately improve the health of future generations. Repurposing EHR data for research is not without challenges, however. In this paper, we describe the processes and considerations necessary to successfully access and utilize a data warehouse for research. Although imperfect, data warehouses are a powerful tool for harnessing a large amount of data to phenotype disease. They will have increasing relevance and applications in clinical research with growing sophistication in processes for EHR data abstraction, biobank integration, and cross-institutional linkage.