Comparison of EHR-based diagnosis documentation locations to a gold standard for risk stratification in patients with multiple chronic conditions

How excellent can centres of excellence be? The impact of prevalence on service quality

Centres of Excellence (CEs) are thought to provide better quality services for their speciality than Generic Services (GS). However, clinical test theory suggests this may arise from differences in the prevalence of these specialities' conditions in their referral populations, which affects the services' ability to detect diagnoses accurately, even with similar diagnostic sensitivities and specificities. Furthermore, GS' insensitivity to rarer diagnoses is necessary to avoid serious overdiagnosis despite using skills equivalent to CEs. Good GS can perform as well as CEs for disorders of 15% to 20% or greater prevalence in their referral populations, depending on the Minimal Clinically Important Difference (MCID) decided for their diagnoses' positive predictive values or degree of bias. CEs are necessary for rare disorders and have a role in determining MCIDs and the sensitivity and specificity of new measures. Sensitivity, specificity, positive & negative predictive values, and true diagnostic prevalence should be routine outcome measures.

Preparing for future pandemics: Automated intensive care electronic health record data extraction to accelerate clinical insights

Background

Manual data abstraction from electronic health records (EHRs) for research on intensive care patients is time-intensive and challenging, especially during high-pressure periods such as pandemics. Automated data extraction is a potential alternative but may raise quality concerns. This study assessed the feasibility and credibility of automated data extraction during the coronavirus disease 2019 (COVID-19) pandemic.

Methods

We retrieved routinely collected data from the COVID-Predict Dutch Data Warehouse, a multicenter database containing the following data on intensive care patients with COVID-19: demographic, medication, laboratory results, and data from monitoring and life support devices. These data were sourced from EHRs using automated data extraction. We used these data to determine indices of wasted ventilation and their prognostic value and compared our findings to a previously published original study that relied on manual data abstraction largely from the same hospitals.

Results

Using automatically extracted data, we replicated the original study. Among 1515 patients intubated for over 2 days, Harris-Benedict (HB) estimates of dead space fraction increased over time and were higher in non-survivors at each time point: at the start of ventilation (0.70±0.13 vs. 0.67±0.15, P <0.001), day 1 (0.74±0.10 vs. 0.71±0.11, P<0.001), day 2 (0.77±0.09 vs. 0.73±0.11, P<0.001), and day 3 (0.78±0.09 vs. 0.74±0.10, P<0.001). Patients with HB dead space fraction above the median had an increased mortality rate of 13.5%, compared to 10.1% in those with values below the median (P<0.005). Ventilatory ratio showed similar trends, with mortality increasing from 10.8% to 12.9% (P=0.040). Conversely, the end-tidal-to-arterial partial pressure of carbon dioxide (PaCO₂) ratio was inversely related to mortality, with a lower 28-day mortality in the higher than median group (8.5% vs. 15.1%, P<0.001). After adjusting for base risk, impaired ventilation markers showed no significant association with 28-day mortality.

Conclusion

Manual data abstraction from EHRs may be unnecessary for reliable research on intensive care patients, highlighting the feasibility and credibility of automated data extraction as a trustworthy and scalable solution to accelerate clinical insights, especially during future pandemics.

Diabetes Complications Among Community-Based Health Center Patients with Varying Multimorbidity Patterns

BACKGROUND

Multimorbidity with diabetes mellitus (DM and ≥ 1 chronic disease) presents challenges to maintaining adequate DM control.

OBJECTIVE

This study evaluates the risk of DM-related complications associated with various multimorbidity/DM patterns for patients seen in community-based health centers (CHCs).

DESIGN

Retrospective cohort study from the ADVANCE multi-state practice-based clinical data network.

PARTICIPANTS

Study included data from 132,765 patients age ≥ 45 years with DM seen in 2493 CHCs across 26 states from 10/01/2015 to 12/31/2019.

MAIN MEASURES

We assessed accrual of conditions and risk of experiencing DM complications during follow-up. Primary outcome of DM complication was categorized into acute, microvascular, microvascular (end-stage), macrovascular, or other. Key exposures included mutually exclusive multimorbidity categories: (1) DM + cardiometabolic, (2) DM + other somatic, (3) DM + mental, (4) DM + mental + somatic.

KEY RESULTS

At baseline, 17.2% of patients had DM only, 55.0% had DM + cardiometabolic multimorbidity, 2.3% had DM + other somatic multimorbidity, 3.0% had DM + mental multimorbidity, and 22.5% had DM + mental + somatic multimorbidity. Most DM-only patients (76.5%) developed multimorbidity with DM by study end. Compared with DM-only, adjusted risk differences of DM complications ranged from 1.4% (acute) to 8.8% (microvascular). DM + mental + somatic multimorbidity was associated with 13.4% (95%CI 12.8-14.1%) higher adjusted risk of experiencing any DM complication.

CONCLUSIONS

CHCs care for increasingly complex populations of patients with DM. Tailoring disease management strategies to address comorbid cardiovascular and/or mental health conditions may be important to prevent acute, microvascular, and macrovascular complications in these settings.

Electronic health record data quality assessment and tools: a systematic review

OBJECTIVE

We extended a 2013 literature review on electronic health record (EHR) data quality assessment approaches and tools to determine recent improvements or changes in EHR data quality assessment methodologies.

MATERIALS AND METHODS

We completed a systematic review of PubMed articles from 2013 to April 2023 that discussed the quality assessment of EHR data. We screened and reviewed papers for the dimensions and methods defined in the original 2013 manuscript. We categorized papers as data quality outcomes of interest, tools, or opinion pieces. We abstracted and defined additional themes and methods though an iterative review process.

RESULTS

We included 103 papers in the review, of which 73 were data quality outcomes of interest papers, 22 were tools, and 8 were opinion pieces. The most common dimension of data quality assessed was completeness, followed by correctness, concordance, plausibility, and currency. We abstracted conformance and bias as 2 additional dimensions of data quality and structural agreement as an additional methodology.

DISCUSSION

There has been an increase in EHR data quality assessment publications since the original 2013 review. Consistent dimensions of EHR data quality continue to be assessed across applications. Despite consistent patterns of assessment, there still does not exist a standard approach for assessing EHR data quality.

CONCLUSION

Guidelines are needed for EHR data quality assessment to improve the efficiency, transparency, comparability, and interoperability of data quality assessment. These guidelines must be both scalable and flexible. Automation could be helpful in generalizing this process.

Accessing and utilizing clinical and genomic data from an electronic health record data warehouse

Electronic health records (EHRs) and linked biobanks have tremendous potential to advance biomedical research and ultimately improve the health of future generations. Repurposing EHR data for research is not without challenges, however. In this paper, we describe the processes and considerations necessary to successfully access and utilize a data warehouse for research. Although imperfect, data warehouses are a powerful tool for harnessing a large amount of data to phenotype disease. They will have increasing relevance and applications in clinical research with growing sophistication in processes for EHR data abstraction, biobank integration, and cross-institutional linkage.

Electronic Health Record Optimization for Artificial Intelligence

Laboratory clinical decision support (CDS) typically relies on data from the electronic health record (EHR). The implementation of a sustainable, effective laboratory CDS program requires a commitment to standardization and harmonization of key EHR data elements that are the foundation of laboratory CDS. The direct use of artificial intelligence algorithms in CDS programs will be limited unless key elements of the EHR are structured. The identification, curation, maintenance, and preprocessing steps necessary to implement robust laboratory-based algorithms must account for the heterogeneity of data present in a typical EHR.

Developing Automated Computer Algorithms to Phenotype Periodontal Disease Diagnoses in Electronic Dental Records

OBJECTIVE

 Our objective was to phenotype periodontal disease (PD) diagnoses from three different sections (diagnosis codes, clinical notes, and periodontal charting) of the electronic dental records (EDR) by developing two automated computer algorithms.

METHODS

 We conducted a retrospective study using EDR data of patients (n = 27,138) who received care at Temple University Maurice H. Kornberg School of Dentistry from January 1, 2017 to August 31, 2021. We determined the completeness of patient demographics, periodontal charting, and PD diagnoses information in the EDR. Next, we developed two automated computer algorithms to automatically diagnose patients' PD statuses from clinical notes and periodontal charting data. Last, we phenotyped PD diagnoses using automated computer algorithms and reported the improved completeness of diagnosis.

RESULTS

 The completeness of PD diagnosis from the EDR was as follows: periodontal diagnosis codes 36% (n = 9,834), diagnoses in clinical notes 18% (n = 4,867), and charting information 80% (n = 21,710). After phenotyping, the completeness of PD diagnoses improved to 100%. Eleven percent of patients had healthy periodontium, 43% were with gingivitis, 3% with stage I, 36% with stage II, and 7% with stage III/IV periodontitis.

CONCLUSIONS

 We successfully developed, tested, and deployed two automated algorithms on big EDR datasets to improve the completeness of PD diagnoses. After phenotyping, EDR provided 100% completeness of PD diagnoses of 27,138 unique patients for research purposes. This approach is recommended for use in other large databases for the evaluation of their EDR data quality and for phenotyping PD diagnoses and other relevant variables.

Comparing ascertainment of chronic condition status with problem lists versus encounter diagnoses from electronic health records

OBJECTIVE

To assess and compare electronic health record (EHR) documentation of chronic disease in problem lists and encounter diagnosis records among Community Health Center (CHC) patients.

MATERIALS AND METHODS

We assessed patient EHR data in a large clinical research network during 2012-2019. We included CHCs who provided outpatient, older adult primary care to patients age ≥45 years, with ≥2 office visits during the study. Our study sample included 1 180 290 patients from 545 CHCs across 22 states. We used diagnosis codes from 39 Chronic Condition Warehouse algorithms to identify chronic conditions from encounter diagnoses only and compared against problem list records. We measured correspondence including agreement, kappa, prevalence index, bias index, and prevalence-adjusted bias-adjusted kappa.

RESULTS

Overlap of encounter diagnosis and problem list ascertainment was 59.4% among chronic conditions identified, with 12.2% of conditions identified only in encounters and 28.4% identified only in problem lists. Rates of coidentification varied by condition from 7.1% to 84.4%. Greatest agreement was found in diabetes (84.4%), HIV (78.1%), and hypertension (74.7%). Sixteen conditions had <50% agreement, including cancers and substance use disorders. Overlap for mental health conditions ranged from 47.4% for anxiety to 59.8% for depression.

DISCUSSION

Agreement between the 2 sources varied substantially. Conditions requiring regular management in primary care settings may have a higher agreement than those diagnosed and treated in specialty care.

CONCLUSION

Relying on EHR encounter data to identify chronic conditions without reference to patient problem lists may under-capture conditions among CHC patients in the United States.

Comparing reliability of ICD-10-based COVID-19 comorbidity data to manual chart review, a retrospective cross-sectional study

International Statistical Classification of Disease and Related Health Problems, 10th Revision codes (ICD‐10) are used to characterize cohort comorbidities. Recent literature does not demonstrate standardized extraction methods. Objective: Compare COVID‐19 cohort manual‐chart‐review and ICD‐10‐based comorbidity data; characterize the accuracy of different methods of extracting ICD‐10‐code‐based comorbidity, including the temporal accuracy with respect to critical time points such as day of admission. Design: Retrospective cross‐sectional study. Measurements: ICD‐10‐based‐data performance characteristics relative to manual‐chart‐review. Results: Discharge billing diagnoses had a sensitivity of 0.82 (95% confidence interval [CI]: 0.79–0.85; comorbidity range: 0.35–0.96). The past medical history table had a sensitivity of 0.72 (95% CI: 0.69–0.76; range: 0.44–0.87). The active problem list had a sensitivity of 0.67 (95% CI: 0.63–0.71; range: 0.47–0.71). On day of admission, the active problem list had a sensitivity of 0.58 (95% CI: 0.54–0.63; range: 0.30–0.68)and past medical history table had a sensitivity of 0.48 (95% CI: 0.43–0.53; range: 0.30–0.56). Conclusions and Relevance: ICD‐10‐based comorbidity data performance varies depending on comorbidity, data source, and time of retrieval; there are notable opportunities for improvement. Future researchers should clearly outline comorbidity data source and validate against manual‐chart‐review.

Subcategorizing EHR diagnosis codes to improve clinical application of machine learning models

BACKGROUND

Electronic health record (EHR) data is commonly used for secondary purposes such as research and clinical decision support. However, reuse of EHR data presents several challenges including but not limited to identifying all diagnoses associated with a patient's clinical encounter. The purpose of this study was to assess the feasibility of developing a schema to identify and subclassify all structured diagnosis codes for a patient encounter.

METHODS

To develop a subclassification schema we used EHR data from an interhospital transport data repository that contained complete hospital encounter level data. Eight discrete data sources containing structured diagnosis codes were identified. Diagnosis codes were normalized using the Unified Medical Language System and additional EHR data were combined with standardized terminologies to create and validate the subcategories. We then employed random forest to assess the usefulness of the new subcategorized diagnoses to predict post-interhospital transfer mortality by building 2 models, one using standard diagnosis codes, and one using the new subcategorized diagnosis codes.

RESULTS

Six subcategories of diagnoses were identified and validated. The subcategories included: primary or admitting diagnoses (10%), past medical, surgical or social history (9%), problem list (20%), comorbidity (24%), discharge diagnoses (6%), and unmapped diagnoses (31%). The subcategorized model outperformed the standard model, achieving a training AUROC of 0.97 versus 0.95 and testing model AUROC of 0.81 versus 0.46.

DISCUSSION

Our work demonstrates that merging structured diagnosis codes with additional EHR data and secondary data sources provides additional information to understand the role of diagnosis throughout a clinical encounter and improves predictive model performance. Further work is necessary to assess if subcategorizing produces benefits in interpreting the results of prognostic models and/or operationalizing the results in clinical decision support applications.

A systematic review of risk stratification tools internationally used in primary care settings

BACKGROUND AND AIMS

In our current healthcare situation, burden on healthcare services is increasing, with higher costs and increased utilization. Structured population health management has been developed as an approach to balance quality with increasing costs. This approach identifies sub-populations with comparable health risks, to tailor interventions for those that will benefit the most. Worldwide, the use of routine healthcare data extracted from electronic health registries for risk stratification approaches is increasing. Different risk stratification tools are used on different levels of the healthcare continuum. In this systematic literature review, we aimed to explore which tools are used in primary healthcare settings and assess their performance.

METHODS

We performed a systematic literature review of studies applying risk stratification tools with health outcomes in primary care populations. Studies in Organisation for Economic Co-operation and Development countries published in English-language journals were included. Search engines were utilized with keywords, for example, "primary care," "risk stratification," and "model." Risk stratification tools were compared based on different measures: area under the curve (AUC) and C-statistics for dichotomous outcomes and R 2 for continuous outcomes.

RESULTS

The search provided 4718 articles. Specific election criteria such as primary care populations, generic health utilization outcomes, and routinely collected data sources identified 61 articles, reporting on 31 different models. The three most frequently applied models were the Adjusted Clinical Groups (ACG, n = 23), the Charlson Comorbidity Index (CCI, n = 19), and the Hierarchical Condition Categories (HCC, n = 7). Most AUC and C-statistic values were above 0.7, with ACG showing slightly improved scores compared with the CCI and HCC (typically between 0.6 and 0.7).

CONCLUSION

Based on statistical performance, the validity of the ACG was the highest, followed by the CCI and the HCC. The ACG also appeared to be the most flexible, with the use of different international coding systems and measuring a wider variety of health outcomes.

Characteristics and Disease Severity of US Children and Adolescents Diagnosed With COVID-19

This cohort study uses data from the Premier Healthcare Database Special COVID-19 Release to assess the association of demographic and clinical characteristics with severe COVID-19 illness among hospitalized US pediatric patients with COVID-19.

Primary care practices' ability to predict future risk of expenditures and hospitalization using risk stratification and segmentation

Background

Patients with complex health care needs may suffer adverse outcomes from fragmented and delayed care, reducing well-being and increasing health care costs. Health reform efforts, especially those in primary care, attempt to mitigate risk of adverse outcomes by better targeting resources to those most in need. However, predicting who is susceptible to adverse outcomes, such as unplanned hospitalizations, ED visits, or other potentially avoidable expenditures, can be difficult, and providing intensive levels of resources to all patients is neither wanted nor efficient. Our objective was to understand if primary care teams can predict patient risk better than standard risk scores.

Methods

Six primary care practices risk stratified their entire patient population over a 2-year period, and worked to mitigate risk for those at high risk through care management and coordination. Individual patient risk scores created by the practices were collected and compared to a common risk score (Hierarchical Condition Categories) in their ability to predict future expenditures, ED visits, and hospitalizations. Accuracy of predictions, sensitivity, positive predictive values (PPV), and c-statistics were calculated for each risk scoring type. Analyses were stratified by whether the practice used intuition alone, an algorithm alone, or adjudicated an algorithmic risk score.

Results

In all, 40,342 patients were risk stratified. Practice scores had 38.6% agreement with HCC scores on identification of high-risk patients. For the 3,381 patients with reliable outcomes data, accuracy was high (0.71–0.88) but sensitivity and PPV were low (0.16–0.40). Practice-created scores had 0.02–0.14 lower sensitivity, specificity and PPV compared to HCC in prediction of outcomes. Practices using adjudication had, on average, .16 higher sensitivity.

Conclusions

Practices using simple risk stratification techniques had slightly worse accuracy in predicting common outcomes than HCC, but adjudication improved prediction.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01455-4.

Measuring problem prescription opioid use among patients receiving long-term opioid analgesic treatment: development and evaluation of an algorithm for use in EHR and claims data

Objective

Opioid surveillance in response to the opioid epidemic will benefit from scalable, automated algorithms for identifying patients with clinically documented signs of problem prescription opioid use. Existing algorithms lack accuracy. We sought to develop a high-sensitivity, high-specificity classification algorithm based on widely available structured health data to identify patients receiving chronic extended-release/long-acting (ER/LA) therapy with evidence of problem use to support subsequent epidemiologic investigations.

Methods

Outpatient medical records of a probability sample of 2,000 Kaiser Permanente Washington patients receiving ≥60 days’ supply of ER/LA opioids in a 90-day period from 1 January 2006 to 30 June 2015 were manually reviewed to determine the presence of clinically documented signs of problem use and used as a reference standard for algorithm development. Using 1,400 patients as training data, we constructed candidate predictors from demographic, enrollment, encounter, diagnosis, procedure, and medication data extracted from medical claims records or the equivalent from electronic health record (EHR) systems, and we used adaptive least absolute shrinkage and selection operator (LASSO) regression to develop a model. We evaluated this model in a comparable 600-patient validation set. We compared this model to ICD-9 diagnostic codes for opioid abuse, dependence, and poisoning. This study was registered with ClinicalTrials.gov as study NCT02667262 on 28 January 2016.

Results

We operationalized 1,126 potential predictors characterizing patient demographics, procedures, diagnoses, timing, dose, and location of medication dispensing. The final model incorporating 53 predictors had a sensitivity of 0.582 at positive predictive value (PPV) of 0.572. ICD-9 codes for opioid abuse, dependence, and poisoning had a sensitivity of 0.390 at PPV of 0.599 in the same cohort.

Conclusions

Scalable methods using widely available structured EHR/claims data to accurately identify problem opioid use among patients receiving long-term ER/LA therapy were unsuccessful. This approach may be useful for identifying patients needing clinical evaluation.

Towards augmenting structured EHR data: a comparison of manual chart review and patient self-report

Structured electronic health record (EHR) data are often used for quality measurement and improvement, clinical research, and other secondary uses. These data, however, are known to suffer from quality problems. There may be value in augmenting structured EHR data to improve data quality, thereby improving the reliability and validity of the conclusions drawn from those data. Focusing on five diagnoses related to cardiovascular care, this paper considers the added value of two alternative data sources: manual chart abstraction and patient self-report. We assess the overall agreement between structured EHR problem list data, abstracted EHR data, and patient self- report; and explore possible causes of disagreement between those sources. Our findings suggest that both chart abstraction and patient self-report contain significantly more diagnoses than the problem list, but that the information they capture is different. Methods for collecting and validating self-reported medical data require further consideration and exploration.

A Framework for Contracted Health IT Performance Assessments

Limited research exists which aids in structuring health IT contracts in an era of performance-based payments. We provide an assessment of common approaches to contracting and measuring of performance in practice. We conducted a review of existing literature and compliment this approach with a survey of healthcare professionals directly involved with health IT systems to further understand and classify current approaches. We identified architypes for structuring healthcare IT performance contracts to include: (1) internal operations, (2) external evaluation and (3) joint agreement for the delivery of value-based care.