The impact of electronic medical records data sources on an adverse drug event quality measure

Comparison of Administrative Database-Derived and Hospital-Derived Data for Monitoring Blood Culture Use in the Pediatric Intensive Care Unit

BACKGROUND

Optimizing blood culture practices requires monitoring of culture use. Collecting culture data from electronic medical records can be resource intensive. Our objective was to determine whether administrative data could serve as a data source to measure blood culture use in pediatric intensive care units (PICUs).

METHODS

Using data from a national diagnostic stewardship collaborative to reduce blood culture use in PICUs, we compared the monthly number of blood cultures and patient-days collected from sites (site-derived) and the Pediatric Health Information System (PHIS, administrative-derived), an administrative data warehouse, for 11 participating sites. The collaborative's reduction in blood culture use was compared using administrative-derived and site-derived data.

RESULTS

Across all sites and months, the median of the monthly relative blood culture rate (ratio of administrative- to site-derived data) was 0.96 (Q1: 0.77, Q3: 1.24). The administrative-derived data produced an estimate of blood culture reduction over time that was attenuated toward the null compared with site-derived data.

CONCLUSIONS

Administrative data on blood culture use from the PHIS database correlates unpredictably with hospital-derived PICU data. The limitations of administrative billing data should be carefully considered before use for ICU-specific data.

Increased reporting but decreased mortality associated with adverse events in patients undergoing lung cancer surgery: Competing forces in an era of heightened focus on care quality?

INTRODUCTION

Advances in surgical techniques have improved clinical outcomes and decreased complications. At the same time, heightened attention to care quality has resulted in increased identification of hospital-acquired adverse events. We evaluated these divergent effects on the reported safety of lung cancer resection.

METHODS AND MATERIALS

We analyzed hospital-acquired adverse events in patients undergoing lung cancer resection using the National Hospital Discharge Survey (NHDS) database from 2001-2010. Demographics, diagnoses, and procedures data were abstracted using ICD-9 codes. We used the Agency for Healthcare Research and Quality (AHRQ) Patient Safety Indicators (PSI) to identify hospital-acquired adverse events. Weighted analyses were performed using t-tests and chi-square.

RESULTS

A total of 302,444 hospitalizations for lung cancer resection and were included in the analysis. Incidence of PSI increased over time (28% in 2001-2002 vs 34% in 2009-2010; P<0.001). Those with one or more PSI had increased in-hospital mortality (aOR = 11.1; 95% CI, 4.7-26.1; P<0.001) and prolonged hospitalization (12.5 vs 7.8 days; P<0.001). However, among those with PSI, in-hospital mortality decreased over time, from 17% in 2001-2002 to 2% in 2009-2010.

CONCLUSIONS

In a recent ten-year period, documented rates of adverse events associated with lung cancer resection increased. Despite this increase in safety events, we observed that mortality decreased. Because such metrics may be incorporated into hospital rankings and reimbursement considerations, adverse event coding consistency and content merit further evaluation.

Barriers and facilitators to data quality of electronic health records used for clinical research in China: a qualitative study

OBJECTIVES

There is an increasing trend in the use of electronic health records (EHRs) for clinical research. However, more knowledge is needed on how to assure and improve data quality. This study aimed to explore healthcare professionals' experiences and perceptions of barriers and facilitators of data quality of EHR-based studies in the Chinese context.

SETTING

Four tertiary hospitals in Beijing, China.

PARTICIPANTS

Nineteen healthcare professionals with experience in using EHR data for clinical research participated in the study.

METHODS

A qualitative study based on face-to-face semistructured interviews was conducted from March to July 2018. The interviews were audiorecorded and transcribed verbatim. Data analysis was performed using the inductive thematic analysis approach.

RESULTS

The main themes included factors related to healthcare systems, clinical documentation, EHR systems and researchers. The perceived barriers to data quality included heavy workload, staff rotations, lack of detailed information for specific research, variations in terminology, limited retrieval capabilities, large amounts of unstructured data, challenges with patient identification and matching, problems with data extraction and unfamiliar with data quality assessment. To improve data quality, suggestions from participants included: better staff training, providing monetary incentives, performing daily data verification, improving software functionality and coding structures as well as enhancing multidisciplinary cooperation.

CONCLUSIONS

These results provide a basis to begin to address current barriers and ultimately to improve validity and generalisability of research findings in China.

Visualizing nationwide variation in medicare Part D prescribing patterns

Background

To characterize the regional and national variation in prescribing patterns in the Medicare Part D program using dimensional reduction visualization methods.

Methods

Using publicly available Medicare Part D claims data, we identified and visualized regional and national provider prescribing profile variation with unsupervised clustering and t-distributed stochastic neighbor embedding (t-SNE) dimensional reduction techniques. Additionally, we examined differences between regionally representative prescribing patterns for major metropolitan areas.

Results

Distributions of prescribing volume and medication diversity were highly skewed among over 800,000 Medicare Part D providers. Medical specialties had characteristic prescribing patterns. Although the number of Medicare providers in each state was highly correlated with the number of Medicare Part D enrollees, some states were enriched for providers with > 10,000 prescription claims annually. Dimension-reduction, hierarchical clustering and t-SNE visualization of drug- or drug-class prescribing patterns revealed that providers cluster strongly based on specialty and sub-specialty, with large regional variations in prescribing patterns. Major metropolitan areas had distinct prescribing patterns that tended to group by major geographical divisions.

Conclusions

This work demonstrates that unsupervised clustering, dimension-reduction and t-SNE visualization can be used to analyze and visualize variation in provider prescribing patterns on a national level across thousands of medications, revealing substantial prescribing variation both between and within specialties, regionally, and between major metropolitan areas. These methods offer an alternative system-wide and pattern-centric view of such data for hypothesis generation, visualization, and pattern identification.

Electronic supplementary material

The online version of this article (10.1186/s12911-018-0670-2) contains supplementary material, which is available to authorized users.

Navigating a ship with a broken compass: evaluating standard algorithms to measure patient safety

Objective

Agency for Healthcare Research and Quality (AHRQ) software applies standardized algorithms to hospital administrative data to identify patient safety indicators (PSIs). The objective of this study was to assess the validity of PSI flags and report reasons for invalid flagging.

Material and Methods

At a 6-hospital academic medical center, a retrospective analysis was conducted of all PSIs flagged in fiscal year 2014. A multidisciplinary PSI Quality Team reviewed each flagged PSI based on quarterly reports. The positive predictive value (PPV, the percent of clinically validated cases) was calculated for 12 PSI categories. The documentation for each reversed case was reviewed to determine the reasons for PSI reversal.

Results

Of 657 PSI flags, 185 were reversed. Seven PSI categories had a PPV below 75%. Four broad categories of reasons for reversal were AHRQ algorithm limitations (38%), coding misinterpretations (45%), present upon admission (10%), and documentation insufficiency (7%). AHRQ algorithm limitations included 2 subcategories: an "incident" was inherent to the procedure, or highly likely (eg, vascular tumor bleed), or an "incident" was nonsignificant, easily controlled, and/or no intervention was needed.

Discussion

These findings support previous research highlighting administrative data problems. Additionally, AHRQ algorithm limitations was an emergent category not considered in previous research. Herein we present potential solutions to address these issues.

Conclusions

If, despite poor validity, US policy continues to rely on PSIs for incentive and penalty programs, improvements are needed in the quality of administrative data and the standardized PSI algorithms. These solutions require national motivation, research attention, and dissemination support.

Reporting Data Quality Assessment Results: Identifying Individual and Organizational Barriers and Solutions

Introduction:

Electronic health record (EHR) data are known to have significant data quality issues, yet the practice and frequency of assessing EHR data is unknown. We sought to understand current practices and attitudes towards reporting data quality assessment (DQA) results by data professionals.

Methods:

The project was conducted in four Phases: (1) examined current DQA practices among informatics/CER stakeholders via engagement meeting (07/2014); (2) characterized organizations conducting DQA by interviewing key personnel and data management professionals (07-08/2014); (3) developed and administered an anonymous survey to data professionals (03-06/2015); and (4) validated survey results during a follow-up informatics/CER stakeholder engagement meeting (06/2016).

Results:

The first engagement meeting identified the theme of unintended consequences as a primary barrier to DQA. Interviewees were predominantly medical groups serving distributed networks with formalized DQAs. Consistent with the interviews, most survey (N=111) respondents utilized DQA processes/programs. A lack of resources and clear definitions of how to judge the quality of a dataset were the most commonly cited individual barriers. Vague quality action plans/expectations and data owners not trained in problem identification and problem-solving skills were the most commonly cited organizational barriers. Solutions included allocating resources for DQA, establishing standards and guidelines, and changing organizational culture.

Discussion:

Several barriers affecting DQA and reporting were identified. Community alignment towards systematic DQA and reporting is needed to overcome these barriers.

Conclusion:

Understanding barriers and solutions to DQA reporting is vital for establishing trust in the secondary use of EHR data for quality improvement and the pursuit of personalized medicine.

Extracting autism spectrum disorder data from the electronic health record

BACKGROUND

Little is known about the health care utilization patterns of individuals with pediatric autism spectrum disorder (ASD).

OBJECTIVES

Electronic health record (EHR) data provide an opportunity to study medical utilization and track outcomes among children with ASD.  Methods: Using a pediatric, tertiary, academic hospital's Epic EHR, search queries were built to identify individuals aged 2-18 with International Classification of Diseases, Ninth Revision (ICD-9) codes, 299.00, 299.10, and 299.80 in their records. Codes were entered in the EHR using four different workflows: (1) during an ambulatory visit, (2) abstracted by Health Information Management (HIM) for an encounter, (3) recorded on the patient problem list, or (4) added as a chief complaint during an Emergency Department visit. Once individuals were identified, demographics, scheduling, procedures, and prescribed medications were extracted for all patient-related encounters for the period October 2010 through September 2012.

RESULTS

There were 100,000 encounters for more than 4,800 unique individuals. Individuals were most frequently identified with an HIM abstracted code (82.6%) and least likely to be identified by a chief complaint (45.8%). Categorical frequency for reported race (2 = 816.5, p < 0.001); payor type (2 = 354.1, p < 0.001); encounter type (2 = 1497.0, p < 0.001); and department (2 = 3722.8, p < 0.001) differed by search query. Challenges encountered included, locating available discrete data elements and missing data.

CONCLUSIONS

This study identifies challenges inherent in designing inclusive algorithms for identifying individuals with ASD and demonstrates the utility of employing multiple extractions to improve the completeness and quality of EHR data when conducting research.

Ambulatory Computerized Prescribing and Preventable Adverse Drug Events

BACKGROUND

Adverse drug events (ADEs) represent a significant cause of injury in the ambulatory care setting. Computerized physician order entry reduces rates of serious medication errors that can lead to ADEs in the inpatient setting, but few studies have evaluated whether computerized prescribing in the ambulatory setting reduces preventable ADE rates in ambulatory care.

OBJECTIVE

To determine the rates of preventable ADEs before and after the implementation of computerized prescribing with basic clinical decision support for ordering medications.

DESIGN

Before-after study of ADE rates in practices implementing computer order entry.

PARTICIPANTS

Adult patients seeking care in primary care practices at academic medical centers in Boston, Massachusetts (n = 41,819), and Indianapolis, Indiana (n = 9128).

MAIN MEASURES

We attempted to standardize the medication-related decision support knowledge base provided at the 2 sites, although the electronic records and presentation layers used at the 2 sites differed. The primary outcome was preventable ADEs identified based on structured results or symptoms defined by extracting symptom concepts from provider notes; potential ADEs were a secondary outcome.

RESULTS

Computerized prescribing did not significantly change the rate of preventable ADEs at either site. Compared with Boston practices, the rate of potential ADEs was more than seven-fold greater at Indianapolis (6.4/10,000 patient-months vs. 49.5/10,000 patient-months, P < 0.001). Computerized prescribing was associated with a 56% decrease in the potential ADE rate at Indianapolis (49.5 to 21.9/10,000 patient-months, P < 0.001) but a 104% increase at Boston (6.4 to 13.0/10,000 patient-months, P < 0.001). Preventable ADEs that occurred after computerized prescribing was implemented were due to patient education issues, physicians ignoring feedback from CDSS, and incomplete computerized knowledge base was incomplete (34%, 33%, and 33% in Indianapolis and 44%, 28%, and 28% in Boston).

CONCLUSIONS

The implementation of computerized prescribing in the ambulatory setting was not associated with any change in preventable ADEs but was associated with a decrease in potential ADEs at Indianapolis but an increase at Boston, although the absolute rate of ADEs was much lower in Boston.

A Harmonized Data Quality Assessment Terminology and Framework for the Secondary Use of Electronic Health Record Data

Objective:

Harmonized data quality (DQ) assessment terms, methods, and reporting practices can establish a common understanding of the strengths and limitations of electronic health record (EHR) data for operational analytics, quality improvement, and research. Existing published DQ terms were harmonized to a comprehensive unified terminology with definitions and examples and organized into a conceptual framework to support a common approach to defining whether EHR data is ‘fit’ for specific uses.

Materials and Methods:

DQ publications, informatics and analytics experts, managers of established DQ programs, and operational manuals from several mature EHR-based research networks were reviewed to identify potential DQ terms and categories. Two face-to-face stakeholder meetings were used to vet an initial set of DQ terms and definitions that were grouped into an overall conceptual framework. Feedback received from data producers and users was used to construct a draft set of harmonized DQ terms and categories. Multiple rounds of iterative refinement resulted in a set of terms and organizing framework consisting of DQ categories, subcategories, terms, definitions, and examples. The harmonized terminology and logical framework’s inclusiveness was evaluated against ten published DQ terminologies.

Results:

Existing DQ terms were harmonized and organized into a framework by defining three DQ categories: (1) Conformance (2) Completeness and (3) Plausibility and two DQ assessment contexts: (1) Verification and (2) Validation. Conformance and Plausibility categories were further divided into subcategories. Each category and subcategory was defined with respect to whether the data may be verified with organizational data, or validated against an accepted gold standard, depending on proposed context and uses. The coverage of the harmonized DQ terminology was validated by successfully aligning to multiple published DQ terminologies.

Discussion:

Existing DQ concepts, community input, and expert review informed the development of a distinct set of terms, organized into categories and subcategories. The resulting DQ terms successfully encompassed a wide range of disparate DQ terminologies. Operational definitions were developed to provide guidance for implementing DQ assessment procedures. The resulting structure is an inclusive DQ framework for standardizing DQ assessment and reporting. While our analysis focused on the DQ issues often found in EHR data, the new terminology may be applicable to a wide range of electronic health data such as administrative, research, and patient-reported data.

Conclusion:

A consistent, common DQ terminology, organized into a logical framework, is an initial step in enabling data owners and users, patients, and policy makers to evaluate and communicate data quality findings in a well-defined manner with a shared vocabulary. Future work will leverage the framework and terminology to develop reusable data quality assessment and reporting methods.

Role of Pharmacovigilance in India: An overview

Pharmacovigilance (PV) plays a key role in the healthcare system through assessment, monitoring and discovery of interactions amongst drugs and their effects in human. Pharmaceutical and biotechnological medicines are designed to cure, prevent or treat diseases; however, there are also risks particularly adverse drug reactions (ADRs) can cause serious harm to patients. Thus, for safety medication ADRs monitoring required for each medicine throughout its life cycle, during development of drug such as pre-marketing including early stages of drug design, clinical trials, and post-marketing surveillance. PV is concerns with the detection, assessment, understanding and prevention of ADRs. Pharmacogenetics and pharmacogenomics are an indispensable part of the clinical research. Variation in the human genome is a cause of variable response to drugs and susceptibility to diseases are determined, which is important for early drug discovery to PV. Moreover, PV has traditionally involved in mining spontaneous reports submitted to national surveillance systems. The research focus is shifting toward the use of data generated from platforms outside the conventional framework such as electronic medical records, biomedical literature, and patient-reported data in health forums. The emerging trend in PV is to link premarketing data with human safety information observed in the post-marketing phase. The PV system team obtains valuable additional information, building up the scientific data contained in the original report and making it more informative. This necessitates an utmost requirement for effective regulations of the drug approval process and conscious pre and post approval vigilance of the undesired effects, especially in India. Adverse events reported by PV system potentially benefit to the community due to their proximity to both population and public health practitioners, in terms of language and knowledge, enables easy contact with reporters by electronically. Hence, PV helps to the patients get well and to manage optimally or ideally, avoid illness is a collective responsibility of industry, drug regulators, clinicians and other healthcare professionals to enhance their contribution to public health. This review summarized objectives and methodologies used in PV with critical overview of existing PV in India, challenges to overcome and future prospects with respect to Indian context.

Analyzing complex patients' temporal histories: new frontiers in temporal data mining

In recent years, data coming from hospital information systems (HIS) and local healthcare organizations have started to be intensively used for research purposes. This rising amount of available data allows reconstructing the compete histories of the patients, which have a strong temporal component. This chapter introduces the major challenges faced by temporal data mining researchers in an era when huge quantities of complex clinical temporal data are becoming available. The analysis is focused on the peculiar features of this kind of data and describes the methodological and technological aspects that allow managing such complex framework. The chapter shows how heterogeneous data can be processed to derive a homogeneous representation. Starting from this representation, it illustrates different techniques for jointly analyze such kind of data. Finally, the technological strategies that allow creating a common data warehouse to gather data coming from different sources and with different formats are presented.

Variation in outcomes of quality measurement by data source

OBJECTIVE

To evaluate selected Children's Health Insurance Program Reauthorization Act claims-based quality measures using claims data alone, electronic health record (EHR) data alone, and both data sources combined.

METHODS

Our population included pediatric patients from 46 clinics in the OCHIN network of community health centers, who were continuously enrolled in Oregon's public health insurance program during 2010. Within this population, we calculated selected pediatric care quality measures according to the Children's Health Insurance Program Reauthorization Act technical specifications within administrative claims. We then calculated these measures in the same cohort, by using EHR data, by using the technical specifications plus clinical data previously shown to enhance capture of a given measure. We used the k statistic to determine agreement in measurement when using claims versus EHR data. Finally, we measured quality of care delivered to the study population, when using a combined dataset of linked, patient-level administrative claims and EHR data.

RESULTS

When using administrative claims data, 1.0% of children (aged 3-17) had a BMI percentile recorded, compared with 71.9% based on the EHR data (k agreement [k] # 0.01), and 72.0% in the combined dataset. Among children turning 2 in 2010, 20.2% received all recommended immunizations according to the administrative claims data, 17.2% according to the EHR data (k = 0.82), and 21.4% according to the combined dataset.

CONCLUSIONS

Children's care quality measures may not be accurate when assessed using only administrative claims. Adding EHR data to administrative claims data may yield more complete measurement.

Caveats for the use of operational electronic health record data in comparative effectiveness research

The growing amount of data in operational electronic health record systems provides unprecedented opportunity for its reuse for many tasks, including comparative effectiveness research. However, there are many caveats to the use of such data. Electronic health record data from clinical settings may be inaccurate, incomplete, transformed in ways that undermine their meaning, unrecoverable for research, of unknown provenance, of insufficient granularity, and incompatible with research protocols. However, the quantity and real-world nature of these data provide impetus for their use, and we develop a list of caveats to inform would-be users of such data as well as provide an informatics roadmap that aims to insure this opportunity to augment comparative effectiveness research can be best leveraged.

e-Measures: insight into the challenges and opportunities of automating publicly reported quality measures

Using electronic health records (EHR) to automate publicly reported quality measures is receiving increasing attention and is one of the promises of EHR implementation. Kaiser Permanente has fully or partly automated six of 13 the joint commission measure sets. We describe our experience with automation and the resulting time savings: a reduction by approximately 50% of abstractor time required for one measure set alone (surgical care improvement project). However, our experience illustrates the gap between the current and desired states of automated public quality reporting, which has important implications for measure developers, accrediting entities, EHR vendors, public/private payers, and government.

The Analytic Information Warehouse (AIW): a platform for analytics using electronic health record data

OBJECTIVE

To create an analytics platform for specifying and detecting clinical phenotypes and other derived variables in electronic health record (EHR) data for quality improvement investigations.

MATERIALS AND METHODS

We have developed an architecture for an Analytic Information Warehouse (AIW). It supports transforming data represented in different physical schemas into a common data model, specifying derived variables in terms of the common model to enable their reuse, computing derived variables while enforcing invariants and ensuring correctness and consistency of data transformations, long-term curation of derived data, and export of derived data into standard analysis tools. It includes software that implements these features and a computing environment that enables secure high-performance access to and processing of large datasets extracted from EHRs.

RESULTS

We have implemented and deployed the architecture in production locally. The software is available as open source. We have used it as part of hospital operations in a project to reduce rates of hospital readmission within 30days. The project examined the association of over 100 derived variables representing disease and co-morbidity phenotypes with readmissions in 5years of data from our institution's clinical data warehouse and the UHC Clinical Database (CDB). The CDB contains administrative data from over 200 hospitals that are in academic medical centers or affiliated with such centers.

DISCUSSION AND CONCLUSION

A widely available platform for managing and detecting phenotypes in EHR data could accelerate the use of such data in quality improvement and comparative effectiveness studies.

Diabetes and asthma case identification, validation, and representativeness when using electronic health data to construct registries for comparative effectiveness and epidemiologic research

BACKGROUND

Advances in health information technology and widespread use of electronic health data offer new opportunities for development of large scale multisite disease-specific patient registries. Such registries use existing data, can be constructed at relatively low cost, include large numbers of patients, and once created can be used to address many issues with a short time between posing a question and obtaining an answer. Potential applications include comparative effectiveness research, public health surveillance, mapping and improving quality of clinical care, and others.

OBJECTIVE AND DISCUSSION

This paper describes selected conceptual and practical challenges related to development of multisite diabetes and asthma registries, including development of case definitions, validation of case identification methods, variation in electronic health data sources; representativeness of registry populations, including the impact of attrition. Specific challenges are illustrated with data from actual registries.

Clinical research informatics: a conceptual perspective

Clinical research informatics is the rapidly evolving sub-discipline within biomedical informatics that focuses on developing new informatics theories, tools, and solutions to accelerate the full translational continuum: basic research to clinical trials (T1), clinical trials to academic health center practice (T2), diffusion and implementation to community practice (T3), and ‘real world’ outcomes (T4). We present a conceptual model based on an informatics-enabled clinical research workflow, integration across heterogeneous data sources, and core informatics tools and platforms. We use this conceptual model to highlight 18 new articles in the JAMIA special issue on clinical research informatics.

Validity of electronic health record-derived quality measurement for performance monitoring

Background

Since 2007, New York City's primary care information project has assisted over 3000 providers to adopt and use a prevention-oriented electronic health record (EHR). Participating practices were taught to re-adjust their workflows to use the EHR built-in population health monitoring tools, including automated quality measures, patient registries and a clinical decision support system. Practices received a comprehensive suite of technical assistance, which included quality improvement, EHR customization and configuration, privacy and security training, and revenue cycle optimization. These services were aimed at helping providers understand how to use their EHR to track and improve the quality of care delivered to patients.

Materials and Methods

Retrospective electronic chart reviews of 4081 patient records across 57 practices were analyzed to determine the validity of EHR-derived quality measures and documented preventive services.

Results

Results from this study show that workflow and documentation habits have a profound impact on EHR-derived quality measures. Compared with the manual review of electronic charts, EHR-derived measures can undercount practice performance, with a disproportionately negative impact on the number of patients captured as receiving a clinical preventive service or meeting a recommended treatment goal.

Conclusion

This study provides a cautionary note in using EHR-derived measurement for public reporting of provider performance or use for payment.