Reporting Data Quality Assessment Results: Identifying Individual and Organizational Barriers and Solutions

Systematically assessing the quality of dental electronic health record data for an investigation into oral health care disparities

OBJECTIVES

This work describes the process by which the quality of electronic health care data for a public health study was determined. The objectives were to adapt, develop, and implement data quality assessments (DQAs) based on the National Institutes of Health Pragmatic Trials Collaboratory (NIHPTC) data quality framework within the three domains of completeness, accuracy, and consistency, for an investigation into oral health care disparities of a preventive care program.

METHODS

Electronic health record data for eligible children in a dental accountable care organization of 30 offices, in Oregon, were extracted iteratively from January 1, 2014, through March 31, 2022. Baseline eligibility criteria included: children ages 0-18 with a baseline examination, Oregon home address, and either Medicaid or commercial dental benefits at least once between 2014 and 2108. Using the NIHPTC framework as a guide, DQAs were conducted throughout data element identification, extraction, staging, profiling, review, and documentation.

RESULTS

The data set included 91,487 subjects, 11 data tables comprising 75 data variables (columns), with a total of 6,861,525 data elements. Data completeness was 97.2%, the accuracy of EHR data elements in extracts was 100%, and consistency between offices was strong; 29 of 30 offices within 2 standard deviations of the mean (s = 94%).

CONCLUSIONS

The NIHPTC framework proved to be a useful approach, to identify, document, and characterize the dataset. The concepts of completeness, accuracy, and consistency were adapted by the multidisciplinary research team and the overall quality of the data are demonstrated to be of high quality.

Digital Health Data Quality Issues: Systematic Review

BACKGROUND

The promise of digital health is principally dependent on the ability to electronically capture data that can be analyzed to improve decision-making. However, the ability to effectively harness data has proven elusive, largely because of the quality of the data captured. Despite the importance of data quality (DQ), an agreed-upon DQ taxonomy evades literature. When consolidated frameworks are developed, the dimensions are often fragmented, without consideration of the interrelationships among the dimensions or their resultant impact.

OBJECTIVE

The aim of this study was to develop a consolidated digital health DQ dimension and outcome (DQ-DO) framework to provide insights into 3 research questions: What are the dimensions of digital health DQ? How are the dimensions of digital health DQ related? and What are the impacts of digital health DQ?

METHODS

Following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines, a developmental systematic literature review was conducted of peer-reviewed literature focusing on digital health DQ in predominately hospital settings. A total of 227 relevant articles were retrieved and inductively analyzed to identify digital health DQ dimensions and outcomes. The inductive analysis was performed through open coding, constant comparison, and card sorting with subject matter experts to identify digital health DQ dimensions and digital health DQ outcomes. Subsequently, a computer-assisted analysis was performed and verified by DQ experts to identify the interrelationships among the DQ dimensions and relationships between DQ dimensions and outcomes. The analysis resulted in the development of the DQ-DO framework.

RESULTS

The digital health DQ-DO framework consists of 6 dimensions of DQ, namely accessibility, accuracy, completeness, consistency, contextual validity, and currency; interrelationships among the dimensions of digital health DQ, with consistency being the most influential dimension impacting all other digital health DQ dimensions; 5 digital health DQ outcomes, namely clinical, clinician, research-related, business process, and organizational outcomes; and relationships between the digital health DQ dimensions and DQ outcomes, with the consistency and accessibility dimensions impacting all DQ outcomes.

CONCLUSIONS

The DQ-DO framework developed in this study demonstrates the complexity of digital health DQ and the necessity for reducing digital health DQ issues. The framework further provides health care executives with holistic insights into DQ issues and resultant outcomes, which can help them prioritize which DQ-related problems to tackle first.

The IeDEA harmonist data toolkit: A data quality and data sharing solution for a global HIV research consortium

We describe the design, implementation, and impact of a data harmonization, data quality checking, and dynamic report generation application in an international observational HIV research network. The IeDEA Harmonist Data Toolkit is a web-based application written in the open source programming language R, employs the R/Shiny and RMarkdown packages, and leverages the REDCap data collection platform for data model definition and user authentication. The Toolkit performs data quality checks on uploaded datasets, checks for conformance with the network's common data model, displays the results both interactively and in downloadable reports, and stores approved datasets in secure cloud storage for retrieval by the requesting investigator. Including stakeholders and users in the design process was key to the successful adoption of the application. A survey of regional data managers as well as initial usage metrics indicate that the Toolkit saves time and results in improved data quality, with a 61% mean reduction in the number of error records in a dataset. The generalized application design allows the Toolkit to be easily adapted to other research networks.

Developing a Data Quality Standard Primer for Cardiovascular Risk Assessment from Electronic Health Record Data Using the DataGauge Process

The learning health systems aim to support the needs of patients with chronic diseases, which require methods that account for electronic health recorded (EHR) data limitations. EHR data is often used to calculate cardiovascular risk scores. However, it is unclear whether EHR data presents high enough quality to provide accurate estimates. Still, there is currently no open standard available to assess data quality for such applications. We applied the DataGauge process to develop a data quality standard based on expert clinical, analytical and informatics knowledge by conducting four interviews and one focus group that produced 61 individual data quality requirements. These requirements covered all standard data quality dimensions and uncovered 705 quality issues in EHR data for 456 patients. These requirements will be expanded and further validated in future work. Our work initiates the development of open and explicit data quality standards for specific secondary uses of clinical data.

Levels of Trimethylamine N-Oxide Remain Elevated Long Term After Left Ventricular Assist Device and Heart Transplantation and Are Independent From Measures of Inflammation and Gut Dysbiosis

BACKGROUND

Trimethylamine N-oxide (TMAO)-a gut-derived metabolite-is elevated in heart failure (HF) and linked to poor prognosis. We investigated variations in TMAO in HF, left ventricular assist device (LVAD), and heart transplant (HT) and assessed its relation with inflammation, endotoxemia, oxidative stress, and gut dysbiosis.

METHODS

We enrolled 341 patients. TMAO, CRP (C-reactive protein), IL (interleukin)-6, TNF-α (tumor necrosis factor alpha), ET-1 (endothelin-1), adiponectin, lipopolysaccharide, soluble CD14, and isoprostane were measured in 611 blood samples in HF (New York Heart Association class I-IV) and at multiple time points post-LVAD and post-HT. Gut microbiota were assessed via 16S rRNA sequencing among 327 stool samples. Multivariable regression models were used to assess the relationship between TMAO and (1) New York Heart Association class; (2) pre- versus post-LVAD or post-HT; (3) biomarkers of inflammation, endotoxemia, oxidative stress, and microbial diversity.

RESULTS

ln-TMAO was lower among HF New York Heart Association class I (1.23 [95% CI, 0.52-1.94] µM) versus either class II, III, or IV (1.99 [95% CI, 1.68-2.30], 1.97 [95% CI, 1.71-2.24], and 2.09 [95% CI, 1.83-2.34] µM, respectively; all P<0.05). In comparison to class II-IV, ln-TMAO was lower 1 month post-LVAD (1.58 [95% CI, 1.32-1.83] µM) and 1 week and 1 month post-HT (0.97 [95% CI, 0.60-1.35] and 1.36 [95% CI, 1.01-1.70] µM). ln-TMAO levels in long-term LVAD (>6 months: 1.99 [95% CI, 1.76-2.22] µM) and HT (>6 months: 1.86 [95% CI, 1.66-2.05] µM) were not different from symptomatic HF. After multivariable adjustments, TMAO was not associated with biomarkers of inflammation, endotoxemia, oxidative stress, or microbial diversity.

CONCLUSIONS

TMAO levels are increased in symptomatic HF patients and remain elevated long term after LVAD and HT. TMAO levels were independent from measures of inflammation, endotoxemia, oxidative stress, and gut dysbiosis.

Validation process of a high-resolution database in a paediatric intensive care unit-Describing the perpetual patient's validation

RATIONALE

High data quality is essential to ensure the validity of clinical and research inferences based on it. However, these data quality assessments are often missing even though these data are used in daily practice and research.

AIMS AND OBJECTIVES

Our objective was to evaluate the data quality of our high-resolution electronic database (HRDB) implemented in our paediatric intensive care unit (PICU).

METHODS

We conducted a prospective validation study of a HRDB in a 32-bed paediatric medical, surgical, and cardiac PICU in a tertiary care freestanding maternal-child health centre in Canada. All patients admitted to the PICU with at least one vital sign monitored using a cardiorespiratory monitor connected to the central monitoring station.

RESULTS

Between June 2017 and August 2018, data from 295 patient days were recorded from medical devices and 4645 data points were video recorded and compared to the corresponding data collected in the HRDB. Statistical analysis showed an excellent overall correlation (R² = 1), accuracy (100%), agreement (bias = 0, limits of agreement = 0), completeness (2% missing data), and reliability (ICC = 1) between recorded and collected data within clinically significant pre-defined limits of agreement. Divergent points could all be explained.

CONCLUSIONS

This prospective validation of a representative sample showed an excellent overall data quality.

Gut microbiota, endotoxemia, inflammation, and oxidative stress in patients with heart failure, left ventricular assist device, and transplant

BACKGROUND

Gut microbial imbalance may contribute to endotoxemia, inflammation, and oxidative stress in heart failure (HF). Changes occurring in the intestinal microbiota and inflammatory/oxidative milieu during HF progression and following left ventricular assist device (LVAD) or heart transplantation (HT) are unknown. We aimed to investigate variation in gut microbiota and circulating biomarkers of endotoxemia, inflammation, and oxidative stress in patients with HF (New York Heart Association, Class I-IV), LVAD, and HT.

METHODS

We enrolled 452 patients. Biomarkers of endotoxemia (lipopolysaccharide and soluble [sCD14]), inflammation (C-reactive protein, interleukin-6, tumor necrosis factor-α, and endothelin-1 adiponectin), and oxidative stress (isoprostane) were measured in 644 blood samples. A total of 304 stool samples were analyzed using 16S rRNA sequencing.

RESULTS

Gut microbial community measures of alpha diversity were progressively lower across worsening HF class and were similarly reduced in patients with LVAD and HT (p < 0.05). Inflammation and oxidative stress were elevated in patients with Class IV HF vs all other groups (all p < 0.05). Lipopolysaccharide was elevated in patients with Class IV HF (vs Class I-III) as well as in patients with LVAD and HT (p < 0.05). sCD14 was elevated in patients with Class IV HF and LVAD (vs Class I-III, p < 0.05) but not in patients with HT.

CONCLUSIONS

Reduced gut microbial diversity and increased endotoxemia, inflammation, and oxidative stress are present in patients with Class IV HF. Inflammation and oxidative stress are lower among patients with LVAD and HT relative to patients with Class IV HF, whereas reduced gut diversity and endotoxemia persist in LVAD and HT.

A tale of three subspecialties: Diagnosis recording patterns are internally consistent but Specialty-Dependent

BACKGROUND

Structured diagnosis (DX) are crucial for secondary use of electronic health record (EHR) data. However, they are often suboptimally recorded. Our previous work showed initial evidence of variable DX recording patterns in oncology charts even after biopsy records are available.

OBJECTIVE

We verified this finding's internal and external validity. We hypothesized that this recording pattern would be preserved in a larger cohort of patients for the same disease. We also hypothesized that this effect would vary across subspecialties.

METHODS

We extracted DX data from EHRs of patients treated for brain, lung, and pancreatic neoplasms, identified through clinician-led chart reviews. We used statistical methods (i.e., binomial and mixed model regressions) to test our hypotheses.

RESULTS

We found variable recording patterns in brain neoplasm DX (i.e., larger number of distinct DX-OR = 2.2, P < 0.0001, higher descriptive specificity scores-OR = 1.4, P < 0.0001-and much higher entropy after the BX-OR = 3.8 P = 0.004 and OR = 8.0, P < 0.0001), confirming our initial findings. We also found strikingly different patterns for lung and pancreas DX. Although both seemed to have much lower DX sequence entropy after the BX-OR = 0.198, P = 0.015 and OR = 0.099, P = 0.015, respectively compared to OR = 3.8 P = 0.004). We also found statistically significant differences between the brain dataset and both the lung (P < 0.0001) and pancreas (0.009

CONCLUSION

Our results suggest that disease-specific DX entry patterns exist and are established differently by clinical subspecialty. These differences should be accounted for during clinical data reuse and data quality assessments but also during EHR entry system design to maximize accurate, precise and consistent data entry likelihood.

Collapse

Key Words

• clinical data management
• data quality
• electronic health records
• learning healthcare system
• secondary use of clinical data

Collapse

MESH Headings Collapse

Grants

• K12 GM102773 NIGMS NIH HHS
• UL1 TR001420 NCATS NIH HHS

Collapse

Affiliation(s)

Jose-Franck Diaz-Garelli Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Roy Strowd Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Tamjeed Ahmed Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Brian J Wells Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Rebecca Merrill Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Javier Laurini Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Boris Pasche Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA
Umit Topaloglu Wake Forest Baptist Medical Center, Winston Salem, North Carolina, USA

Collapse

Towards a content agnostic computable knowledge repository for data quality assessment

BACKGROUND AND OBJECTIVE

In recent years, several data quality conceptual frameworks have been proposed across the Data Quality and Information Quality domains towards assessment of quality of data. These frameworks are diverse, varying from simple lists of concepts to complex ontological and taxonomical representations of data quality concepts. The goal of this study is to design, develop and implement a platform agnostic computable data quality knowledge repository for data quality assessments.

METHODS

We identified computable data quality concepts by performing a comprehensive literature review of articles indexed in three major bibliographic data sources. From this corpus, we extracted data quality concepts, their definitions, applicable measures, their computability and identified conceptual relationships. We used these relationships to design and develop a data quality meta-model and implemented it in a quality knowledge repository.

RESULTS

We identified three primitives for programmatically performing data quality assessments: data quality concept, its definition, its measure or rule for data quality assessment, and their associations. We modeled a computable data quality meta-data repository and extended this framework to adapt, store, retrieve and automate assessment of other existing data quality assessment models.

CONCLUSION

We identified research gaps in data quality literature towards automating data quality assessments methods. In this process, we designed, developed and implemented a computable data quality knowledge repository for assessing quality and characterizing data in health data repositories. We leverage this knowledge repository in a service-oriented architecture to perform scalable and reproducible framework for data quality assessments in disparate biomedical data sources.

Lost in Translation: Diagnosis Records Show More Inaccuracies After Biopsy in Oncology Care EHRs

The use of diagnosis (DX) data is crucial to secondary use of electronic health record (EHR) data, yet accessible structured DX data often lack in accuracy. DX descriptions associated with structured DX codes vary even after recording biopsy results; this may indicate poor data quality. We hypothesized that biopsy reports in cancer care charts do not improve intrinsic DX data quality. We analyzed DX data for a manually well-annotated cohort of patients with brain neoplasms. We built statistical models to predict the number of fully-accurate (i.e., correct neoplasm type and anatomical location) and inaccurate DX (i.e. type or location contradicts cohort data) descriptions. We found some evidence of statistically larger numbers of fully-accurate (RR=3.07, p=0.030) but stronger evidence of much larger numbers of inaccurate DX (RR=12.3, p=0.001 and RR=19.6, p<0.0001) after biopsy result recording. Still, 65.9% of all DX records were neither fully-accurate nor fully-inaccurate. These results suggest EHRs must be modified to support more reliable DX data recording and secondary use of EHR data.

Clinical data quality: a data life cycle perspective

Clinical data is the staple of modern learning health systems. It promises to accelerate biomedical discovery and improves the efficiency of clinical and translational research but is also fraught with significant data quality issues. This paper aims to provide a life cycle perspective of clinical data quality issues along with recommendations for establishing appropriate expectations for research based on real-world clinical data and best practices for reusing clinical data as a secondary data source.

Adapting data management education to support clinical research projects in an academic medical center

Background

Librarians and researchers alike have long identified research data management (RDM) training as a need in biomedical research. Despite the wealth of libraries offering RDM education to their communities, clinical research is an area that has not been targeted. Clinical RDM (CRDM) is seen by its community as an essential part of the research process where established guidelines exist, yet educational initiatives in this area are unknown.

Case Presentation

Leveraging my academic library's experience supporting CRDM through informationist grants and REDCap training in our medical center, I developed a 1.5 hour CRDM workshop. This workshop was designed to use established CRDM guidelines in clinical research and address common questions asked by our community through the library's existing data support program. The workshop was offered to the entire medical center 4 times between November 2017 and July 2018. This case study describes the development, implementation, and evaluation of this workshop.

Conclusions

The 4 workshops were well attended and well received by the medical center community, with 99% stating that they would recommend the class to others and 98% stating that they would use what they learned in their work. Attendees also articulated how they would implement the main competencies they learned from the workshop into their work. For the library, the effort to support CRDM has led to the coordination of a larger institutional collaborative training series to educate researchers on best practices with data, as well as the formation of institution-wide policy groups to address researcher challenges with CRDM, data transfer, and data sharing.