A basic model for assessing primary health care electronic medical record data quality

Transforming Primary Care Data Into the Observational Medical Outcomes Partnership Common Data Model: Development and Usability Study

Background

Patient-monitoring software generates a large amount of data that can be reused for clinical audits and scientific research. The Observational Health Data Sciences and Informatics (OHDSI) consortium developed the Observational Medical Outcomes Partnership (OMOP) Common Data Model (CDM) to standardize electronic health record data and promote large-scale observational and longitudinal research.

Objective

This study aimed to transform primary care data into the OMOP CDM format.

Methods

We extracted primary care data from electronic health records at a multidisciplinary health center in Wattrelos, France. We performed structural mapping between the design of our local primary care database and the OMOP CDM tables and fields. Local French vocabularies concepts were mapped to OHDSI standard vocabularies. To validate the implementation of primary care data into the OMOP CDM format, we applied a set of queries. A practical application was achieved through the development of a dashboard.

Results

Data from 18,395 patients were implemented into the OMOP CDM, corresponding to 592,226 consultations over a period of 20 years. A total of 18 OMOP CDM tables were implemented. A total of 17 local vocabularies were identified as being related to primary care and corresponded to patient characteristics (sex, location, year of birth, and race), units of measurement, biometric measures, laboratory test results, medical histories, and drug prescriptions. During semantic mapping, 10,221 primary care concepts were mapped to standard OHDSI concepts. Five queries were used to validate the OMOP CDM by comparing the results obtained after the completion of the transformations with the results obtained in the source software. Lastly, a prototype dashboard was developed to visualize the activity of the health center, the laboratory test results, and the drug prescription data.

Conclusions

Primary care data from a French health care facility have been implemented into the OMOP CDM format. Data concerning demographics, units, measurements, and primary care consultation steps were already available in OHDSI vocabularies. Laboratory test results and drug prescription data were mapped to available vocabularies and structured in the final model. A dashboard application provided health care professionals with feedback on their practice.

Systematically assessing the quality of dental electronic health record data for an investigation into oral health care disparities

OBJECTIVES

This work describes the process by which the quality of electronic health care data for a public health study was determined. The objectives were to adapt, develop, and implement data quality assessments (DQAs) based on the National Institutes of Health Pragmatic Trials Collaboratory (NIHPTC) data quality framework within the three domains of completeness, accuracy, and consistency, for an investigation into oral health care disparities of a preventive care program.

METHODS

Electronic health record data for eligible children in a dental accountable care organization of 30 offices, in Oregon, were extracted iteratively from January 1, 2014, through March 31, 2022. Baseline eligibility criteria included: children ages 0-18 with a baseline examination, Oregon home address, and either Medicaid or commercial dental benefits at least once between 2014 and 2108. Using the NIHPTC framework as a guide, DQAs were conducted throughout data element identification, extraction, staging, profiling, review, and documentation.

RESULTS

The data set included 91,487 subjects, 11 data tables comprising 75 data variables (columns), with a total of 6,861,525 data elements. Data completeness was 97.2%, the accuracy of EHR data elements in extracts was 100%, and consistency between offices was strong; 29 of 30 offices within 2 standard deviations of the mean (s = 94%).

CONCLUSIONS

The NIHPTC framework proved to be a useful approach, to identify, document, and characterize the dataset. The concepts of completeness, accuracy, and consistency were adapted by the multidisciplinary research team and the overall quality of the data are demonstrated to be of high quality.

From Scientific Research to Practical Implementations: Applications to Improve Data Quality in Child Welfare

Child welfare decisions have life-impacting consequences which, often times, are underpinned by limited or inadequate data and poor quality. Though research of data quality has gained popularity and made advancements in various practical areas, it has not made significant inroads for child welfare fields or data systems. Poor data quality can hinder service decision-making, impacting child behavioral health and well-being as well as increasing unnecessary expenditure of time and resources. Poor data quality can also undermine the validity of research and slow policymaking processes. The purpose of this commentary is to summarize the data quality research base in other fields, describe obstacles and uniqueness to improve data quality in child welfare, and propose necessary steps to scientific research and practical implementation that enables researchers and practitioners to improve the quality of child welfare services based on the enhanced quality of data.

Electronic Medical Record Data Missingness and Interruption in Antiretroviral Therapy Among Adults and Children Living With HIV in Haiti: Retrospective Longitudinal Study

Background

Children (aged 0-14 years) living with HIV often experience lower rates of HIV diagnosis, treatment, and viral load suppression. In Haiti, only 63% of children living with HIV know their HIV status (compared to 85% overall), 63% are on treatment (compared to 85% overall), and 48% are virally suppressed (compared to 73% overall). Electronic medical records (EMRs) can improve HIV care and patient outcomes, but these benefits are largely dependent on providers having access to quality and nonmissing data.

Objective

We sought to understand the associations between EMR data missingness and interruption in antiretroviral therapy treatment by age group (pediatric vs adult).

Methods

We assessed associations between patient intake record data missingness and interruption in treatment (IIT) status at 6 and 12 months post antiretroviral therapy initiation using patient-level data drawn from iSanté, the most widely used EMR in Haiti. Missingness was assessed for tuberculosis diagnosis, World Health Organization HIV stage, and weight using a composite score indicator (ie, the number of indicators of interest missing). Risk ratios were estimated using marginal parameters from multilevel modified Poisson models with robust error variances and random intercepts for the facility to account for clustering.

Results

Data were drawn from 50 facilities and comprised 31,457 patient records from people living with HIV, of which 1306 (4.2%) were pediatric cases. Pediatric patients were more likely than adult patients to experience IIT (n=431, 33% vs n=7477, 23.4% at 6 months; P<.001). Additionally, pediatric patient records had higher data missingness, with 581 (44.5%) pediatric records missing at least 1 indicator of interest, compared to 7812 (25.9%) adult records (P<.001). Among pediatric patients, each additional indicator missing was associated with a 1.34 times greater likelihood of experiencing IIT at 6 months (95% CI 1.08-1.66; P=.008) and 1.24 times greater likelihood of experiencing IIT at 12 months (95% CI 1.05-1.46; P=.01). These relationships were not statistically significant for adult patients. Compared to pediatric patients with 0 missing indicators, pediatric patients with 1, 2, or 3 missing indicators were 1.59 (95% CI 1.26-2.01; P<.001), 1.74 (95% CI 1.02-2.97; P=.04), and 2.25 (95% CI 1.43-3.56; P=.001) times more likely to experience IIT at 6 months, respectively. Among adult patients, compared to patients with 0 indicators missing, having all 3 indicators missing was associated with being 1.32 times more likely to experience IIT at 6 months (95% CI 1.03-1.70; P=.03), while there was no association with IIT status for other levels of missingness.

Conclusions

These findings suggest that both EMR data quality and quality of care are lower for children living with HIV in Haiti. This underscores the need for further research into the mechanisms by which EMR data quality impacts the quality of care and patient outcomes among this population. Efforts to improve both EMR data quality and quality of care should consider prioritizing pediatric patients.

Understanding data provenance when using electronic medical records for research: Lessons learned from the Deliver Primary Healthcare Information (DELPHI) database

Introduction

We set out to assess the impact of Choosing Wisely Canada recommendations (2014) on reducing unnecessary health investigations and interventions in primary care across Southwestern Ontario.

Methods

We used the Deliver Primary Healthcare Information (DELPHI) database, which stores deidentified electronic medical records (EMR) of nearly 65,000 primary care patients across Southwestern Ontario. When conducting research using EMR data, data provenance (i.e., how the data came to be) should first be established. We first considered DELPHI data provenance in relation to longitudinal analyses, flagging a change in EMR software that occurred during 2012 and 2013. We attempted to link records between EMR databases produced by different software using probabilistic linkage and inspected 10 years of data in the DELPHI database (2009 to 2019) for data quality issues, including comparability over time.

Results

We encountered several issues resulting from this change in EMR software. These included limited linkage of records between software without a common identifier; data migration issues that distorted procedure dates; and unusual changes in laboratory test and medication prescription volumes.

Conclusion

This study reinforces the necessity of assessing data provenance and quality for new research projects. By understanding data provenance, we can anticipate related data quality issues such as changes in EMR data over time-which represent a growing concern as longitudinal data analyses increase in feasibility and popularity.

Electronic health record data quality assessment and tools: a systematic review

OBJECTIVE

We extended a 2013 literature review on electronic health record (EHR) data quality assessment approaches and tools to determine recent improvements or changes in EHR data quality assessment methodologies.

MATERIALS AND METHODS

We completed a systematic review of PubMed articles from 2013 to April 2023 that discussed the quality assessment of EHR data. We screened and reviewed papers for the dimensions and methods defined in the original 2013 manuscript. We categorized papers as data quality outcomes of interest, tools, or opinion pieces. We abstracted and defined additional themes and methods though an iterative review process.

RESULTS

We included 103 papers in the review, of which 73 were data quality outcomes of interest papers, 22 were tools, and 8 were opinion pieces. The most common dimension of data quality assessed was completeness, followed by correctness, concordance, plausibility, and currency. We abstracted conformance and bias as 2 additional dimensions of data quality and structural agreement as an additional methodology.

DISCUSSION

There has been an increase in EHR data quality assessment publications since the original 2013 review. Consistent dimensions of EHR data quality continue to be assessed across applications. Despite consistent patterns of assessment, there still does not exist a standard approach for assessing EHR data quality.

CONCLUSION

Guidelines are needed for EHR data quality assessment to improve the efficiency, transparency, comparability, and interoperability of data quality assessment. These guidelines must be both scalable and flexible. Automation could be helpful in generalizing this process.

Automating Electronic Health Record Data Quality Assessment

Information systems such as Electronic Health Record (EHR) systems are susceptible to data quality (DQ) issues. Given the growing importance of EHR data, there is an increasing demand for strategies and tools to help ensure that available data are fit for use. However, developing reliable data quality assessment (DQA) tools necessary for guiding and evaluating improvement efforts has remained a fundamental challenge. This review examines the state of research on operationalising EHR DQA, mainly automated tooling, and highlights necessary considerations for future implementations. We reviewed 1841 articles from PubMed, Web of Science, and Scopus published between 2011 and 2021. 23 DQA programs deployed in real-world settings to assess EHR data quality (n = 14), and a few experimental prototypes (n = 9), were identified. Many of these programs investigate completeness (n = 15) and value conformance (n = 12) quality dimensions and are backed by knowledge items gathered from domain experts (n = 9), literature reviews and existing DQ measurements (n = 3). A few DQA programs also explore the feasibility of using data-driven techniques to assess EHR data quality automatically. Overall, the automation of EHR DQA is gaining traction, but current efforts are fragmented and not backed by relevant theory. Existing programs also vary in scope, type of data supported, and how measurements are sourced. There is a need to standardise programs for assessing EHR data quality, as current evidence suggests their quality may be unknown.

Documenting cannabis use in primary care: a descriptive cross-sectional study using electronic medical record data in Alberta, Canada

OBJECTIVE

Documenting cannabis use is important for patient care, but no formal requirements for consistent reporting exist in primary care. The objective of this study was to understand how cannabis use is documented in primary care electronic medical record (EMR) data.

RESULTS

This was a cross-sectional study using de-identified EMR data from over 398,000 patients and 333 primary care providers in Alberta, Canada. An automated pattern-matching algorithm was developed to identify text and ICD-9 diagnostic codes indicating cannabis use in the EMR. There was a total of 11,724 records indicating cannabis use from 4652 patients, representing approximately 1.2% of the patient sample. Commonly used terms and ICD-9 codes included cannabis, marijuana/marihuana, THC, 304.3 and 305.2. Nabilone was the most frequently prescribed cannabinoid medication. Slightly more males and those with a chronic condition had cannabis use recorded more often. Overall, very few patients have cannabis use recorded in primary care EMR data and this is not captured in a systematic way. We propose several strategies to improve the documentation of cannabis use to facilitate more effective clinical care, research, and surveillance.

Data Quality in Health Care: Main Concepts and Assessment Methodologies

INTRODUCTION

 In the health care environment, a huge volume of data is produced on a daily basis. However, the processes of collecting, storing, sharing, analyzing, and reporting health data usually face with numerous challenges that lead to producing incomplete, inaccurate, and untimely data. As a result, data quality issues have received more attention than before.

OBJECTIVE

 The purpose of this article is to provide an insight into the data quality definitions, dimensions, and assessment methodologies.

METHODS

 In this article, a scoping literature review approach was used to describe and summarize the main concepts related to data quality and data quality assessment methodologies. Search terms were selected to find the relevant articles published between January 1, 2012 and September 31, 2022. The retrieved articles were then reviewed and the results were reported narratively.

RESULTS

 In total, 23 papers were included in the study. According to the results, data quality dimensions were various and different methodologies were used to assess them. Most studies used quantitative methods to measure data quality dimensions either in paper-based or computer-based medical records. Only two studies investigated respondents' opinions about data quality.

CONCLUSION

 In health care, high-quality data not only are important for patient care, but also are vital for improving quality of health care services and better decision making. Therefore, using technical and nontechnical solutions as well as constant assessment and supervision is suggested to improve data quality.

Describing a complex primary health care population to support future decision support initiatives

Introduction

Developing decision support tools using data from a health care organization, to support care within that organization, is a promising paradigm to improve care delivery and population health. Descriptive epidemiology may be a valuable supplement to stakeholder input towards selection of potential initiatives and to inform methodological decisions throughout tool development. We additionally propose that to properly characterize complex populations in large-scale descriptive studies, both simple statistical and machine learning techniques can be useful.

Objective

To describe sociodemographic, clinical, and health care use characteristics of primary care clients served by the Alliance for Healthier Communities, which provides team-based primary health care through Community Health Centres (CHCs) across Ontario, Canada.

Methods

We used electronic health record data from adult ongoing primary care clients served by CHCs in 2009-2019. We performed traditional table-based summaries for each characteristic; and applied three unsupervised learning techniques to explore patterns of common condition co-occurrence, care provider teams, and care frequency.

Results

There were 221,047 eligible clients. Sociodemographics: We described 13 characteristics, stratified by CHC type and client multimorbidity status. Clinical characteristics: Eleven-year prevalence of 24 investigated conditions ranged from 1% (Hepatitis C) to 63% (chronic musculoskeletal problem) with non-uniform risk across the care history; multimorbidity was common (81%) with variable co-occurrence patterns. Health care use characteristics: Most care was provided by physician and nursing providers, with heterogeneous combinations of other provider types. A subset of clients had many issues addressed within single-visits and there was within- and between-client variability in care frequency. In addition to substantive findings, we discuss methodological considerations for future decision support initiatives.

Conclusions

We demonstrated the use of methods from statistics and machine learning, applied with an epidemiological lens, to provide an overview of a complex primary care population and lay a foundation for stakeholder engagement and decision support tool development.

Lifting Hospital Electronic Health Record Data Treasures: Challenges and Opportunities

Electronic health records (EHRs) have been successfully used in data science and machine learning projects. However, most of these data are collected for clinical use rather than for retrospective analysis. This means that researchers typically face many different issues when attempting to access and prepare the data for secondary use. We aimed to investigate how raw EHRs can be accessed and prepared in retrospective data science projects in a disciplined, effective, and efficient way. We report our experience and findings from a large-scale data science project analyzing routinely acquired retrospective data from the Kepler University Hospital in Linz, Austria. The project involved data collection from more than 150,000 patients over a period of 10 years. It included diverse data modalities, such as static demographic data, irregularly acquired laboratory test results, regularly sampled vital signs, and high-frequency physiological waveform signals. Raw medical data can be corrupted in many unexpected ways that demand thorough manual inspection and highly individualized data cleaning solutions. We present a general data preparation workflow, which was shaped in the course of our project and consists of the following 7 steps: obtain a rough overview of the available EHR data, define clinically meaningful labels for supervised learning, extract relevant data from the hospital’s data warehouses, match data extracted from different sources, deidentify them, detect errors and inconsistencies therein through a careful exploratory analysis, and implement a suitable data processing pipeline in actual code. Only few of the data preparation issues encountered in our project were addressed by generic medical data preprocessing tools that have been proposed recently. Instead, highly individualized solutions for the specific data used in one’s own research seem inevitable. We believe that the proposed workflow can serve as a guidance for practitioners, helping them to identify and address potential problems early and avoid some common pitfalls.

Database quality assessment in research in paramedicine: a scoping review protocol

INTRODUCTION

The paramedic practice environment presents unique challenges to data documentation and access, as well as linkage to other parts of the healthcare system. Variable or unknown data quality can influence the validity of research in paramedicine. A number of database quality assessment (DQA) frameworks have been developed and used to evaluate data quality in other areas of healthcare. The extent these or other DQA practices have been applied to paramedic research is not known. Accordingly, this scoping review aims to describe the range, extent and nature of DQA practices within research in paramedicine.

METHODS AND ANALYSIS

This scoping review will follow established methods for the conduct (Johanna Briggs Institute; Arksey and O'Malley) and reporting (Preferred Reporting Items in Systematic Reviews and Meta-Analyses extension for scoping reviews) of scoping reviews. In consultation with a professional librarian, a search strategy was developed representing the applicable population, concept and context. This strategy will be applied to MEDLINE (National Library of Medicine), Embase (Elsevier), Scopus (Elsevier) and CINAHL (EBSCO) to identify studies published from 2011 through 2021 that assess paramedic data quality as a stated goal. Studies will be included if they report quantitative results of DQA using data that relate primarily to the paramedic practice environment. Protocols, commentaries, case studies, interviews, simulations and experimental data-processing techniques will be excluded. No restrictions will be placed on language. Study selection will be performed by two reviewers, with a third available to resolve conflicts. Data will be extracted from included studies using a data-charting form piloted and iteratively revised based on studies known to be relevant. Results will be summarised in a chart of study characteristics, DQA-specific outcomes and key findings.

ETHICS AND DISSEMINATION

Ethical approval is not required. Results will be submitted to relevant conferences and peer-reviewed journals.

TRIAL REGISTRATION

10.17605/OSF.IO/Z287T.

Quality Assessment of Services in Primary Healthcare in Iran: A Systematic Review and Meta-analysis

Background

Primary healthcare (PHC) plays an important role in achieving universal health coverage (UHC). The SERVQUAL instrument is the tool for evaluating the quality of services in the health sector. The main purpose of this study is to evaluate the quality of services provided in PHC in Iran using the SERVQUAL instrument.

Materials and Method

We searched eight databases from January 2000 to September 2021. We analyzed the mean of various SERVQUAL instrument items using the DerSimonian-Laird approach via a random model with 95% confidence interval. Also, we used I2 to evaluate the heterogeneity of the studies.

Results

Finally, 17 studies were chosen for analysis in the present study. There were 8,767 study participants, out of which 8,237 were female and 530 were male. The mean dimensions of perception were as follows: Tangibles = 3.71, reliability = 4, responsiveness = 3.79, assurance = 3.83, and empathy = 3.86. For the expectation, the mean dimension were: Tangibles = 4.46, reliability = 4.46, responsiveness = 4.36, assurance = 4.36, and empathy = 4.36 respectively. The total gap quality between perception and expectation was -0.53.

Conclusion

All dimensions of quality based on SERVQUAL were negative, and the quality of service in PHC is not satisfactory. Therefore, policymakers must adopt serious and effective programs to improve services in this area. We also recommend that quality management of services in PHC in Iran should move toward comprehensive optimization in all areas, and quality in this area should be a priority.

Developing a systematic approach to assessing data quality in secondary use of clinical data based on intended use

INTRODUCTION

Secondary use of electronic health record (EHR) data for research requires that the data are fit for use. Data quality (DQ) frameworks have traditionally focused on structural conformance and completeness of clinical data extracted from source systems. In this paper, we propose a framework for evaluating semantic DQ that will allow researchers to evaluate fitness for use prior to analyses.

METHODS

We reviewed current DQ literature, as well as experience from recent multisite network studies, and identified gaps in the literature and current practice. Derived principles were used to construct the conceptual framework with attention to both analytic fitness and informatics practice.

RESULTS

We developed a systematic framework that guides researchers in assessing whether a data source is fit for use for their intended study or project. It combines tools for evaluating clinical context with DQ principles, as well as factoring in the characteristics of the data source, in order to develop semantic DQ checks.

CONCLUSIONS

Our framework provides a systematic process for DQ development. Further work is needed to codify practices and metadata around both structural and semantic data quality.

Improved preventive care clinical decision-making efficiency: leveraging a point-of-care clinical decision support system

Background

Electronic medical records are widely used in family practices across Canada and can improve health outcomes. However, recent reports indicate that physicians using electronic medical records work longer and have less direct patient contact which may contribute to burnout. Therefore, new and innovative digital tools are essential to reduce physician workloads and improve patient-physician interaction to address physician burnout. The objective of this study was to assess the efficiency and accuracy of clinical decision-making when using a new preventive care point-of-care clinical decision support system (CDSS). An estimate of the potential annual time savings was also determined. This study also assessed physician reported perceived usefulness and ease of use of the CDSS.

Methods

Quantitative and qualitative data were collected during this study. Each participant evaluated two simulated patient charts and identified which preventive care metrics were due. The participants recorded their decisions and the time required to assess each chart. Participants then completed a Technology Acceptance Model survey regarding the perceived usefulness and ease of use of the CDSS, which included qualitative feedback. The amount of time saved was determined and participants’ clinical decision-making accuracy was scored against current Canadian preventive care guidelines. The number of preventive care specific visits completed per year was determined using clinic billing data.

Results

The preventive care CDSS saved an average of 195.7 s of chart review time (249.5 s vs 445.2 s; P < 0.001). A total of 1520 preventive visits were performed at Primrose and Bruyère Family Medicine Centres. Extrapolated across the organization, implementation of the new tool could save 82.6 h per year. Decision-making accuracy was not affected by the new tool (78.4% vs 80.9%, P > 0.05). Participants rated the perceived ease of use and usefulness to be very high.

Conclusions

New digital tools may reduce providers’ workload without impacting clinical decision-making accuracy. Participants indicated that the preventive care CDSS was useful and easy to use. Further software development and clinical studies are required to further improve and characterize the effect this new CDSS has when implemented in clinical practice.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12911-021-01675-8.

An automated data cleaning method for Electronic Health Records by incorporating clinical knowledge

BACKGROUND

The use of Electronic Health Records (EHR) data in clinical research is incredibly increasing, but the abundancy of data resources raises the challenge of data cleaning. It can save time if the data cleaning can be done automatically. In addition, the automated data cleaning tools for data in other domains often process all variables uniformly, meaning that they cannot serve well for clinical data, as there is variable-specific information that needs to be considered. This paper proposes an automated data cleaning method for EHR data with clinical knowledge taken into consideration.

METHODS

We used EHR data collected from primary care in Flanders, Belgium during 1994-2015. We constructed a Clinical Knowledge Database to store all the variable-specific information that is necessary for data cleaning. We applied Fuzzy search to automatically detect and replace the wrongly spelled units, and performed the unit conversion following the variable-specific conversion formula. Then the numeric values were corrected and outliers were detected considering the clinical knowledge. In total, 52 clinical variables were cleaned, and the percentage of missing values (completeness) and percentage of values within the normal range (correctness) before and after the cleaning process were compared.

RESULTS

All variables were 100% complete before data cleaning. 42 variables had a drop of less than 1% in the percentage of missing values and 9 variables declined by 1-10%. Only 1 variable experienced large decline in completeness (13.36%). All variables had more than 50% values within the normal range after cleaning, of which 43 variables had a percentage higher than 70%.

CONCLUSIONS

We propose a general method for clinical variables, which achieves high automation and is capable to deal with large-scale data. This method largely improved the efficiency to clean the data and removed the technical barriers for non-technical people.

Joint Imaging Platform for Federated Clinical Data Analytics

PURPOSE

Image analysis is one of the most promising applications of artificial intelligence (AI) in health care, potentially improving prediction, diagnosis, and treatment of diseases. Although scientific advances in this area critically depend on the accessibility of large-volume and high-quality data, sharing data between institutions faces various ethical and legal constraints as well as organizational and technical obstacles.

METHODS

The Joint Imaging Platform (JIP) of the German Cancer Consortium (DKTK) addresses these issues by providing federated data analysis technology in a secure and compliant way. Using the JIP, medical image data remain in the originator institutions, but analysis and AI algorithms are shared and jointly used. Common standards and interfaces to local systems ensure permanent data sovereignty of participating institutions.

RESULTS

The JIP is established in the radiology and nuclear medicine departments of 10 university hospitals in Germany (DKTK partner sites). In multiple complementary use cases, we show that the platform fulfills all relevant requirements to serve as a foundation for multicenter medical imaging trials and research on large cohorts, including the harmonization and integration of data, interactive analysis, automatic analysis, federated machine learning, and extensibility and maintenance processes, which are elementary for the sustainability of such a platform.

CONCLUSION

The results demonstrate the feasibility of using the JIP as a federated data analytics platform in heterogeneous clinical information technology and software landscapes, solving an important bottleneck for the application of AI to large-scale clinical imaging data.

Assessing the quality of clinical and administrative data extracted from hospitals: the General Medicine Inpatient Initiative (GEMINI) experience

OBJECTIVE

Large clinical databases are increasingly used for research and quality improvement. We describe an approach to data quality assessment from the General Medicine Inpatient Initiative (GEMINI), which collects and standardizes administrative and clinical data from hospitals.

METHODS

The GEMINI database contained 245 559 patient admissions at 7 hospitals in Ontario, Canada from 2010 to 2017. We performed 7 computational data quality checks and iteratively re-extracted data from hospitals to correct problems. Thereafter, GEMINI data were compared to data that were manually abstracted from the hospital's electronic medical record for 23 419 selected data points on a sample of 7488 patients.

RESULTS

Computational checks flagged 103 potential data quality issues, which were either corrected or documented to inform future analysis. For example, we identified the inclusion of canceled radiology tests, a time shift of transfusion data, and mistakenly processing the chemical symbol for sodium ("Na") as a missing value. Manual validation identified 1 important data quality issue that was not detected by computational checks: transfusion dates and times at 1 site were unreliable. Apart from that single issue, across all data tables, GEMINI data had high overall accuracy (ranging from 98%-100%), sensitivity (95%-100%), specificity (99%-100%), positive predictive value (93%-100%), and negative predictive value (99%-100%) compared to the gold standard.

DISCUSSION AND CONCLUSION

Computational data quality checks with iterative re-extraction facilitated reliable data collection from hospitals but missed 1 critical quality issue. Combining computational and manual approaches may be optimal for assessing the quality of large multisite clinical databases.

The Adoption of Electronic Health Records in Primary Healthcare Settings

Little is known about the adoption rate of electronic health records in primary healthcare settings. This study aimed to estimate this adoption rate in Jordan, using a national survey with a descriptive cross-sectional design. The first step was to review the annual report of the Jordanian Ministry of Health as a basis for identifying the sample, which are primary healthcare settings. Then, Electronic Health Solutions, the company that vends electronic health record systems in Jordan, was used to determine which primary healthcare settings were using these electronic record systems. The final task was to determine which functionalities of the system were being used in these settings, and for this, a telephone survey of key personnel was conducted. It was found that 21.6% of the primary healthcare settings were using electronic health records, while the other 78.4% were still relying on paper records. The results also showed that the adoption rate of electronic health records was significantly associated with the type of the setting (P < .001), its size (P < .001), location (P < .001), and region (P = .04). As a conclusion, the adoption rate in Jordan is still in its infancy compared with those in developed countries. Policy and decision makers should therefore be focused on minimizing any challenges or obstacles to such adoption.

Assessing the practice of data quality evaluation in a national clinical data research network through a systematic scoping review in the era of real-world data

Objective

To synthesize data quality (DQ) dimensions and assessment methods of real-world data, especially electronic health records, through a systematic scoping review and to assess the practice of DQ assessment in the national Patient-centered Clinical Research Network (PCORnet).

Materials and Methods

We started with 3 widely cited DQ literature—2 reviews from Chan et al (2010) and Weiskopf et al (2013a) and 1 DQ framework from Kahn et al (2016)—and expanded our review systematically to cover relevant articles published up to February 2020. We extracted DQ dimensions and assessment methods from these studies, mapped their relationships, and organized a synthesized summarization of existing DQ dimensions and assessment methods. We reviewed the data checks employed by the PCORnet and mapped them to the synthesized DQ dimensions and methods.

Results

We analyzed a total of 3 reviews, 20 DQ frameworks, and 226 DQ studies and extracted 14 DQ dimensions and 10 assessment methods. We found that completeness, concordance, and correctness/accuracy were commonly assessed. Element presence, validity check, and conformance were commonly used DQ assessment methods and were the main focuses of the PCORnet data checks.

Discussion

Definitions of DQ dimensions and methods were not consistent in the literature, and the DQ assessment practice was not evenly distributed (eg, usability and ease-of-use were rarely discussed). Challenges in DQ assessments, given the complex and heterogeneous nature of real-world data, exist.

Conclusion

The practice of DQ assessment is still limited in scope. Future work is warranted to generate understandable, executable, and reusable DQ measures.

A data quality assessment to inform hypertension surveillance using primary care electronic medical record data from Alberta, Canada

BACKGROUND

Hypertension is a common chronic condition affecting nearly a quarter of Canadians. Hypertension surveillance in Canada typically relies on administrative data and/or national surveys. Routinely-captured data from primary care electronic medical records (EMRs) are a complementary source for chronic disease surveillance, with longitudinal patient-level details such as sociodemographics, blood pressure, weight, prescribed medications, and behavioural risk factors. As EMR data are generated from patient care and administrative tasks, assessing data quality is essential before using for secondary purposes. This study evaluated the quality of primary care EMR data from one province in Canada within the context of hypertension surveillance.

METHODS

We conducted a cross-sectional, descriptive study using primary care EMR data collected by two practice-based research networks in Alberta, Canada. There were 48,377 adults identified with hypertension from 53 clinics as of June 2018. Summary statistics were used to examine the quality of data elements considered relevant for hypertension surveillance.

RESULTS

Patient year of birth and sex were complete, but other sociodemographic information (ethnicity, occupation, education) was largely incomplete and highly variable. Height, weight, body mass index and blood pressure were complete for most patients (over 90%), but a small proportion of outlying values indicate data inaccuracies were present. Most patients had a relevant laboratory test present (e.g. blood glucose/glycated hemoglobin, lipid profile), though a very small proportion of values were outside a biologically plausible range. Details of prescribed antihypertensive medication, such as start date, strength, dose, frequency, were mostly complete. Nearly 80% of patients had a smoking status recorded, though only 66% had useful information (i.e. categorized as current, past, or never), and less than half had their alcohol use described; information related to amount, frequency or duration was not available.

CONCLUSIONS

Blood pressure and prescribed medications in primary care EMR data demonstrated good completeness and plausibility, and contribute valuable information for hypertension epidemiology and surveillance. The use of other clinical, laboratory, and sociodemographic variables should be used carefully due to variable completeness and suspected data errors. Additional strategies to improve these data at the point of entry and after data extraction (e.g. statistical methods) are required.

Development of comparable algorithms to measure primary care indicators using administrative health data across three Canadian provinces

INTRODUCTION

Performance measurement has been recognized as key to transforming primary care (PC). Yet, performance reporting in PC lags behind even though high-performing PC is foundational to an effective and efficient health care system.

OBJECTIVES

We used administrative data from three Canadian provinces, British Columbia, Ontario and Nova Scotia, to: 1) identify and develop a core set of PC performance indicators using administrative data and 2) examine their ability to capture PC performance.

METHODS

Administrative data used included Physician Billings, Discharge Abstract Database, the National Ambulatory Care and Reporting System database, Census and Vital Statistics. Indicators were compiled based on a literature review of PC indicators previously developed with administrative data available in Canada (n=158). We engaged in iterative discussions to assess data conformity, completeness, and plausibility of results in all jurisdictions. Challenges to creating comparable algorithms were examined through content analysis and research team discussions, which included clinicians, analysts, and health services researchers familiar with PC.

RESULTS

Our final list included 21 PC performance indicators pertaining to 1) technical care (n=4), 2) continuity of care (n=6), and 3) health services utilization (n=11). Establishing comparable algorithms across provinces was possible though time intensive. A major challenge was inconsistent data elements. Ease of data access, and a deep understanding of the data and practice context, was essential for selecting the most appropriate data elements.

CONCLUSIONS

This project is unique in creating algorithms to measure PC performance across provinces. It was essential to balance internal validity of the indicators within a province and external validity across provinces. The intuitive desire of having the exact same coding across provinces was infeasible due to lack of standardized PC data. Rather, a context-tailored definition was developed for each jurisdiction. This work serves as an example for developing comparable PC performance indicators across different provincial/territorial jurisdictions.

Development and evaluation of an osteoarthritis risk model for integration into primary care health information technology

BACKGROUND

We developed and evaluated a prognostic prediction model that estimates osteoarthritis risk for use by patients and practitioners that is designed to be appropriate for integration into primary care health information technology systems. Osteoarthritis, a joint disorder characterized by pain and stiffness, causes significant morbidity among older Canadians. Because our prognostic prediction model for osteoarthritis risk uses data that are readily available in primary care settings, it supports targeting of interventions delivered as part of clinical practice that are aimed at risk reduction.

METHODS

We used the CPCSSN (Canadian Primary Sentinel Surveillance Network) database, which contains aggregated electronic health information from a cohort of primary care practices, to develop and evaluate a prognostic prediction model to estimate 5-year osteoarthritis risk, addressing contextual challenges of data availability and missingness. We constructed a retrospective cohort of 383,117 eligible primary care patients who were included in the cohort if they had an encounter with their primary care practitioner between 1 January 2009 and 31 December 2010. Patients were excluded if they had a diagnosis of osteoarthritis prior to their first visit in this time period. Incident cases of osteoarthritis were observed. The model was constructed to predict incident osteoarthritis based on age, sex, BMI, previous leg injury, and osteoporosis. Evaluation of the model used internal 10-fold cross-validation; we argue that internal validation is particularly appropriate for a model that is to be integrated into the same context from which the data were derived.

RESULTS

The resulting prediction model for 5-year risk of osteoarthritis diagnosis demonstrated state-of-the-art discrimination (estimated AUROC 0.84) and good calibration (assessed visually.) The model relies only on information that is readily available in Canadian primary care settings, and hence is appropriate for integration into Canadian primary care health information technology.

CONCLUSIONS

If the contextual challenges arising when using primary care electronic medical record data are appropriately addressed, highly discriminative models for osteoarthritis risk may be constructed using only data commonly available in primary care. Because the models are constructed from data in the same setting where the model is to be applied, internal validation provides strong evidence that the resulting model will perform well in its intended application.