Data Quality in Electronic Health Records Research: Quality Domains and Assessment Methods

The challenges and opportunities of continuous data quality improvement for healthcare administration data

Background

Various data quality issues have prevented healthcare administration data from being fully utilized when dealing with problems ranging from COVID-19 contact tracing to controlling healthcare costs.

Objectives

(i) Describe the currently adopted approaches and practices for understanding and improving the quality of healthcare administration data. (ii) Explore the challenges and opportunities to achieve continuous quality improvement for such data.

Materials and Methods

We used a qualitative approach to obtain rich contextual data through semi-structured interviews conducted at a state health agency regarding Medicaid claims and reimbursement data. We interviewed all data stewards knowledgeable about the data quality issues experienced at the agency. The qualitative data were analyzed using the Framework method.

Results

Sixteen themes emerged from our analysis, collected under 4 categories: (i) Defect characteristics: Data defects showed variability, frequently remained obscure, and led to negative outcomes. Detecting and resolving them was often difficult, and the work required often exceeded the organizational boundaries. (ii) Current process and people issues: The agency adopted primarily ad-hoc, manual approaches to resolving data quality problems leading to work frustration. (iii) Challenges: Communication and lack of knowledge about legacy software systems and the data maintained in them constituted challenges, followed by different standards used by various organizations and vendors, and data verification difficulties. (iv) Opportunities: Training, tool support, and standardization of data definitions emerged as immediate opportunities to improve data quality.

Conclusions

Our results can be useful to similar agencies on their journey toward becoming learning health organizations leveraging data assets effectively and efficiently.

Data quality in hospital information systems: Lessons learned from analyzing 30 years of patient data in a regional German hospital

BACKGROUND

The integration of Hospital Information Systems (HIS) into healthcare delivery has significantly enhanced patient care and operational efficiency. Nonetheless, the rapid acceleration of digital transformation has led to a substantial increase in the volume of data managed by these systems. This emphasizes the need for robust mechanisms for data management and quality assurance.

OBJECTIVE

This study addresses data quality issues related to patient identifiers within the Hospital Information System (HIS) of a regional German hospital, focusing on improving the accuracy and consistency of these administrative data entries.

METHODS

Employing a combination of data analysis and expert interviews, this study reviews and programmatically cleanses a dataset with over 2,000,000 patient data entries extracted from the HIS. The areas of investigation are patient admissions, discharges, and geographical data.

RESULTS

The analysis revealed that roughly 25% of the dataset was rendered unusable by errors and inconsistencies. By implementing a thorough data cleansing process, we significantly enhanced the utility of the dataset. In doing so, we identified the primary issues affecting data quality, including ambiguities among similar variables and a gap between the intended and actual use of the system.

CONCLUSION

The findings highlight the critical importance of enhancing data quality in healthcare information systems. This study shows the necessity of a careful review of data extracted from the HIS before it can be reliably utilized for machine learning tasks, thereby rendering the data more usable for both clinical and analytical purposes.

Common data quality elements for health information systems: a systematic review

BACKGROUND

Data quality in health information systems has a complex structure and consists of several dimensions. This research conducted for identify Common data quality elements for health information systems.

METHODS

A literature review was conducted and search strategies run in Web of Knowledge, Science Direct, Emerald, PubMed, Scopus and Google Scholar search engine as an additional source for tracing references. We found 760 papers, excluded 314 duplicates, 339 on abstract review and 167 on full-text review; leaving 58 papers for critical appraisal.

RESULTS

Current review shown that 14 criteria are categorized as the main dimensions for data quality for health information system include: Accuracy, Consistency, Security, Timeliness, Completeness, Reliability, Accessibility, Objectivity, Relevancy, Understandability, Navigation, Reputation, Efficiency and Value- added. Accuracy, Completeness, and Timeliness, were the three most-used dimensions in literature.

CONCLUSIONS

At present, there is a lack of uniformity and potential applicability in the dimensions employed to evaluate the data quality of health information system. Typically, different approaches (qualitative, quantitative and mixed methods) were utilized to evaluate data quality for health information system in the publications that were reviewed. Consequently, due to the inconsistency in defining dimensions and assessing methods, it became imperative to categorize the dimensions of data quality into a limited set of primary dimensions.

Assessing disparities through missing race and ethnicity data: results from a juvenile arthritis registry

Introduction

Ensuring high-quality race and ethnicity data within the electronic health record (EHR) and across linked systems, such as patient registries, is necessary to achieving the goal of inclusion of racial and ethnic minorities in scientific research and detecting disparities associated with race and ethnicity. The project goal was to improve race and ethnicity data completion within the Pediatric Rheumatology Care Outcomes Improvement Network and assess impact of improved data completion on conclusions drawn from the registry.

Methods

This is a mixed-methods quality improvement study that consisted of five parts, as follows: (1) Identifying baseline missing race and ethnicity data, (2) Surveying current collection and entry, (3) Completing data through audit and feedback cycles, (4) Assessing the impact on outcome measures, and (5) Conducting participant interviews and thematic analysis.

Results

Across six participating centers, 29% of the patients were missing data on race and 31% were missing data on ethnicity. Of patients missing data, most patients were missing both race and ethnicity. Rates of missingness varied by data entry method (electronic vs. manual). Recovered data had a higher percentage of patients with Other race or Hispanic/Latino ethnicity compared with patients with non-missing race and ethnicity data at baseline. Black patients had a significantly higher odds ratio of having a clinical juvenile arthritis disease activity score (cJADAS10) of ≥5 at first follow-up compared with White patients. There was no significant change in odds ratio of cJADAS10 ≥5 for race and ethnicity after data completion. Patients missing race and ethnicity were more likely to be missing cJADAS values, which may affect the ability to detect changes in odds ratio of cJADAS ≥5 after completion.

Conclusions

About one-third of the patients in a pediatric rheumatology registry were missing race and ethnicity data. After three audit and feedback cycles, centers decreased missing data by 94%, primarily via data recovery from the EHR. In this sample, completion of missing data did not change the findings related to differential outcomes by race. Recovered data were not uniformly distributed compared with those with non-missing race and ethnicity data at baseline, suggesting that differences in outcomes after completing race and ethnicity data may be seen with larger sample sizes.

Developing and testing a framework for coding general practitioners' free-text diagnoses in electronic medical records - a reliability study for generating training data in natural language processing

BACKGROUND

Diagnoses entered by general practitioners into electronic medical records have great potential for research and practice, but unfortunately, diagnoses are often in uncoded format, making them of little use. Natural language processing (NLP) could assist in coding free-text diagnoses, but NLP models require local training data to unlock their potential. The aim of this study was to develop a framework of research-relevant diagnostic codes, to test the framework using free-text diagnoses from a Swiss primary care database and to generate training data for NLP modelling.

METHODS

The framework of diagnostic codes was developed based on input from local stakeholders and consideration of epidemiological data. After pre-testing, the framework contained 105 diagnostic codes, which were then applied by two raters who independently coded randomly drawn lines of free text (LoFT) from diagnosis lists extracted from the electronic medical records of 3000 patients of 27 general practitioners. Coding frequency and mean occurrence rates (n and %) and inter-rater reliability (IRR) of coding were calculated using Cohen's kappa (Κ).

RESULTS

The sample consisted of 26,980 LoFT and in 56.3% no code could be assigned because it was not a specific diagnosis. The most common diagnostic codes were, 'dorsopathies' (3.9%, a code covering all types of back problems, including non-specific lower back pain, scoliosis, and others) and 'other diseases of the circulatory system' (3.1%). Raters were in almost perfect agreement (Κ ≥ 0.81) for 69 of the 105 diagnostic codes, and 28 codes showed a substantial agreement (K between 0.61 and 0.80). Both high coding frequency and almost perfect agreement were found in 37 codes, including codes that are particularly difficult to identify from components of the electronic medical record, such as musculoskeletal conditions, cancer or tobacco use.

CONCLUSION

The coding framework was characterised by a subset of very frequent and highly reliable diagnostic codes, which will be the most valuable targets for training NLP models for automated disease classification based on free-text diagnoses from Swiss general practice.

Assessing readiness to use electronic health record data for outcome ascertainment in clinical trials - A case study

BACKGROUND

Variable data quality poses a challenge to using electronic health record (EHR) data to ascertain acute clinical outcomes in multi-site clinical trials. Differing EHR platforms and data comprehensiveness across clinical trial sites, especially if patients received care outside of the clinical site's network, can also affect validity of results. Overcoming these challenges requires a structured approach.

METHODS

We propose a framework and create a checklist to assess the readiness of clinical sites to contribute EHR data to a clinical trial for the purpose of outcome ascertainment, based on our experience with the Strategies to Reduce Injuries and Develop Confidence in Elders (STRIDE) study, which enrolled 5451 participants in 86 primary care practices across 10 healthcare systems (sites).

RESULTS

The site readiness checklist includes assessment of the infrastructure (i.e., size and structure of the site's healthcare system or clinical network), data procurement (i.e., quality of the data), and cost of obtaining study data. The checklist emphasizes the importance of understanding how data are captured and integrated across a site's catchment area and having a protocol in place for data procurement to ensure consistent and uniform extraction across each site.

CONCLUSIONS

We suggest rigorous, prospective vetting of the data quality and infrastructure of each clinical site before launching a multi-site trial dependent on EHR data. The proposed checklist serves as a guiding tool to help investigators ensure robust and unbiased data capture for their clinical trials.

ORIGINAL TRIAL REGISTRATION NUMBER

NCT02475850.

Exploring phenotype-based ventilator parameter optimization to mitigate postoperative pulmonary complications: a retrospective observational cohort study

PURPOSE

To identify tidal volume (VT) and positive end-expiratory pressure (PEEP) associated with the lowest incidence and severity of postoperative pulmonary complications (PPCs) for each phenotype based on preoperative characteristics.

METHODS

The subjects of this retrospective observational cohort study were 34,910 adults who underwent surgery, using general anesthesia with mechanical ventilation. Initially, the least absolute shrinkage and selection operator regression was employed to select relevant preoperative characteristics. Then, the classification and regression tree (CART) was built to identify phenotypes. Finally, we computed the area under the receiver operating characteristic curves from logistic regressions to identify VT and PEEP associated with the lowest incidence and severity of PPCs for each phenotype.

RESULTS

CARTs classified seven phenotypes for each outcome. A probability of the development of PPCs ranged from the lowest (3.51%) to the highest (68.57%), whereas the probability of the development of the highest level of PPC severity ranged from 3.3% to 91.0%. Across all phenotypes, the VT and PEEP associated with the most desirable outcomes were within a small range of VT 7-8 ml/kg predicted body weight with PEEP of between 6 and 8 cmH2O.

CONCLUSIONS

The ranges of optimal VT and PEEP were small, regardless of the phenotypes, which had a wide range of risk profiles.

Appropriateness of Hysterectomy as Treatment for Benign Gynecological Conditions

Objective: To assess the appropriateness of hysterectomies performed at a large tertiary health system using the 1997 RAND appropriateness classification system and an updated algorithm. Design: We abstracted structured and unstructured data from electronic medical records on patient demographics, primary indication(s) for hysterectomy, diagnosis codes associated with the hysterectomy, previous treatments, and laboratory results. Subjects: Patients aged 18-44 years. Exposure: Receipt of hysterectomy for benign and nonobstetric conditions from October 2014 to December 2017. Main Outcome Measures: Using these data, we provided a RAND-based (dichotomous: inappropriate/appropriate) and Wright-based (3-level: inappropriate/ambiguous/appropriate) appropriateness rating and characterized missing information patterns associated with inappropriate ratings. Results: We analyzed 1,829 hysterectomies across 30 nonmutually exclusive primary indications for surgery. Nearly a third (32.8%) of surgeries had only one primary indication for surgery. Using the RAND-based classifier, 31.3% of hysterectomies were rated as appropriate and 68.7% as inappropriate. Using the Wright-based algorithm, 58.1% of hysterectomies were rated as appropriate, 15.7% as ambiguous, and 26.2% as inappropriate. Missing information on diagnostic procedures was the most common characteristic related to both RAND-based (46.1%) and Wright-based (51.2%) inappropriate ratings. Conclusions: The 1997 RAND classification lacked guidance for several contemporary indications, including gender-affirming care. RAND also has an outdated requirement for diagnostic surgeries such as laparoscopies, which have decreased in practice as diagnostic imaging has improved. Sensitivity analyses suggest that inappropriate surgeries cannot all be attributed to bias from missing electronic medical record data. Accurately documenting care delivery for benign gynecological conditions is key to ensuring quality and equity in gynecological care.

Bariatric surgery and COVID-19 outcomes: results from the PaTH to Health: Diabetes study

BACKGROUND

Obesity and type 2 diabetes mellitus (T2DM) are risk factors for severe COVID-19 infection. Bariatric surgery (BSG) is an effective treatment of obesity through weight loss and may reduce COVID-19 severity.

OBJECTIVES

We examined the effect of BSG on COVID-19 outcomes in patients with or at risk of T2DM.

SETTING

Electronic health record data from the PaTH Clinical Data Research Network, a partnership of 5 health systems reviewed from March 1, 2020, to December 31, 2020.

METHODS

Ambulatory and in-hospital patient encounters with COVID-19 diagnosis and obesity were identified. We constructed 2 patient groups: BSG and non-BSG (NBSG). The BSG group included patients with at least 1 encounter for the BSG procedure code and/or 1 BSG diagnosis code; the NBSG group included patients with no procedure or diagnosis code for BSG with body mass index (BMI) ≥40 or BMI ≥35 and at least 2 obesity-related co-morbidities. We matched 1 patient in the BSG group to 2 patients in the NBSG group based on age, gender (sex defined at birth), race and ethnicity, group (T2DM and at risk of T2DM), and site. The primary outcome was 30-day outcomes of COVID-19 severity.

RESULTS

After matching, we found that patients with BSG had lower odds of respiratory failure (41%) and ventilation/intensive care unit (ICU) admission/death (52%). Patients in the BSG group had lower odds of hospitalization, pneumonia, respiratory failure, and the most severe COVID-19 outcomes combined (ventilation/ICU admission/death). T2DM was identified as a risk factor for COVID-19 severity in the BSG group.

CONCLUSIONS

This retrospective, matched-cohort analysis found BSG to have a protective effect against severe COVID-19 outcomes.

Cohort profile: The Social media, smartphone use and Self-harm in Young People (3S-YP) study-A prospective, observational cohort study of young people in contact with mental health services

OBJECTIVES

The Social media, Smartphone use and Self-Harm (3S-YP) study is a prospective observational cohort study to investigate the mechanisms underpinning associations between social media and smartphone use and self-harm in a clinical youth sample. We present here a comprehensive description of the cohort from baseline data and an overview of data available from baseline and follow-up assessments.

METHODS

Young people aged 13-25 years were recruited from a mental health trust in England and followed up for 6 months. Self-report data was collected at baseline and monthly during follow-up and linked with electronic health records (EHR) and user-generated data.

FINDINGS

A total of 362 young people enrolled and provided baseline questionnaire data. Most participants had a history of self-harm according to clinical (n = 295, 81.5%) and broader definitions (n = 296, 81.8%). At baseline, there were high levels of current moderate/severe anxiety (n = 244; 67.4%), depression (n = 255; 70.4%) and sleep disturbance (n = 171; 47.2%). Over half used social media and smartphones after midnight on weekdays (n = 197, 54.4%; n = 215, 59.4%) and weekends (n = 241, 66.6%; n = 263, 72.7%), and half met the cut-off for problematic smartphone use (n = 177; 48.9%). Of the cohort, we have questionnaire data at month 6 from 230 (63.5%), EHR data from 345 (95.3%), social media data from 110 (30.4%) and smartphone data from 48 (13.3%).

CONCLUSION

The 3S-YP study is the first prospective study with a clinical youth sample, for whom to investigate the impact of digital technology on youth mental health using novel data linkages. Baseline findings indicate self-harm, anxiety, depression, sleep disturbance and digital technology overuse are prevalent among clinical youth. Future analyses will explore associations between outcomes and exposures over time and compare self-report with user-generated data in this cohort.

A Standard Framework for Evaluating Large Health Care Data and Related Resources

Since 2000, the availability and use of large health care data and related resources for conducting surveillance, research, and evaluations to guide clinical and public health decision-making has increased rapidly. These trends have been related to transformations in health care information technology and public as well as private-sector efforts for collecting, compiling, and supplying large volumes of data. This growing collection of robust and often timely data has enhanced the capability to increase the knowledge base guiding clinical and public health activities and also has increased the need for effective tools to assess the attributes of these resources and identify the types of scientific questions they are best suited to address. This

MMWR supplement presents a standard framework for evaluating large health care data and related resources, including constructs, criteria, and tools that investigators and evaluators can apply and adapt.

Systematically assessing the quality of dental electronic health record data for an investigation into oral health care disparities

OBJECTIVES

This work describes the process by which the quality of electronic health care data for a public health study was determined. The objectives were to adapt, develop, and implement data quality assessments (DQAs) based on the National Institutes of Health Pragmatic Trials Collaboratory (NIHPTC) data quality framework within the three domains of completeness, accuracy, and consistency, for an investigation into oral health care disparities of a preventive care program.

METHODS

Electronic health record data for eligible children in a dental accountable care organization of 30 offices, in Oregon, were extracted iteratively from January 1, 2014, through March 31, 2022. Baseline eligibility criteria included: children ages 0-18 with a baseline examination, Oregon home address, and either Medicaid or commercial dental benefits at least once between 2014 and 2108. Using the NIHPTC framework as a guide, DQAs were conducted throughout data element identification, extraction, staging, profiling, review, and documentation.

RESULTS

The data set included 91,487 subjects, 11 data tables comprising 75 data variables (columns), with a total of 6,861,525 data elements. Data completeness was 97.2%, the accuracy of EHR data elements in extracts was 100%, and consistency between offices was strong; 29 of 30 offices within 2 standard deviations of the mean (s = 94%).

CONCLUSIONS

The NIHPTC framework proved to be a useful approach, to identify, document, and characterize the dataset. The concepts of completeness, accuracy, and consistency were adapted by the multidisciplinary research team and the overall quality of the data are demonstrated to be of high quality.

Frameworks, Dimensions, Definitions of Aspects, and Assessment Methods for the Appraisal of Quality of Health Data for Secondary Use: Comprehensive Overview of Reviews

BACKGROUND

Health care has not reached the full potential of the secondary use of health data because of-among other issues-concerns about the quality of the data being used. The shift toward digital health has led to an increase in the volume of health data. However, this increase in quantity has not been matched by a proportional improvement in the quality of health data.

OBJECTIVE

This review aims to offer a comprehensive overview of the existing frameworks for data quality dimensions and assessment methods for the secondary use of health data. In addition, it aims to consolidate the results into a unified framework.

METHODS

A review of reviews was conducted including reviews describing frameworks of data quality dimensions and their assessment methods, specifically from a secondary use perspective. Reviews were excluded if they were not related to the health care ecosystem, lacked relevant information related to our research objective, and were published in languages other than English.

RESULTS

A total of 22 reviews were included, comprising 22 frameworks, with 23 different terms for dimensions, and 62 definitions of dimensions. All dimensions were mapped toward the data quality framework of the European Institute for Innovation through Health Data. In total, 8 reviews mentioned 38 different assessment methods, pertaining to 31 definitions of the dimensions.

CONCLUSIONS

The findings in this review revealed a lack of consensus in the literature regarding the terminology, definitions, and assessment methods for data quality dimensions. This creates ambiguity and difficulties in developing specific assessment methods. This study goes a step further by assigning all observed definitions to a consolidated framework of 9 data quality dimensions.

Data completeness and consistency in individual medical records of institutional births: retrospective crossectional study from Northwest Ethiopia, 2022

BACKGROUND

Ensuring the data quality of Individual Medical Records becomes a crucial strategy in mitigating maternal and newborn morbidity and mortality during and around childbirth. However, previous research in Ethiopia primarily focused on studying data quality of institutional birth at the facility level, overlooking the data quality within Individual Medical Records. This study examined the data completeness and consistency within Individual Medical Records of the institutional birth service and associated factors.

METHODS

An institution-based retrospective cross-sectional study was conducted in two districts of Northwest Ethiopia. Data were obtained by reviewing three sets of Individual Medical Records of 651 women: the delivery register, Integrated Individual Folder, and integrated card. The proportions of completeness and consistency were computed. A multilevel binary logistic regression was used to identify factors of completeness and consistency. An odds ratio with a 95% confidence interval was used to assess the level of significance.

RESULTS

Overall, 74.0% of women's Individual Medical Records demonstrated good data completeness ( > = 70%), 95%CI (70.5, 77.3), while 26% exhibited good consistency, 95%CI (22.9, 29.7). The presence of trained providers in data quality (AOR = 2.9, 95%CI: (1.5, 5.7)) and supportive supervision (AOR = 11.5, 95%CI: (4.8, 27.2)) were found to be associated with completeness. Health facilities' practice of root cause analysis on data quality gaps (AOR = 8.7, 9%CI: (1.5, 50.9)) was statistically significantly associated with the consistency.

CONCLUSIONS

Most medical records were found to have good completeness, but nearly only a quarter of them found to contain consistent data. Completeness and consistency varied on the type of medical record. Health facility's root cause analysis of data quality gaps, the presence of trained providers in data quality, and supportive supervision from higher officials were identified as factors affecting data quality in institutional birth service. These results emphasize the importance of focused efforts to enhance data completeness and consistency within Individual Medical Records, particularly through consideration of Individual Medical Records in future provider training, supervision, and the implementation of root cause analysis practices.

Data Quality in Health Research: Integrative Literature Review

BACKGROUND

Decision-making and strategies to improve service delivery must be supported by reliable health data to generate consistent evidence on health status. The data quality management process must ensure the reliability of collected data. Consequently, various methodologies to improve the quality of services are applied in the health field. At the same time, scientific research is constantly evolving to improve data quality through better reproducibility and empowerment of researchers and offers patient groups tools for secured data sharing and privacy compliance.

OBJECTIVE

Through an integrative literature review, the aim of this work was to identify and evaluate digital health technology interventions designed to support the conducting of health research based on data quality.

METHODS

A search was conducted in 6 electronic scientific databases in January 2022: PubMed, SCOPUS, Web of Science, Institute of Electrical and Electronics Engineers Digital Library, Cumulative Index of Nursing and Allied Health Literature, and Latin American and Caribbean Health Sciences Literature. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses checklist and flowchart were used to visualize the search strategy results in the databases.

RESULTS

After analyzing and extracting the outcomes of interest, 33 papers were included in the review. The studies covered the period of 2017-2021 and were conducted in 22 countries. Key findings revealed variability and a lack of consensus in assessing data quality domains and metrics. Data quality factors included the research environment, application time, and development steps. Strategies for improving data quality involved using business intelligence models, statistical analyses, data mining techniques, and qualitative approaches.

CONCLUSIONS

The main barriers to health data quality are technical, motivational, economical, political, legal, ethical, organizational, human resources, and methodological. The data quality process and techniques, from precollection to gathering, postcollection, and analysis, are critical for the final result of a study or the quality of processes and decision-making in a health care organization. The findings highlight the need for standardized practices and collaborative efforts to enhance data quality in health research. Finally, context guides decisions regarding data quality strategies and techniques.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1101/2022.05.31.22275804.

Impact of Imaging Biomarkers and AI on Breast Cancer Management: A Brief Review

Breast cancer stands out as the most frequently identified malignancy, ranking as the fifth leading cause of global cancer-related deaths. The American College of Radiology (ACR) introduced the Breast Imaging Reporting and Data System (BI-RADS) as a standard terminology facilitating communication between radiologists and clinicians; however, an update is now imperative to encompass the latest imaging modalities developed subsequent to the 5th edition of BI-RADS. Within this review article, we provide a concise history of BI-RADS, delve into advanced mammography techniques, ultrasonography (US), magnetic resonance imaging (MRI), PET/CT images, and microwave breast imaging, and subsequently furnish comprehensive, updated insights into Molecular Breast Imaging (MBI), diagnostic imaging biomarkers, and the assessment of treatment responses. This endeavor aims to enhance radiologists' proficiency in catering to the personalized needs of breast cancer patients. Lastly, we explore the augmented benefits of artificial intelligence (AI), machine learning (ML), and deep learning (DL) applications in segmenting, detecting, and diagnosing breast cancer, as well as the early prediction of the response of tumors to neoadjuvant chemotherapy (NAC). By assimilating state-of-the-art computer algorithms capable of deciphering intricate imaging data and aiding radiologists in rendering precise and effective diagnoses, AI has profoundly revolutionized the landscape of breast cancer radiology. Its vast potential holds the promise of bolstering radiologists' capabilities and ameliorating patient outcomes in the realm of breast cancer management.

The data quality and applicability of a Danish prehospital electronic health record: A mixed-methods study

BACKGROUND

Without accurate documentation, it can be difficult to assess the quality of care and the impact of quality improvement initiatives. Prehospital lack of documentation of the basic measurements is associated with a twofold risk of mortality. The aim of this study was to investigate data quality in the electronic prehospital patient record (ePPR) system in the Region of Southern Denmark. In addition, we investigated ambulance professionals' attitudes toward the use of ePPR and identified barriers and facilitators to its use.

METHOD

We used an explanatory sequential mixed-methods design. Phase one consisted of a retrospective assessment of the data quality of ePPR information, and phase two included semi-structured interviews with ambulance professionals combined with observations. We included patients who were acutely transported to an emergency department by ambulance in the Region of Southern Denmark from 2016 to 2020. Data completeness was calculated for each vital sign using a two-way table of frequency. Vital signs were summarised to calculate data correctness. Interviews and observations were analysed using thematic analysis.

RESULTS

Overall, an improvement in data completeness and correctness was observed from 2016-2020. When stratified by age group, children (<12 years) accounted for the majority of missing vital sign registrations. In the thematic analysis, we identified four themes; ambulance professionals' attitudes, emergency setting, training and guidelines, and tablet and software.

CONCLUSION

We found high data quality, but there is room for improvement. The ambulance professionals' attitudes toward the ePPR, working in an emergency setting, a notion of insufficient training in completing the ePPR, and challenges related to the tablet and software could be barriers to data completeness and correctness. It would be beneficial to include the end-user when developing an ePPR system and to consider that the tablet should be used in emergency situations.

Electronic health record data quality assessment and tools: a systematic review

OBJECTIVE

We extended a 2013 literature review on electronic health record (EHR) data quality assessment approaches and tools to determine recent improvements or changes in EHR data quality assessment methodologies.

MATERIALS AND METHODS

We completed a systematic review of PubMed articles from 2013 to April 2023 that discussed the quality assessment of EHR data. We screened and reviewed papers for the dimensions and methods defined in the original 2013 manuscript. We categorized papers as data quality outcomes of interest, tools, or opinion pieces. We abstracted and defined additional themes and methods though an iterative review process.

RESULTS

We included 103 papers in the review, of which 73 were data quality outcomes of interest papers, 22 were tools, and 8 were opinion pieces. The most common dimension of data quality assessed was completeness, followed by correctness, concordance, plausibility, and currency. We abstracted conformance and bias as 2 additional dimensions of data quality and structural agreement as an additional methodology.

DISCUSSION

There has been an increase in EHR data quality assessment publications since the original 2013 review. Consistent dimensions of EHR data quality continue to be assessed across applications. Despite consistent patterns of assessment, there still does not exist a standard approach for assessing EHR data quality.

CONCLUSION

Guidelines are needed for EHR data quality assessment to improve the efficiency, transparency, comparability, and interoperability of data quality assessment. These guidelines must be both scalable and flexible. Automation could be helpful in generalizing this process.

Cardiovascular Care Innovation through Data-Driven Discoveries in the Electronic Health Record

The electronic health record (EHR) represents a rich source of patient information, increasingly being leveraged for cardiovascular research. Although its primary use remains the seamless delivery of health care, the various longitudinally aggregated structured and unstructured data elements for each patient within the EHR can define the computational phenotypes of disease and care signatures and their association with outcomes. Although structured data elements, such as demographic characteristics, laboratory measurements, problem lists, and medications, are easily extracted, unstructured data are underused. The latter include free text in clinical narratives, documentation of procedures, and reports of imaging and pathology. Rapid scaling up of data storage and rapid innovation in natural language processing and computer vision can power insights from unstructured data streams. However, despite an array of opportunities for research using the EHR, specific expertise is necessary to adequately address confidentiality, accuracy, completeness, and heterogeneity challenges in EHR-based research. These often require methodological innovation and best practices to design and conduct successful research studies. Our review discusses these challenges and their proposed solutions. In addition, we highlight the ongoing innovations in federated learning in the EHR through a greater focus on common data models and discuss ongoing work that defines such an approach to large-scale, multicenter, federated studies. Such parallel improvements in technology and research methods enable innovative care and optimization of patient outcomes.

Implementation of the trial emulation approach in medical research: a scoping review

BACKGROUND

When conducting randomised controlled trials is impractical, an alternative is to carry out an observational study. However, making valid causal inferences from observational data is challenging because of the risk of several statistical biases. In 2016 Hernán and Robins put forward the 'target trial framework' as a guide to best design and analyse observational studies whilst preventing the most common biases. This framework consists of (1) clearly defining a causal question about an intervention, (2) specifying the protocol of the hypothetical trial, and (3) explaining how the observational data will be used to emulate it.

METHODS

The aim of this scoping review was to identify and review all explicit attempts of trial emulation studies across all medical fields. Embase, Medline and Web of Science were searched for trial emulation studies published in English from database inception to February 25, 2021. The following information was extracted from studies that were deemed eligible for review: the subject area, the type of observational data that they leveraged, and the statistical methods they used to address the following biases: (A) confounding bias, (B) immortal time bias, and (C) selection bias.

RESULTS

The search resulted in 617 studies, 38 of which we deemed eligible for review. Of those 38 studies, most focused on cardiology, infectious diseases or oncology and the majority used electronic health records/electronic medical records data and cohort studies data. Different statistical methods were used to address confounding at baseline and selection bias, predominantly conditioning on the confounders (N = 18/49, 37%) and inverse probability of censoring weighting (N = 7/20, 35%) respectively. Different approaches were used to address immortal time bias, assigning individuals to treatment strategies at start of follow-up based on their data available at that specific time (N = 21, 55%), using the sequential trial emulations approach (N = 11, 29%) or the cloning approach (N = 6, 16%).

CONCLUSION

Different methods can be leveraged to address (A) confounding bias, (B) immortal time bias, and (C) selection bias. When working with observational data, and if possible, the 'target trial' framework should be used as it provides a structured conceptual approach to observational research.

Electronic health record data quality variability across a multistate clinical research network

Background

Electronic health record (EHR) data have many quality problems that may affect the outcome of research results and decision support systems. Many methods have been used to evaluate EHR data quality. However, there has yet to be a consensus on the best practice. We used a rule-based approach to assess the variability of EHR data quality across multiple healthcare systems.

Methods

To quantify data quality concerns across healthcare systems in a PCORnet Clinical Research Network, we used a previously tested rule-based framework tailored to the PCORnet Common Data Model to perform data quality assessment at 13 clinical sites across eight states. Results were compared with the current PCORnet data curation process to explore the differences between both methods. Additional analyses of testosterone therapy prescribing were used to explore clinical care variability and quality.

Results

The framework detected discrepancies across sites, revealing evident data quality variability between sites. The detailed requirements encoded the rules captured additional data errors with a specificity that aids in remediation of technical errors compared to the current PCORnet data curation process. Other rules designed to detect logical and clinical inconsistencies may also support clinical care variability and quality programs.

Conclusion

Rule-based EHR data quality methods quantify significant discrepancies across all sites. Medication and laboratory sources are causes of data errors.

Developing and testing inter-rater reliability of a data collection tool for patient health records on end-of-life care of neurological patients in an acute hospital ward

AIM

Develop and test a data collection tool-Neurological End-Of-Life Care Assessment Tool (NEOLCAT)-for extracting data from patient health records (PHRs) on end-of-life care of neurological patients in an acute hospital ward.

DESIGN

Instrument development and inter-rater reliability (IRR) assessment.

METHOD

NEOLCAT was constructed from patient care items obtained from clinical guidelines and literature on end-of-life care. Expert clinicians reviewed the items. Using percentage agreement and Fleiss' kappa we calculated IRR on 32 nominal items, out of 76 items.

RESULTS

IRR of NEOLCAT showed 89% (range 83%-95%) overall categorical percentage agreement. The Fleiss' kappa categorical coefficient was 0.84 (range 0.71-0.91). There was fair or moderate agreement on six items, and moderate or almost perfect agreement on 26 items.

CONCLUSION

The NEOLCAT shows promising psychometric properties for studying clinical components of care of neurological patients at the end-of-life on an acute hospital ward but could be further developed in future studies.

Automating Electronic Health Record Data Quality Assessment

Information systems such as Electronic Health Record (EHR) systems are susceptible to data quality (DQ) issues. Given the growing importance of EHR data, there is an increasing demand for strategies and tools to help ensure that available data are fit for use. However, developing reliable data quality assessment (DQA) tools necessary for guiding and evaluating improvement efforts has remained a fundamental challenge. This review examines the state of research on operationalising EHR DQA, mainly automated tooling, and highlights necessary considerations for future implementations. We reviewed 1841 articles from PubMed, Web of Science, and Scopus published between 2011 and 2021. 23 DQA programs deployed in real-world settings to assess EHR data quality (n = 14), and a few experimental prototypes (n = 9), were identified. Many of these programs investigate completeness (n = 15) and value conformance (n = 12) quality dimensions and are backed by knowledge items gathered from domain experts (n = 9), literature reviews and existing DQ measurements (n = 3). A few DQA programs also explore the feasibility of using data-driven techniques to assess EHR data quality automatically. Overall, the automation of EHR DQA is gaining traction, but current efforts are fragmented and not backed by relevant theory. Existing programs also vary in scope, type of data supported, and how measurements are sourced. There is a need to standardise programs for assessing EHR data quality, as current evidence suggests their quality may be unknown.

Data Quality in Health Care: Main Concepts and Assessment Methodologies

INTRODUCTION

 In the health care environment, a huge volume of data is produced on a daily basis. However, the processes of collecting, storing, sharing, analyzing, and reporting health data usually face with numerous challenges that lead to producing incomplete, inaccurate, and untimely data. As a result, data quality issues have received more attention than before.

OBJECTIVE

 The purpose of this article is to provide an insight into the data quality definitions, dimensions, and assessment methodologies.

METHODS

 In this article, a scoping literature review approach was used to describe and summarize the main concepts related to data quality and data quality assessment methodologies. Search terms were selected to find the relevant articles published between January 1, 2012 and September 31, 2022. The retrieved articles were then reviewed and the results were reported narratively.

RESULTS

 In total, 23 papers were included in the study. According to the results, data quality dimensions were various and different methodologies were used to assess them. Most studies used quantitative methods to measure data quality dimensions either in paper-based or computer-based medical records. Only two studies investigated respondents' opinions about data quality.

CONCLUSION

 In health care, high-quality data not only are important for patient care, but also are vital for improving quality of health care services and better decision making. Therefore, using technical and nontechnical solutions as well as constant assessment and supervision is suggested to improve data quality.

Leveraging electronic health records for data science: common pitfalls and how to avoid them

Analysis of electronic health records (EHRs) is an increasingly common approach for studying real-world patient data. Use of routinely collected data offers several advantages compared with other study designs, including reduced administrative costs, the ability to update analysis as practice patterns evolve, and larger sample sizes. Methodologically, EHR analysis is subject to distinct challenges because data are not collected for research purposes. In this Viewpoint, we elaborate on the importance of in-depth knowledge of clinical workflows and describe six potential pitfalls to be avoided when working with EHR data, drawing on examples from the literature and our experience. We propose solutions for prevention or mitigation of factors associated with each of these six pitfalls-sample selection bias, imprecise variable definitions, limitations to deployment, variable measurement frequency, subjective treatment allocation, and model overfitting. Ultimately, we hope that this Viewpoint will guide researchers to further improve the methodological robustness of EHR analysis.

Identifying nursing sensitive indicators from electronic health records in acute cardiac care-Towards intelligent automated assessment of care quality

AIM

The aim of this study is to explore the potential of using electronic health records for assessment of nursing care quality through nursing-sensitive indicators in acute cardiac care.

BACKGROUND

Nursing care quality is a multifaceted phenomenon, making a holistic assessment of it difficult. Quality assessment systems in acute cardiac care units could benefit from big data-based solutions that automatically extract and help interpret data from electronic health records.

METHODS

This is a deductive descriptive study that followed the theory of value-added analysis. A random sample from electronic health records of 230 patients was analysed for selected indicators. The data included documentation in structured and free-text format.

RESULTS

One thousand six hundred seventy-six expressions were extracted and divided into (1) established and (2) unestablished expressions, providing positive, neutral and negative descriptions related to care quality.

CONCLUSIONS

Electronic health records provide a potential source of information for information systems to support assessment of care quality. More research is warranted to develop, test and evaluate the effectiveness of such tools in practice.

IMPLICATIONS FOR NURSING MANAGEMENT

Knowledge-based health care management would benefit from the development and implementation of advanced information systems, which use continuously generated already available real-time big data for improved data access and interpretation to better support nursing management in quality assessment.

Tolerance to oral anticancer agent treatment in older adults with cancer: a secondary analysis of data from electronic health records and a pilot study of patient-reported outcomes

Background

More than 60% of cancer cases occur in older adults, and many are treated with oral anticancer agents. Yet, the treatment tolerability in older adults has not been fully understood due to their underrepresentation in oncology clinical trials, creating challenges for treatment decision-making and symptom management. The objective of this study was to investigate the tolerance of capecitabine, an example of oral chemotherapy, among older adults with cancer and explore factors associated with capecitabine-related side effects and treatment changes, to enhance supportive care.

Methods

A secondary analysis used combined data from electronic health records and a pilot study of patient-reported outcomes, with a total of 97 adult patients taking capecitabine during 2016–2017, including older adult patients aged 65 years or older (n = 43). The data extracted included patient socio-demographics, capecitabine information, side effects, and capecitabine treatment changes (dose reductions and dose interruptions). Bivariate correlations, negative binomial regression, and multiple linear regression were conducted for data analysis.

Results

Older adults were more likely to experience fatigue (86% vs. 51%, p = .001) and experienced more severe fatigue (β = 0.44, p = 0.03) and hand-foot syndrome (HFS) (β = 1.15, p = 0.004) than younger adults. The severity of fatigue and HFS were associated with the number of outpatient medications (β = 0.06, p = 0.006) and the duration of treatment (β = 0.50, p = 0.009), respectively. Correlations among side effects presented different patterns between younger and older adults. Although more older adults experienced dose reductions (21% vs. 13%) and dose interruptions (33% vs. 28%) than younger adults, the differences were not statistically different. Female sex, breast cancer diagnosis, capecitabine monotherapy, and severe HFS were found to be associated with dose reductions (p-values < 0.05).

Conclusions

Older adults were less likely to tolerate capecitabine treatment and had different co-occurring side effects compared to younger adults. While dose reductions are common among older adults, age 65 years or older may not be an independent factor of treatment changes. Other socio-demographic and clinical factors may be more likely to be associated. Future studies can be conducted to further explore older adults’ tolerance to a variety of oral anticancer agents to generate more evidence to support optimal treatment decision-making and symptom management.

Challenges and recommendations for high quality research using electronic health records

Harnessing Real World Data is vital to improve health care in the 21st Century. Data from Electronic Health Records (EHRs) are a rich source of patient centred data, including information on the patient's clinical condition, laboratory results, diagnoses and treatments. They thus reflect the true state of health systems. However, access and utilisation of EHR data for research presents specific challenges. We assert that using data from EHRs effectively is dependent on synergy between researchers, clinicians and health informaticians, and only this will allow state of the art methods to be used to answer urgent and vital questions for patient care. We propose that there needs to be a paradigm shift in the way this research is conducted - appreciating that the research process is iterative rather than linear. We also make specific recommendations for organisations, based on our experience of developing and using EHR data in trusted research environments.

Analyses and identification of ICD codes for dementias in the research based on the NHIRD: a scoping review protocol

INTRODUCTION

Studies based on health claims data (HCD) have been increasingly adopted in medical research for their strengths in large sample size and abundant information, and the Taiwan National Health Insurance Research Database (NHIRD) has been widely used in medical research across disciplines, including dementia. How the diagnostic codes are applied to define the diseases/conditions of interest is pivotal in HCD-related research, but the consensus on the issue that diagnostic codes most appropriately define dementias in the NHIRD is lacking. The objectives of this scoping review are (1) to investigate the relevant characteristics in the published reports targeting dementias based on the NHIRD, and (2) to address the diversity by a case study.

METHODS AND ANALYSIS

This scoping review protocol follows the methodological framework of the Joanna Briggs Institute Reviewer's Manual and the guidance of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for scoping reviews. The review will be performed between 1 March and 31 December 2022 in five stages, including identifying the relevant studies, developing search strategies, individually screening and selecting evidence, collecting and extracting data, and summarising and reporting the results. The electronic databases of MEDLINE, EMBASE, CENTRAL, CINAHL, and PsycINFO, Airiti Library Academic Database, the National Health Insurance Administration's repository, and Taiwan Government Research Bulletin will be searched. We will perform narrative syntheses of the results to address research questions and will analyse the prevalence across the included individual studies as a case study.

ETHICS AND DISSEMINATION

Our scoping review is a review of the published reports and ethical approval is not required. The results will provide a panorama of the dementia studies based on the NHIRD. We will disseminate our findings through peer-reviewed journals and conferences, and share with stakeholders by distributing the summaries in social media and emails.

Database quality assessment in research in paramedicine: a scoping review protocol

INTRODUCTION

The paramedic practice environment presents unique challenges to data documentation and access, as well as linkage to other parts of the healthcare system. Variable or unknown data quality can influence the validity of research in paramedicine. A number of database quality assessment (DQA) frameworks have been developed and used to evaluate data quality in other areas of healthcare. The extent these or other DQA practices have been applied to paramedic research is not known. Accordingly, this scoping review aims to describe the range, extent and nature of DQA practices within research in paramedicine.

METHODS AND ANALYSIS

This scoping review will follow established methods for the conduct (Johanna Briggs Institute; Arksey and O'Malley) and reporting (Preferred Reporting Items in Systematic Reviews and Meta-Analyses extension for scoping reviews) of scoping reviews. In consultation with a professional librarian, a search strategy was developed representing the applicable population, concept and context. This strategy will be applied to MEDLINE (National Library of Medicine), Embase (Elsevier), Scopus (Elsevier) and CINAHL (EBSCO) to identify studies published from 2011 through 2021 that assess paramedic data quality as a stated goal. Studies will be included if they report quantitative results of DQA using data that relate primarily to the paramedic practice environment. Protocols, commentaries, case studies, interviews, simulations and experimental data-processing techniques will be excluded. No restrictions will be placed on language. Study selection will be performed by two reviewers, with a third available to resolve conflicts. Data will be extracted from included studies using a data-charting form piloted and iteratively revised based on studies known to be relevant. Results will be summarised in a chart of study characteristics, DQA-specific outcomes and key findings.

ETHICS AND DISSEMINATION

Ethical approval is not required. Results will be submitted to relevant conferences and peer-reviewed journals.

TRIAL REGISTRATION

10.17605/OSF.IO/Z287T.

Infants With Congenital Muscular Torticollis: Demographic Factors, Clinical Characteristics, and Physical Therapy Episode of Care

PURPOSE

To describe demographic factors, baseline characteristics, and physical therapy episodes in infants with congenital muscular torticollis (CMT), examine groups based on physical therapy completion, and identify implications for clinical practice.

METHODS

Retrospective data were extracted from a single-site registry of 445 infants with CMT.

RESULTS

Most infants were male (57%), Caucasian (63%), and firstborn (50%), with torticollis detected by 3 months old (89%) with a left (51%), mild (72%) CMT presentation. Cervical range of motion (ROM) limitations were greatest in passive lateral flexion and active rotation. Sixty-seven percent of infants completed an episode of physical therapy, 25% completed a partial episode, and 8% did not attend visits following the initial examination. Age at examination, ROM, and muscle function differed significantly between groups.

CONCLUSIONS

Physical therapists may use clinical registry data to inform practice for timing of referral, frequency of care, and clinician training to manage infants with CMT.

Women's Health Care in the Deployed Setting 2013-2020: A Health Services Research Approach

INTRODUCTION

With the management and oversight of MTFs moving under the authority of the Defense Health Agency, coupled with a careful examination of the composition of uniformed medical personnel, it is imperative to ensure that active duty servicewomen who are in deployed settings receive timely, appropriate, and quality health care. This study sought to examine the amount and types of gynecological and obstetric care provided in the deployed setting and to examine that data by the socioeconomic and demographic characteristics of the women receiving that care.

MATERIALS AND METHODS

Using the Military Health System's Theater Medical Data Store, we identified women aged 15 to 54 years old who received care at a theater-based MTF between 2013 and 2020. Within our study population, we subsequently identified obstetric and gynecologic (OBGYN) health services during the study period, and ran descriptive statistics on patient demographics (age group, race, rank, and U.S. military branch of service) and OBGYN health services. Patient age was assessed at the time of data extraction and race was categorized as Black, White, Other, and Unknown. The military branch of service was categorized as Army, Navy/Marines, Air Force, and Other. Rank was used as a proxy for socioeconomic status and categorized as Junior Enlisted, Senior Enlisted, Junior Officer, Senior Officer, Warrant Officer, and Other. Multivariable logistic regressions were also conducted and used to assess the odds of OBGYN health service utilization, with all patient demographics included as predictor variables.

RESULTS

A total of 490,482 women were identified and received OBGYN health services at theater-based MTFs between 2013 and 2020. The majority of our population consisted of women aged 25 to 34 years (56.98%), associated with a Junior Enlisted rank (39.27%) and with the Navy/Marines (37.27%). Race was severely underreported, with 51.58% associated with an unknown race; however, 20.88% of our population were White women, 16.81% were Black women, and 10.72% of women identified their race as Other. The top five diagnoses for women seen in the deployed environment were for a contraceptive prescription (12.13%), followed by sexually transmitted infection (STI) screening (8.14%), breast disorder (7.89%), GYN exam (6.86%), and menstrual abnormalities (6.35%). Compared to White women, Black women had higher odds of seeking the contraceptive prescription (3.03 OR, 2.91-3.17 95% CI), obtaining STI screening (5.34 OR, 5.16-5.54 95% CI), being seen for a breast disorder (4.88 OR, 4.71-5.06 95% CI), GYN exam (3.21 OR, 3.10-3.32 95% CI), and menstrual abnormalities (3.71 OR, 3.58-3.85 95% CI).

CONCLUSIONS

Almost consistently, senior officers were more likely to receive OBGYN services during deployment. Policymakers and health-care providers need to identify interventions to close this care gap, particularly in preventive OBGYN services (contraception, GYN exams, STI screenings). Fully implementing the Comprehensive Contraceptive Counseling and Access to the Full Range of Methods of Contraception policy and developing one standard Defense Health Affairs policy on pre-deployment evaluation standards and deployment follow-up care for women's health care may also assist in closing care gaps.

Developing a systematic approach to assessing data quality in secondary use of clinical data based on intended use

INTRODUCTION

Secondary use of electronic health record (EHR) data for research requires that the data are fit for use. Data quality (DQ) frameworks have traditionally focused on structural conformance and completeness of clinical data extracted from source systems. In this paper, we propose a framework for evaluating semantic DQ that will allow researchers to evaluate fitness for use prior to analyses.

METHODS

We reviewed current DQ literature, as well as experience from recent multisite network studies, and identified gaps in the literature and current practice. Derived principles were used to construct the conceptual framework with attention to both analytic fitness and informatics practice.

RESULTS

We developed a systematic framework that guides researchers in assessing whether a data source is fit for use for their intended study or project. It combines tools for evaluating clinical context with DQ principles, as well as factoring in the characteristics of the data source, in order to develop semantic DQ checks.

CONCLUSIONS

Our framework provides a systematic process for DQ development. Further work is needed to codify practices and metadata around both structural and semantic data quality.

Understanding challenges of using routinely collected health data to address clinical care gaps: a case study in Alberta, Canada

High-quality data are fundamental to healthcare research, future applications of artificial intelligence and advancing healthcare delivery and outcomes through a learning health system. Although routinely collected administrative health and electronic medical record data are rich sources of information, they have significant limitations. Through four example projects from the Physician Learning Program in Edmonton, Alberta, Canada, we illustrate barriers to using routinely collected health data to conduct research and engage in clinical quality improvement. These include challenges with data availability for variables of clinical interest, data completeness within a clinical visit, missing and duplicate visits, and variability of data capture systems. We make four recommendations that highlight the need for increased clinical engagement to improve the collection and coding of routinely collected data. Advancing the quality and usability of health systems data will support the continuous quality improvement needed to achieve the quintuple aim.

Estimated impact of COVID-19 on preventive care service delivery: an observational cohort study

BACKGROUND

COVID-19 has caused significant healthcare service disruptions. Surgical backlogs have been estimated but not for other healthcare services. This study aims to estimate the backlog of preventive care services caused by COVID-19.

METHODS

This observational study assessed preventive care screening rates at three primary care clinics in Ottawa, Ontario from March to November 2020 using data from 22,685 electronic medical records. The change in cervical cancer, colorectal cancer, and type 2 diabetes screening rates were crudely estimated using 2016 census data, estimating the volume of key services delayed by COVID-19 across Ontario and Canada.

RESULTS

The mean percentage of patients appropriately screened for cervical cancer decreased by 7.5% (- 0.3% to - 14.7%; 95% CI), colorectal cancer decreased by 8.1% (- 0.3% to - 15.8%; 95% CI), and type 2 diabetes decreased by 4.5% (- 0.2% to - 8.7%; 95% CI). Crude estimates imply 288,000 cervical cancer (11,000 to 565,000; 95% CI), 326,000 colorectal cancer (13,000 to 638,000; 95% CI), and 274,000 type 2 diabetes screenings (13,000 to 535,000; 95% CI) may be overdue in Ontario. Nationally the deficits may be tripled these numbers. Re-opening measures have not reversed these trends.

INTERPRETATION

COVID-19 decreased the delivery of preventive care services, which may cause delayed diagnoses, increased mortality, and increased health care costs. Virtual care and reopening measures have not restored the provision of preventive care services. Electronic medical record data could be leveraged to improve screening via panel management. Additional, system-wide primary care and laboratory capacity will be needed to restore pre-COVID-19 screening rates.

An automated data cleaning method for Electronic Health Records by incorporating clinical knowledge

BACKGROUND

The use of Electronic Health Records (EHR) data in clinical research is incredibly increasing, but the abundancy of data resources raises the challenge of data cleaning. It can save time if the data cleaning can be done automatically. In addition, the automated data cleaning tools for data in other domains often process all variables uniformly, meaning that they cannot serve well for clinical data, as there is variable-specific information that needs to be considered. This paper proposes an automated data cleaning method for EHR data with clinical knowledge taken into consideration.

METHODS

We used EHR data collected from primary care in Flanders, Belgium during 1994-2015. We constructed a Clinical Knowledge Database to store all the variable-specific information that is necessary for data cleaning. We applied Fuzzy search to automatically detect and replace the wrongly spelled units, and performed the unit conversion following the variable-specific conversion formula. Then the numeric values were corrected and outliers were detected considering the clinical knowledge. In total, 52 clinical variables were cleaned, and the percentage of missing values (completeness) and percentage of values within the normal range (correctness) before and after the cleaning process were compared.

RESULTS

All variables were 100% complete before data cleaning. 42 variables had a drop of less than 1% in the percentage of missing values and 9 variables declined by 1-10%. Only 1 variable experienced large decline in completeness (13.36%). All variables had more than 50% values within the normal range after cleaning, of which 43 variables had a percentage higher than 70%.

CONCLUSIONS

We propose a general method for clinical variables, which achieves high automation and is capable to deal with large-scale data. This method largely improved the efficiency to clean the data and removed the technical barriers for non-technical people.

A new fuzzy approach for managing data governance implementation relevant activities

Purpose

The purpose of this paper is to propose an approach for managing relevant factors and activities for implementing data governance in an organization. The process of assessing the establishment of data governance in an organization is intrinsically imprecise, due to the characteristics of new problem settings, particularly in relation to newly generated alternatives or vaguely defined qualitative assessment criteria.

Design/methodology/approach

To reject the inherent subjectiveness and imprecision involved in the evaluation process, the authors use the concept of fuzzy logic in this approach for developing the assessment model and analyzing the model for allocating the management efforts in the most efficient way to improve the data governance deployment level.

Findings

This paper identifies relevant factors and activities for implementing data governance in an organization and evaluates the state of data governance based on causal relationships between influential factors. In this study, factors are prioritized for effective allocation of limited management efforts in any improvement plan.

Research limitations/implications

The interrelationships among factors are contextual and based on the perceptions of experts who may be biased as per their background and area of expertise. Meanwhile, lack of a data governance plan may cause failure during its implementation in an organization, as the worth of an organization's data will not be determined precisely. The paper has tremendous practical implications for organizations that intend to implement the data governance program and evaluate its state to design an improvement plan.

Originality/value

The paper proposes an approach for implementing data governance in an organization faced with limited resources for improvement.

Validity of Incident Opioid Use Disorder (OUD) Diagnoses in Administrative Data: a Chart Verification Study

BACKGROUND

An important strategy to address the opioid overdose epidemic involves identifying people at elevated risk of overdose, particularly those with opioid use disorder (OUD). However, it is unclear to what degree OUD diagnoses in administrative data are inaccurate.

OBJECTIVE

To estimate the prevalence of inaccurate diagnoses of OUD among patients with incident OUD diagnoses.

SUBJECTS

A random sample of 90 patients with incident OUD diagnoses associated with an index in-person encounter between October 1, 2016, and June 1, 2018, in three Veterans Health Administration medical centers.

DESIGN

Direct chart review of all encounter notes, referrals, prescriptions, and laboratory values within a 120-day window before and after the index encounter. Using all available chart data, we determined whether the diagnosis of OUD was likely accurate, likely inaccurate, or of indeterminate accuracy. We then performed a bivariate analysis to assess demographic or clinical characteristics associated with likely inaccurate diagnoses.

KEY RESULTS

We identified 1337 veterans with incident OUD diagnoses. In the chart verification subsample, we assessed 26 (29%) OUD diagnoses as likely inaccurate; 20 due to systems error and 6 due to clinical error; additionally, 8 had insufficient information to determine accuracy. Veterans with likely inaccurate diagnoses were more likely to be younger and prescribed opioids for pain. Clinical settings associated with likely inaccurate diagnoses were non-mental health clinical settings, group visits, and non-patient care settings.

CONCLUSIONS

Our study identified significant levels of likely inaccurate OUD diagnoses among veterans with incident OUD diagnoses. The majority of these cases reflected readily addressable systems errors. The smaller proportion due to clinical errors and those with insufficient documentation may be addressed by increased training for clinicians. If these inaccuracies are prevalent throughout the VHA, they could complicate health services research and health systems responses.

Electronic patient record and its effects on social aspects of interprofessional collaboration and clinical workflows in hospitals (eCoCo): a mixed methods study protocol

Background

The need for and usage of electronic patient records within hospitals has steadily increased over the last decade for economic reasons as well as the proceeding digitalization. While there are numerous benefits from this system, the potential risks of using electronic patient records for hospitals, patients and healthcare professionals must also be discussed. There is a lack in research, particularly regarding effects on healthcare professionals and their daily work in health services. The study eCoCo aims to gain insight into changes in interprofessional collaboration and clinical workflows resulting from introducing electronic patient records.

Methods

eCoCo is a multi-center case study integrating mixed methods from qualitative and quantitative social research. The case studies include three hospitals that undergo the process of introducing electronic patient records. Data are collected before and after the introduction of electronic patient records using participant observation, interviews, focus groups, time measurement, patient and employee questionnaires and a questionnaire to measure the level of digitalization. Furthermore, documents (patient records) as well as structural and administrative data are gathered. To analyze the interprofessional collaboration qualitative network analyses, reconstructive-hermeneutic analyses and document analyses are conducted. The workflow analyses, patient and employee assessment analyses and classification within the clinical adoption meta-model are conducted to provide insights into clinical workflows.

Discussion

This study will be the first to investigate the effects of introducing electronic patient records on interprofessional collaboration and clinical workflows from the perspective of healthcare professionals. Thereby, it will consider patients’ safety, legal and ethical concerns and quality of care. The results will help to understand the organization and thereby improve the performance of health services working with electronic patient records.

Trial registration

The study was registered at the German clinical trials register (DRKS00023343, Pre-Results) on November 17, 2020.

Assessing the practice of data quality evaluation in a national clinical data research network through a systematic scoping review in the era of real-world data

Objective

To synthesize data quality (DQ) dimensions and assessment methods of real-world data, especially electronic health records, through a systematic scoping review and to assess the practice of DQ assessment in the national Patient-centered Clinical Research Network (PCORnet).

Materials and Methods

We started with 3 widely cited DQ literature—2 reviews from Chan et al (2010) and Weiskopf et al (2013a) and 1 DQ framework from Kahn et al (2016)—and expanded our review systematically to cover relevant articles published up to February 2020. We extracted DQ dimensions and assessment methods from these studies, mapped their relationships, and organized a synthesized summarization of existing DQ dimensions and assessment methods. We reviewed the data checks employed by the PCORnet and mapped them to the synthesized DQ dimensions and methods.

Results

We analyzed a total of 3 reviews, 20 DQ frameworks, and 226 DQ studies and extracted 14 DQ dimensions and 10 assessment methods. We found that completeness, concordance, and correctness/accuracy were commonly assessed. Element presence, validity check, and conformance were commonly used DQ assessment methods and were the main focuses of the PCORnet data checks.

Discussion

Definitions of DQ dimensions and methods were not consistent in the literature, and the DQ assessment practice was not evenly distributed (eg, usability and ease-of-use were rarely discussed). Challenges in DQ assessments, given the complex and heterogeneous nature of real-world data, exist.

Conclusion

The practice of DQ assessment is still limited in scope. Future work is warranted to generate understandable, executable, and reusable DQ measures.

Validation process of a high-resolution database in a paediatric intensive care unit-Describing the perpetual patient's validation

RATIONALE

High data quality is essential to ensure the validity of clinical and research inferences based on it. However, these data quality assessments are often missing even though these data are used in daily practice and research.

AIMS AND OBJECTIVES

Our objective was to evaluate the data quality of our high-resolution electronic database (HRDB) implemented in our paediatric intensive care unit (PICU).

METHODS

We conducted a prospective validation study of a HRDB in a 32-bed paediatric medical, surgical, and cardiac PICU in a tertiary care freestanding maternal-child health centre in Canada. All patients admitted to the PICU with at least one vital sign monitored using a cardiorespiratory monitor connected to the central monitoring station.

RESULTS

Between June 2017 and August 2018, data from 295 patient days were recorded from medical devices and 4645 data points were video recorded and compared to the corresponding data collected in the HRDB. Statistical analysis showed an excellent overall correlation (R² = 1), accuracy (100%), agreement (bias = 0, limits of agreement = 0), completeness (2% missing data), and reliability (ICC = 1) between recorded and collected data within clinically significant pre-defined limits of agreement. Divergent points could all be explained.

CONCLUSIONS

This prospective validation of a representative sample showed an excellent overall data quality.

Pain Management in Clinical Practice Research Using Electronic Health Records

BACKGROUND

The use of electronic health record (EHR) systems encourages and facilitates the use of data for the development and surveillance of quality indicators, including pain management.

AIM

to conduct an integrative review on pain management research using data extracted from EHR in order to synthesize and analyze the following elements: pain management (assessments, interventions, and outcomes) and study results with potential clinical implications, data source, clinical sample characteristics, and method description.

DESIGN

An integrative review of the literature was undertaken to identify exemplars of scientific research studies that explore pain management using data from EHR, using Cooper's framework.

RESULTS

Our search of 1,061 records from PubMed, Scopus, and Cinahl was narrowed down to 28 eligible articles to be analyzed.

CONCLUSION

Results of this integrative review will make a critical contribution, assisting others in developing research proposals and sound research methods, as well as providing an overview of such studies over the past 10 years. Through this review it is therefore possible to guide new research on clinical pain management using EHR.

Aligning EHR Data for Pediatric Leukemia With Standard Protocol Therapy

Children with acute lymphoblastic leukemia (ALL) are treated according to risk-based protocols defined by the Children's Oncology Group (COG). Alignment between real-world clinical practice and protocol milestones is not widely understood. Aggregate deidentified electronic health record (EHR) data offer a useful resource to evaluate real-world clinical practice.

Treatment effects in randomised trials using routinely collected data for outcome assessment versus traditional trials: meta-research study

OBJECTIVE

To compare effect estimates of randomised clinical trials that use routinely collected data (RCD-RCT) for outcome ascertainment with traditional trials not using routinely collected data.

DESIGN

Meta-research study.

DATA SOURCE

Studies included in the same meta-analysis in a Cochrane review.

ELIGIBILITY CRITERIA FOR STUDY SELECTION

Randomised clinical trials using any type of routinely collected data for outcome ascertainment, including from registries, electronic health records, and administrative databases, that were included in a meta-analysis of a Cochrane review on any clinical question and any health outcome together with traditional trials not using routinely collected data for outcome measurement.

REVIEW METHODS

Effect estimates from trials using or not using routinely collected data were summarised in random effects meta-analyses. Agreement of (summary) treatment effect estimates from trials using routinely collected data and those not using such data was expressed as the ratio of odds ratios. Subgroup analyses explored effects in trials based on different types of routinely collected data. Two investigators independently assessed the quality of each data source.

RESULTS

84 RCD-RCTs and 463 traditional trials on 22 clinical questions were included. Trials using routinely collected data for outcome ascertainment showed 20% less favourable treatment effect estimates than traditional trials (ratio of odds ratios 0.80, 95% confidence interval 0.70 to 0.91, I2=14%). Results were similar across various types of outcomes (mortality outcomes: 0.92, 0.74 to 1.15, I2=12%; non-mortality outcomes: 0.71, 0.60 to 0.84, I2=8%), data sources (electronic health records: 0.81, 0.59 to 1.11, I2=28%; registries: 0.86, 0.75 to 0.99, I2=20%; administrative data: 0.84, 0.72 to 0.99, I2=0%), and data quality (high data quality: 0.82, 0.72 to 0.93, I2=0%).

CONCLUSIONS

Randomised clinical trials using routinely collected data for outcome ascertainment show smaller treatment benefits than traditional trials not using routinely collected data. These differences could have implications for healthcare decision making and the application of real world evidence.

Comparison of Nurse-Midwife and Physician Birth Outcomes in the Military Health System

The purpose of this study is to identify the socioeconomic and demographic characteristics of women cared for by Certified Nurse-Midwives (CNMs) versus physicians in the Military Health System (MHS) and compare birth outcomes between provider types. The MHS is one of America's largest and most complex health care systems. Using the Military Health System Data Repository, this retrospective study examined TRICARE beneficiaries who gave birth during 2012-2014. Analysis included frequency of patients by perinatal services, descriptive statistics, and logistic regression analysis by provider type. To account for differences in patient and pregnancy risk, odds ratios were calculated for both high-risk and general risk population. There were 136,848 births from 2012 to 2014, and 30.8% were delivered by CNMs. Low-risk women whose births were attended by CNMs had lower odds of a cesarean birth, induction/augmentation of labor, complications of birth, postpartum hemorrhage, endometritis, and preterm birth and higher odds of a vaginal birth, vaginal birth after cesarean, and breastfeeding than women whose births were attended by physicians. These results have implications for the composition of the women's health workforce. In the MHS, where CNMs work to the fullest scope of their authority, CNMs attended almost 4 times more births than our national average. An example to other U.S. systems and high-income countries, this study adds to the growing body of evidence demonstrating that when CNMs practice to the fullest extent of their education, they provide quality health outcomes to more women.

Characterization of bariatric surgery and outcomes using administrative claims data in the research network of a nationwide commercial health plan

Background

The supplementation of electronic health records data with administrative claims data may be used to capture outcome events more comprehensively in longitudinal observational studies. This study investigated the utility of administrative claims data to identify outcomes across health systems using a comparative effectiveness study of different types of bariatric surgery as a model.

Methods

This observational cohort study identified patients who had bariatric surgery between 2007 and 2015 within the HealthCore Anthem Research Network (HCARN) database in the National Patient-Centered Clinical Research Network (PCORnet) common data model. Patients whose procedures were performed in a member facility affiliated with PCORnet Clinical Research Networks (CRNs) were selected. The outcomes included a 30-day composite adverse event (including venous thromboembolism, percutaneous/operative intervention, failure to discharge and death), and all-cause hospitalization, abdominal operation or intervention, and in-hospital death up to 5 years after the procedure. Outcomes were classified as occurring within or outside PCORnet CRN health systems using facility identifiers.

Results

We identified 4899 patients who had bariatric surgery in one of the PCORnet CRN health systems. For 30-day composite adverse event, the inclusion of HCARN multi-site claims data marginally increased the incidence rate based only on HCARN single-site claims data for PCORnet CRNs from 3.9 to 4.2%. During the 5-year follow-up period, 56.8% of all-cause hospitalizations, 31.2% abdominal operations or interventions, and 32.3% of in-hospital deaths occurred outside PCORnet CRNs. Incidence rates (events per 100 patient-years) were significantly lower when based on claims from a single PCORnet CRN only compared to using claims from all health systems in the HCARN: all-cause hospitalization, 11.0 (95% Confidence Internal [CI]: 10.4, 11.6) to 25.3 (95% CI: 24.4, 26.3); abdominal operations or interventions, 4.2 (95% CI: 3.9, 4.6) to 6.1 (95% CI: 5.7, 6.6); in-hospital death, 0.2 (95% CI: 0.11, 0.27) to 0.3 (95% CI: 0.19, 0.38).

Conclusions

Short-term inclusion of multi-site claims data only marginally increased the incidence rate computed from single-site claims data alone. Longer-term follow up captured a notable number of events outside of PCORnet CRNs. The findings suggest that supplementing claims data improves the outcome ascertainment in longitudinal observational comparative effectiveness studies.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06074-3.

Utilization of intensive behavioural treatment for obesity in patients with diabetes

Obesity is a leading public health concern. The Centers for Medicare and Medicaid Services implemented a healthcare procedure code for intensive behavioural therapy (IBT) in 2012 to facilitate payment for addressing obesity in primary care settings, followed by universal coverage by insurers for all adults. Our objective was to understand utilization of IBT from 2009 to 2017 in patients with a diabetes diagnosis. Leveraging electronic health record data from the PaTH Clinical Data Research Network (CDRN), a partnership of six health systems, utilization of IBT was summarized at a yearly basis. The trend of IBT prevalence was examined for patients with diabetes by gender, race, age (>=65 vs <65) and rurality. A total of 205, 913 patients were included. While utilization of IBT is low (0.24% in 2017), use of IBT increased among patients with commercial insurance and Medicaid (codes S9449 and S9470) in 2011, and among patients with Medicare (code G0447) in 2012. IBT users tended to be less than 65 years of age, female, non-White (Black or Hispanic), and reside in urban areas. Overall, use of IBT in patients with diabetes remains low. Future work is necessary to understand the impact of IBT and, if effective, how to increase use within primary care.