Measuring data reliability for preventive services in electronic medical records

Case Identification of Depression in Inpatient Electronic Medical Records: Scoping Review

Background

Electronic medical records (EMRs) contain large amounts of detailed clinical information. Using medical record review to identify conditions within large quantities of EMRs can be time-consuming and inefficient. EMR-based phenotyping using machine learning and natural language processing algorithms is a continually developing area of study that holds potential for numerous mental health disorders.

Objective

This review evaluates the current state of EMR-based case identification for depression and provides guidance on using current algorithms and constructing new ones.

Methods

A scoping review of EMR-based algorithms for phenotyping depression was completed. This research encompassed studies published from January 2000 to May 2023. The search involved 3 databases: Embase, MEDLINE, and APA PsycInfo. This was carried out using selected keywords that fell into 3 categories: terms connected with EMRs, terms connected to case identification, and terms pertaining to depression. This study adhered to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) guidelines.

Results

A total of 20 papers were assessed and summarized in the review. Most of these studies were undertaken in the United States, accounting for 75% (15/20). The United Kingdom and Spain followed this, accounting for 15% (3/20) and 10% (2/20) of the studies, respectively. Both data-driven and clinical rule-based methodologies were identified. The development of EMR-based phenotypes and algorithms indicates the data accessibility permitted by each health system, which led to varying performance levels among different algorithms.

Conclusions

Better use of structured and unstructured EMR components through techniques such as machine learning and natural language processing has the potential to improve depression phenotyping. However, more validation must be carried out to have confidence in depression case identification algorithms in general.

A cohort study of engagement in telehealth psychotherapy versus in-person services

OBJECTIVE

Although telehealth psychotherapies have been studied for over 20 years, mental health services remained largely delivered in person until the COVID-19 pandemic forced clinics to reconsider the utility of telehealth psychotherapy. This study aims to compare patient engagement in in-person versus telehealth services in outpatient psychotherapy for mood and anxiety disorders.

METHOD

A cohort investigation was conducted, using a propensity score matched sample, extracted from an electronic health record (EHR) to compare engagement in psychotherapy for 762 patients who used in-person services before the pandemic to a cohort of 762 patients who used telehealth psychotherapy after the onset of COVID-19. The authors compared cohorts on initial engagement in psychotherapy services following an initial intake, number of psychotherapy sessions attended, and the rate of missed sessions.

RESULTS

There was a 26% increase in the total number of individual psychotherapy sessions attended when the clinics transitioned to telehealth services (p < .001). In addition, patients who received telehealth psychotherapy were five times more likely to not cancel or miss any scheduled sessions (p < .001).

CONCLUSION

These results indicate that telehealth services may result in improved treatment engagement for outpatient centers focused on brief evidence-based psychotherapies for mood and anxiety disorders.

Engaging the collective voice of physicians: Optimizing participation in research and policy development in the context of COVID-19 and physician burnout

Physicians and governments work collaboratively to determine optimal healthcare policy options. Physicians are also engaged by health researchers to participate in studies. Physician engagement can be impeded by limits on physician time and remuneration for engagement, and the impact of physician burnout (exacerbated by COVID-19). Doctors Nova Scotia engaged physicians on various research and policy items throughout the pandemic. Strategies included integrating physicians into research teams, remunerating engagement activities, and leveraging existing tools and networks. Health researchers and policy-makers can improve physician engagement through physician champions, reduction of research duplication, valuing of physician contributions, and integrating networks.

Data quality assessment in emergency medical services: an objective approach

BACKGROUND

In emergency medical services, high quality data are of great importance for patient care. Due to the unique nature of this type of services, the purpose of this study was to assess data quality in emergency medical services using an objective approach.

METHODS

This was a retrospective quantitative study conducted in 2019. The research sample included the emergency medical records of patients who referred to three emergency departments by the pre-hospital emergency care services (n = 384). Initially a checklist was designed based on the data elements of the triage form, pre-hospital emergency care form, and emergency medical records. Then, data completeness, accuracy and timeliness were assessed.

RESULTS

Data completeness in the triage form, pre-hospital emergency care form, and emergency medical records was 52.3%, 70% and 57.3%, respectively. Regarding data accuracy, most of the data elements were consistent. Measuring data timeliness showed that in some cases, paper-based ordering and computer-based data entry was not sequential.

CONCLUSION

Data quality in emergency medical services was not satisfactory and there were some weaknesses in the documentation processes. The results of this study can inform the clinical and administrative staff to pay more attentions to these weaknesses and plan for data quality improvement.

Data Quality in Health Care: Main Concepts and Assessment Methodologies

INTRODUCTION

 In the health care environment, a huge volume of data is produced on a daily basis. However, the processes of collecting, storing, sharing, analyzing, and reporting health data usually face with numerous challenges that lead to producing incomplete, inaccurate, and untimely data. As a result, data quality issues have received more attention than before.

OBJECTIVE

 The purpose of this article is to provide an insight into the data quality definitions, dimensions, and assessment methodologies.

METHODS

 In this article, a scoping literature review approach was used to describe and summarize the main concepts related to data quality and data quality assessment methodologies. Search terms were selected to find the relevant articles published between January 1, 2012 and September 31, 2022. The retrieved articles were then reviewed and the results were reported narratively.

RESULTS

 In total, 23 papers were included in the study. According to the results, data quality dimensions were various and different methodologies were used to assess them. Most studies used quantitative methods to measure data quality dimensions either in paper-based or computer-based medical records. Only two studies investigated respondents' opinions about data quality.

CONCLUSION

 In health care, high-quality data not only are important for patient care, but also are vital for improving quality of health care services and better decision making. Therefore, using technical and nontechnical solutions as well as constant assessment and supervision is suggested to improve data quality.

Reliability of trauma coding with ICD-10

PURPOSE

The reliability of trauma coding is essential in establishing the reliable trauma data and adopting efficient control and monitoring policies. The present study aimed to determine the reliability of trauma coding in educational hospitals affiliated to Shahid Beheshti University of Medical Sciences, Iran.

METHODS

In this descriptive cross-sectional study, 591 coded medical records with a trauma diagnosis in 2018 were selected and recoded by two coders. The reliability of trauma coding was calculated using Cohen's kappa. The data were recorded in a checklist, in which the validity of the content had been confirmed by experts.

RESULTS

The reliability of the coding related to the nature of trauma in research units was 0.75-0.77, indicating moderate reliability. Also, the reliability of the coding of external causes of trauma was 0.57-0.58, suggesting poor reliability.

CONCLUSION

The reliability of trauma coding both in terms of the nature of trauma and the external causes of trauma does not have a good status in the research units. This can be due to the complex coding of trauma, poor documentation of the cases, and not studying the entire case. Therefore, holding training courses for coders, offering training on the accurate documentation to other service providers, and periodically auditing the medical coding are recommended.

Data quality for federated medical data lakes

Purpose

Medical research requires biological material and data collected through biobanks in reliable processes with quality assurance. Medical studies based on data with unknown or questionable quality are useless or even dangerous, as evidenced by recent examples of withdrawn studies. Medical data sets consist of highly sensitive personal data, which has to be protected carefully and is available for research only after the approval of ethics committees. The purpose of this research is to propose an architecture to support researchers to efficiently and effectively identify relevant collections of material and data with documented quality for their research projects while observing strict privacy rules.

Design/methodology/approach

Following a design science approach, this paper develops a conceptual model for capturing and relating metadata of medical data in biobanks to support medical research.

Findings

This study describes the landscape of biobanks as federated medical data lakes such as the collections of samples and their annotations in the European federation of biobanks (Biobanking and Biomolecular Resources Research Infrastructure – European Research Infrastructure Consortium, BBMRI-ERIC) and develops a conceptual model capturing schema information with quality annotation. This paper discusses the quality dimensions for data sets for medical research in-depth and proposes representations of both the metadata and data quality documentation with the aim to support researchers to effectively and efficiently identify suitable data sets for medical studies.

Originality/value

This novel conceptual model for metadata for medical data lakes has a unique focus on the high privacy requirements of the data sets contained in medical data lakes and also stands out in the detailed representation of data quality and metadata quality of medical data sets.

Patient self-report versus medical records for smoking status and alcohol consumption at Aboriginal Community Controlled Health Services

OBJECTIVE

This study assessed the level of agreement, and predictors of agreement, between patient self-report and medical records for smoking status and alcohol consumption among patients attending one of four Aboriginal Community Controlled Health Service (ACCHSs).

METHODS

A convenience sample of 110 ACCHS patients self-reported whether they were current smokers or currently consumed alcohol. ACCHS staff completed a medical record audit for corresponding items for each patient. The level of agreement was evaluated using the kappa statistic. Factors associated with levels of agreement were explored using logistic regression.

RESULTS

The level of agreement between self-report and medical records was strong for smoking status (kappa=0.85; 95%CI: 0.75-0.96) and moderate for alcohol consumption (kappa=0.74; 95%CI: 0.60-0.88). None of the variables explored were significantly associated with levels of agreement for smoking status or alcohol consumption.

CONCLUSIONS

Medical records showed good agreement with patient self-report for smoking and alcohol status and are a reliable means of identifying potentially at-risk ACCHS patients. Implications for public health: ACCHS medical records are accurate for identifying smoking and alcohol risk factors for their patients. However, strategies to increase documentation and reduce missing data in the medical records are needed.

Conducting Population Health Research during the COVID-19 Pandemic: Impacts and Recommendations

The COVID-19 pandemic has resulted in many changes, including restrictions on indoor gatherings and visitation to residential aged care facilities, hospitals and certain communities. Coupled with potential restrictions imposed by health services and academic institutions, these changes may significantly impact the conduct of population health research. However, the continuance of population health research is beneficial for the provision of health services and sometimes imperative. This paper discusses the impact of COVID-19 restrictions on the conduct of population health research. This discussion unveils important ethical considerations, as well as potential impacts on recruitment methods, face-to-face data collection, data quality and validity. In addition, this paper explores potential recruitment and data collection methods that could replace face-to-face methods. The discussion is accompanied by reflections on the challenges experienced by the authors in their own research at an oral health service during the COVID-19 pandemic and alternative methods that were utilised in place of face-to-face methods. This paper concludes that, although COVID-19 presents challenges to the conduct of population health research, there is a range of alternative methods to face-to-face recruitment and data collection. These alternative methods should be considered in light of project aims to ensure data quality is not compromised.

Impact of the diabetes Canada guideline dissemination strategy on dispensed vascular protective medications for older patients in Ontario, Canada: a linked EMR and administrative data study

BACKGROUND

The 2013 Diabetes Canada guidelines recommended routinely using vascular protective medications for most patients with diabetes. These medications included statins and angiotensin-converting enzyme inhibitors (ACEIs) or angiotensin receptor blockers (ARBs). Antiplatelet agents were only recommended for secondary prevention of cardiovascular disease. Using Electronic Medical Record (EMR) data, we previously found that guideline dissemination efforts were not associated with an increase in the rate of primary care prescriptions of these medications. However, this needs confirmation: patients can receive prescriptions from different sources including specialists and they may not always fill these prescriptions. Using both EMR and administrative health data, we examined whether guideline dissemination impacted the dispensing of vascular protective medications to patients.

METHODS

The study population included patients with diabetes aged 66 or over in Ontario, Canada. We created two cohorts using two different approaches: an Electronic Medical Record (EMR) algorithm for diabetes using linked EMR-administrative data and an administrative algorithm using population level administrative data. We examined data from January 2010 to December 2016. Patients with diabetes were deemed to be likely taking a medication (or covered) during a quarter if the daily amount for a dispensed medication would last for at least 75% of days in any given quarter. An interrupted time series analysis was used to assess the proportion of patients covered by each medication class. Proton pump inhibitors (PPIs) were used as a reference.

RESULTS

There was no increase in the rate of change for medication coverage following guideline release in either the EMR or the administrative diabetes cohorts. For statins, the change in trend was - 0.03, p = 0.7 (EMR) and - 0.12, p = 0.04(administrative). For ACEI/ARBs, this was 0.03, p = 0.6 (EMR) and 0, p = 1(administrative). For antiplatelets, this was 0.001, P = .97 (EMR) and - 0.03, p = 0.03 (administrative). The comparator PPI was - 0.07, p = 0.4 (EMR) and - 0.11, p = 0.002 (administrative).

CONCLUSIONS

Using both EMR and administrative health data, we confirmed that the Diabetes Canada 2013 guideline dissemination strategy did not lead to an increased rate of coverage for vascular protective medications. Alternative strategies are needed to effect change in practice.

Applications of Machine Learning Using Electronic Medical Records in Spine Surgery

Developments in machine learning in recent years have precipitated a surge in research on the applications of artificial intelligence within medicine. Machine learning algorithms are beginning to impact medicine broadly, and the field of spine surgery is no exception. Electronic medical records are a key source of medical data that can be leveraged for the creation of clinically valuable machine learning algorithms. This review examines the current state of machine learning using electronic medical records as it applies to spine surgery. Studies across the electronic medical record data domains of imaging, text, and structured data are reviewed. Discussed applications include clinical prognostication, preoperative planning, diagnostics, and dynamic clinical assistance, among others. The limitations and future challenges for machine learning research using electronic medical records are also discussed.

Prevalence of drooling, swallowing, and feeding problems in cerebral palsy across the lifespan: a systematic review and meta-analyses

AIM

To determine the prevalence of drooling, swallowing, and feeding problems in persons with cerebral palsy (CP) across the lifespan.

METHOD

A systematic review was conducted using five different databases (AMED, CINAHL, Embase, MEDLINE, and PubMed). The selection process was completed by two independent researchers and the methodological quality of included studies was assessed using the STROBE and AXIS guidelines. Meta-analyses were conducted to determine pooled prevalence estimates of drooling, swallowing, and feeding problems with stratified group analyses by type of assessment and Gross Motor Function Classification System level.

RESULTS

A total of 42 studies were included. Substantial variations in selected outcome measures and variables were observed, and data on adults were limited. Pooled prevalence estimates determined by meta-analyses were as high as 44.0% (95% confidence interval [CI] 35.6-52.7) for drooling, 50.4% (95% CI 36.0-64.8) for swallowing problems, and 53.5% (95% CI 40.7-65.9) for feeding problems. Group analyses for type of assessments were non-significant; however, more severely impaired functioning in CP was associated with concomitant problems of increased drooling, swallowing, and feeding.

INTERPRETATION

Drooling, swallowing, and feeding problems are very common in people with CP. Consequently, they experience increased risks of malnutrition and dehydration, aspiration pneumonia, and poor quality of life.

WHAT THIS PAPER ADDS

Drooling, swallowing, and feeding problems are very common in persons with cerebral palsy (CP). The prevalence of drooling, swallowing, and feeding problems is 44.0%, 50.4%, and 53.5% respectively. There are limited data on the prevalence of drooling, swallowing, and feeding problems in adults. Higher Gross Motor Function Classification System levels are associated with higher prevalence of drooling, swallowing, and feeding problems. There is increased risk for malnutrition, dehydration, aspiration pneumonia, and poor quality of life in CP.

Population segmentation based on healthcare needs: a systematic review

BACKGROUND

Healthcare needs-based population segmentation is a promising approach for enabling the development and evaluation of integrated healthcare service models that meet healthcare needs. However, healthcare policymakers interested in understanding adult population healthcare needs may not be aware of suitable population segmentation tools available for use in the literature and barring better-known alternatives, may reinvent the wheel by creating and validating their own tools rather than adapting available tools in the literature. Therefore, we undertook a systematic review to identify all available tools which operationalize healthcare need-based population segmentation, to help inform policymakers developing population-level health service programmes.

METHODS

Using search terms reflecting concepts of population, healthcare need and segmentation, we systematically reviewed and included articles containing healthcare need-based adult population segmentation tools in PubMed, CINAHL and Web of Science databases. We included tools comprising mutually exclusive segments with prognostic value for clinically relevant outcomes. An updated secondary search on the PubMed database was also conducted as the last search was conducted 2 years ago. All identified tools were characterized in terms of segment formulation, segmentation base, whether they received peer-reviewed validation, requirement for comprehensive electronic medical records, proprietary status and number of segments.

RESULTS

A total of 16 unique tools were identified from systematically reviewing 9970 articles. Peer-reviewed validation studies were found for 9 of these tools.

DISCUSSION AND CONCLUSIONS

The underlying segmentation basis of most identified tools was found to be conceptually comparable to each other which suggests a broad recognition of archetypical patient overall healthcare need profiles. While many tools operate based on administrative record data, it is noted that healthcare systems without comprehensive electronic medical records would benefit from tools which segment populations through primary data collection. Future work could therefore include development and validation of such primary data collection-based tools. While this study is limited by exclusion of non-English literature, the identified and characterized tools will nonetheless facilitate efforts by policymakers to improve patient-centred care through development and evaluation of services tailored for specific populations segmented by these tools.

A basic model for assessing primary health care electronic medical record data quality

Background

The increased use of electronic medical records (EMRs) in Canadian primary health care practice has resulted in an expansion of the availability of EMR data. Potential users of these data need to understand their quality in relation to the uses to which they are applied. Herein, we propose a basic model for assessing primary health care EMR data quality, comprising a set of data quality measures within four domains. We describe the process of developing and testing this set of measures, share the results of applying these measures in three EMR-derived datasets, and discuss what this reveals about the measures and EMR data quality. The model is offered as a starting point from which data users can refine their own approach, based on their own needs.

Methods

Using an iterative process, measures of EMR data quality were created within four domains: comparability; completeness; correctness; and currency. We used a series of process steps to develop the measures. The measures were then operationalized, and tested within three datasets created from different EMR software products.

Results

A set of eleven final measures were created. We were not able to calculate results for several measures in one dataset because of the way the data were collected in that specific EMR. Overall, we found variability in the results of testing the measures (e.g. sensitivity values were highest for diabetes, and lowest for obesity), among datasets (e.g. recording of height), and by patient age and sex (e.g. recording of blood pressure, height and weight).

Conclusions

This paper proposes a basic model for assessing primary health care EMR data quality. We developed and tested multiple measures of data quality, within four domains, in three different EMR-derived primary health care datasets. The results of testing these measures indicated that not all measures could be utilized in all datasets, and illustrated variability in data quality. This is one step forward in creating a standard set of measures of data quality. Nonetheless, each project has unique challenges, and therefore requires its own data quality assessment before proceeding.

Electronic supplementary material

The online version of this article (10.1186/s12911-019-0740-0) contains supplementary material, which is available to authorized users.

Reliability of a Canadian database for primary care nursing services' clinical and administrative data

BACKGROUND

The use of electronic clinical and administrative data can be an advantageous source of information for assessing nursing performance in primary care. In Québec (Canada), the I-CLSC electronic database could be used to measure performance indicators. However, little is known about the reliability of the data contained in this database. The objective of this study was to assess the reliability of the clinical and administrative data contained in the I-CLSC electronic database based on the data entered in medical records.

METHODS

We used a longitudinal design for this study. A sample of 100 patients who had experienced 107 episodes of wound care were randomly selected from all patients who had two or more consultations during the year 2015. The paper records were used as reference. We collected data regarding eight nursing sensitive indicators from both sources. We assessed the concordance between the electronic data and the paper records by measuring inter-rater agreement.

RESULTS

Six of the eight indicators showed a percentage agreement ≥ 85%, and kappa scores between 0.7 and 1.00 (p < 0.001), indicating high to perfect levels of agreement between the two data sources. Two indicators presented fair kappa scores.

CONCLUSION

This database provides reliable data relating to the organization of care but shows lower reliability for specific acts performed by nurses in primary care. This existing database can be used to assess, manage and improve certain dimensions of nursing performance in primary care.

Evaluation of data quality of interRAI assessments in home and community care

BACKGROUND

The aim of this project is to describe the quality of assessment data regularly collected in home and community, with techniques adapted from an evaluation of the quality of long-term care data in Canada.

METHODS

Data collected using the Resident Assessment Instrument - Home Care (RAI-HC) in Ontario and British Columbia (BC) as well as the interRAI Community Health Assessment (CHA) in Ontario were analyzed using descriptive statistics, Pearson's r correlation, and Cronbach's alpha in order to assess trends in population characteristics, convergent validity, and scale reliability.

RESULTS

Results indicate that RAI-HC data from Ontario and BC behave in a consistent manner, with stable trends in internal consistency providing evidence of good reliability (alpha values range from 0.72-0.94, depending on the scale and province). The associations between various scales, such as those reflecting functional status and cognition, were found to be as expected and stable over time within each setting (r values range from 0.42-0.45 in Ontario and 0.41-0.43 in BC). These trends in convergent validity demonstrate that constructs in the data behave as they should, providing evidence of good data quality. In most cases, CHA data quality matches that of RAI-HC data quality and shows evidence of good validity and reliability. The findings are comparable to the findings observed in the evaluation of data from the long-term care sector.

CONCLUSIONS

Despite an increasingly complex client population in the home and community care sectors, the results from this work indicate that data collected using the RAI-HC and the CHA are of an overall quality that may be trusted when used to inform decision-making at the organizational- or policy-level. High quality data and information are vital when used to inform steps taken to improve quality of care and enhance quality of life. This work also provides evidence that a method used to evaluate the quality of data obtained in the long-term care setting may be used to evaluate the quality of data obtained through community-based measures.

Data quality in electronic medical records in Manitoba: Do problem lists reflect chronic disease as defined by prescriptions?

OBJECTIVE

To determine if the problem list (health conditions) in primary care electronic medical records (EMRs) accurately reflects the conditions for which chronic medications are prescribed in the EMR.

DESIGN

A retrospective analysis of EMR data.

SETTING

Eighteen primary care clinics across rural and urban Manitoba using the Accuro EMR.

PARTICIPANTS

Data from the EMRs of active patients seen in an 18-month period (December 18, 2011, to June 18, 2013, or December 3, 2012, to June 3, 2014) were used.

MAIN OUTCOME MEASURES

The likelihood of documentation in the EMR problem list of those specific chronic diseases for which drug prescriptions were documented in the EMR. Regression modeling was performed to determine the effect of clinic patient load and remuneration type on the completeness of EMR problem lists.

RESULTS

Overall problem-list completeness was low but was highest for diabetes and lowest for insomnia. Fee-for-service clinics generally had lower problem-list completeness than salaried clinics did for all prescription medications examined. Panel size did not affect problem-list completeness rates.

CONCLUSION

The low EMR problem-list completeness suggests that this field is not reliable for use in quality improvement initiatives or research until higher reliability has been demonstrated. Further research is recommended to explore the reasons for the poor quality and to support improvement efforts.

Quality of routine health data collected by health workers using smartphone at primary health care in Ethiopia

BACKGROUND

Mobile phone based applications are considered by many as potentially useful for addressing challenges and improving the quality of data collection in developing countries. Yet very little evidence is available supporting or refuting the potential and widely perceived benefits on the use of electronic forms on smartphones for routine patient data collection by health workers at primary health care facilities.

METHODS

A facility based cross sectional study using a structured paper checklist was prepared to assess the completeness and accuracy of 408 electronic records completed and submitted to a central database server using electronic forms on smartphones by 25 health workers. The 408 electronic records were selected randomly out of a total of 1772 maternal health records submitted by the health workers to the central database over a period of six months. Descriptive frequencies and percentages of data completeness and error rates were calculated.

RESULTS

When compared to paper records, the use of electronic forms significantly improved data completeness by 209 (8%) entries. Of a total 2622 entries checked for completeness, 2602 (99.2%) electronic record entries were complete, while 2393 (91.3%) paper record entries were complete. A very small percentage of error rates, which was easily identifiable, occurred in both electronic and paper forms although the error rate in the electronic records was more than double that of paper records (2.8% vs. 1.1%). More than half of entry errors in the electronic records related to entering a text value.

CONCLUSIONS

With minimal training, supervision, and no incentives, health care workers were able to use electronic forms for patient assessment and routine data collection appropriately and accurately with a very small error rate. Minimising the number of questions requiring text responses in electronic forms would be helpful in minimizing data errors.

Improving Completeness of Inpatient Medical Records in Menelik II Referral Hospital, Addis Ababa, Ethiopia

Introduction. The incompleteness of medical records is a significant problem that affects the quality of health care services in many hospitals of Ethiopia. Improving the completeness of patient’s records is an important step towards improving the quality of healthcare. Methods. Pre- and postintervention study was conducted to assess improvement of inpatient medical record completeness in Menelik II Referral Hospital from September 2015 to April 2016. Simple random sampling technique was used. Data was collected using data extraction checklist and independent sample t-test was used to compare statistical difference that exists between pre- and postintervention outcomes at confidence interval of 95% and P value less than 0.05 was considered statistically significant. Result. The overall inpatient medical record completeness was found to be 84% after intervention. An enhancement of completeness and reporting of inpatient medical record completeness increased significantly from the baseline 73% to 84% during postintervention evaluation at P value < 0.05. Conclusion and Recommendation. The finding of this project suggests that a simple set of interventions comprising inpatient medical record format and training healthcare provider showed a significant improvement in inpatient medical record completeness. The Quality Officer and Chief Executive Officer of the study hospital are recommended to design and launch intervention programs to improve medical record completeness.

Improving the quality of EHR recording in primary care: a data quality feedback tool

OBJECTIVE

Electronic health record (EHR) data are used to exchange information among health care providers. For this purpose, the quality of the data is essential. We developed a data quality feedback tool that evaluates differences in EHR data quality among practices and software packages as part of a larger intervention.

METHODS

The tool was applied in 92 practices in the Netherlands using different software packages. Practices received data quality feedback in 2010 and 2012.

RESULTS

We observed large differences in the quality of recording. For example, the percentage of episodes of care that had a meaningful diagnostic code ranged from 30% to 100%. Differences were highly related to the software package. A year after the first measurement, the quality of recording had improved significantly and differences decreased, with 67% of the physicians indicating that they had actively changed their recording habits based on the results of the first measurement. About 80% found the feedback helpful in pinpointing recording problems. One of the software vendors made changes in functionality as a result of the feedback.

CONCLUSIONS

Our EHR data quality feedback tool is capable of highlighting differences among practices and software packages. As such, it also stimulates improvements. As substantial variability in recording is related to the software package, our study strengthens the evidence that data quality can be improved substantially by standardizing the functionalities of EHR software packages.

Monitoring Prevalence, Treatment, and Control of Metabolic Conditions in New York City Adults Using 2013 Primary Care Electronic Health Records: A Surveillance Validation Study

Introduction:

Electronic health records (EHRs) can potentially extend chronic disease surveillance, but few EHR-based initiatives tracking population-based metrics have been validated for accuracy. We designed a new EHR-based population health surveillance system for New York City (NYC) known as NYC Macroscope. This report is the third in a 3-part series describing the development and validation of that system. The first report describes governance and technical infrastructure underlying the NYC Macroscope. The second report describes validation methods and presents validation results for estimates of obesity, smoking, depression and influenza vaccination. In this third paper we present validation findings for metabolic indicators (hypertension, hyperlipidemia, diabetes).

Methods:

We compared EHR-based estimates to those from a gold standard surveillance source - the 2013–2014 NYC Health and Nutrition Examination Survey (NYC HANES) - overall and stratified by sex and age group, using the two one-sided test of equivalence and other validation criteria.

Results:

EHR-based hypertension prevalence estimates were highly concordant with NYC HANES estimates. Diabetes prevalence estimates were highly concordant when measuring diagnosed diabetes but less so when incorporating laboratory results. Hypercholesterolemia prevalence estimates were less concordant overall. Measures to assess treatment and control of the 3 metabolic conditions performed poorly.

Discussion:

While indicator performance was variable, findings here confirm that a carefully constructed EHR-based surveillance system can generate prevalence estimates comparable to those from gold-standard examination surveys for certain metabolic conditions such as hypertension and diabetes.

Conclusions:

Standardized EHR metrics have potential utility for surveillance at lower annual costs than surveys, especially as representativeness of contributing clinical practices to EHR-based surveillance systems increases.

Prevalence of toddler, child and adolescent overweight and obesity derived from primary care electronic medical records: an observational study

BACKGROUND

Population monitoring and surveillance of objectively measured child weight data in Canada is limited to national surveys with poor regional applicability, and no healthy weight data are available for children less than 2 years of age. We aimed to determine the prevalence of childhood overweight and obesity using objective measures derived from primary care electronic medical records.

METHODS

Observational data included all height and weight records for children less than 20 years of age, between 2004 and 2013, from 3 Ontario primary care research networks. We calculated body mass index (BMI)-for-age and weight-for-length using the World Health Organization Growth Standards and Reference to assign growth status indicator categories by age group. Descriptive data and prevalence estimates were generated for 2013. We also compared weight-for-length for children less than 2 years of age with a corresponding billing code for known well-child visits.

RESULTS

Our study included 8261 children with a corresponding growth status indicator, a sample close to 4 times larger than the national survey sample. In 2013, 28.4% of children aged 5-19 years, and 6% of children aged 0-5 years, were categorized as overweight or obese. Between 2008 and 2013, the total number of 18-month well baby visit billing codes was 1152; 6.9% of this group were categorized as overweight or obese; 19.2% were categorized as having risk of overweight.

INTERPRETATION

Primary care electronic medical records show good potential for ongoing population monitoring of overweight and obesity, particularly for very young children for whom early intervention is likely to show the greatest positive health impact.

Using a Data Quality Framework to Clean Data Extracted from the Electronic Health Record: A Case Study

Objectives:

We examine the following: (1) the appropriateness of using a data quality (DQ) framework developed for relational databases as a data-cleaning tool for a data set extracted from two EPIC databases, and (2) the differences in statistical parameter estimates on a data set cleaned with the DQ framework and data set not cleaned with the DQ framework.

Background:

The use of data contained within electronic health records (EHRs) has the potential to open doors for a new wave of innovative research. Without adequate preparation of such large data sets for analysis, the results might be erroneous, which might affect clinical decision-making or the results of Comparative Effectives Research studies.

Methods:

Two emergency department (ED) data sets extracted from EPIC databases (adult ED and children ED) were used as examples for examining the five concepts of DQ based on a DQ assessment framework designed for EHR databases. The first data set contained 70,061 visits; and the second data set contained 2,815,550 visits. SPSS Syntax examples as well as step-by-step instructions of how to apply the five key DQ concepts these EHR database extracts are provided.

Conclusions:

SPSS Syntax to address each of the DQ concepts proposed by Kahn et al. (2012)1 was developed. The data set cleaned using Kahn’s framework yielded more accurate results than the data set cleaned without this framework. Future plans involve creating functions in R language for cleaning data extracted from the EHR as well as an R package that combines DQ checks with missing data analysis functions.

Consistency of denominator data in electronic health records in Australian primary healthcare services: enhancing data quality

The quality of data derived from primary healthcare electronic systems has been subjected to little critical systematic analysis, especially in relation to the purported benefits and substantial investment in electronic information systems in primary care. Many indicators of quality of care are based on numbers of certain types of patients as denominators. Consistency of denominator data is vital for comparison of indicators over time and between services. This paper examines the consistency of denominator data extracted from electronic health records (EHRs) for monitoring of access and quality of primary health care. Data collection and analysis were conducted as part of a prospective mixed-methods formative evaluation of the Commonwealth Government's Indigenous Chronic Disease Package. Twenty-six general practices and 14 Aboriginal Health Services (AHSs) located in all Australian States and Territories and in urban, regional and remote locations were purposively selected within geographically defined locations. Percentage change in reported number of regular patients in general practices ranged between -50% and 453% (average 37%). The corresponding figure for AHSs was 1% to 217% (average 31%). In approximately half of general practices and AHSs, the change was ≥ 20%. There were similarly large changes in reported numbers of patients with a diagnosis of diabetes or coronary heart disease (CHD), and Indigenous patients. Inconsistencies in reported numbers were due primarily to limited capability of staff in many general practices and AHSs to accurately enter, manage, and extract data from EHRs. The inconsistencies in data required for the calculation of many key indicators of access and quality of care places serious constraints on the meaningful use of data extracted from EHRs. There is a need for greater attention to quality of denominator data in order to realise the potential benefits of EHRs for patient care, service planning, improvement, and policy. We propose a quality improvement approach for enhancing data quality.

Utility of linking primary care electronic medical records with Canadian census data to study the determinants of chronic disease: an example based on socioeconomic status and obesity

Background

Electronic medical records (EMRs) used in primary care contain a breadth of data that can be used in public health research. Patient data from EMRs could be linked with other data sources, such as a postal code linkage with Census data, to obtain additional information on environmental determinants of health. While promising, successful linkages between primary care EMRs with geographic measures is limited due to ethics review board concerns. This study tested the feasibility of extracting full postal code from primary care EMRs and linking this with area-level measures of the environment to demonstrate how such a linkage could be used to examine the determinants of disease. The association between obesity and area-level deprivation was used as an example to illustrate inequalities of obesity in adults.

Methods

The analysis included EMRs of 7153 patients aged 20 years and older who visited a single, primary care site in 2011. Extracted patient information included demographics (date of birth, sex, postal code) and weight status (height, weight). Information extraction and management procedures were designed to mitigate the risk of individual re-identification when extracting full postal code from source EMRs. Based on patients’ postal codes, area-based deprivation indexes were created using the smallest area unit used in Canadian censuses. Descriptive statistics and socioeconomic disparity summary measures of linked census and adult patients were calculated.

Results

The data extraction of full postal code met technological requirements for rendering health information extracted from local EMRs into anonymized data. The prevalence of obesity was 31.6 %. There was variation of obesity between deprivation quintiles; adults in the most deprived areas were 35 % more likely to be obese compared with adults in the least deprived areas (Chi-Square = 20.24(1), p < 0.0001). Maps depicting spatial representation of regional deprivation and obesity were created to highlight high risk areas.

Conclusions

An area based socio-economic measure was linked with EMR-derived objective measures of height and weight to show a positive association between area-level deprivation and obesity. The linked dataset demonstrates a promising model for assessing health disparities and ecological factors associated with the development of chronic diseases with far reaching implications for informing public health and primary health care interventions and services.

Mixed Influence of Electronic Health Record Implementation on Diabetes Order Patterns for Michigan Medicaid Adults

BACKGROUND

These 2011-2013 analyses examined the authors' hypothesis that relative diabetes care order changes would be measured after electronic health record (EHR) implementation for 291 Medicaid adults who received all of their office-based care at one midwestern federally qualified health center (FQHC) over a 24-month period (n = 2727 encounters, 2489 claims).

METHOD

Beneficiary sociodemographic, clinical, and claims data were validated with clinic EHR and state Medicaid claims linked to providers' national identifier numbers. Overall pre-post order rate comparisons, and a series of controlled within group binary logistic models were conducted under penalized maximum likelihood estimation terms.

RESULTS

After EHR implementation, both the overall order rates and odds ratios of per beneficiary hemoglobin A1C (HbA1C) orders increased significantly (ie, from mean of 0.65 [SD = 1.19] annual tests to 0.96 tests [SD = 1.24] [P < .001]). Although the overall post-EHR order rates of dilated eye exams and microalbumin urine tests appeared fairly stable, the odds of eye exam orders being placed at the claims level decreased significantly (OR = 0.774, P = .0030).

CONCLUSIONS

These mixed results provide evidence of the varied diabetes care ordering patterns likely seen from increased office use of EHR technologies. The authors attempt to explain these post-EHR differences (or lack of) that generally resemble some of the authors' results from another funded project. Ideally, these findings provide Medicaid and health care officials with a more realistic indication of how EHRs may, or may not, influence diabetes care ordering patterns for vulnerable lower-income primary health care consumers.

Are family physicians comprehensively using electronic medical records such that the data can be used for secondary purposes? A Canadian perspective

Background

With the introduction and implementation of a variety of government programs and policies to encourage adoption of electronic medical records (EMRs), EMRs are being increasingly adopted in North America. We sought to evaluate the completeness of a variety of EMR fields to determine if family physicians were comprehensively using their EMRs and the suitability of use of the data for secondary purposes in Ontario, Canada.

Methods

We examined EMR data from a convenience sample of family physicians distributed throughout Ontario within the Electronic Medical Record Administrative data Linked Database (EMRALD) as extracted in the summer of 2012. We identified all physicians with at least one year of EMR use. Measures were developed and rates of physician documentation of clinical encounters, electronic prescriptions, laboratory tests, blood pressure and weight, referrals, consultation letters, and all fields in the cumulative patient profile were calculated as a function of physician and patient time since starting on the EMR.

Results

Of the 167 physicians with at least one year of EMR use, we identified 186,237 patients. Overall, the fields with the highest level of completeness were for visit documentations and prescriptions (>70 %). Improvements were observed with increasing trends of completeness overtime for almost all EMR fields according to increasing physician time on EMR. Assessment of the influence of patient time on EMR demonstrated an increasing likelihood of the population of EMR fields overtime, with the largest improvements occurring between the first and second years.

Conclusions

All of the data fields examined appear to be reasonably complete within the first year of adoption with the biggest increase occurring the first to second year. Using all of the basic functions of the EMR appears to be occurring in the current environment of EMR adoption in Ontario. Thus the data appears to be suitable for secondary use.

Variability in screening prevention activities in primary care in Spain: a multilevel analysis

BACKGROUND

Despite evidence of the benefits of prevention activities, studies have reported only partial integration and great variability of screening in daily clinical practice. The study objectives were: 1) To describe Primary Health Care (PHC) screening for arterial hypertension, dyslipidaemia, obesity, tobacco use, and excessive alcohol consumption in 2008 in 2 regions of Spain, based on electronic health records, and 2) To assess and quantify variability in screening, and identify factors (of patient, general practitioners and PHC team) associated with being screened, that are common throughout the PHC population.

METHODS

Multicentre, cross-sectional study of individuals aged ≥ 16 years (N = 468,940) who visited the 426 general practitioners (GPs) in 44 PHC teams in Catalonia and Navarre in 2008.

OUTCOMES

screening for hypertension, dyslipidaemia, obesity, tobacco use, and excessive alcohol consumption. Other variables were considered at the individual (sociodemographics, visits, health problems), GP and PHC team (region among others). Individual and contextual factors associated with the odds of being screened and the variance attributable to each level were identified using the SAS PROC GLIMMIX macro.

RESULTS

The most prevalent screenings were for dyslipidaemia (64.4%) and hypertension (50.8%); the least prevalent was tobacco use (36.6%). Overall, the odds of being screened were higher for women, older patients, those with more comorbidities, more cardiovascular risk factors, and more frequent office visits, and those assigned to a female GP, a GP with a lower patient load, or a PHC team with a lower percentage of patients older than 65 years. On average, individuals in Navarre were less likely to be screened than those in Catalonia. Hypertension and dyslipidaemia screenings had the least unexplained variability between PHC teams and GPs, respectively, after adjusting for individual and contextual factors.

CONCLUSIONS

Of the studied screenings, those for obesity, tobacco, and alcohol use were the least prevalent. Attention to screening, especially for tobacco and alcohol, can be greatly improved in the PHC setting.

Reasons for non-response to a direct-mailed FIT kit program: lessons learned from a pragmatic colorectal-cancer screening study in a federally sponsored health center

Colorectal cancer screening rates are below optimal. As part of a pilot clinic-based pragmatic study aiming to raise rates of colorectal-cancer screening, we explored patients' reasons for not responding to a direct-mailed screening invitation. We conducted telephone interviews with patients who were mailed a fecal immunochemical test (FIT) but who did not return it to the lab. Interviews were audio-recorded, transcribed, and coded for thematic analysis. We met our goal of 20 interviews (10 in English and 10 Spanish; 75 % female). Reasons for not completing tests were fear of results or cost of follow-up colonoscopy (n = 9); not having received the test in the mail (n = 7); concerns about mailing fecal matter or that test results could be mixed up (n = 6); and being busy or forgetful (n = 4). Efforts to improve uptake of colorectal cancer screening in a direct-mailed program ought to address concerns identified in our study.

Identification of Relationships Between Patients Through Elements in a Data Warehouse Using the Familial, Associational, and Incidental Relationship (FAIR) Initiative: A Pilot Study

Background

Over the last several years there has been widespread development of medical data warehouses. Current data warehouses focus on individual cases, but lack the ability to identify family members that could be used for dyadic or familial research. Currently, the patient’s family history in the medical record is the only documentation we have to understand the health status and social habits of their family members. Identifying familial linkages in a phenotypic data warehouse can be valuable in cohort identification and in beginning to understand the interactions of diseases among families.

Objective

The goal of the Familial, Associational, & Incidental Relationships (FAIR) initiative is to identify an index set of patients’ relationships through elements in a data warehouse.

Methods

Using a test set of 500 children, we measured the sensitivity and specificity of available linkage algorithm identifiers (eg, insurance identification numbers and phone numbers) and validated this tool/algorithm through a manual chart audit.

Results

Of all the children, 52.4% (262/500) were male, and the mean age of the cohort was 8 years old (SD 5). Of the children, 51.6% (258/500) were identified as white in race. The identifiers used for FAIR were available for the majority of patients: insurance number (483/500, 96.6%), phone number (500/500, 100%), and address (497/500, 99.4%). When utilizing the FAIR tool and various combinations of identifiers, sensitivity ranged from 15.5% (62/401) to 83.8% (336/401), and specificity from 72% (71/99) to 100% (99/99). The preferred method was matching patients using insurance or phone number, which had a sensitivity of 72.1% (289/401) and a specificity of 94% (93/99). Using the Informatics for Integrating Biology and the Bedside (i2b2) warehouse infrastructure, we have now developed a Web app that facilitates FAIR for any index population.

Conclusions

FAIR is a valuable research and clinical resource that extends the capabilities of existing data warehouses and lays the groundwork for family-based research. FAIR will expedite studies that would otherwise require registry or manual chart abstraction data sources.

Electronic health records and community health surveillance of childhood obesity

BACKGROUND

Childhood obesity remains a public health concern, and tracking local progress may require local surveillance systems. Electronic health record data may provide a cost-effective solution.

PURPOSE

To demonstrate the feasibility of estimating childhood obesity rates using de-identified electronic health records for the purpose of public health surveillance and health promotion.

METHODS

Data were extracted from the Public Health Information Exchange (PHINEX) database. PHINEX contains de-identified electronic health records from patients primarily in south central Wisconsin. Data on children and adolescents (aged 2-19 years, 2011-2012, n=93,130) were transformed in a two-step procedure that adjusted for missing data and weighted for a national population distribution. Weighted and adjusted obesity rates were compared to the 2011-2012 National Health and Nutrition Examination Survey (NHANES). Data were analyzed in 2014.

RESULTS

The weighted and adjusted obesity rate was 16.1% (95% CI=15.8, 16.4). Non-Hispanic white children and adolescents (11.8%, 95% CI=11.5, 12.1) had lower obesity rates compared to non-Hispanic black (22.0%, 95% CI=20.7, 23.2) and Hispanic (23.8%, 95% CI=22.4, 25.1) patients. Overall, electronic health record-derived point estimates were comparable to NHANES, revealing disparities from preschool onward.

CONCLUSIONS

Electronic health records that are weighted and adjusted to account for intrinsic bias may create an opportunity for comparing regional disparities with precision. In PHINEX patients, childhood obesity disparities were measurable from a young age, highlighting the need for early intervention for at-risk children. The electronic health record is a cost-effective, promising tool for local obesity prevention efforts.

Data Integration Protocol In Ten-steps (DIPIT): a new standard for medical researchers

INTRODUCTION

The exponential increase in data, computing power and the availability of readily accessible analytical software has allowed organisations around the world to leverage the benefits of integrating multiple heterogeneous data files for enterprise-level planning and decision making. Benefits from effective data integration to the health and medical research community include more trustworthy research, higher service quality, improved personnel efficiency, reduction of redundant tasks, facilitation of auditing and more timely, relevant and specific information. The costs of poor quality processes elevate the risk of erroneous outcomes, an erosion of confidence in the data and the organisations using these data. To date there are no documented set of standards for best practice integration of heterogeneous data files for research purposes. Therefore, the aim of this paper is to describe a set of clear protocol for data file integration (Data Integration Protocol In Ten-steps; DIPIT) translational to any field of research.

METHODS AND RESULTS

The DIPIT approach consists of a set of 10 systematic methodological steps to ensure the final data are appropriate for the analysis to meet the research objectives, legal and ethical requirements are met, and that data definitions are clear, concise, and comprehensive. This protocol is neither file specific nor software dependent, but aims to be transportable to any data-merging situation to minimise redundancy and error and translational to any field of research. DIPIT aims to generate a master data file that is of the optimal integrity to serve as the basis for research analysis.

CONCLUSION

With linking of heterogeneous data files becoming increasingly common across all fields of medicine, DIPIT provides a systematic approach to a potentially complex task of integrating a large number of files and variables. The DIPIT protocol will ensure the final integrated data is consistent and of high integrity for the research requirements, useful for practical application across all fields of medical research.

Agreement of Medicaid claims and electronic health records for assessing preventive care quality among adults

To compare the agreement of electronic health record (EHR) data versus Medicaid claims data in documenting adult preventive care. Insurance claims are commonly used to measure care quality. EHR data could serve this purpose, but little information exists about how this source compares in service documentation. For 13 101 Medicaid-insured adult patients attending 43 Oregon community health centers, we compared documentation of 11 preventive services, based on EHR versus Medicaid claims data. Documentation was comparable for most services. Agreement was highest for influenza vaccination (κ =  0.77; 95% CI 0.75 to 0.79), cholesterol screening (κ = 0.80; 95% CI 0.79 to 0.81), and cervical cancer screening (κ = 0.71; 95% CI 0.70 to 0.73), and lowest on services commonly referred out of primary care clinics and those that usually do not generate claims. EHRs show promise for use in quality reporting. Strategies to maximize data capture in EHRs are needed to optimize the use of EHR data for service documentation.

Co-medication of pravastatin and paroxetine-a categorical study

Electronic Medical Records (EMRs) are wealthy storehouses of patient information, to which data mining techniques can be prudently applied to reveal clinically significant patterns. Detecting patterns in drug-drug interactions, leading to adverse drug reactions is a powerful application of EMR data mining. Adverse effects of drug treatments can be investigated by mining clinical laboratory tests data which are reliable indicators of abnormal physiological functions. We report here the co-medication effects of pravastatin (HMG-CoA reductase inhibitor) and paroxetine (selective serotonin reuptake inhibitor (SSRI) anti-depressant) on significant clinical parameters, identified through a data mining analysis conducted on the Allscripts data warehouse. We found that the concomitant drug treatments of pravastatin and paroxetine increased the mean values of glucose serum from 113.2 to 132.1 mg/dL and international normalized ratio (INR) from 2.18 to 2.52, respectively. It also decreased the mean values of estimated glomerular filtration rate (eGFR) from 43 to 37 mL/min/1.73 m(3) and blood CO2 levels from 24.8 to 23.9 mEq/L respectively. Our findings indicate that co-medication of pravastatin and paroxetine might have significant impact on blood anti-coagulation, kidney function, and glucose homeostasis. Our methodology can be applied to any EMR data set to reveal co-medication effects of any drug pairs.

An evaluation of data quality in Canada's Continuing Care Reporting System (CCRS): secondary analyses of Ontario data submitted between 1996 and 2011

Background

Evidence informed decision making in health policy development and clinical practice depends on the availability of valid and reliable data. The introduction of interRAI assessment systems in many countries has provided valuable new information that can be used to support case mix based payment systems, quality monitoring, outcome measurement and care planning. The Continuing Care Reporting System (CCRS) managed by the Canadian Institute for Health Information has served as a data repository supporting national implementation of the Resident Assessment Instrument (RAI 2.0) in Canada for more than 15 years. The present paper aims to evaluate data quality for the CCRS using an approach that may be generalizable to comparable data holdings internationally.

Methods

Data from the RAI 2.0 implementation in Complex Continuing Care (CCC) hospitals/units and Long Term Care (LTC) homes in Ontario were analyzed using various statistical techniques that provide evidence for trends in validity, reliability, and population attributes. Time series comparisons included evaluations of scale reliability, patterns of associations between items and scales that provide evidence about convergent validity, and measures of changes in population characteristics over time.

Results

Data quality with respect to reliability, validity, completeness and freedom from logical coding errors was consistently high for the CCRS in both CCC and LTC settings. The addition of logic checks further improved data quality in both settings. The only notable change of concern was a substantial inflation in the percentage of long term care home residents qualifying for the Special Rehabilitation level of the Resource Utilization Groups (RUG-III) case mix system after the adoption of that system as part of the payment system for LTC.

Conclusions

The CCRS provides a robust, high quality data source that may be used to inform policy, clinical practice and service delivery in Ontario. Only one area of concern was noted, and the statistical techniques employed here may be readily used to target organizations with data quality problems in that (or any other) area. There was also evidence that data quality was good in both CCC and LTC settings from the outset of implementation, meaning data may be used from the entire time series. The methods employed here may continue to be used to monitor data quality in this province over time and they provide a benchmark for comparisons with other jurisdictions implementing the RAI 2.0 in similar populations.