The effect of electronic audits and feedback in primary care and factors that contribute to their effectiveness: a systematic review

Electronic Health Record Nudges and Health Care Quality and Outcomes in Primary Care: A Systematic Review

Importance

Nudges have been increasingly studied as a tool for facilitating behavior change and may represent a novel way to modify the electronic health record (EHR) to encourage evidence-based care.

Objective

To evaluate the association between EHR nudges and health care outcomes in primary care settings and describe implementation facilitators and barriers.

Evidence Review

On June 9, 2023, an electronic search was performed in PubMed, Embase, PsycINFO, CINAHL, and Web of Science for all articles about clinician-facing EHR nudges. After reviewing titles, abstracts, and full texts, the present review was restricted to articles that used a randomized clinical trial (RCT) design, focused on primary care settings, and evaluated the association between EHR nudges and health care quality and patient outcome measures. Two reviewers abstracted the following elements: country, targeted clinician types, medical conditions studied, length of evaluation period, study design, sample size, intervention conditions, nudge mechanisms, implementation facilitators and barriers encountered, and major findings. The findings were qualitatively reported by type of health care quality and patient outcome and type of primary care condition targeted. The Risk of Bias 2.0 tool was adapted to evaluate the studies based on RCT design (cluster, parallel, crossover). Studies were scored from 0 to 5 points, with higher scores indicating lower risk of bias.

Findings

Fifty-four studies met the inclusion criteria. Overall, most studies (79.6%) were assessed to have a moderate risk of bias. Most or all descriptive (eg, documentation patterns) (30 of 38) or patient-centeredness measures (4 of 4) had positive associations with EHR nudges. As for other measures of health care quality and patient outcomes, few had positive associations between EHR nudges and patient safety (4 of 12), effectiveness (19 of 48), efficiency (0 of 4), patient-reported outcomes (0 of 3), patient adherence (1 of 2), or clinical outcome measures (1 of 7).

Conclusions and Relevance

This systematic review found low- and moderate-quality evidence that suggested that EHR nudges were associated with improved descriptive measures (eg, documentation patterns). Meanwhile, it was unclear whether EHR nudges were associated with improvements in other areas of health care quality, such as effectiveness and patient safety outcomes. Future research is needed using longer evaluation periods, a broader range of primary care conditions, and in deimplementation contexts.

Clinical audit of quality of care among patients with viral hepatitis in primary care in a low endemic region

BACKGROUND

The current hepatitis B (HBV) and hepatitis C virus (HCV) screening practices may fail to detect many infected patients who could benefit from new therapeutic agents to limit progression to cirrhosis and hepatocellular carcinoma.

OBJECTIVES

This study assessed the test positivity rate and cascade of care of viral hepatitis patients in primary care in a low endemic region as well as the testing policy of abnormal alanine aminotransferase (ALT) level.

METHODS

This is a retrospective clinical audit among primary health care practices in Flanders, Belgium, assessing patients with an active medical file between 2019 and 2021.

RESULTS

A total of 84/89 (94.4%) primary health care practices participated representing 621,573 patients of which 1069 patients (0.17%) were registered as having viral hepatitis, not further specified. Detailed information was available from 38 practices representing 243,723/621,573 (39.2%) patients of which 169 (0.07%) were HBsAg positive and 99 (0.04%) anti-HCV positive. A total of 96/134(71.6%) chronic HBV-infected and 31/77(40.3%) chronic HCV-infected patients were referred to a hepatologist. A total of 30,573/621,573(4.9%) patients had an abnormal ALT level, and by at random selection, more detailed information was obtained on 211 patients. Information on high-risk groups was missing in up to 60%. In patients with abnormal ALT level, HBsAg and anti-HCV testing were conducted in 37/211(17.5%) and 25/211(11.8%), respectively.

CONCLUSION

In a low endemic region, the testing rate and cascade of care of HBV and HCV-infected patients can be improved in primary care, especially in high-risk groups and patients with abnormal ALT levels.

Model-based disease mapping using primary care registry data

BACKGROUND

Spatial modeling of disease risk using primary care registry data is promising for public health surveillance. However, it remains unclear to which extent challenges such as spatially disproportionate sampling and practice-specific reporting variation affect statistical inference.

METHODS

Using lower respiratory tract infection data from the INTEGO registry, modeled with a logistic model incorporating patient characteristics, a spatially structured random effect at municipality level, and an unstructured random effect at practice level, we conducted a case and simulation study to assess the impact of these challenges on spatial trend estimation.

RESULTS

Even with spatial imbalance and practice-specific reporting variation, the model performed well. Performance improved with increasing spatial sample balance and decreasing practice-specific variation.

CONCLUSION

Our findings indicate that, with correction for reporting efforts, primary care registries are valuable for spatial trend estimation. The diversity of patient locations within practice populations plays an important role.

Association between non-registration of chronic kidney disease and mortality and cardiovascular outcome: a time-to-event analysis of retrospective primary care data

OBJECTIVE

Patients with impaired kidney function and increased albuminuria are at risk of developing cardiovascular disease (CVD). Previous research has revealed that a substantial proportion of patients with chronic kidney disease (CKD) do not get a registered diagnosis in the electronic health record of the general practitioner. The aim of this study was to investigate the association between non-registration of CKD and all-cause mortality and cardiovascular outcome.

DESIGN AND SETTING

A retrospective study in primary care.

METHODS

The analyses were carried out in the INTEGO database, a general practice-based morbidity registration network in Flanders, Belgium. The study used INTEGO data from the year 2018 for all patients ≥18 years old, including 10 551 patients. To assess the risk of mortality and CVD, a time-to-event analysis was performed. Cox proportional hazard model was used to evaluate the association between non-registration and incidence of all-cause mortality and cardiovascular events with mortality as a competing risk. Subgroup analyses were performed for estimated glomerular filtration rate stages (3A, 3B, 4 and 5). Multiple imputation was done following the methodology of Mamouris et al. RESULTS: Mortality was higher in patients with non-registered CKD compared with patients with registered CKD (HR 1.29, 95% CI 1.19 to 1.41). Non-registration of CKD was not associated with an increased risk for the development of CVD (HR 0.92, 95% CI 0.77 to 1.11).

CONCLUSION

An association between non-registration and all-cause mortality was identified, although no such association was apparent for CVD.

Developing a questionnaire to evaluate an automated audit & feedback intervention: a Rand-modified Delphi method

BACKGROUND

Audit and feedback (A&F) is a widely used implementation strategy to evaluate and improve medical practice. The optimal design of an A&F system is uncertain and structured process evaluations are currently lacking. This study aimed to develop and validate a questionnaire to evaluate the use of automated A&F systems.

METHODS

Based on the Clinical Performance Feedback Intervention Theory (CP-FIT) and the REFLECT-52 (REassessing audit & Feedback interventions: a tooL for Evaluating Compliance with suggested besT practices) evaluation tool a questionnaire was designed for the purpose of evaluating automated A&F systems. A Rand-modified Delphi method was used to develop the process evaluation and obtain validation. Fourteen experts from different domains in primary care consented to participate and individually scored the questions on a 9-point Likert scale. Afterwards, the questions were discussed in a consensus meeting. After approval, the final questionnaire was compiled.

RESULTS

A 34-question questionnaire composed of 57 items was developed and presented to the expert panel. The consensus meeting resulted in a selection of 31 questions, subdivided into 43 items. A final list of 30 questions consisting of 42 items was obtained.

CONCLUSION

A questionnaire consisting of 30 questions was drawn up for the assessment and improvement of automated A&F systems, based on CP-FIT and REFLECT-52 theory and approved by experts. Next steps will be piloting and implementation of the questionnaire.

Quality in screening and measuring blood pressure in China's primary health care: a national cross-sectional study using unannounced standardized patients

Background

This study aims to evaluate primary care providers' adherence to the standard of measuring blood pressure for people aged 35 or above during their initial visit, as per Chinese guidelines, and to identify factors affecting their practices.

Methods

We developed 11 standardized patients (SP) cases as tracer conditions to evaluate primary care, and deployed trained SPs for unannounced visits to randomly selected providers in seven provinces of China. The SPs used a checklist based on guidelines to record whether and how blood pressure was measured. Data were analyzed descriptively and regression analysis was performed to examine the association between outcomes and factors such as provider, patient, facility, and clinical case characteristics.

Findings

The SPs conducted 1201 visits and found that less than one-third of USPs ≥35 had their blood pressure measured. Only 26.9% of migraine and 15.4% of diabetes cases received blood pressure measurements. Additionally, these measurements did not follow the proper guidelines and recommended steps. On average, 55.6% of the steps were followed with few providers considering influencing factors before measurement and only 6.0% of patients received both-arm measurements. The use of wrist sphygmomanometers was associated with poor blood pressure measurement.

Interpretation

In China, primary care hypertension screening practices fall short of guidelines, with infrequent initiation of blood pressure measurements and inadequate adherence to proper measurement steps. To address this, priority should be placed on adopting, implementing, and upholding guidelines for hypertension screening and measurement.

Funding

National Natural Science Foundation of China, Swiss Agency for Development and Cooperation, Doctoral Fund Project of Inner Mongolia Medical University, China Postdoctoral Science Foundation.

Qualitative study of international key informants' perspectives on the current and future state of healthcare quality measurement and feedback

OBJECTIVES

The aim of this study is to explore the current and future state of quality measurement and feedback and identify factors influencing measurement feedback systems, including the barriers and enablers to their effective design, implementation, use and translation into quality improvement.

DESIGN

This qualitative study used semistructured interviews with key informants. A deductive framework analysis was conducted to code transcripts to the Theoretical Domains Framework (TDF). An inductive analysis was used to produce subthemes and belief statements within each TDF domain.

SETTING

All interviews were conducted by videoconference and audio-recorded.

PARTICIPANTS

Key informants were purposively sampled experts in quality measurement and feedback, including clinical (n=5), government (n=5), research (n=4) and health service leaders (n=3) from Australia (n=7), the USA (n=4), the UK (n=2), Canada (n=2) and Sweden (n=2).

RESULTS

A total of 17 key informants participated in the study. The interview length ranged from 48 to 66 min. 12 theoretical domains populated by 38 subthemes were identified as relevant to measurement feedback systems. The most populous domains included environmental context and resources, memory, attention and decision-making, and social influences. The most populous subthemes included 'quality improvement culture', 'financial and human resource support' and 'patient-centred measurement'. There were minimal conflicting beliefs outside of 'data quality and completeness'. Conflicting beliefs in these subthemes were predominantly between government and clinical leaders.

CONCLUSIONS

Multiple factors were found to influence measurement feedback systems and future considerations are presented within this manuscript. The barriers and enablers that impact these systems are complex. While there are some clear modifiable factors in the design of measurement and feedback processes, influential factors described by key informants were largely socioenvironmental. Evidence-based design and implementation, coupled with a deeper understanding of the implementation context, may lead to enhanced quality measurement feedback systems and ultimately improved care delivery and patient outcomes.

How do healthcare providers use national audit data for improvement?

BACKGROUND

Substantial resources are invested by Health Departments worldwide in introducing National Clinical Audits (NCAs). Yet, there is variable evidence on the NCAs' effectiveness and little is known on factors underlying the successful use of NCAs to improve local practice. This study will focus on a single NCA (the National Audit of Inpatient Falls -NAIF 2017) to explore: (i) participants' perspectives on the NCA reports, local feedback characteristics and actions undertaken following the feedback underpinning the effective use of the NCA feedback to improve local practice; (ii) reported changes in local practice following the NCA feedback in England and Wales.

METHODS

Front-line staff perspectives were gathered through interviews. An inductive qualitative approach was used. Eighteen participants were purposefully sampled from 7 of the 85 participating hospitals in England and Wales. Analysis was guided by constant comparative techniques.

RESULTS

Regarding the NAIF annual report, interviewees valued performance benchmarking with other hospitals, the use of visual representations and the inclusion of case studies and recommendations. Participants stated that feedback should target front-line healthcare professionals, be straightforward and focused, and be delivered through an encouraging and honest discussion. Interviewees highlighted the value of using other relevant data sources alongside NAIF feedback and the importance of continuous data monitoring. Participants reported that engagement of front-line staff in the NAIF and following improvement activities was critical. Leadership, ownership, management support and communication at different organisational levels were perceived as enablers, while staffing level and turnover, and poor quality improvement (QI) skills, were perceived as barriers to improvement. Reported changes in practice included increased awareness and attention to patient safety issues and greater involvement of patients and staff in falls prevention activities.

CONCLUSIONS

There is scope to improve the use of NCAs by front-line staff. NCAs should not be seen as isolated interventions but should be fully embedded and integrated into the QI strategic and operational plans of NHS trusts. The use of NCAs could be optimised, but knowledge of them is poor and distributed unevenly across different disciplines. More research is needed to provide guidance on key elements to consider throughout the whole improvement process at different organisational levels.

Characterizing the Gaps Between Best-Practice Implementation Strategies and Real-world Implementation: Qualitative Study Among Family Physicians Who Engaged With Audit and Feedback Reports

BACKGROUND

In Ontario, Canada, a government agency known as Ontario Health is responsible for making audit and feedback reports available to all family physicians to encourage ongoing quality improvement. The confidential report provides summary data on 3 key areas of practice: safe prescription, cancer screening, and diabetes management.

OBJECTIVE

This report was redesigned to improve its usability in line with evidence. The objective of this study was to explore how the redesign was perceived, with an emphasis on recipients' understanding of the report and their engagement with it.

METHODS

We conducted qualitative semistructured interviews with family physicians who had experience with both versions of the report recruited through purposeful and snowball sampling. We analyzed the transcripts following an emergent and iterative approach.

RESULTS

Saturation was reached after 17 family physicians participated. In total, 2 key themes emerged as factors that affected the perceived usability of the report: alignment between the report and the recipients' expectations and capacity to engage in quality improvement. Family physicians expected the report and its quality indicators to reflect best practices and to be valid and accurate. They also expected the report to offer feedback on the clinical activities they perceived to be within their control to change. Furthermore, family physicians expected the goal of the report to be aligned with their perspective on feasible quality improvement activities. Most of these expectations were not met, limiting the perceived usability of the report. The capacity to engage with audit and feedback was hindered by several organizational and physician-level barriers, including the lack of fit with the existing workflow, competing priorities, time constraints, and insufficient skills for bridging the gaps between their data and the corresponding desired actions.

CONCLUSIONS

Despite recognized improvements in the design of the report to better align with best practices, it was not perceived as highly usable. Improvements in the presentation of the data could not overcome misalignment with family physicians' expectations or the limited capacity to engage with the report. Integrating iterative evaluations informed by user-centered design can complement evidence-based guidance for implementation strategies. Creating a space for bringing together audit and feedback designers and recipients may help improve usability and effectiveness.

A Systematic Review of Electronic Medical Record Driven Quality Measurement and Feedback Systems

Historically, quality measurement analyses utilize manual chart abstraction from data collected primarily for administrative purposes. These methods are resource-intensive, time-delayed, and often lack clinical relevance. Electronic Medical Records (EMRs) have increased data availability and opportunities for quality measurement. However, little is known about the effectiveness of Measurement Feedback Systems (MFSs) in utilizing EMR data. This study explores the effectiveness and characteristics of EMR-enabled MFSs in tertiary care. The search strategy guided by the PICO Framework was executed in four databases. Two reviewers screened abstracts and manuscripts. Data on effect and intervention characteristics were extracted using a tailored version of the Cochrane EPOC abstraction tool. Due to study heterogeneity, a narrative synthesis was conducted and reported according to PRISMA guidelines. A total of 14 unique MFS studies were extracted and synthesized, of which 12 had positive effects on outcomes. Findings indicate that quality measurement using EMR data is feasible in certain contexts and successful MFSs often incorporated electronic feedback methods, supported by clinical leadership and action planning. EMR-enabled MFSs have the potential to reduce the burden of data collection for quality measurement but further research is needed to evaluate EMR-enabled MFSs to translate and scale findings to broader implementation contexts.

A Population Health Approach to Transplant Access: Challenging the Status Quo

Transplant referral and evaluation are critical steps to waitlisting yet remain an elusive part of the transplant process. Despite calls for more data collection on pre-waitlisting steps, there are currently no national surveillance data to aid in understanding the causes and potential solutions for the extreme variation in access to transplantation. As population health scientists, epidemiologists, clinicians, and ethicists we submit that the transplant community has an obligation to better understand disparities in transplant access as a first necessary step to effectively mitigating these inequities. Our position is grounded in a population health approach, consistent with several new overarching national policy and quality initiatives. The purpose of this Perspective is to (1) provide an overview of how a population health approach should inform current multisystem policies impacting kidney transplantation and demonstrate how these efforts could be enhanced with national data collection on pre-waitlisting steps; (2) demonstrate the feasibility and concrete next steps for pre-waitlisting data collection; and (3) identify potential opportunities to use these data to implement effective population-level interventions, policies, and quality measures to improve equity in access to kidney transplantation.

Do electronic reminders alter recorded diagnoses in primary care office-hours practices of health centers: A register-based study in a Finnish city

Objectives

One purpose of electronic reminders is improvement of the quality of documentation in office-hours primary care. The aim of this study was to evaluate how implementation of electronic reminders alters the rate and/or content of diagnostic data recorded by primary care physicians in office-hours practices in primary care health centers.

Methods

The present work is a register-based longitudinal follow-up study with a before-and-after design. An electronic reminder was installed in the electronic health record system of the primary health care of a Finnish city to remind physicians to include the diagnosis code of the visit in the health record. The report generator of the electronic health record system provided monthly figures for the number of various recorded diagnoses by using the International Classification of Diseases, 10th edition, and the total number of visits to primary care physicians, thus allowing the calculation of the recording rate of diagnoses on a monthly basis. The distribution of diagnoses before and after implementing ERs was also compared.

Results

After the introduction of the electronic reminder, the rate of diagnosis recording by primary care physicians increased clearly from 39.7% to 87.2% (p < 0.001). The intervention enhanced the recording rate of symptomatic diagnoses (group R) and some chronic diseases such as hypertension, type 2 diabetes and other soft tissue disorders. Recording rate of diagnoses related to diseases of the respiratory system (group J), injuries, poisoning and certain other consequences of external causes (group S), and diseases of single body region of the musculoskeletal system and connective tissue (group M) decreased after the implementation of electronic reminders.

Conclusion

Electronic reminders may alter the contents and extent of recorded diagnosis data in office-hours practices of the primary care health centers. They were found to have an influence on the recording rates of diagnoses related to chronic diseases. Electronic reminders may be a useful tool in primary health care when attempting to change the behavior of primary care physicians.

The effect of automated audit and feedback on data completeness in the electronic health record of the general physician: protocol for a cluster randomized controlled trial

Background

The electronic health record (EHR) of the general physician (GP) is an important tool that can be used to assess and improve the quality of healthcare. However, there are some problems when (re) using the data gathered in the EHR for quality assessments. One problem is the lack of data completeness in the EHR. Audit and feedback (A&F) is a well-known quality intervention that can improve the quality of healthcare. We hypothesize that an automated A&F intervention can be adapted to improve the data completeness of the EHR of the GP, more specifically, the number of correctly registered diagnoses of type 2 diabetes and chronic kidney disease.

Methods

This study is a pragmatic cluster randomized controlled trial with an intervention at the level of GP practice. The intervention consists of an audit and extended electronically delivered feedback with multiple components that will be delivered 4 times electronically to general practices over 12 months. The data will be analyzed on an aggregated level (per GP practice). The primary outcome is the percentage of correctly registered diagnoses of type 2 diabetes. The key secondary outcome is the registration of chronic kidney disease. Exploratory secondary outcomes are the registration of heart failure, biometric data and lifestyle habits, and the evolution of 4 different EHR-extractable quality indicators.

Discussion

This cluster randomized controlled trial intends to primarily improve the registration of type 2 diabetes in the EHR of the GP and to secondarily improve the registration of chronic kidney disease. In addition, the registration of heart failure, lifestyle parameters, and biometric data in the EHR of the GP are explored together with 4 EHR-extractable quality indicators. By doing so, this study aims to improve the data completeness of the EHR, paving the way for future quality assessments.

Trial registration

ClinicalTrials.gov NCT04388228. Registered on May 14, 2020.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-021-05259-9.