Beyond quality improvement: exploring why primary care teams engage in a voluntary audit and feedback program

Reconsidering performance management to support innovative changes in health care services

PURPOSE

A large number of studies indicate that coercive forms of organizational control and performance management in health care services often backfire and initiate dysfunctional consequences. The purpose of this article is to discuss new approaches to performance management in health care services when the purpose is to support innovative changes in the delivery of services.

DESIGN/METHODOLOGY/APPROACH

The article represents cross-boundary work as the theoretical and empirical material used to discuss and reconsider performance management comes from several relevant research disciplines, including systematic reviews of audit and feedback interventions in health care and extant theories of human motivation and organizational control.

FINDINGS

An enabling approach to performance management in health care services can potentially contribute to innovative changes. Key design elements to operationalize such an approach are a formative and learning-oriented use of performance measures, an appeal to self- and social-approval mechanisms when providing feedback and support for local goals and action plans that fit specific conditions and challenges.

ORIGINALITY/VALUE

The article suggests how to operationalize an enabling approach to performance management in health care services. The framework is consistent with new governance and managerial approaches emerging in public sector organizations more generally, supporting a higher degree of professional autonomy and the use of nonfinancial incentives.

A realist synthesis of multicentre comparative audit implementation: exploring what works and in which healthcare contexts

BACKGROUND

Multicentre comparative clinical audits have the potential to improve patient care, allow benchmarking and inform resource allocation. However, implementing effective and sustainable large-scale audit can be difficult within busy and resource-constrained contemporary healthcare settings. There are little data on what facilitates the successful implementation of multicentre audits. As healthcare environments are complex sociocultural organisational environments, implementing multicentre audits within them is likely to be highly context dependent.

OBJECTIVE

We aimed to examine factors that were influential in the implementation process of multicentre comparative audits within healthcare contexts-what worked, why, how and for whom?

METHODS

A realist review was conducted in accordance with the Realist and Meta-narrative Evidence Syntheses: Evolving Standards reporting standards. A preliminary programme theory informed two systematic literature searches of peer-reviewed and grey literature. The main context-mechanism-outcome (CMO) configurations underlying the implementation processes of multicentre audits were identified and formed a final programme theory.

RESULTS

69 original articles were included in the realist synthesis. Four discrete CMO configurations were deduced from this synthesis, which together made up the final programme theory. These were: (1) generating trustworthy data; (2) encouraging audit participation; (3) ensuring audit sustainability; and (4) facilitating audit cycle completion.

CONCLUSIONS

This study elucidated contexts, mechanisms and outcomes influential to the implementation processes of multicentre or national comparative audits in healthcare. The relevance of these contextual factors and generative mechanisms were supported by established theories of behaviour and findings from previous empirical research. These findings highlight the importance of balancing reliability with pragmatism within complex adaptive systems, generating and protecting human capital, ensuring fair and credible leadership and prioritising change facilitation.

Making the most of audit and feedback to improve diabetes care: a qualitative study of the perspectives of Australian Diabetes Centres

BACKGROUND

Diabetes has high burden on the health system and the individual, and many people living with diabetes struggle to optimally manage their condition. In Australia, people living with diabetes attend a mixture of primary, secondary and tertiary care centres. Many of these Diabetes Centres participate in the Australian National Diabetes Audit (ANDA), a quality improvement (QI) activity that collects clinical information (audit) and feeds back collated information to participating sites (feedback). Despite receiving this feedback, many process and care outcomes for Diabetes Centres continue to show room for improvement. The purpose of this qualitative study was to inform improvement of the ANDA feedback, identify the needs of those receiving feedback and elicit the barriers to and enablers of optimal feedback use.

METHODS

Semi-structured interviews were conducted with representatives of Australian Diabetes Centres, underpinned by the Consolidated Framework for Implementation Research (CFIR). De-identified transcripts were analysed thematically, underpinned by the domains and constructs of the CFIR.

RESULTS

Representatives from 14 Diabetes centres participated in this study, including a diverse range of staff typical of the Diabetes Centres who take part in ANDA. In general, participants wanted a shorter report with a more engaging, simplified data visualisation style. Identified barriers to use of feedback were time or resource constraints, as well as access to knowledge about how to use the data provided to inform the development of QI activities. Enablers included leadership engagement, peer mentoring and support, and external policy and incentives. Potential cointerventions to support use include exemplars from clinical change champions and peer leaders, and educational resources to help facilitate change.

CONCLUSIONS

This qualitative study supported our contention that the format of ANDA feedback presentation can be improved. Healthcare professionals suggested actionable changes to current feedback to optimise engagement and potential implementation of QI activities. These results will inform redesign of the ANDA feedback to consider the needs and preferences of end users and to provide feedback and other supportive cointerventions to improve care, and so health outcomes for people with diabetes. A subsequent cluster randomised trial will enable us to evaluate the impact of these changes.

Impact of Team-Based Care on Emergency Department Use

PURPOSE

We sought to assess the impact of team-based care on emergency department (ED) use in the context of physicians transitioning from fee-for-service payment to capitation payment in Ontario, Canada.

METHODS

We conducted an interrupted time series analysis to assess annual ED visit rates before and after transition from an enhanced fee-for-service model to either a team capitation model or a nonteam capitation model. We included Ontario residents aged 19 years and older who had at least 3 years of outcome data both pretransition and post-transition (N = 2,524,124). We adjusted for age, sex, income quintile, immigration status, comorbidity, and morbidity, and we stratified by rurality. A sensitivity analysis compared outcomes for team vs nonteam patients matched on year of transition, age, sex, rurality, and health region.

RESULTS

We compared 387,607 team and 1,399,103 nonteam patients in big cities, 213,394 team and 380,009 nonteam patients in small towns, and 65,289 team and 78,722 nonteam patients in rural areas. In big cities, after adjustment, the ED visit rate increased by 2.4% (95% CI, 2.2% to 2.6%) per year for team patients and 5.2% (95% CI, 5.1% to 5.3%) per year for nonteam patients in the years after transition (P <.001). Similarly, there was a slower increase in ED visits for team relative to nonteam patients in small towns (0.9% [95% CI, 0.7% to 1.1%] vs 2.9% [95% CI, 2.8% to 3.1%], P <.001) and rural areas (‒0.5% [95% CI, -0.8% to 0.2%] vs 1.3% [95% CI, 1.0% to 1.6%], P <.001). Results were much the same in the matched analysis.

CONCLUSIONS

Adoption of team-based primary care may reduce ED use. Further research is needed to understand optimal team composition and roles.

Audit feedback interventions to address high-risk prescriptions in long-term care homes: a costing study and return on investment analysis

BACKGROUND

Audit and feedback is a common implementation strategy, but few studies describe its costs. 'MyPractice' is a province-wide audit and feedback initiative to improve prescribing in nursing homes. This study sought to estimate the costs of 'MyPractice' and assess whether the financial benefit of 'MyPractice' offsets those costs.

METHODS

We conducted a costing study from the perspective of the Ontario government. Total cost of 'MyPractice' was calculated as the sum of the costs of producing and disseminating the reports (covering three report releases) which were obtained from Ontario Health staff interviews and document reviews. Return on investment (ROI) was calculated as the ratio of net cost-savings and the intervention cost. Cost savings were based on the effectiveness of 'MyPractice' derived from a published cohort study. Cost-savings attributable to 'MyPractice' were estimated from the changes in the rates of antipsychotics over time between physicians who signed up and viewed the reports and those who did not sign up to the reports.

RESULTS

Total intervention costs were C$223,691 (C$838 per physician and C$74,564 per release). Costs incurred during the development phase accounted for 74% of the total cost (C$166,117), while implementation costs for three report releases were responsible for 26% of the total costs (C$57,575). The ROI for every C$1 spent on the 'MyPractice' intervention was 1.02 (95% CI 0.51, 1.93) for three report releases.

CONCLUSION

'MyPractice' report offers a good return on investment and the value for money could improve with greater number of report releases.

Are quality improvement plans perceived to improve the quality of primary care in Ontario? Qualitative study

OBJECTIVE

To explore primary care administrators' perceptions of provincially mandated quality improvement plans, and barriers to and facilitators of using quality improvement plans as tools for improving the quality of primary care.

DESIGN

Qualitative descriptive study using semistructured interviews.

SETTING

Ontario.

PARTICIPANTS

Eleven primary care administrators (ie, executive directors, director of clinical services, office administrators) at 7 family health teams and 4 community health centres.

METHODS

All interviews were audiotaped and transcribed verbatim. Data were analyzed deductively to generate a framework based on a conceptual model of structural, organizational, individual, and innovation-related factors that influence the success of improvement initiatives and, inductively, to generate additional themes.

MAIN FINDINGS

Provincially mandated quality improvement plans seem to have raised awareness of and provided an overall focus on quality improvement, and have contributed to primary care organizations implementing initiatives to address quality gaps. Four factors that have contributed to the success of quality improvement plans relate to attributes of the quality improvement plans (adaptability and compatibility) and contextual factors (leadership and organizational culture). However, participants expressed that the use of quality improvement plans have not yet led to substantial improvements in the quality of primary care in Ontario, which may be owing to several challenges: poor data quality, lack of staff and physician engagement and buy-in, and lack of resources to support measurement and quality improvement.

CONCLUSION

Awareness of and focused attention on the need for high-quality patient care may have increased, but participants expressed that substantial improvements in quality care have yet to be achieved in Ontario. The lack of perceived improvements is likely the result of multifaceted and complex challenges primary care organizations face when trying to improve patient care. To effect positive change, organization- and health system-level efforts are needed to improve measurement capabilities, improve staff and physician engagement, and increase capacity for quality improvement among organizations.

The current and potential uses of Electronic Medical Record (EMR) data for primary health care performance measurement in the Canadian context: a qualitative analysis

BACKGROUND

Electronic Medical Records (EMRs) are a rich data source to measure and improve quality of care. As Canadian primary health care (PHC) EMRs mature, there is increasing potential use of EMR data for performance measurement. This study identifies and describes current uses of EMR data for performance measurement and considerations to further its potential in the Canadian context.

METHODS

We applied a qualitative case study design and descriptive assessment in three phases, consulting multiple data sources including scientific and grey literature, system leaders (n = 41), and clinician/researchers (n = 20). Phases included a multimethod approach to identify initiatives using EMR data for performance measurement across Canadian jurisdictions; in-depth review of current initiatives identified from a healthcare performance intelligence lens; and triangulation and thematic analysis across data sources to explore considerations for advancing performance measurement uses of EMR data in the Canadian context.

RESULTS

Six initiatives of EMR data use for performance measurement were identified: one multi-jurisdictional; five jurisdiction-specific in the provinces of British Columbia, Manitoba and Ontario. EMR data uses were predominately for micro-level PHC physician and team performance improvement, with some use for meso-level organization/network-wide improvement. Indicator sets varied in number, though shared emphasis on chronic disease management and prevention/screening and to a lesser extent medication management. Key considerations for governing, resourcing and implementing EMR data for performance measurement were identified.

CONCLUSIONS

The extent of EMR data use for performance measurement varies across Canada. To further its potential, pan-Canadian data and privacy standards, performance intelligence competencies and renewed core PHC indicators should be prioritized. Experiences across countries, coupled with increasing momentum for performance measurement using real-world data, should be leveraged to avoid unnecessarily slow progress in Canada and abroad.

Unpacking the intention to action gap: a qualitative study understanding how physicians engage with audit and feedback

Background

Audit and feedback (A&F) often successfully enhances health professionals’ intentions to improve quality of care but does not consistently lead to practice changes. Recipients often cite data credibility and limited resources as barriers impeding their ability to act upon A&F, suggesting the intention-to-action gap manifests while recipients are interacting with their data. While attention has been paid to the role feedback and contextual variables play in contributing to (or impeding) success, we lack a nuanced understanding of how healthcare professionals interact with and process clinical performance data.

Methods

We used qualitative, semi-structured interviews guided by Normalization Process Theory (NPT). Questions explored the role of data in quality improvement, experiences with the A&F report, perceptions of the data, and interpretations and reflections. Interviews were audio-recorded and transcribed verbatim. Data were analyzed using a combination of inductive and deductive strategies using reflexive thematic analysis informed by a constructivist paradigm.

Results

Healthcare professional characteristics (individual quality improvement capabilities and beliefs about data) seem to influence engagement with A&F to a greater degree than feedback variables (i.e., delivered by peers) and observed contextual factors (i.e., strong quality improvement culture). Most participants lacked the capabilities to interpret practice-level data in an actionable way despite a motivation to engage meaningfully. Reasons for the intention-to-action gap included challenges interpreting longitudinal data, appreciating the nuances of common data sources, understanding how aggregate data provides insights into individualized care, and identifying practice-level actions to improve quality. These factors limited effective cognitive participation and collective action, as outlined in NPT.

Conclusions

A well-designed A&F intervention is necessary but not sufficient to inform practice changes. A&F initiatives must include co-interventions to address recipient characteristics (i.e., beliefs and capabilities) and context to optimize impact. Effective strategies to overcome the intention-to-action gap may include modelling how to use A&F to inform practice change, providing opportunities for social interaction relating to the A&F, and circulating examples of effective actions taken in response to A&F. More broadly, undergraduate medical education and post-graduate training must ensure physicians are equipped with QI capabilities, with an emphasis on the skills required to interpret and act on practice-level data.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-021-01088-1.

Exploring implementation processes in general practice in a feedback intervention aiming to reduce potentially inappropriate prescribing: a qualitative study among general practitioners

Background

Potentially inappropriate prescribing (PIP) has been linked with adverse health outcomes and increased healthcare costs. Feedback interventions targeting PIP have shown promising results. However, translation from research to everyday practice remains a challenge. With the Normalisation Process Theory (NPT) as overarching framework, we aimed to explore the implementation processes performed by general practices in a real-life, quality improvement intervention using feedback on practice-level prescribing.

Methods

All 376 general practices in the Central Denmark Region received a prescribing feedback intervention targeting selected types of PIP. Six months later, they received an evaluation questionnaire, to which 45% responded. Among 102 practices reporting to have made changes in response to the intervention, we conducted individual, semi-structured interviews with ten GPs. Maximum variation was sought in terms of baseline prescribing status, implementation activities, practice type and geographical location. The interviews were analysed thematically using NPT.

Results

The implementation processes in general practice reflected the four NPT constructs. Key motivators for implementation included the GPs’ professional values and interests, but pragmatic considerations were also of importance (coherence). A collective versus an individual approach to the engagement and planning of the implementation process (cognitive participation) was observed. Similarly, a distinction was evident between practice-level actions involving the entire practice team as opposed to individual-level actions performed by the individual GP (collective action). Several challenges to the implementation processes were identified, including patient influences and competing priorities at multiple levels (reflexive monitoring). Additionally, internal evaluation and normalisation of new practices occurred in varying degrees.

Conclusion

NPT provided a useful framework for understanding implementation processes in general practice. Our results emphasise that clear professional aims and feasible content of interventions are key for GP motivation. This may be ensured through cooperation with GPs’ professional organisation, which may strengthen intervention legitimacy and uptake. Two main implementation strategies were identified: practice-level and GP-level strategies. Intervention developers need to recognise both strategies to deliver intervention content and implementation support that promote sustainable improvements in prescribing practice. Competing demands and patient influences remain important challenges that need to be addressed in future studies to further facilitate the reduction of PIPs.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-020-00106-5.

Effectiveness of confidential reports to physicians on their prescribing of antipsychotic medications in nursing homes

BACKGROUND

Antipsychotic medication use in nursing homes is associated with potential for harms. In Ontario, Canada, an agency of the provincial government offers nursing home physicians quarterly audit and feedback on their antipsychotic prescribing. We compared the characteristics of physicians who did and did not engage with the intervention, and assessed early changes in prescribing.

METHODS

This population-level, retrospective cohort study used linked administrative databases to track prescribing practices in nursing homes pre-intervention (baseline), immediately post-initiative (3 months), and at follow-up (6 months). Exposure variables identified whether a physician signed up to participate (or not) or viewed the feedback following sign up (or not). Differences in the proportion of days that residents received antipsychotic medications at 6 months compared to baseline by exposure(s) were assessed using a linear mixed effects regression analysis to adjust for a range of resident, physician, and nursing home factors. Benzodiazepine and statin prescribing were assessed as a balance and tracer measures, respectively.

RESULTS

Of 944 eligible physicians, 210 (22.3%) signed up to recieve the feedback report and 132 (13.9%) viewed their feedback. Physicians who signed up for feedback were more likely to have graduated from a Canadian medical school, work in urban nursing homes, and care for a larger number of residents. The clinical and functional characteristics of residents were similar across physician exposure groups. At 6 months, antipsychotic prescribing had decreased in all exposure groups. Those who viewed their feedback report had a signicantly greater reduction in antipsychotic prescribing than those who did not sign up (0.94% patient-days exposed; 95% CI 0.35 to 1.54%, p = 0.002). Trends in prescribing patterns across exposure groups for benzodiazepines and statins were not statistically significant.

INTERPRETATION

Almost a quarter of eligible physicians engaged early in a voluntary audit and feedback intervention related to antipsychotic prescribing in nursing homes. Those who viewed their feedback achieved a small but statistically significant change in prescribing, equivalent to approximately 14,000 fewer days that nursing home residents received antipsychotic medications over 6 months. This study adds to the literature regarding the role of audit and feedback interventions to improve quality of care.

Ten-Year Follow-Up of Clinical Governance Implementation in Primary Care: Improving Screening, Diagnosis and Control of Cardiovascular Risk Factors

Current improvement strategies for the control of cardiovascular risk factors (CRFs) in Europe are based on quality management policies. With the aim of understanding the effect of interventions delivered by primary healthcare systems, we evaluated the impact of clinical governance on cardiovascular health after ten years of implementation in Catalonia. A cohort study that included 1878 patients was conducted in 19 primary care centres (PCCs). Audits that comprised 13 cardiovascular health indicators were performed and general practitioners received periodic (annual, biannual or monthly) feedback about their clinical practice. We evaluated improvement in screening, diagnosis and control of the main CRFs and the effects of the feedback on cardiovascular risk (CR), incidence of cardiovascular disease (CVD) and mortality, comparing baseline data with data at the end of the study (after a 10-year follow-up). The impact of the intervention was assessed globally and with respect to feedback frequency. General improvement was observed in screening, percentage of diagnoses and control of CRFs. At the end of the study, few clinically significant differences in CRFs were observed between groups. However, the reduction in CR was greater in the group receiving high frequency feedback, specifically in relation to smoking and control of diabetes and cholesterol (Low Density Lipoprotein (LDL) and High Density Lipoprotein (HDL)). A protective effect of having a cardiovascular event (hazard ratio (HR) = 0.64, 95% confidence interval (CI) = 0.44-0.94) or death (HR = 0.55, 95% CI = 0.35-0.88) was observed in patients from centres where general practitioners received high frequency feedback. Additionally, these PCCs presented improved cardiovascular health indicators and lower incidence and mortality by CVD, illustrating the impact of this intervention.

Measurement without management: qualitative evaluation of a voluntary audit & feedback intervention for primary care teams

Background

The use of clinical performance feedback to support quality improvement (QI) activities is based on the sound rationale that measurement is necessary to improve quality of care. However, concerns persist about the reliability of this strategy, known as Audit and Feedback (A&F) to support QI. If successfully implemented, A&F should reflect an iterative, self-regulating QI process. Whether and how real-world A&F initiatives result in this type of feedback loop are scarcely reported. This study aimed to identify barriers or facilitators to implementation in a team-based primary care context.

Methods

Semi-structured interviews were conducted with key informants from team-based primary care practices in Ontario, Canada. At the time of data collection, practices could have received up to three iterations of the voluntary A&F initiative. Interviews explored whether, how, and why practices used the feedback to guide their QI activities. The Consolidated Framework for Implementation Research was used to code transcripts and the resulting frameworks were analyzed inductively to generate key themes.

Results

Twenty-five individuals representing 18 primary care teams participated in the study. Analysis of how the A&F intervention was used revealed that implementation reflected an incomplete feedback loop. Participation was facilitated by the reliance on an external resource to facilitate the practice audit. The frequency of feedback, concerns with data validity, the design of the feedback report, the resource requirements to participate, and the team relationship were all identified as barriers to implementation of A&F.

Conclusions

The implementation of a real-world, voluntary A&F initiative did not lead to desired QI activities despite substantial investments in performance measurement. In small primary care teams, it may take long periods of time to develop capacity for QI and future evaluations may reveal shifts in the implementation state of the initiative. Findings from the present study demonstrate that the potential mechanism of action of A&F may be deceptively clear; in practice, moving from measurement to action can be complex.

Electronic supplementary material

The online version of this article (10.1186/s12913-019-4226-7) contains supplementary material, which is available to authorized users.