Revitalising audit and feedback to improve patient care

Practice visitations in primary care to improve performance of cardiovascular risk management: an observational study

BACKGROUND

Despite programmatic protocolised care and structured support, considerable variation is observed in completeness of registration and achieving targets of cardiovascular risk management (CVRM) between individual GPs in the Netherlands.

AIM

To determine whether completeness of registration and achieved targets of cardiovascular risk factors improves with practice visitation.

DESIGN & SETTING

Observational study utilising the care group's database (2016-2019), comparing changes in registration and achieved targets in non-visited practices and visited practices.

METHOD

We compared completeness scores of registration and scores of targets achieved before visitation and 1 year after visitation. Data were analysed on patient level and GP level. Separate analyses were performed among GPs who were ranked in the lower 25% of score distributions.

RESULTS

We observed no clinically relevant improvements in completeness of registration and targets achieved in 2017, 2018, and 2019 that could be attributed to visitations in the previous year, both on individual patient level and on aggregated level per general practice. In practices ranked in the lower 25% of the distribution, improvements over time were clinically relevant and larger than the overall changes. Yet, these findings were irrespective of the number of practice visitations.

CONCLUSION

Practice visitations in our setting did not seem to lead to improvements in practice performance, nor in completeness of registration of risk factors or in reaching predefined target goals for cardiovascular risk factors.

Using Implementation Science to Improve Short Peripheral Intravenous Catheter Outcomes

Short peripheral intravenous catheters (short PIVCs) are commonly used in acute care, guided by evidence-based policy with interventions to limit premature failure. Research on how nurses use evidence and change processes to optimize outcomes is needed. The study objective was to use a theory-based implementation science approach to evaluate and improve short PIVC insertion and care processes and reduce removals for adverse outcomes in acute care. This mixed-methods study was conducted with inpatient nursing units (n = 23) at a large urban quaternary medical center. Units identified and implemented one PIVC care intervention that could lower catheter removals for adverse outcomes over 3 months. Data from multiple sources were convergently analyzed to evaluate process and outcomes postintervention. Although overall frequency of PIVC removals for adverse outcomes was unchanged, several units improved their outcomes using implementation strategies. The determinant framework provides a plausible explanation for the study results. While adverse outcome rates remained below published rates, some units had limited success improving outcomes with traditional change strategies. Implementation strategies and readily accessible data can offer nursing units a new approach to effectively deploy, monitor, and maintain interventions to achieve improved outcomes.

Exploring Anesthesia Provider Preferences for Precision Feedback: Preference Elicitation Study

Background

Health care professionals must learn continuously as a core part of their work. As the rate of knowledge production in biomedicine increases, better support for health care professionals' continuous learning is needed. In health systems, feedback is pervasive and is widely considered to be essential for learning that drives improvement. Clinical quality dashboards are one widely deployed approach to delivering feedback, but engagement with these systems is commonly low, reflecting a limited understanding of how to improve the effectiveness of feedback about health care. When coaches and facilitators deliver feedback for improving performance, they aim to be responsive to the recipient's motivations, information needs, and preferences. However, such functionality is largely missing from dashboards and feedback reports. Precision feedback is the delivery of high-value, motivating performance information that is prioritized based on its motivational potential for a specific recipient, including their needs and preferences. Anesthesia care offers a clinical domain with high-quality performance data and an abundance of evidence-based quality metrics.

Objective

The objective of this study is to explore anesthesia provider preferences for precision feedback.

Methods

We developed a test set of precision feedback messages with balanced characteristics across 4 performance scenarios. We created an experimental design to expose participants to contrasting message versions. We recruited anesthesia providers and elicited their preferences through analysis of the content of preferred messages. Participants additionally rated their perceived benefit of preferred messages to clinical practice on a 5-point Likert scale.

Results

We elicited preferences and feedback message benefit ratings from 35 participants. Preferences were diverse across participants but largely consistent within participants. Participants' preferences were consistent for message temporality (α=.85) and display format (α=.80). Ratings of participants' perceived benefit to clinical practice of preferred messages were high (mean rating 4.27, SD 0.77).

Conclusions

Health care professionals exhibited diverse yet internally consistent preferences for precision feedback across a set of performance scenarios, while also giving messages high ratings of perceived benefit. A "one-size-fits-most approach" to performance feedback delivery would not appear to satisfy these preferences. Precision feedback systems may hold potential to improve support for health care professionals' continuous learning by accommodating feedback preferences.

Funding and resource availability for home parenteral nutrition in Australia: A national cross-sectional survey

BACKGROUND

Home parenteral nutrition (HPN) is a life-saving therapy required for the management of type III intestinal failure, one of the rarest organ failures. It requires a multidisciplinary approach to manage the complexity of the underlying medical, surgical, and nutrition issues, but the current levels of healthcare funding in Australia are unknown. This study aimed to quantify the caseload, staffing, and capacity of existing HPN centers nationally.

METHODS

This was a cross-sectional survey inviting centers known to provide HPN care. The survey was designed to capture metrics related to the national framework for the delivery of HPN. These centered on staffing levels, patient load, capacity to audit key outcomes, and service challenges.

RESULTS

A total of 24 (89%) of 27 invited centers responded to the survey. There were 17 (71%) adult centers and 7 (29%) pediatric centers. Adult centers managed a median of 12 (interquartile range [IQR]: 6-25) patients vs 16 (IQR: 9-17) in pediatric centers. Several centers did not have dedicated funding for core team members. The total funded clinician time each week per patient was 7 min (IQR: 0-12 min) in adult centers and 14 min (IQR: 10-21 min) in pediatric centers. Fewer than half of centers reported having sufficient resources to regularly audit key metrics.

CONCLUSION

The availability of dedicated expertise to manage the highly complex needs of people living with type III intestinal failure is lacking in Australia. Current funding of HPN services falls well short of being sufficient to meet the requirements outlined in the national quality framework.

Impact of Medical Oncology-Focused Quality Programs on Cancer Care Around the World

Quality cancer care is efficient, accessible, coordinated, and evidence-based. Recognizing the necessary key components, development of pathways and guidelines to incorporate these key domains, and finally respectful adaptation to cultural differences can ensure that cancer care globally is of the highest quality. This quality care should be judged not only on how it optimizes health outcomes for patients, but also its impact on the care providers and the global community.

Low risk squamous cell carcinoma and appropriate follow up

A recent article in the BJD postulated that it may be "Time to reconsider skin cancer-related follow-up visits". In our unit, we too have been seeing too many patient's unnecessarily and we put in place measures to reduce the numbers of outpatient appointments thereby diverting the resources saved into professional development.

Developing a questionnaire to evaluate an automated audit & feedback intervention: a Rand-modified Delphi method

BACKGROUND

Audit and feedback (A&F) is a widely used implementation strategy to evaluate and improve medical practice. The optimal design of an A&F system is uncertain and structured process evaluations are currently lacking. This study aimed to develop and validate a questionnaire to evaluate the use of automated A&F systems.

METHODS

Based on the Clinical Performance Feedback Intervention Theory (CP-FIT) and the REFLECT-52 (REassessing audit & Feedback interventions: a tooL for Evaluating Compliance with suggested besT practices) evaluation tool a questionnaire was designed for the purpose of evaluating automated A&F systems. A Rand-modified Delphi method was used to develop the process evaluation and obtain validation. Fourteen experts from different domains in primary care consented to participate and individually scored the questions on a 9-point Likert scale. Afterwards, the questions were discussed in a consensus meeting. After approval, the final questionnaire was compiled.

RESULTS

A 34-question questionnaire composed of 57 items was developed and presented to the expert panel. The consensus meeting resulted in a selection of 31 questions, subdivided into 43 items. A final list of 30 questions consisting of 42 items was obtained.

CONCLUSION

A questionnaire consisting of 30 questions was drawn up for the assessment and improvement of automated A&F systems, based on CP-FIT and REFLECT-52 theory and approved by experts. Next steps will be piloting and implementation of the questionnaire.

EASY-NET Program: Effectiveness of an Audit and Feedback Intervention in the Emergency Care for Acute Conditions in the Lazio Region

The EASY-NET network program (NET-2016-02364191)-effectiveness of audit and feedback (A&F) strategies to improve health practice and equity in various clinical and organizational settings), piloted a novel and more structured A&F strategy. This study compared the effectiveness of the novel strategy against the sole periodic dissemination of indicators in enhancing the appropriateness and timeliness of emergency health interventions for patients diagnosed with acute myocardial infarction (AMI) and ischemic stroke in the Lazio Region. The efficacy of the intervention was assessed through a prospective quasi-experimental design employing a pre- and post-intervention (2021-2022) comparison with a control group. Participating hospitals in the Lazio Region, where professional teams voluntarily engaged in the intervention, constituted the exposed group, while the control group exclusively engaged in routine reporting activities. Effectiveness analysis was conducted at the patient level, utilizing regional health information systems to compute process and outcome indicators. The effectiveness of the intervention was evaluated using difference-in-difference models, comparing pre- and post-intervention periods between exposed and control groups. Estimates were calculated in terms of the difference in percentage points (PP) between absolute risks. Sixteen facilities for the AMI pathway and thirteen for the stroke pathway participated in the intervention. The intervention yielded a reduction in the proportion of 30-day readmissions following hospitalization for ischemic stroke by 0.54 pp in the exposed patients demonstrating a significant difference of -3.80 pp (95% CI: -6.57; -1.03; 5453 patients, 63.7% cases) in the exposed group compared to controls. However, no statistically significant differences attributable to the implemented A&F intervention were observed in other indicators considered. These results represent the first evidence in Italy of the impact of A&F interventions in an emergency setting, utilizing aggregated data from hospitals involved in the Lazio Region's emergency network.

Unwarranted variation and the goal of net zero for the NHS in England: exploring the link between efficiency working, patient outcomes and carbon footprint

In 2020 the NHS in England set a target of reaching net zero carbon emissions by 2040. Progress has already been made towards this goal, with substantial reductions in the use of environmentally harmful anaesthetic gases, such as desflurane, in recent years. Where an effective replacement already exists, changing practice to use low carbon alternatives is relatively easy to achieve, but much greater challenges lie ahead. The Getting It Right First Time (GIRFT) programme is a clinically-led, data-driven clinical improvement initiative with a focus on reducing unwarranted variation in clinical practice and patient outcomes. Reducing unwarranted variation can improve patient care and service efficiency, and can also support the drive to net zero. In this article we set out what the GIRFT programme is doing to support sustainable healthcare in England, why it is uniquely positioned to support this goal and what the future challenges, barriers, enablers and opportunities are likely to be in the drive to net zero.

Quality indicators in surgical oncology: systematic review of measures used to compare quality across hospitals

BACKGROUND

Measurement and reporting of quality indicators at the hospital level has been shown to improve outcomes and support patient choice. Although there are many studies validating individual quality indicators, there has been no systematic approach to understanding what quality indicators exist for surgical oncology and no standardization for their use. The aim of this study was to review quality indicators used to assess variation in quality in surgical oncology care across hospitals or regions. It also sought to describe the aims of these studies and what, if any, feedback was offered to the analysed groups.

METHODS

A literature search was performed to identify studies published between 1 January 2000 and 23 October 2023 that applied surgical quality indicators to detect variation in cancer care at the hospital or regional level.

RESULTS

A total of 89 studies assessed 91 unique quality indicators that fell into the following Donabedian domains: process indicators (58; 64%); outcome indicators (26; 29%); structure indicators (6; 7%); and structure and outcome indicators (1; 1%). Purposes of evaluating variation included: identifying outliers (43; 48%); comparing centres with a benchmark (14; 16%); and supplying evidence of practice variation (29; 33%). Only 23 studies (26%) reported providing the results of their analyses back to those supplying data.

CONCLUSION

Comparisons of quality in surgical oncology within and among hospitals and regions have been undertaken in high-income countries. Quality indicators tended to be process measures and reporting focused on identifying outlying hospitals. Few studies offered feedback to data suppliers.

How can strategies based on performance measurement and feedback support changes in nursing practice? A theoretical reflection drawing on Habermas' social perspective

Strategies based on performance measurement and feedback are commonly used to support quality improvement among nurses. These strategies require practice change, which, for nurses, rely to a large extent on their capacity to coordinate with each other effectively. However, the levers for coordinated action are difficult to mobilize. This discussion paper offers a theoretical reflection on the challenges related to coordinating nurses' actions in the context of practice changes initiated by performance measurement and feedback strategies. We explore how Jürgen Habermas' theory of Communicative Action may shed light on the issues underlying nurses' collective actions and self-determination in practice change and the implications for the design of strategies based on performance measurement and feedback. Based on this theory, we propose differences between communicative and functional coordination according to the nature of the actions and the purposes involved. The domains of action underlying these coordination processes, which Habermas referred to as the lifeworld and the system, are then used to draw a parallel with aspects of nursing practice. Further exploration of these concepts allows us to consider the tensions between the demands of the system and the self-determination of nurses within their practice.

Performance Data Advocacy for Continuing Professional Development in Health Professions

ABSTRACT

Efforts to optimize continuing professional development (CPD) are ongoing and include advocacy for the use of clinician performance data. Several educational and quality-based frameworks support the use of performance data to achieve intended improvement outcomes. Although intuitively appealing, the role of performance data for CPD has been uncertain and its utility mainly assumed. In this Scholarly Perspective, the authors briefly review and trace arguments that have led to the conclusion that performance data are essential for CPD. In addition, they summarize and synthesize a recent and ongoing research program exploring the relationship physicians have with performance data. They draw on Collins, Onwuegbuzie, and Johnson's legitimacy model and Dixon-Woods' integrative approach to generate inferences and ways of moving forward. This interpretive approach encourages questioning or raising of assumptions about related concepts and draws on the perspectives (i.e., interpretive work) of the research team to identify the most salient points to guide future work. The authors identify 6 stimuli for future programs of research intended to support broader and better integration of performance data for CPD. Their aims are to contribute to the discourse on data advocacy for CPD by linking conceptual, methodologic, and analytic processes and to stimulate discussion on how to proceed on the issue of performance data for CPD purposes. They hope to move the field from a discussion on the utility of data for CPD to deeper integration of relevant conceptual frameworks.

Development of theoretically informed audit and feedback: An exemplar from a complex implementation strategy to improve asthma self-management in UK primary care

RATIONALE

Audit and feedback is an evidence-based implementation strategy, but studies reporting the use of theory to guide design elements are limited.

AIMS AND OBJECTIVES

Within the context of a programme of research aiming to improve the implementation of supported asthma self-management in UK primary care (IMPlementing IMProved Asthma self-management as RouTine [IMP2 ART]), we aimed to design and develop theoretically-informed audit and feedback that highlighted supported asthma self-management provision and areas for improvement in primary care general practices.

METHOD

Aligned with the Medical Research Council (MRC) complex intervention framework, the audit and feedback was developed in three phases: (1) Development: literature and theory exploration, and prototype audit and feedback design; (2) Feasibility: eliciting feedback on the audit and feedback from general practice staff (n = 9); (3) Prepiloting: delivering the audit and feedback within the IMP2 ART implementation strategy (incorporating patient and professional resources and an asthma review template) and eliciting clinician feedback (n = 9).

RESULTS

Audit and feedback design was guided by and mapped to existing literature suggestions and theory (e.g., Theoretical Domains Framework, Behaviour Change Technique Taxonomy). Feedback on the prototype audit and feedback confirmed feasibility but identified some refinements (a need to highlight supporting self-management and importance of asthma action plans). Prepiloting informed integration with other IMP2 ART programme strategies (e.g., patient resources and professional education).

CONCLUSION

We conclude that a multistage development process including theory exploration and mapping, contributed to the design and delivery of the audit and feedback. Aligned with the MRC framework, the IMP2 ART strategy (incorporating the audit and feedback) is now being tested in a UK-wide cluster randomised controlled trial.

Quality indicators for evaluating cancer care in low-income and middle-income country settings: a multinational modified Delphi study

This Policy Review sourced opinions from experts in cancer care across low-income and middle-income countries (LMICs) to build consensus around high-priority measures of care quality. A comprehensive list of quality indicators in medical, radiation, and surgical oncology was identified from systematic literature reviews. A modified Delphi study consisting of three 90-min workshops and two international electronic surveys integrating a global range of key clinical, policy, and research leaders was used to derive consensus on cancer quality indicators that would be both feasible to collect and were high priority for cancer care systems in LMICs. Workshop participants narrowed the list of 216 quality indicators from the literature review to 34 for inclusion in the subsequent surveys. Experts' responses to the surveys showed consensus around nine high-priority quality indicators for measuring the quality of hospital-based cancer care in LMICs. These quality indicators focus on important processes of care delivery from accurate diagnosis (eg, histologic diagnosis via biopsy and TNM staging) to adequate, timely, and appropriate treatment (eg, completion of radiotherapy and appropriate surgical intervention). The core indicators selected could be used to implement systems of feedback and quality improvement.

Harnessing the potential of data-driven strategies to optimise transfusion practice

No one doubts the significant variation in the practice of transfusion medicine. Common examples are the variability in transfusion thresholds and the use of tranexamic acid for surgery with likely high blood loss despite evidence-based standards. There is a long history of applying different strategies to address this variation, including education, clinical guidelines, audit and feedback, but the effectiveness and cost-effectiveness of these initiatives remains unclear. Advances in computerised decision support systems and the application of novel electronic capabilities offer alternative approaches to improving transfusion practice. In England, the National Institute for Health and Care Research funded a Blood and Transplant Research Unit (BTRU) programme focussing on 'A data-enabled programme of research to improve transfusion practices'. The overarching aim of the BTRU is to accelerate the development of data-driven methods to optimise the use of blood and transfusion alternatives, and to integrate them within routine practice to improve patient outcomes. One particular area of focus is implementation science to address variation in practice.

Current and Future Perspectives on Computed Tomography Screening for Lung Cancer: A Roadmap From 2023 to 2027 From the International Association for the Study of Lung Cancer

Low-dose computed tomography (LDCT) screening for lung cancer substantially reduces mortality from lung cancer, as revealed in randomized controlled trials and meta-analyses. This review is based on the ninth CT screening symposium of the International Association for the Study of Lung Cancer, which focuses on the major themes pertinent to the successful global implementation of LDCT screening and develops a strategy to further the implementation of lung cancer screening globally. These recommendations provide a 5-year roadmap to advance the implementation of LDCT screening globally, including the following: (1) establish universal screening program quality indicators; (2) establish evidence-based criteria to identify individuals who have never smoked but are at high-risk of developing lung cancer; (3) develop recommendations for incidentally detected lung nodule tracking and management protocols to complement programmatic lung cancer screening; (4) Integrate artificial intelligence and biomarkers to increase the prediction of malignancy in suspicious CT screen-detected lesions; and (5) standardize high-quality performance artificial intelligence protocols that lead to substantial reductions in costs, resource utilization and radiologist reporting time; (6) personalize CT screening intervals on the basis of an individual's lung cancer risk; (7) develop evidence to support clinical management and cost-effectiveness of other identified abnormalities on a lung cancer screening CT; (8) develop publicly accessible, easy-to-use geospatial tools to plan and monitor equitable access to screening services; and (9) establish a global shared education resource for lung cancer screening CT to ensure high-quality reading and reporting.

Alarm with care-a de-implementation strategy to reduce fall prevention alarm use in US hospitals: a study protocol for a hybrid 2 effectiveness-implementation trial

BACKGROUND

Fall prevention alarms are commonly used among US hospitals as a fall prevention strategy despite limited evidence of effectiveness. Further, fall prevention alarms are harmful to healthcare staff (e.g., alarm fatigue) and patients (e.g., sleep disturbance, mobility restriction). There is a need for research to develop and test strategies for reducing use of fall prevention alarms in US hospitals.

METHODS

To address this gap, we propose testing the effectiveness and implementation of Alarm with Care, a de-implementation strategy to reduce fall prevention alarm use using a stepped-wedge randomized controlled trial among 30 adult medical or medical surgical units from nonfederal US acute care hospitals. Guided by the Choosing Wisely De-Implementation Framework, we will (1) identify barriers to fall prevention alarm de-implementation and develop tailored de-implementation strategies for each unit and (2) compare the implementation and effectiveness of high- versus low-intensity coaching to support site-specific de-implementation of fall prevention alarms. We will evaluate effectiveness and implementation outcomes and examine the effect of multi-level (e.g., hospital, unit, and patient) factors on effectiveness and implementation. Rate of fall prevention alarm use is the primary outcome. Balancing measures will include fall rates and fall-related injuries. Implementation outcomes will include feasibility, acceptability, appropriateness, and fidelity.

DISCUSSION

Findings from this line of research could be used to support scale-up of fall prevention alarm de-implementation in other healthcare settings. Further, research generated from this proposal will advance the field of de-implementation science by determining the extent to which low-intensity coaching is an effective and feasible de-implementation strategy.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT06089239 . Date of registration: October 17, 2023.

What would other emergency stroke teams do? Using explainable machine learning to understand variation in thrombolysis practice

INTRODUCTION

The aim of this work was to understand between-hospital variation in thrombolysis use among emergency stroke admissions in England and Wales.

PATIENTS

A total of 88,928 patients who arrived at all 132 emergency stroke hospitals in England Wales within 4 h of stroke onset, from 2016 to 2018.

METHODS

Machine learning was applied to the Sentinel Stroke National Audit Programme (SSNAP) data set, to learn which patients in each hospital would likely receive thrombolysis. We used XGBoost machine learning models, coupled with a SHAP model for explainability; Shapley (SHAP) values, providing estimates of how patient features, and hospital identity, influence the odds of receiving thrombolysis.

RESULTS

Thrombolysis use in patients arriving within 4 h of known or estimated stroke onset ranged 7% -49% between hospitals. The odds of receiving thrombolysis reduced 9-fold over the first 120 min of arrival-to-scan time, varied 30-fold with stroke severity, reduced 3-fold with estimated rather than precise stroke onset time, fell 6-fold with increasing pre-stroke disability, fell 4-fold with onset during sleep, fell 5-fold with use of anticoagulants, fell 2-fold between 80 and 110 years of age, reduced 3-fold between 120 and 240 min of onset-to-arrival time and varied 13-fold between hospitals. The majority of between-hospital variance was explained by the hospital, rather than the differences in local patient populations.

CONCLUSIONS

Using explainable machine learning, we identified that the majority of the between-hospital variation in thrombolysis use in England and Wales may be explained by differences in in-hospital processes and differences in attitudes to judging suitability for thrombolysis.

Addressing social determinants of health in primary care: a quasi-experimental study using unannounced standardised patients to evaluate the impact of audit/feedback on physicians' rates of identifying and responding to social needs

BACKGROUND

Although efforts are underway to address social determinants of health (SDOH), little is known about physicians' SDOH practices despite evidence that failing to fully elicit and respond to social needs can compromise patient safety and undermine both the quality and effectiveness of treatment. In particular, interventions designed to enhance response to social needs have not been assessed using actual practice behaviour. In this study, we evaluate the degree to which providing primary care physicians with feedback on their SDOH practice behaviours is associated with increased rates of eliciting and responding to housing and social isolation needs.

METHODS

Unannounced standardised patients (USPs), actors trained to consistently portray clinical scenarios, were sent, incognito, to all five primary care teams in an urban, safety-net healthcare system. Scenarios involved common primary care conditions and each included an underlying housing (eg, mould in the apartment, crowding) and social isolation issue and USPs assessed whether the physician fully elicited these needs and if so, whether or not they addressed them. The intervention consisted of providing physicians with audit/feedback reports of their SDOH practices, along with brief written educational material. A prepost comparison group design was used to evaluate the intervention; four teams received the intervention and one team served as a 'proxy' comparison (no intervention). Preintervention (February 2017 to December 2017) rates of screening for and response to the scripted housing and social needs were compared with intervention period (January 2018 to March 2019) rates for both intervention and comparison teams.

RESULTS

108 visits were completed preintervention and 183 during the intervention period. Overall, social needs were not elicited half of the time and fully addressed even less frequently. Rates of identifying the housing issue increased for teams that received audit/feedback reports (46%-60%; p=0.045) and declined for the proxy comparison (61%-42%; p=0.174). Rates of responding to housing needs increased significantly for intervention teams (15%-41%; p=0.004) but not for the comparison team (21%-29%; p=0.663). Social isolation was identified more frequently postintervention (53%) compared with baseline (39%; p=0.041) among the intervention teams but remained unchanged for the comparison team (39% vs 32%; p=0.601). Full exploration of social isolation remained low for both intervention and comparison teams.

CONCLUSIONS

Results suggest that physicians may not be consistently screening for or responding to social needs but that receiving feedback on those practices, along with brief targeted education, can improve rates of SDOH screening and response.

A cluster randomised controlled trial, process and economic evaluation of quality improvement collaboratives aligned to a national audit to improve the care for people with diabetes (EQUIPD): study protocol

BACKGROUND

People with type 1 diabetes and raised glucose levels are at greater risk of retinopathy, nephropathy, neuropathy, cardiovascular disease, sexual health problems and foot disease. The UK National Institute for Health and Care Excellence (NICE) recommends continuous subcutaneous 'insulin pump' therapy for people with type 1 diabetes whose HbA1c is above 69 mmol/mol. Insulin pump use can improve quality of life, cut cardiovascular risk and increase treatment satisfaction. About 90,000 people in England and Wales meet NICE criteria for insulin pumps but do not use one. Insulin pump use also varies markedly by deprivation, ethnicity, sex and location. Increasing insulin pump use is a key improvement priority. Audit and feedback is a common but variably effective intervention. Limited capabilities of healthcare providers to mount effective responses to feedback from national audits, such as the National Diabetes Audit (NDA), undermines efforts to improve care. We have co-developed a theoretically and empirically informed quality improvement collaborative (QIC) to strengthen local responses to feedback with patients and carers, national audits and healthcare providers. We will evaluate whether the QIC improves the uptake of insulin pumps following NDA feedback.

METHODS

We will undertake an efficient cluster randomised trial using routine data. The QIC will be delivered alongside the NDA to specialist diabetes teams in England and Wales. Our primary outcome will be the proportion of people with type 1 diabetes and an HbA1c above 69 mmol/mol who start and continue insulin pump use during the 18-month intervention period. Secondary outcomes will assess change in glucose control and duration of pump use. Subgroup analyses will explore impacts upon inequalities by ethnicity, sex, age and deprivation. A theory-informed process evaluation will explore diabetes specialist teams' engagement, implementation, fidelity and tailoring through observations, interviews, surveys and documentary analysis. An economic evaluation will micro-cost the QIC, estimate cost-effectiveness of NDA feedback with QIC and estimate the budget impact of NHS-wide QIC roll out.

DISCUSSION

Our study responds to a need for more head-to-head trials of different ways of reinforcing feedback delivery. Our findings will have implications for other large-scale audit and feedback programmes.

TRIAL REGISTRATION

ISRCTN82176651 Registered 18 October 2022.

Implementing Innovative Approaches to Improve Health Care Delivery Systems for Integrating Communicable and Non-Communicable Diseases Using Tuberculosis and Diabetes as a Model in Tanzania

Background: Many evidence-based health interventions, particularly in low-income settings, have failed to deliver the expected impact. We designed an Adaptive Diseases Control Expert Programme in Tanzania (ADEPT) to address systemic challenges in health care delivery and examined the feasibility, acceptability and effectiveness of the model using tuberculosis (TB) and diabetes mellitus (DM) as a prototype. Methods: This was an effectiveness-implementation hybrid type-3 design that was implemented in Dar es Salaam, Iringa and Kilimanjaro regions. The strategy included a stepwise training approach with web-based platforms adapting the Gibbs' reflective cycle. Health facilities with TB services were supplemented with DM diagnostics, including glycated haemoglobin A1c (HbA1c). The clinical audit was deployed as a measure of fidelity. Retrospective and cross-sectional designs were used to assess the fidelity, acceptability and feasibility of the model. Results: From 2019-2021, the clinical audit showed that ADEPT intervention health facilities more often identified median 8 (IQR 6-19) individuals with dual TB and DM, compared with control health facilities, median of 1 (IQR 0-3) (p = 0.02). Likewise, the clinical utility of HbA1c on intervention sites was 63% (IQR:35-75%) in TB/DM individuals compared to none in the control sites at all levels, whereas other components of the standard of clinical management of patients with dual TB and DM did not significantly differ. The health facilities showed no difference in screening for additional comorbidities such as hypertension and malnutrition. The stepwise training enrolled a total of 46 nurse officers and medical doctors/specialists for web-based training and 40 (87%) attended the workshop. Thirty-one (67%), 18 nurse officers and 13 medical doctors/specialists, implemented the second step of training others and yielded a total of 519 additional front-line health care workers trained: 371 nurses and 148 clinicians. Overall, the ADEPT model was scored as feasible by metrics applied to both front-line health care providers and health facilities. Conclusions: It was feasible to use a stepwise training and clinical audit to support the integration of TB and DM management and it was largely acceptable and effective in differing regions within Tanzania. When adapted in the Tanzania health system context, the model will likely improve quality of services.

What is the role of randomised trials in implementation science?

BACKGROUND

There is a consistent demand for implementation science to inform global efforts to close the gap between evidence and practice. Key evaluation questions for any given implementation strategy concern the assessment and understanding of effects. Randomised trials are generally accepted as offering the most trustworthy design for establishing effectiveness but may be underused in implementation science.

MAIN BODY

There is a continuing debate about the primacy of the place of randomised trials in evaluating implementation strategies, especially given the evolution of more rigorous quasi-experimental designs. Further critiques of trials for implementation science highlight that they cannot provide 'real world' evidence, address urgent and important questions, explain complex interventions nor understand contextual influences. We respond to these critiques of trials and highlight opportunities to enhance their timeliness and relevance through innovative designs, embedding within large-scale improvement programmes and harnessing routine data. Our suggestions for optimising the conditions for randomised trials of implementation strategies include strengthening partnerships with policy-makers and clinical leaders to realise the long-term value of rigorous evaluation and accelerating ethical approvals and decluttering governance procedures for lower risk studies.

CONCLUSION

Policy-makers and researchers should avoid prematurely discarding trial designs when evaluating implementation strategies and work to enhance the conditions for their conduct.

Feedback for Emergency Ambulance Staff: A National Review of Current Practice Informed by Realist Evaluation Methodology

Research suggests that feedback in Emergency Medical Services (EMS) positively affects quality of care and professional development. However, the mechanisms by which feedback achieves its effects still need to be better understood across healthcare settings. This study aimed to understand how United Kingdom (UK) ambulance services provide feedback for EMS professionals and develop a programme theory of how feedback works within EMS, using a mixed-methods, realist evaluation framework. A national cross-sectional survey was conducted to identify feedback initiatives in UK ambulance services, followed by four in-depth case studies involving qualitative interviews and documentary analysis. We used qualitative content analysis and descriptive statistics to analyse survey responses from 40 prehospital feedback initiatives, alongside retroductive analysis of 17 interviews and six documents from case study sites. Feedback initiatives mainly provided individual patient outcome feedback through "pull" initiatives triggered by staff requests. Challenges related to information governance were identified. Our programme theory of feedback to EMS professionals encompassed context (healthcare professional and organisational characteristics), mechanisms (feedback and implementation characteristics, psychological reasoning) and outcomes (implementation, staff and service outcomes). This study suggests that most UK ambulance services use a range of feedback initiatives and provides 24 empirically based testable hypotheses for future research.

Best practice guidance for antibiotic audit and feedback interventions in primary care: a modified Delphi study from the Joint Programming Initiative on Antimicrobial resistance: Primary Care Antibiotic Audit and Feedback Network (JPIAMR-PAAN)

BACKGROUND

Primary care is a critical partner for antimicrobial stewardship efforts given its high human antibiotic usage. Peer comparison audit and feedback (A&F) is often used to reduce inappropriate antibiotic prescribing. The design and implementation of A&F may impact its effectiveness. There are no best practice guidelines for peer comparison A&F in antibiotic prescribing in primary care.

OBJECTIVE

To develop best practice guidelines for peer comparison A&F for antibiotic prescribing in primary care in high income countries by leveraging international expertise via the Joint Programming Initiative on Antimicrobial Resistance-Primary Care Antibiotic Audit and Feedback Network.

METHODS

We used a modified Delphi process to achieve convergence of expert opinions on best practice statements for peer comparison A&F based on existing evidence and theory. Three rounds were performed, each with online surveys and virtual meetings to enable discussion and rating of each best practice statement. A five-point Likert scale was used to rate consensus with a median threshold score of 4 to indicate a consensus statement.

RESULTS

The final set of guidelines include 13 best practice statements in four categories: general considerations (n = 3), selecting feedback recipients (n = 1), data and indicator selection (n = 4), and feedback delivery (n = 5).

CONCLUSION

We report an expert-derived best practice recommendations for designing and evaluating peer comparison A&F for antibiotic prescribing in primary care. These 13 statements can be used by A&F designers to optimize the impact of their quality improvement interventions, and improve antibiotic prescribing in primary care.

A scoping review to inform an auditing framework evaluating healthcare environments for inclusion of people with intellectual disability and/or autism

People with intellectual disability and/or autism are likely to be in hospital more often, for longer, and have poorer health outcomes. Few audit tools exist to identify their barriers in mainstream healthcare environments. This study aimed to identify evidence of audit characteristics of healthcare contexts specifically for people with intellectual disability and/or autism, for conceptual development of an auditing framework. A scoping review of evaluations of healthcare environments was completed in January 2023. Findings were presented using the PAGER framework. Of the sixteen studies identified, most originated in the UK, nine focused on intellectual disability, four on autism, and three were concerned with mixed diagnosis. Six domains for auditing healthcare environments were identified: care imperatives, communication to individuals, understanding communication from individuals, providing supportive environments of care, supporting positive behaviour, and actions to make things go well. Further research is recommended to refine an audit framework.

Bridging the gap: Study protocol for development of an implementation strategy for evidence-informed reablement and rehabilitation for community-dwelling people with dementia

BACKGROUND

Extensive research supports the use of goal-directed reablement and rehabilitation interventions to address a range of physical, functional, cognitive and behavioural needs of people living with dementia. Despite this, evidence-informed multidisciplinary reablement and rehabilitation interventions are not being offered in usual dementia care across Australia. An examination is needed of how best to implement reablement and rehabilitation interventions within the community-based dementia care sector.

METHODS

Drawing on implementation science, this study uses a four-phase mixed-methods retrospective and prospective approach: (1) clinical audit to evaluate current clinical practice, and through focus groups with practitioners, identify practitioner-led goals and targets for practice change; (2) Delphi survey to converge opinions from the diverse stakeholders involved in reablement in dementia, to reach national consensus around an implementation strategy; (3) hybrid pragmatic effectiveness-implementation pilot will facilitate testing of the implementation strategy in parallel with exploring effectiveness of the reablement intervention specifically within a real-world Australian community aged care context; (4) implementation capacity building.

DISCUSSION

This study will result in a freely available, nationally relevant implementation protocol, designed and tailored via input from key stakeholders over a series of iterative project activities. By testing this protocol via a pilot implementation-effectiveness study, we will generate national information about effectiveness of evidence-informed reablement programs for people living with dementia across various community aged care settings. Outcomes have potential to influence policy and drive widespread practice change, increasing access to evidence-informed reablement and rehabilitation for people living with dementia across Australia.

Clinical audit in European radiology: current status and recommendations for improvement endorsed by the European Society of Radiology (ESR)

Clinical audit is an important quality improvement activity and has significant benefits for patients in terms of enhanced care, safety, experience and outcomes. Clinical audit in support of radiation protection is mandated within the European Council Basic Safety Standards Directive (BSSD), 2013/59/Euratom. The European Society of Radiology (ESR) has recognised clinical audit as an area of particular importance in the delivery of safe and effective health care. The ESR, alongside other European organisations and professional bodies, has developed a range of clinical audit-related initiatives to support European radiology departments in developing a clinical audit infrastructure and fulfilling their legal obligations. However, work by the European Commission, the ESR and other agencies has demonstrated a persisting variability in clinical audit uptake and implementation across Europe and a lack of awareness of the BSSD clinical audit requirements. In recognition of these findings, the European Commission supported the QuADRANT project, led by the ESR and in partnership with ESTRO (European Association of Radiotherapy and Oncology) and EANM (European Association of Nuclear Medicine). QuADRANT was a 30-month project which completed in Summer 2022, aiming to provide an overview of the status of European clinical audit and identifying barriers and challenges to clinical audit uptake and implementation. This paper summarises the current position of European radiological clinical audit and considers the barriers and challenges that exist. Reference is made to the QuADRANT project, and a range of potential solutions are suggested to enhance radiological clinical audit across Europe.

How do healthcare providers use national audit data for improvement?

BACKGROUND

Substantial resources are invested by Health Departments worldwide in introducing National Clinical Audits (NCAs). Yet, there is variable evidence on the NCAs' effectiveness and little is known on factors underlying the successful use of NCAs to improve local practice. This study will focus on a single NCA (the National Audit of Inpatient Falls -NAIF 2017) to explore: (i) participants' perspectives on the NCA reports, local feedback characteristics and actions undertaken following the feedback underpinning the effective use of the NCA feedback to improve local practice; (ii) reported changes in local practice following the NCA feedback in England and Wales.

METHODS

Front-line staff perspectives were gathered through interviews. An inductive qualitative approach was used. Eighteen participants were purposefully sampled from 7 of the 85 participating hospitals in England and Wales. Analysis was guided by constant comparative techniques.

RESULTS

Regarding the NAIF annual report, interviewees valued performance benchmarking with other hospitals, the use of visual representations and the inclusion of case studies and recommendations. Participants stated that feedback should target front-line healthcare professionals, be straightforward and focused, and be delivered through an encouraging and honest discussion. Interviewees highlighted the value of using other relevant data sources alongside NAIF feedback and the importance of continuous data monitoring. Participants reported that engagement of front-line staff in the NAIF and following improvement activities was critical. Leadership, ownership, management support and communication at different organisational levels were perceived as enablers, while staffing level and turnover, and poor quality improvement (QI) skills, were perceived as barriers to improvement. Reported changes in practice included increased awareness and attention to patient safety issues and greater involvement of patients and staff in falls prevention activities.

CONCLUSIONS

There is scope to improve the use of NCAs by front-line staff. NCAs should not be seen as isolated interventions but should be fully embedded and integrated into the QI strategic and operational plans of NHS trusts. The use of NCAs could be optimised, but knowledge of them is poor and distributed unevenly across different disciplines. More research is needed to provide guidance on key elements to consider throughout the whole improvement process at different organisational levels.

An Exploration of Critical Care Professionals' Strategies to Enhance Daily Implementation of the Assess, Prevent, and Manage Pain; Both Spontaneous Awakening and Breathing Trials; Choice of Analgesia and Sedation; Delirium Assess, Prevent, and Manage; Early Mobility and Exercise; and Family Engagement and Empowerment: A Group Concept Mapping Study

The goals of this exploratory study were to engage professionals from the Society for Critical Care Medicine ICU Liberation Collaborative ICUs to: 1) conceptualize strategies to enhance daily implementation of the Assess, prevent, and manage pain; Both spontaneous awakening and breathing trials; Choice of analgesia and sedation; Delirium assess, prevent, and manage; Early mobility and exercise; and Family engagement and empowerment (ABCDEF) bundle from different perspectives and 2) identify strategies to prioritize for implementation.

DESIGN

Mixed-methods group concept mapping over 8 months using an online method. Participants provided strategies in response to a prompt about what was needed for successful daily ABCDEF bundle implementation. Responses were summarized into a set of unique statements and then rated on a 5-point scale on degree of necessity (essential) and degree to which currently used.

SETTING

Sixty-eight academic, community, and federal ICUs.

PARTICIPANTS

A total of 121 ICU professionals consisting of frontline and leadership professionals.

INTERVENTIONS

None.

MEASUREMENTS AND MAIN RESULTS

A final set of 76 strategies (reduced from 188 responses) were suggested: education (16 strategies), collaboration (15 strategies), processes and protocols (13 strategies), feedback (10 strategies), sedation/pain practices (nine strategies), education (eight strategies), and family (five strategies). Nine strategies were rated as very essential but infrequently used: adequate staffing, adequate mobility equipment, attention to (patient's) sleep, open discussion and collaborative problem solving, nonsedation methods to address ventilator dyssynchrony, specific expectations for night and day shifts, education of whole team on interdependent nature of the bundle, and effective sleep protocol.

CONCLUSIONS

In this concept mapping study, ICU professionals provided strategies that spanned a number of conceptual implementation clusters. Results can be used by ICU leaders for implementation planning to address context-specific interdisciplinary approaches to improve ABCDEF bundle implementation.

Publishing quality improvement studies: learning to share and sharing to learn

This editorial welcomes the decision of BJA Open to publish quality improvement (QI) studies. It summarises the current problems with conducting, evaluating, and publishing QI studies. It highlights existing guidance for prospective authors to follow regarding the reporting of QI interventions, their context(s), underlying theories, and evaluation. In so doing, we hope to encourage the publication of more QI studies of sufficient quality to facilitate learning or replication elsewhere.

'The way that we are collecting and using data has evolved' evaluating the Australian National Stroke Audit programme to inform strategic direction

BACKGROUND

The National Stroke Audit has been used to audit and provide feedback to health professionals and stroke care services in Australia since 2007. The Australian Stroke Clinical Registry was piloted in 2009 and numbers of hospitals participating in the registry are increasing. Considering the changing data landscape in Australia, we designed this study to evaluate the stroke audit and to inform strategic direction.

METHODS

We conducted a rapid review of published literature to map features of successful data programmes, followed by a mixed-methods study, comprising national surveys and interviews with clinicians and administrators about the stroke audit. We analysed quantitative data descriptively and analysed open-ended survey responses and interview data using qualitative content analysis. We integrated data from the two sources.

RESULTS

We identified 47 Australian data programs, successful programs were usually funded by government sources or professional associations and typically provided twice yearly or yearly reports.106 survey participants, 14 clinician and 5 health administrator interview participants were included in the evaluation. The Stroke Audit was consistently perceived as useful for benchmarking, but there were mixed views about its value for local quality improvement. Time to enter data was the most frequently reported barrier to participation (88% of survey participants), due to the large number of datapoints and features of the audit software.Opportunities to improve the Stroke Audit included refining Audit questions, developing ways to automatically export data from electronic medical records and capturing accurate data for patients who transferred between hospitals.

CONCLUSION

While the Stroke Audit was not perceived by all users to be beneficial for traditional quality improvement purposes, the ability to benchmark national stroke services and use these data in advocacy activities was a consistently reported benefit. Modifications were suggested to improve usability and usefulness for participating sites.

Guidelines for Perioperative Care for Liver Surgery: Enhanced Recovery After Surgery (ERAS) Society Recommendations 2022

BACKGROUND

Enhanced Recovery After Surgery (ERAS) has been widely applied in liver surgery since the publication of the first ERAS guidelines in 2016. The aim of the present article was to update the ERAS guidelines in liver surgery using a modified Delphi method based on a systematic review of the literature.

METHODS

A systematic literature review was performed using MEDLINE/PubMed, Embase, and the Cochrane Library. A modified Delphi method including 15 international experts was used. Consensus was judged to be reached when >80% of the experts agreed on the recommended items. Recommendations were based on the Grading of Recommendations, Assessment, Development and Evaluations system.

RESULTS

A total of 7541 manuscripts were screened, and 240 articles were finally included. Twenty-five recommendation items were elaborated. All of them obtained consensus (>80% agreement) after 3 Delphi rounds. Nine items (36%) had a high level of evidence and 16 (64%) a strong recommendation grade. Compared to the first ERAS guidelines published, 3 novel items were introduced: prehabilitation in high-risk patients, preoperative biliary drainage in cholestatic liver, and preoperative smoking and alcohol cessation at least 4 weeks before hepatectomy.

CONCLUSIONS

These guidelines based on the best available evidence allow standardization of the perioperative management of patients undergoing liver surgery. Specific studies on hepatectomy in cirrhotic patients following an ERAS program are still needed.

A Systematic Review of Electronic Medical Record Driven Quality Measurement and Feedback Systems

Historically, quality measurement analyses utilize manual chart abstraction from data collected primarily for administrative purposes. These methods are resource-intensive, time-delayed, and often lack clinical relevance. Electronic Medical Records (EMRs) have increased data availability and opportunities for quality measurement. However, little is known about the effectiveness of Measurement Feedback Systems (MFSs) in utilizing EMR data. This study explores the effectiveness and characteristics of EMR-enabled MFSs in tertiary care. The search strategy guided by the PICO Framework was executed in four databases. Two reviewers screened abstracts and manuscripts. Data on effect and intervention characteristics were extracted using a tailored version of the Cochrane EPOC abstraction tool. Due to study heterogeneity, a narrative synthesis was conducted and reported according to PRISMA guidelines. A total of 14 unique MFS studies were extracted and synthesized, of which 12 had positive effects on outcomes. Findings indicate that quality measurement using EMR data is feasible in certain contexts and successful MFSs often incorporated electronic feedback methods, supported by clinical leadership and action planning. EMR-enabled MFSs have the potential to reduce the burden of data collection for quality measurement but further research is needed to evaluate EMR-enabled MFSs to translate and scale findings to broader implementation contexts.

Interventions targeting healthcare providers to optimise use of caesarean section: a qualitative comparative analysis to identify important intervention features

BACKGROUND

Rapid increases in caesarean section (CS) rates have been observed globally; however, CS rates exceeding 15% at a population-level have limited benefits for women and babies. Many interventions targeting healthcare providers have been developed to optimise use of CS, typically aiming to improve and monitor clinical decision-making. However, interventions are often complex, and effectiveness is varied. Understanding intervention and implementation features that likely lead to optimised CS use is important to optimise benefits. The aim of this study was to identify important components that lead to successful interventions to optimise CS, focusing on interventions targeting healthcare providers.  METHODS: We used Qualitative Comparative Analysis (QCA) to identify if certain combination of important intervention features (e.g. type of intervention, contextual characteristics, and how the intervention was delivered) are associated with a successful intervention as reflected in a reduction of CS. We included 21 intervention studies targeting healthcare providers to reduce CS, comprising of 34 papers reporting on these interventions. To develop potential theories driving intervention success, we used existing published qualitative evidence syntheses on healthcare providers' perspectives and experiences of interventions targeted at them to reduce CS.

RESULTS

We identified five important components that trigger successful interventions targeting healthcare providers: 1) training to improve providers' knowledge and skills, 2) active dissemination of CS indications, 3) actionable recommendations, 4) multidisciplinary collaboration, and 5) providers' willingness to change. Importantly, when one or more of these components are absent, dictated nature of intervention, where providers are enforced to adhere to the intervention, is needed to prompt successful interventions. Unsuccessful interventions were characterised by the absence of these components.

CONCLUSION

We identified five important intervention components and combinations of intervention components which can lead to successful interventions targeting healthcare providers to optimise CS use. Health facility managers, researchers, and policy-makers aiming to improve providers' clinical decision making and reduce CS may consider including the identified components to optimise benefits.

'Give me something meaningful': GPs perspectives on how to improve an audit and feedback report provided by health insurers - an exploratory qualitative study

BACKGROUND

Audit and feedback (A&F) is a valuable quality improvement strategy, which can contribute to de-implementation of low-value care. In the Netherlands, all health insurers collaboratively provide A&F to general practitioners (GPs), the 'Primary Care Practice Report' (PCPR). Unfortunately, the use of this report by GPs is limited. This study examined the thoughts of GPs on the usability of the PCPR and GPs recommendations for improving the PCPR.

METHOD

We used an interpretative qualitative design, with think-aloud tasks to uncover thoughts of GPs on the usability of the PCPR and semistructured interview questions to ask GPs' recommendations for improvement of the PCPR. Interviews were audiorecorded and transcribed ad verbatim. Data were analysed using thematic content analysis.

RESULTS

We identified two main themes: 'poor usability of the PCPR', and 'minimal motivation to change based on the PCPR'. The GPs found the usability of the PCPR poor due to the feedback not being clinically meaningful, the data not being recent, individual and reliable, the performance comparators offer insufficient guidance to assess clinical performance, the results are not discussed with peers and the definitions and visuals are unclear. The GPs recommended improving these issues. The GPs motivation to change based on the PCPR was minimal.

CONCLUSIONS

The GPs evaluated the PCPR as poorly usable and were minimally motivated to change. The PCPR seems developed from the perspective of the reports' commissioners, health insurers, and does not meet known criteria for effective A&F design and user-centred design. Importantly, the GPs did state that well-designed feedback could contribute to their motivation to improve clinical performance.Furthermore, the GPs stated that they receive a multitude of A&F reports, which they hardly use. Thus, we see a need for policy makers to invest in less, but more usable A&F reports.

A scoping review of local quality improvement using data from UK perioperative National Clinical Audits

Background

Significant resources are invested in the UK to collect data for National Clinical Audits (NCAs), but it is unclear whether and how they facilitate local quality improvement (QI). The perioperative setting is a unique context for QI due to its multidisciplinary nature and history of measurement. It is unclear which NCAs evaluate perioperative care, to what extent their data have been used for QI, and which factors influence this usage.

Methods

NCAs were identified from the directories held by Healthcare Quality Improvement Partnership (HQIP), Scottish Healthcare Audits and the Welsh National Clinical Audit and Outcome Review Advisory Committee. QI reports were identified by the following: systematically searching MEDLINE, CINAHL Plus, Web of Science, Embase, Google Scholar and HMIC up to December 2019, hand-searching grey literature and consulting relevant stakeholders. We charted features describing both the NCAs and the QI reports and summarised quantitative data using descriptive statistics and qualitative themes using framework analysis.

Results

We identified 36 perioperative NCAs in the UK and 209 reports of local QI which used data from 19 (73%) of these NCAs. Six (17%) NCAs contributed 185 (89%) of these reports. Only one NCA had a registry of local QI projects. The QI reports were mostly brief, unstructured, often published by NCAs themselves and likely subject to significant reporting bias. Factors reported to influence local QI included the following: perceived data validity, measurement of clinical processes as well as outcomes, timely feedback, financial incentives, sharing of best practice, local improvement capabilities and time constraints of clinicians.

Conclusions

There is limited public reporting of UK perioperative NCA data for local QI, despite evidence of improvement of most NCA metrics at the national level. It is therefore unclear how these improvements are being made, and it is likely that opportunities are being missed to share learning between local sites. We make recommendations for how NCAs could better support the conduct, evaluation and reporting of local QI and suggest topics which future research should investigate.

Trial registration

The review was registered with the International Prospective Register of Systematic Reviews (PROSPERO: CRD42018092993).

Supplementary Information

The online version contains supplementary material available at 10.1186/s13741-022-00273-0.

Optimization of a Quality Improvement Tool for Cancer Diagnosis in Primary Care: Qualitative Study

Background

The most common route to a diagnosis of cancer is through primary care. Delays in diagnosing cancer occur when an opportunity to make a timely diagnosis is missed and is evidenced by patients visiting the general practitioner (GP) on multiple occasions before referral to a specialist. Tools that minimize prolonged diagnostic intervals and reduce missed opportunities to investigate patients for cancer are therefore a priority.

Objective

This study aims to explore the usefulness and feasibility of a novel quality improvement (QI) tool in which algorithms flag abnormal test results that may be indicative of undiagnosed cancer. This study allows for the optimization of the cancer recommendations before testing the efficacy in a randomized controlled trial.

Methods

GPs, practice nurses, practice managers, and consumers were recruited to participate in individual interviews or focus groups. Participants were purposively sampled as part of a pilot and feasibility study, in which primary care practices were receiving recommendations relating to the follow-up of abnormal test results for prostate-specific antigen, thrombocytosis, and iron-deficiency anemia. The Clinical Performance Feedback Intervention Theory (CP-FIT) was applied to the analysis using a thematic approach.

Results

A total of 17 interviews and 3 focus groups (n=18) were completed. Participant themes were mapped to CP-FIT across the constructs of context, recipient, and feedback variables. The key facilitators to use were alignment with workflow, recognized need, the perceived importance of the clinical topic, and the GPs’ perception that the recommendations were within their control. Barriers to use included competing priorities, usability and complexity of the recommendations, and knowledge of the clinical topic. There was consistency between consumer and practitioner perspectives, reporting language concerns associated with the word cancer, the need for more patient-facing resources, and time constraints of the consultation to address patients’ worries.

Conclusions

There was a recognized need for the QI tool to support the diagnosis of cancer in primary care, but barriers were identified that hindered the usability and actionability of the recommendations in practice. In response, the tool has been refined and is currently being evaluated as part of a randomized controlled trial. Successful and effective implementation of this QI tool could support the detection of patients at risk of undiagnosed cancer in primary care and assist in preventing unnecessary delays.

A multi-faceted intervention to improve screening of erectile dysfunction for men with diabetes mellitus in public health clinics: A pilot study

INTRODUCTION

Erectile dysfunction (ED) is common in men with diabetes and is associated with coronary artery disease and psychological distress. However, discussion of ED in primary care consultations is uncommon. Interventions, such as audit and feedback, mandate of management, and workshop on men's sexual health, have been proposed to improve ED screening in public health clinics. This study aimed to pilot test a multi-faceted intervention to increase ED screening among men with diabetes and to improve knowledge and confidence in ED screening among primary healthcare providers (PHPs).

METHODS

We conducted an interrupted time-series quasi-experimental study with PHPs at public health clinics in Perak, Malaysia from February 2019 to February 2020. Doctors, nurses, and assistant medical officers involved in diabetes care were recruited. The intervention was conducted between July and September 2019 and comprised two phases: audit and feedback, and mandate from management (phase 1), and an interactive face-to-face workshop (phase 2). The primary outcome measure was monthly ED screening rate, and the data were retrieved from health records and the clinical diabetes registry. The PHPs completed a questionnaire on ED knowledge and confidence in ED screening before and after the workshop.

RESULTS

The total number of attendances by patients with diabetes was 50,325 during the study period, of which 21,413 were by men. A total of 30 PHPs participated in the training workshop: 7 (23.3%) doctors, 12 (40%) medical assistants, and 11 (36.7%) nurses. The mean duration of employment at a health clinic was 7 ± 5 years. The majority of the participants were male (56.7%), Malay (80%), married (76.7%), and had no experience with working in a urology department (76.7%). There was a significant improvement in the mean knowledge score (39.0 ± 11.4 vs. 63.6 ± 6.2, p<0.001) and median confidence level in ED screening from 2 (IQR 2-3) to 4 (IQR 4-4) after the workshop (p<0.001). After phase 1, the rate of ED screening increased from a baseline of between 10.3% (February 2019) and 12.7% (June 2019) before intervention to between 15% (July 2019) and 18.9% (September 2019) and was maintained between 18% (October 2019) and 17.9% (December 2019) after phase 2. Subsequently, the ED screening rate declined from 14.6% (January 2020) to 10.9% (February 2020).

CONCLUSION

This study found that audit and feedback with mandate from management increased ED screening and detection rate. The workshop improved the participants' knowledge and confidence in ED screening but did not further increased ED screening. The improved practice was sustained for 6 months after the intervention, while the detection rate of ED persisted. We propose regular audit and feedback with mandate from management to sustain the practice of ED screening in men with diabetes.

‘More than just numbers on a page?’ A qualitative exploration of the use of data collection and feedback in youth mental health services

Objectives

This study aimed to explore current data collection and feedback practice, in the form of monitoring and evaluation, among youth mental health (YMH) services and healthcare commissioners; and to identify barriers and enablers to this practice.

Design

Qualitative semi-structured interviews were conducted via Zoom videoconferencing software. Data collection and analysis were informed by the Theoretical Domains Framework (TDF). Data were deductively coded to the 14 domains of the TDF and inductively coded to generate belief statements.

Setting

Healthcare commissioning organisations and YMH services in Australia.

Participants

Twenty staff from healthcare commissioning organisations and twenty staff from YMH services.

Results

The umbrella behaviour ‘monitoring and evaluation’ (ME) can be sub-divided into 10 specific sub-behaviours (e.g. planning and preparing, providing technical assistance, reviewing and interpreting data) performed by healthcare commissioners and YMH services. One hundred belief statements relating to individual, social, or environmental barriers and enablers were generated. Both participant groups articulated a desire to improve the use of ME for quality improvement and had particular interest in understanding the experiences of young people and families. Identified enablers included services and commissioners working in partnership, data literacy (including the ability to set appropriate performance indicators), relational skills, and provision of meaningful feedback. Barriers included data that did not adequately depict service performance, problems with data processes and tools, and the significant burden that data collection places on YMH services with the limited resources they have to do it.

Conclusions

Importantly, this study illustrated that the use of ME could be improved. YMH services, healthcare commissioners should collaborate on ME plans and meaningfully involve young people and families where possible. Targets, performance indicators, and outcome measures should explicitly link to YMH service quality improvement; and ME plans should include qualitative data. Streamlined data collection processes will reduce unnecessary burden, and YMH services should have the capability to interrogate their own data and generate reports. Healthcare commissioners should also ensure that they provide meaningful feedback to their commissioned services, and local and national organisations collecting youth mental health data should facilitate the sharing of this data. The results of the study should be used to design theory-informed strategies to improve ME use.

Enhancing national audit through addressing the quality improvement capabilities of feedback recipients: a multi-phase intervention development study

Background

National audits are a common, but variably effective, intervention to improve services. This study aimed to design an intervention to increase the effectiveness of national audit.

Methods

We used interviews, documentary analysis, observations, co-design and stakeholder engagement methods. The intervention was described in an intervention manual and illustrated using a logic model. Phase 1 described the current hospital response to a national audit. Phase 2 identified potential enhancements. Phase 3 developed a strategy to implement the enhancements. Phase 4 explored the feasibility of the intervention alongside the National Audit of Dementia and refined the intervention. Phase 5 adapted the intervention to a second national audit (National Diabetes Audit). Phase 6 explored the feasibility and fidelity of the intervention alongside the National Diabetes Audit and used the findings to further refine the intervention.

Results

The developed intervention is a quality improvement collaborative (QIC), containing virtual educational workshop, virtual outreach for local team leads and virtual facilitation of a learning collaborative delivered after feedback has been received. The QIC aims to support national audit recipients to undertake improvement actions tailored to their local context. The target audience is clinical and clinical governance leaders. We found that actions from national audit were constrained by what the clinical lead perceived they deliver personally, these actions were not aligned to identified influences upon performance. We found that the hospital response could be enhanced by targeting low baseline performance, identifying and addressing influences upon to performance, developing trust and credibility, addressing recipient priorities, presenting meaningful comparisons, developing a conceptual model, involving stakeholders and considering the opportunity cost. Phase 3 found that an educational workshop and outreach strategy could support implementation of the enhancements through developing coherence and cognitive participation. We found feasibility could be increased by revising the content, re-naming the intervention, amending activities to address time commitment, incorporating a more structured analysis of influences, supporting collaboration and developing local feedback mechanisms. Phase 5 found adaptation to a second national audit involved reflecting differences in the clinical topic, context and contractual requirements. We found that the behaviour change techniques identified in the manual were delivered by facilitators. Participants reported positive attitudes towards the intervention and that the intervention was appropriate.

Conclusions

The QIC supports local teams to tailor their actions to local context and develop change commitment. Future work will evaluate the effectiveness of the intervention as an adjunct to the National Diabetes Audit.

A framework for implementing antibiotic stewardship in ambulatory care: Lessons learned from the Agency for Healthcare Research and Quality Safety Program for Improving Antibiotic Use

Antibiotic overuse is common in ambulatory care settings, underscoring the importance of outpatient antibiotic stewardship to ensure safe and effective antibiotic prescription. In response to this need, the Agency for Healthcare Research and Quality (AHRQ) developed the AHRQ Safety Program for Improving Antibiotic Use in Ambulatory Care. The Safety Program successfully assisted 389 outpatient practices across the United States to establish ambulatory antibiotic stewardship. Herein, we have used lessons learned from the AHRQ Safety Program to describe a step-by-step framework to assist practices with establishing antibiotic stewardship in the outpatient setting. Steps include obtaining support from practice leadership; establishing an antibiotic stewardship team; garnering support from practice members; determining how to access antibiotic prescribing data; building communication skills around antibiotic use in the practice; implementing educational content around an infectious syndrome; monitoring antibiotic prescription data; and implementing a sustainability plan.

A randomised fractional factorial screening experiment to predict effective features of audit and feedback

Background

Audit and feedback aims to improve patient care by comparing healthcare performance against explicit standards. It is used to monitor and improve patient care, including through National Clinical Audit (NCA) programmes in the UK. Variability in effectiveness of audit and feedback is attributed to intervention design; separate randomised trials to address multiple questions about how to optimise effectiveness would be inefficient. We evaluated different feedback modifications to identify leading candidates for further “real-world” evaluation.

Methods

Using an online fractional factorial screening experiment, we randomised recipients of feedback from five UK NCAs to different combinations of six feedback modifications applied within an audit report excerpt: use effective comparators, provide multimodal feedback, recommend specific actions, provide optional detail, incorporate the patient voice, and minimise cognitive load. Outcomes, assessed immediately after exposure to the online modifications, included intention to enact audit standards (primary outcome, ranked on a scale of −3 to +3, tailored to the NCA), comprehension, user experience, and engagement.

Results

We randomised 1241 participants (clinicians, managers, and audit staff) between April and October 2019. Inappropriate repeated participant completion occurred; we conservatively excluded participant entries during the relevant period, leaving a primary analysis population of 638 (51.4%) participants.

None of the six feedback modifications had an independent effect on intention across the five NCAs. We observed both synergistic and antagonistic effects across outcomes when modifications were combined; the specific NCA and whether recipients had a clinical role had dominant influences on outcome, and there was an antagonistic interaction between multimodal feedback and optional detail. Among clinical participants, predicted intention ranged from 1.22 (95% confidence interval 0.72, 1.72) for the least effective combination in which multimodal feedback, optional detail, and reduced cognitive load were applied within the audit report, up to 2.40 (95% CI 1.88, 2.93) for the most effective combination including multimodal feedback, specific actions, patient voice, and reduced cognitive load.

Conclusion

Potentially important synergistic and antagonistic effects were identified across combinations of feedback modifications, audit programmes, and recipients, suggesting that feedback designers must explicitly consider how different features of feedback may interact to achieve (or undermine) the desired effects.

Trial registration

International Standard Randomised Controlled Trial Number: ISRCTN41584028

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-022-01208-5.

A Scalable Service to Improve Health Care Quality Through Precision Audit and Feedback: Proposal for a Randomized Controlled Trial

BACKGROUND

Health care delivery organizations lack evidence-based strategies for using quality measurement data to improve performance. Audit and feedback (A&F), the delivery of clinical performance summaries to providers, demonstrates the potential for large effects on clinical practice but is currently implemented as a blunt one size fits most intervention. Each provider in a care setting typically receives a performance summary of identical metrics in a common format despite the growing recognition that precisionizing interventions hold significant promise in improving their impact. A precision approach to A&F prioritizes the display of information in a single metric that, for each recipient, carries the highest value for performance improvement, such as when the metric's level drops below a peer benchmark or minimum standard for the first time, thereby revealing an actionable performance gap. Furthermore, precision A&F uses an optimal message format (including framing and visual displays) based on what is known about the recipient and the intended gist meaning being communicated to improve message interpretation while reducing the cognitive processing burden. Well-established psychological principles, frameworks, and theories form a feedback intervention knowledge base to achieve precision A&F. From an informatics perspective, precision A&F requires a knowledge-based system that enables mass customization by representing knowledge configurable at the group and individual levels.

OBJECTIVE

This study aims to implement and evaluate a demonstration system for precision A&F in anesthesia care and to assess the effect of precision feedback emails on care quality and outcomes in a national quality improvement consortium.

METHODS

We propose to achieve our aims by conducting 3 studies: a requirements analysis and preferences elicitation study using human-centered design and conjoint analysis methods, a software service development and implementation study, and a cluster randomized controlled trial of a precision A&F service with a concurrent process evaluation. This study will be conducted with the Multicenter Perioperative Outcomes Group, a national anesthesia quality improvement consortium with >60 member hospitals in >20 US states. This study will extend the Multicenter Perioperative Outcomes Group quality improvement infrastructure by using existing data and performance measurement processes.

RESULTS

The proposal was funded in September 2021 with a 4-year timeline. Data collection for Aim 1 began in March 2022. We plan for a 24-month trial timeline, with the intervention period of the trial beginning in March 2024.

CONCLUSIONS

The proposed aims will collectively demonstrate a precision feedback service developed using an open-source technical infrastructure for computable knowledge management. By implementing and evaluating a demonstration system for precision feedback, we create the potential to observe the conditions under which feedback interventions are effective.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/34990.

Opportunities to improve the impact of two national clinical audit programmes: a theory-guided analysis

Background

Audit and feedback is widely used in healthcare improvement, with evidence of modest yet potentially important effects upon professional practice. There are approximately 60 national clinical audit programmes in the UK. These programmes often develop and adapt new ways of delivering feedback to optimise impacts on clinical practice. Two such programmes, the National Diabetes Audit (NDA) and the Trauma Audit Research Network (TARN), recently introduced changes to their delivery of feedback. We assessed the extent to which the design of these audit programmes and their recent changes were consistent with best practice according to the Clinical Performance Feedback Intervention Theory (CP-FIT). This comprehensive framework specifies how variables related to the feedback itself, the recipient, and the context operate via explanatory mechanisms to influence feedback success.

Methods

We interviewed 19 individuals with interests in audit and feedback, including researchers, audit managers, healthcare staff, and patient and public representatives. This range of expert perspectives enabled a detailed exploration of feedback from the audit programmes. We structured interviews around the CP-FIT feedback cycle and its component processes (e.g. Data collection and analysis, Interaction). Our rapid analytic approach explored the extent to which both audits applied features consistent with CP-FIT.

Results

Changes introduced by the audit programmes were consistent with CP-FIT. Specifically, the NDA’s increased frequency of feedback augmented existing strengths, such as automated processes (CP-FIT component: Data collection and analysis) and being a credible source of feedback (Acceptance). TARN’s new analytic tool allowed greater interactivity, enabling recipients to interrogate their data (Verification; Acceptance). We also identified scope for improvement in feedback cycles, such as targeting of feedback recipients (Interaction) and feedback complexity (Perception) for the NDA and specifying recommendations (Intention) and demonstrating impact (Clinical performance improvement) for TARN.

Conclusions

The changes made by the two audit programmes appear consistent with suggested best practice, making clinical improvement more likely. However, observed weaknesses in the feedback cycle may limit the benefits of these changes. Applying CP-FIT via a rapid analysis approach helps identify strengths and remediable weaknesses in the design of audit programmes that can be shared with them in a timely manner.

Improving dementia care: insights from audit and feedback in interdisciplinary primary care sites

Background

Many primary care sites have implemented models to improve detection, diagnosis, and management of dementia, as per Canadian guidelines. The aim of this study is to describe the responses of clinicians, managers, and staff of sites that have implemented these models when presented with audit results, their insights on the factors that explain their results, their proposed solutions for improvement and how these align to one another.

Methods

One audit and feedback cycle was carried out in eight purposefully sampled sites in Ontario, Canada, that had previously implemented dementia care models. Audit consisted of a) chart review to assess quality of dementia care indicators, b) questionnaire to assess the physicians’ knowledge, attitudes and practice toward dementia care, and c) semi-structured interviews to understand barriers and facilitators to implementing these models. Feedback was given to clinicians, managers, and staff in the form of graphic and oral presentations, followed by eight focus groups (one per site). Discussions revolved around: what audit results elicited more discussion from the participants, 2) their insights on the factors that explain their audit results, and 3) solutions they propose to improve dementia care. Deductive content and inductive thematic analyses, grounded in causal pathways models’ theory was performed.

Findings

The audit and feedback process allowed the 63 participants to discuss many audit results and share their insights on a) organizational factors (lack of human resources, the importance of organized links with community services, clear roles and support from external memory clinics) and b) clinician factors (perceived competency practice and attitudes on dementia care), that could explain their audit results. Participants also provided solutions to improve dementia care in primary care (financial incentives, having clear pathways, adding tools to improve chart documentation, establish training on dementia care, and the possibility of benchmarking with other institutions). Proposed solutions were well aligned with their insights and further nuanced according to contextual details.

Conclusions

This study provides valuable information on solutions proposed by primary care clinicians, managers, and staff to improve dementia care in primary care. The solutions are grounded in clinical experience and will inform ongoing and future dementia strategies.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-022-07672-5.

Making the most of audit and feedback to improve diabetes care: a qualitative study of the perspectives of Australian Diabetes Centres

BACKGROUND

Diabetes has high burden on the health system and the individual, and many people living with diabetes struggle to optimally manage their condition. In Australia, people living with diabetes attend a mixture of primary, secondary and tertiary care centres. Many of these Diabetes Centres participate in the Australian National Diabetes Audit (ANDA), a quality improvement (QI) activity that collects clinical information (audit) and feeds back collated information to participating sites (feedback). Despite receiving this feedback, many process and care outcomes for Diabetes Centres continue to show room for improvement. The purpose of this qualitative study was to inform improvement of the ANDA feedback, identify the needs of those receiving feedback and elicit the barriers to and enablers of optimal feedback use.

METHODS

Semi-structured interviews were conducted with representatives of Australian Diabetes Centres, underpinned by the Consolidated Framework for Implementation Research (CFIR). De-identified transcripts were analysed thematically, underpinned by the domains and constructs of the CFIR.

RESULTS

Representatives from 14 Diabetes centres participated in this study, including a diverse range of staff typical of the Diabetes Centres who take part in ANDA. In general, participants wanted a shorter report with a more engaging, simplified data visualisation style. Identified barriers to use of feedback were time or resource constraints, as well as access to knowledge about how to use the data provided to inform the development of QI activities. Enablers included leadership engagement, peer mentoring and support, and external policy and incentives. Potential cointerventions to support use include exemplars from clinical change champions and peer leaders, and educational resources to help facilitate change.

CONCLUSIONS

This qualitative study supported our contention that the format of ANDA feedback presentation can be improved. Healthcare professionals suggested actionable changes to current feedback to optimise engagement and potential implementation of QI activities. These results will inform redesign of the ANDA feedback to consider the needs and preferences of end users and to provide feedback and other supportive cointerventions to improve care, and so health outcomes for people with diabetes. A subsequent cluster randomised trial will enable us to evaluate the impact of these changes.

Effectiveness of Enhanced Performance Feedback on Appropriate Use of Blood Transfusions: A Comparison of 2 Cluster Randomized Trials

IMPORTANCE

Auditing and feedback are frequently used to improve patient care. However, it remains unclear how to optimize feedback effectiveness for the appropriate use of treatments such as blood transfusion, a common but costly procedure that is more often overused than underused.

OBJECTIVE

To evaluate 2 theoretically informed feedback interventions to improve the appropriate use of blood transfusions.

DESIGN, SETTING, AND PARTICIPANTS

Two sequential, linked 2 × 2 cluster randomized trials were performed in hospitals in the UK participating in national audits of transfusion for perioperative anemia and management of hematological disorders. Data were collected for a surgical trial from October 1, 2014, to October 31, 2016, with follow-up completed on October 31, 2016. Data were collected for a hematological trial through follow-up from July 1, 2015, to June 30, 2017. Trial data were analyzed from November 1, 2016, to June 1, 2019.

INTERVENTIONS

Hospitals were randomized to standard content or enhanced content to improve feedback clarity and usability and to standard support or enhanced support for staff to act on feedback.

MAIN OUTCOMES AND MEASURES

The primary end point was appropriateness of transfusions audited at 12 months. Secondary end points included volume of transfusions (aiming for reductions at patient and cluster levels) and transfusion-related adverse events and reactions.

RESULTS

One hundred thirty-five of 152 eligible clusters participated in the surgical audit (2714 patients; mean [SD] age, 74.9 [14.0] years; 1809 women [66.7%]), and 134 of 141 participated in the hematological audit (4439 patients; median age, 72.0 [IQR, 64.0-80.0] years; 2641 men [59.5%]). Fifty-seven of 69 clusters (82.6%) in the surgical audit randomized to enhanced content downloaded reports compared with 52 of 66 clusters (78.8%) randomized to standard reports. Fifty-nine of 68 clusters (86.8%) randomized to enhanced support logged onto the toolkit. The proportion of patients with appropriate transfusions was 0.184 for standard content and 0.176 for enhanced content (adjusted odds ratio [OR], 0.91 [97.5% CI, 0.61-1.36]) and 0.181 for standard support and 0.180 for enhanced support (adjusted OR, 1.05 [97.5% CI, 0.68-1.61]). For the hematological audit, 53 of 66 clusters (80.3%) randomized to enhanced content downloaded the reports compared with 53 of 68 clusters (77.9%) randomized to standard content. Forty-nine of 67 clusters sites (73.1%) assigned to enhanced support logged into the toolkit at least once. The proportion of patients with appropriate transfusions was 0.744 for standard content and 0.714 for enhanced content (adjusted OR, 0.81 [97.5% CI, 0.56-1.12]), and 0.739 for standard support and 0.721 for enhanced support (adjusted OR, 0.96 [97.5% CI, 0.67-1.38]).

CONCLUSIONS AND RELEVANCE

This comparison of cluster randomized trials found that interventions to improve feedback usability and guide local action were no more effective than standard feedback in increasing the appropriate use of blood transfusions. Auditing and feedback delivered at scale is a complex and costly program; therefore, effective responses may depend on developing robust local quality improvement arrangements, which can be evaluated using rigorous experimental designs embedded within national programs.

TRIAL REGISTRATION

isrctn.org Identifier: ISRCTN15490813.

OUP accepted manuscript

Objectives

(1) Systematically review the literature on computerized audit and feedback (e-A&F) systems in healthcare. (2) Compare features of current systems against e-A&F best practices. (3) Generate hypotheses on how e-A&F systems may impact patient care and outcomes.

Methods

We searched MEDLINE (Ovid), EMBASE (Ovid), and CINAHL (Ebsco) databases to December 31, 2020. Two reviewers independently performed selection, extraction, and quality appraisal (Mixed Methods Appraisal Tool). System features were compared with 18 best practices derived from Clinical Performance Feedback Intervention Theory. We then used realist concepts to generate hypotheses on mechanisms of e-A&F impact. Results are reported in accordance with the PRISMA statement.

Results

Our search yielded 4301 unique articles. We included 88 studies evaluating 65 e-A&F systems, spanning a diverse range of clinical areas, including medical, surgical, general practice, etc. Systems adopted a median of 8 best practices (interquartile range 6–10), with 32 systems providing near real-time feedback data and 20 systems incorporating action planning. High-confidence hypotheses suggested that favorable e-A&F systems prompted specific actions, particularly enabled by timely and role-specific feedback (including patient lists and individual performance data) and embedded action plans, in order to improve system usage, care quality, and patient outcomes.

Conclusions

e-A&F systems continue to be developed for many clinical applications. Yet, several systems still lack basic features recommended by best practice, such as timely feedback and action planning. Systems should focus on actionability, by providing real-time data for feedback that is specific to user roles, with embedded action plans.

Protocol Registration

PROSPERO CRD42016048695.