A qualitative study of the variable effects of audit and feedback in the ICU

Identifying the impact of audit and feedback on the professional role of the nurse and psychological well-being: An integrative systematic review

This systematic review aimed to critically synthesis evidence to identify the impact that audit and feedback processes have on the professional role of the nurse and psychological well-being. Little is known about the extent to which audit and feedback processes can positively or negatively impact the professional role of the nurse and psychological well-being. An integrative systematic review was conducted. Covidence systematic review software was used to manage the screening process. Data extraction and methodological quality appraisal were conducted in parallel, and a narrative synthesis was conducted. Nurse participation and responsiveness to audit and feedback processes depended on self-perceived motivation, content, and delivery; and nurses viewed it as an opportunity for professional development. However, audit was reported to negatively impact nurses' psychological well-being, with impacts on burnout, stress, and demotivation in the workplace. Targeting framing, delivery, and content of audit and feedback is critical to nurses' satisfaction and successful quality improvement.

Interprofessional intensive care unit (ICU) team perspectives on physical restraint practices and minimization strategies in an adult ICU: A qualitative study of contextual influences

BACKGROUND

Guidelines advocate for minimization of physical restraint (PR) use in intensive care units (ICU). Interprofessional team perspectives on PR practices can inform the design and implementation of successful PR minimization interventions.

AIM

To identify ICU staff perspectives of contextual influences on PR practices and minimization strategies.

STUDY DESIGN

A qualitative descriptive study in a single ICU in Toronto, Canada. One-on-one semi-structured interviews were conducted with 14 ICU staff. A deductive content analysis of interviews was undertaken using the integrated-Promoting Action on Research Implementation in Health Services (i-PARIHS) framework.

RESULTS

Five themes were developed: risk-averse culture, leadership, practice monitoring and feedback processes, environmental factors, and facilitation. Participants described a risk-averse culture where prophylactic application of PR for intubated patients was used to prevent unplanned extubation thereby avoiding blame from colleagues. Perceived absence of leadership and interprofessional team involvement situated nurses as the primary decision-maker for restraint application and removal. Insufficient monitoring of restraint practices, lack of access to restraint alternatives, and inability to control environmental contributors to delirium and agitation further increased PR use. Recommendations as to how to minimize restraint use included a nurse facilitator to advance leadership-team collaboration, availability of restraints alternatives, and guidance on situations for applying and removing restraints.

CONCLUSIONS

This analysis of contextual influences on PR practices and minimization using the i-PARIHS framework revealed potentially modifiable barriers to successful PR minimization, including a lack of leadership involvement, gaps in practice monitoring, and collaborative decision-making processes. A team approach to changing behaviour and culture should be considered for successful implementation and sustainability of PR minimization.

RELEVANCE TO PRACTICE

The establishment of an interprofessional facilitation team that addresses risk-averse culture and promotes collaboration among ICU stakeholders will be crucial to the success of any approach to restraint minimization.

The Role of Data Science in Closing the Implementation Gap

Data science has the potential to greatly enhance efforts to translate evidence into practice in critical care. The intensive care unit is a data-rich environment enabling insight into both patient-level care patterns and clinician-level treatment patterns. By applying artificial intelligence to these novel data sources, implementation strategies can be tailored to individual patients, individual clinicians, and individual situations, revealing when evidence-based practices are missed and facilitating context-sensitive clinical decision support. To achieve these goals, technology developers should work closely with clinicians to create unbiased applications that are integrated into the clinical workflow.

Reinventing the Clinical Audit in a Pediatric Oncology Network

Providing equal access to pediatric cancer patients regardless of their geographic location is a major goal of the Affiliate Program at St. Jude Children's Research Hospital (St. Jude). Thirty-five percent of new cancer patients enrolled on St. Jude clinical trials reside in the communities of 1 of the 8 affiliate clinics, which serve 9 states in the Southeast and Midwest United States. The affiliate clinics support participant recruitment for clinical trials and the geographic extension of St. Jude clinical care. To ensure high-quality pediatric cancer care, we instituted on-site clinical audits, however, we did not see improvement in clinical outcomes including the time to antibiotics in febrile immunocompromised patients, consistent hand-off communication, consistent documentation of oral chemotherapy, and adherence to a central line bundle in the ambulatory setting. We then moved to a more comprehensive clinical audit which involved self-reflection of clinic staff members, transparent data sharing, development of local quality champions, and engagement of senior leaders. The comprehensive approach was more successful in improving clinical outcomes including the time to antibiotics, hand-off communication, documentation of oral chemotherapy administration, and adherence to a central line bundle in the ambulatory setting.

A Quality Improvement Initiative to Decrease Central Line-Associated Bloodstream Infections During the COVID-19 Pandemic: A "Zero Harm" Approach

INTRODUCTION

Central line-associated bloodstream infections (CLABSIs) are associated with significant patient harm and health care costs. Central line-associated bloodstream infections are preventable through quality improvement initiatives. The COVID-19 pandemic has caused many challenges to these initiatives. Our community health system in Ontario, Canada, had a baseline rate of 4.62 per 1000 line days during the baseline period.

OBJECTIVES

Our aim was to reduce CLABSIs by 25% by 2023.

METHODS

An interprofessional quality aim committee performed a root cause analysis to identify areas for improvement. Change ideas included improving governance and accountability, education and training, standardizing insertion and maintenance processes, updating equipment, improving data and reporting, and creating a culture of safety. Interventions occurred over 4 Plan-Do-Study-Act cycles. The outcome was CLABSI rate per 1000 central lines: process measures were rate of central line insertion checklists used and central line capped lumens used, and balancing measure was the number of CLABSI readmissions to the critical care unit within 30 days.

RESULTS

Central line-associated bloodstream infections decreased over 4 Plan-Do-Study-Act cycles from a baseline rate of 4.62 (July 2019-February 2020) to 2.34 (December 2021-May 2022) per 1000 line days (51%). The rate of central line insertion checklists used increased from 22.8% to 56.9%, and central line capped lumens used increased from 72% to 94.3%. Mean CLABSI readmissions within 30 days decreased from 1.49 to 0.1798.

CONCLUSIONS

Our multidisciplinary quality improvement interventions reduced CLABSIs by 51% across a health system during the COVID-19 pandemic.

The Spillover Effects of Quality Improvement Beyond Target Populations in Mechanical Ventilation

To assess the impact of a mechanical ventilation quality improvement program on patients who were excluded from the intervention.

DESIGN

Before-during-and-after implementation interrupted time series analysis to assess the effect of the intervention between coronary artery bypass grafting (CABG) surgery patients (included) and left-sided valve surgery patients (excluded).

SETTING

Academic medical center.

PATIENTS

Patients undergoing CABG and left-sided valve procedures were analyzed.

INTERVENTIONS

A postoperative mechanical ventilation quality improvement program was developed for patients undergoing CABG.

MEASUREMENTS AND MAIN RESULTS

Patients undergoing CABG had a median mechanical ventilation time of 11 hours during P0 ("before" phase) and 6.22 hours during P2 ("after" phase; p < 0.001). A spillover effect was observed because mechanical ventilation times also decreased from 10 hours during P0 to 6 hours during P2 among valve patients who were excluded from the protocol (p < 0.001). The interrupted time series analysis demonstrated a significant level of change for ventilation time from P0 to P2 for both CABG (p < 0.0001) and valve patients (p < 0.0001). There was no significant difference in the slope of change between the CABG and valve patient populations across time cohorts (P0 vs P1 [p = 0.8809]; P1 vs P2 [p = 0.3834]; P0 vs P2 [p = 0.7672]), which suggests that the rate of change in mechanical ventilation times was similar between included and excluded patients.

CONCLUSIONS

Decreased mechanical ventilation times for patients who were not included in a protocol suggests a spillover effect of quality improvement and demonstrates that quality improvement can have benefits beyond a target population.

How does nursing-sensitive indicator feedback with nursing or interprofessional teams work and shape nursing performance improvement systems? A rapid realist review

BACKGROUND

Care quality varies between organizations and even units within an organization. Inadequate care can have harmful financial and social consequences, e.g. nosocomial infection, lengthened hospital stays or death. Experts recommend the implementation of nursing performance improvement systems to assess team performance and monitor patient outcomes as well as service efficiency. In practice, these systems provide nursing or interprofessional teams with nursing-sensitive indicator feedback. Feedback is essential since it commits teams to improve their practice, although it appears somewhat haphazard and, at times, overlooked. Research findings suggest that contextual dynamics, initial system performance and feedback modes interact in unknown ways. This rapid review aims to produce a theorization to explain what works in which contexts, and how feedback to nursing or interprofessional teams shape nursing performance improvement systems.

METHODS

Based on theory-driven realist methodology, with reference to an innovative combination of Actor-Network Theory and Critical Realist philosophy principles, this realist rapid review entailed an iterative procedure: 8766 documents in French and English, published between 2010 and 2018, were identified via 5 databases, and 23 were selected and analysed. Two expert panels (scientific and clinical) were consulted to improve the synthesis and systemic modelling of an original feedback theorization.

RESULTS

We identified three hypotheses, subdivided into twelve generative configurations to explain how feedback mobilizes nursing or interprofessional teams. Empirically founded and actionable, these propositions are supported by expert panels. Each configuration specifies contextualized mechanisms that explain feedback and team outcomes. Socially mediated mechanisms are particularly generative of action and agency.

CONCLUSIONS

This rapid realist review provides an informative theoretical proposition to embrace the complexity of nursing-sensitive indicator feedback with nursing or interdisciplinary teams. Building on general explanations previously observed, this review provides insight into a deep explanation of feedback mechanisms.

SYSTEMATIC REVIEW REGISTRATION

Prospero CRD42018110128 .

Individualized Provider Feedback Increased HIV and HCV Screening and Identification in a New York City Emergency Department

Efforts to end the HIV and hepatitis C virus (HCV) epidemics begin with ascertainment of a person's infection status through screening. Despite its importance as a site of testing, missed opportunities for screening in the Emergency Department (ED) are common. We describe the impact of implementing an individualized provider feedback intervention on HIV and HCV testing in a quaternary ED. We conducted an interrupted time series analysis to evaluate the impact of the intervention on weekly HIV and HCV screening in an observational cohort of patients seeking care in the ED. The intervention included a physician champion individualized feedback with peer comparisons to all providers in the ED and an existing HIV/HCV testing and response team. Data were abstracted from the electronic medical record (EMR) for 30 weeks before, during, and after implementing the intervention. We used Poisson regression analysis to estimate changes in the weekly counts and rates of HIV and HCV testing. The incidence rate ratios (IRRs) of HIV testing were 1.94 [95% confidence interval (CI) 1.85-2.04] and 1.38 (95% CI 1.31-1.45) times higher for the intervention and post-intervention period compared with the pre-intervention period. The IRRs of HCV testing was 6.96 (95% CI 6.40-7.58) and 4.70 (95% CI 4.31-5.13) for the intervention and post-intervention periods. There were no meaningful differences in demographic characteristics during the observation period. The intervention meaningfully increased HIV and HCV testing volume and positive case detection, including testing in high-risk groups like young adults and individuals without prior testing. Although diminished, the intervention effect sustained in the 30-week period following implementation.

Implementation of nonpharmacologic physical restraint minimization interventions in the adult intensive care unit: A scoping review

OBJECTIVE

To identify the elements informing the successful implementation of nonpharmacologic physical restraint minimization interventions in adult intensive care unit patients. To map those elements to innovation, context, recipients and facilitation domains of the integrated-Promoting Action on Research Implementation in Health Services (i-PARIHS) framework and to describe the outcomes of those interventions.

METHODOLOGY

A scoping review of studies published in English reporting on restraint minimization interventions in adult intensive care units. We searched seven databases (MEDLINE, CIHAHL, Embase, Web of Science, Cochrane Library, PROSPERO and Joanna Briggs) from inception to 2021. Two authors independently screened articles for inclusion, extracted study characteristics and mapped intervention data to the i-PARIHS domains.

RESULTS

Seven studies met inclusion criteria. Innovations comprised multicomponent interventions including education, decision aids/protocols and restraint alternatives. No studies utilised an implementation science framework to diagnose the baseline practice context. A commonly reported barrier to restraint minimization was a risk averse culture. Change was mostly driven by the external context (i.e. national regulations). Overall, nurses were the primary facilitators and recipients of practice change. Outcomes were changes in restraint incidence and prevalence abstracted from the medical record. However, no study validated the accuracy of restraint documentation. All studies documented an initial decrease in physical restraint use, but no long-term results were reported.

CONCLUSION

Restraint minimization intervention studies report nurse-facilitated multicomponent interventions and short-term practice change. Future restraint minimization research incorporating implementation science frameworks, interprofessional teams and patient/family perspectives is warranted.

How well do critical care audit and feedback interventions adhere to best practice? Development and application of the REFLECT-52 evaluation tool

BACKGROUND

Healthcare Audit and Feedback (A&F) interventions have been shown to be an effective means of changing healthcare professional behavior, but work is required to optimize them, as evidence suggests that A&F interventions are not improving over time. Recent published guidance has suggested an initial set of best practices that may help to increase intervention effectiveness, which focus on the "Nature of the desired action," "Nature of the data available for feedback," "Feedback display," and "Delivering the feedback intervention." We aimed to develop a generalizable evaluation tool that can be used to assess whether A&F interventions conform to these suggestions for best practice and conducted initial testing of the tool through application to a sample of critical care A&F interventions.

METHODS

We used a consensus-based approach to develop an evaluation tool from published guidance and subsequently applied the tool to conduct a secondary analysis of A&F interventions. To start, the 15 suggestions for improved feedback interventions published by Brehaut et al. were deconstructed into rateable items. Items were developed through iterative consensus meetings among researchers. These items were then piloted on 12 A&F studies (two reviewers met for consensus each time after independently applying the tool to four A&F intervention studies). After each consensus meeting, items were modified to improve clarity and specificity, and to help increase the reliability between coders. We then assessed the conformity to best practices of 17 critical care A&F interventions, sourced from a systematic review of A&F interventions on provider ordering of laboratory tests and transfusions in the critical care setting. Data for each criteria item was extracted by one coder and confirmed by a second; results were then aggregated and presented graphically or in a table and described narratively.

RESULTS

In total, 52 criteria items were developed (38 ratable items and 14 descriptive items). Eight studies targeted lab test ordering behaviors, and 10 studies targeted blood transfusion ordering. Items focused on specifying the "Nature of the Desired Action" were adhered to most commonly-feedback was often presented in the context of an external priority (13/17), showed or described a discrepancy in performance (14/17), and in all cases it was reasonable for the recipients to be responsible for the change in behavior (17/17). Items focused on the "Nature of the Data Available for Feedback" were adhered to less often-only some interventions provided individual (5/17) or patient-level data (5/17), and few included aspirational comparators (2/17), or justifications for specificity of feedback (4/17), choice of comparator (0/9) or the interval between reports (3/13). Items focused on the "Nature of the Feedback Display" were reported poorly-just under half of interventions reported providing feedback in more than one way (8/17) and interventions rarely included pilot-testing of the feedback (1/17 unclear) or presentation of a visual display and summary message in close proximity of each other (1/13). Items focused on "Delivering the Feedback Intervention" were also poorly reported-feedback rarely reported use of barrier/enabler assessments (0/17), involved target members in the development of the feedback (0/17), or involved explicit design to be received and discussed in a social context (3/17); however, most interventions clearly indicated who was providing the feedback (11/17), involved a facilitator (8/12) or involved engaging in self-assessment around the target behavior prior to receipt of feedback (12/17).

CONCLUSIONS

Many of the theory-informed best practice items were not consistently applied in critical care and can suggest clear ways to improve interventions. Standardized reporting of detailed intervention descriptions and feedback templates may also help to further advance research in this field. The 52-item tool can serve as a basis for reliably assessing concordance with best practice guidance in existing A&F interventions trialed in other healthcare settings, and could be used to inform future A&F intervention development.

TRIAL REGISTRATION

Not applicable.

Nationwide audit and feedback on implementation of antibiotic stewardship programmes in Norwegian hospitals

Background

Implementation of antibiotic stewardship programmes (ASPs) in hospitals is challenging and there is a knowledge gap on how to pursue this process efficiently.

Objectives

To evaluate whether audit and feedback (A&F) is a feasible and useful methodology to assess and support the implementation of ASPs in hospitals.

Methods

A multidisciplinary team performed document reviews and on-site interviews with professionals involved in the implementation of ASPs. Oral feedback on preliminary findings and areas of improvement were provided on-site, followed by feedback reports summarizing major findings and recommendations. Descriptive statistics were used to present number of hospital trusts, interviewees, professions, disciplines, workload and costs.

Results

All 22 hospital trusts in Norway participated in the A&F conducted October 2017 to April 2019. Altogether, 446 leaders and healthcare workers were interviewed: 110 leaders, 336 health professionals of whom 89 were antimicrobial stewardship team members. Median number of days from audits were performed till reporting were 36 (IQR 30-49). Median workload for auditors per visit was 7 days (6-8). Total costs were €133 952. Main audit findings were that ASP structures were established in most hospital trusts, but leadership commitment and implementation of interventions were often lacking. The hospital trusts received feedback on establishing governance structures, setting local targets, implementing interventions and increased involvement of nurses.

Conclusions

Nationwide A&F provides a unique and comprehensive insight into the implementation of ASPs in hospitals and is feasible with a reasonable amount of resources. This approach can identify targets for improved implementation of ASPs in hospitals.

Understanding how and why audits work in improving the quality of hospital care: A systematic realist review

BACKGROUND

Several types of audits have been used to promote quality improvement (QI) in hospital care. However, in-depth studies into the mechanisms responsible for the effectiveness of audits in a given context is scarce. We sought to understand the mechanisms and contextual factors that determine why audits might, or might not, lead to improved quality of hospital care.

METHODS

A realist review was conducted to systematically search and synthesise the literature on audits. Data from individual papers were synthesised by coding, iteratively testing and supplementing initial programme theories, and refining these theories into a set of context-mechanism-outcome configurations (CMOcs).

RESULTS

From our synthesis of 85 papers, seven CMOcs were identified that explain how audits work: (1) externally initiated audits create QI awareness although their impact on improvement diminishes over time; (2) a sense of urgency felt by healthcare professionals triggers engagement with an audit; (3) champions are vital for an audit to be perceived by healthcare professionals as worth the effort; (4) bottom-up initiated audits are more likely to bring about sustained change; (5) knowledge-sharing within externally mandated audits triggers participation by healthcare professionals; (6) audit data support healthcare professionals in raising issues in their dialogues with those in leadership positions; and (7) audits legitimise the provision of feedback to colleagues, which flattens the perceived hierarchy and encourages constructive collaboration.

CONCLUSIONS

This realist review has identified seven CMOcs that should be taken into account when seeking to optimise the design and usage of audits. These CMOcs can provide policy makers and practice leaders with an adequate conceptual grounding to design contextually sensitive audits in diverse settings and advance the audit research agenda for various contexts.

PROSPERO REGISTRATION

CRD42016039882.

SUMAMOS EXCELENCIA® Project: Results of the Implementation of Best Practice in a Spanish National Health System (NHS)

The use of certain strategies for the implementation of a specific recommendation yields better results in clinical practice. The aim of this study was to assess the effectiveness of an evidence-based model using clinical audits (GRIP model), for the implementation of recommendations in pain and urinary incontinence management as well as fall prevention, in the Spanish National Health System during the period 2015-2018. A quasi-experimental study has been conducted. The subjects were patients treated in hospitals, primary care units and nursing home centers. There were measures related to pain, fall prevention and urinary incontinence. Measurements were taken at baseline and at months 3, 6, 9, and 12. The sample consisted of 22,114 patients. The frequency of pain assessment increased from 59.9% in the first cycle to a mean of 71.6% in the last cycle, assessments of risk of falling increased from 56.8% to 87.8% in the last cycle; and finally, the frequency of assessments of urinary incontinence increased from a 43.4% in the first cycles to a mean of 62.2% in the last cycles. The implementation of specific evidence-based recommendations on pain, fall prevention, and urinary incontinence using a model based on clinical audits improved the frequency of assessments and their documentation.

Audit and Feedback: An Evidence-Based Practice Literature Review of Nursing Report Cards

BACKGROUND

As more hospitals transition to electronic health records (EHR) and rely on technology to inform practice, what is done with that information is increasingly important. Performance report cards for physicians and nurses are not new, yet there is little recent evidence on nurse-specific audit and feedback.

AIM

The aim of the project was to conduct an evidence-based practice (EBP) review to answer the question, "Does implementing an individualized audit and feedback report tool for nurses improve compliance, adherence, and/or performance of nursing tasks?".

METHODS

Evidence was gathered from several databases. Reviewers read and appraised articles that answered the EBP question using the Johns Hopkins Nursing EBP Model. Data were then collated to synthesize and generate recommendations.

RESULTS

Of the initial 613 unique articles, eight (two research and six quality improvement) were included. Six articles demonstrated improvements while two did not. Articles analyzed nursing documentation (n = 3), tasks or skills (n = 2), and best practice compliance (n = 3). One manuscript utilized an EHR-generated report; all others were completed by hand. Overall, there was not consistent and compelling evidence to support individualized audit and feedback report tools in nursing. However, several themes emerged related to sustainability, timing of feedback, audit, and feedback in the context of quality improvement, and the methods of acquiring and distributing data.

LINKING EVIDENCE TO ACTION

The ubiquity and ease of the EHR make providing automated feedback to nurses tempting, yet it is not supported by the literature. More implementation science research is needed to explore audit and feedback reports in nursing. This article adds to the literature by highlighting a significant lack of consistent and compelling positive results from the well-established quality improvement strategy of audit and feedback in the nursing population. The absence of good data is as telling as its presence.

A Scoping Review of Implementation Science in Adult Critical Care Settings

OBJECTIVES

The purpose of this scoping review is to provide a synthesis of the available literature on implementation science in critical care settings. Specifically, we aimed to identify the evidence-based practices selected for implementation, the frequency and type of implementation strategies used to foster change, and the process and clinical outcomes associated with implementation.

DATA SOURCES

A librarian-assisted search was performed using three electronic databases.

STUDY SELECTION

Articles that reported outcomes aimed at disseminating, implementing, or sustaining an evidence-based intervention or practice, used established implementation strategies, and were conducted in a critical care unit were included.

DATA EXTRACTION

Two reviewers independently screened titles, abstracts, and full text of articles to determine eligibility. Data extraction was performed using customized fields established a priori within a systematic review software system.

DATA SYNTHESIS

Of 1,707 citations, 82 met eligibility criteria. Studies included prospective research investigations, quality improvement projects, and implementation science trials. The most common practices investigated were use of a ventilator-associated pneumonia bundle, nutritional support protocols, and the Awakening and Breathing Coordination, Delirium Monitoring/Management, and Early Exercise/Mobility bundle. A variety of implementation strategies were used to facilitate evidence adoption, most commonly educational meetings, auditing and feedback, developing tools, and use of local opinion leaders. The majority of studies (76/82, 93%) reported using more than one implementation strategy. Few studies specifically used implementation science designs and frameworks to systematically evaluate both implementation and clinical outcomes.

CONCLUSIONS

The field of critical care has experienced slow but steady gains in the number of investigations specifically guided by implementation science. However, given the exponential growth of evidence-based practices and guidelines in this same period, much work remains to critically evaluate the most effective mechanisms to integrate and sustain these practices across diverse critical care settings and teams.

Finding the match between healthcare worker and expert for optimal audit and feedback on antimicrobial resistance prevention measures

Background

The potentials of audit and feedback (AF) to improve healthcare are currently not exploited. To unlock the potentials of AF, this study focused on the process of making sense of audit data and translating data into actionable feedback by studying a specific AF-case: limiting antimicrobial resistance (AMR). This was done via audit and feedback of AMR prevention measures (APM) that are executed by healthcare workers (HCW) in their day-to-day contact with patients. This study’s aim was to counterbalance the current predominantly top-down, expert-driven audit and feedback approach for APM, with needs and expectations of HCW.

Methods

Qualitative semi-structured interviews were held with sixteen HCW (i.e. physicians, residents and nurses) from high-risk AMR departments at a regional hospital in The Netherlands. Deductive coding was succeeded by open and axial coding to establish main codes, subcodes and variations within codes.

Results

HCW demand insights from audits into all facets of APM in their working routines (i.e. diagnostics, treatment and infection control), preferably in the form of simple and actionable feedback that invites interdisciplinary discussions, so that substantiated actions for improvement can be implemented. AF should not be seen as an isolated ad-hoc intervention, but as a recurrent, long-term, and organic improvement strategy that balances the primary aims of HCW (i.e. improving quality and safety of care for individual patients and HCW) and AMR-experts (i.e. reducing the burden of AMR).

Conclusions

To unlock the learning and improvement potentials of audit and feedback, HCW’ and AMR-experts’ perspectives should be balanced throughout the whole AF-loop (incl. data collection, analysis, visualization, feedback and planning, implementing and monitoring actions). APM-AF should be flexible, so that both audit (incl. collecting and combining the right data in an efficient and transparent manner) and feedback (incl. persuasive and actionable feedback) can be tailored to the needs of various target groups. To balance HCW’ and AMR-experts’ perspectives a participatory holistic AF development approach is advocated.

Facilitating action planning within audit and feedback interventions: a mixed-methods process evaluation of an action implementation toolbox in intensive care

Background

Audit and feedback (A&F) is more effective if it facilitates action planning, but little is known about how best to do this. We developed an electronic A&F intervention with an action implementation toolbox to improve pain management in intensive care units (ICUs); the toolbox contained suggested actions for improvement. A head-to-head randomised trial demonstrated that the toolbox moderately increased the intervention’s effectiveness when compared with A&F only.

Objective

To understand the mechanisms through which A&F with action implementation toolbox facilitates action planning by ICUs to increase A&F effectiveness.

Methods

We extracted all individual actions from action plans developed by ICUs that received A&F with (n = 10) and without (n = 11) toolbox for 6 months and classified them using Clinical Performance Feedback Intervention Theory. We held semi-structured interviews with participants during the trial. We compared the number and type of planned and completed actions between study groups and explored barriers and facilitators to effective action planning.

Results

ICUs with toolbox planned more actions directly aimed at improving practice (p = 0.037) and targeted a wider range of practice determinants compared to ICUs without toolbox. ICUs with toolbox also completed more actions during the study period, but not significantly (p = 0.142). ICUs without toolbox reported more difficulties in identifying what actions they could take. Regardless of the toolbox, all ICUs still experienced barriers relating to the feedback (low controllability, accuracy) and organisational context (competing priorities, resources, cost).

Conclusions

The toolbox helped health professionals to broaden their mindset about actions they could take to change clinical practice. Without the toolbox, professionals tended to focus more on feedback verification and exploring solutions without developing intentions for actual change. All feedback recipients experienced organisational barriers that inhibited eventual completion of actions.

Trial registration

ClinicalTrials.gov, NCT02922101. Registered on 26 September 2016.

Electronic supplementary material

The online version of this article (10.1186/s13012-019-0937-8) contains supplementary material, which is available to authorized users.

Impact of audit and feedback with action implementation toolbox on improving ICU pain management: cluster-randomised controlled trial

Background

Audit and feedback (A&F) enjoys widespread use, but often achieves only marginal improvements in care. Providing recipients of A&F with suggested actions to overcome barriers (action implementation toolbox) may increase effectiveness.

Objective

To assess the impact of adding an action implementation toolbox to an electronic A&F intervention targeting quality of pain management in intensive care units (ICUs).

Trial design

Two-armed cluster-randomised controlled trial. Randomisation was computer generated, with allocation concealment by a researcher, unaffiliated with the study. Investigators were not blinded to the group assignment of an ICU.

Participants

Twenty-one Dutch ICUs and patients eligible for pain measurement.

Interventions

Feedback-only versus feedback with action implementation toolbox.

Outcome

Proportion of patient-shift observations where pain management was adequate; composed by two process (measuring pain at least once per patient in each shift; re-measuring unacceptable pain scores within 1 hour) and two outcome indicators (acceptable pain scores; unacceptable pain scores normalised within 1 hour).

Results

21 ICUs (feedback-only n=11; feedback-with-toolbox n=10) with a total of 253 530 patient-shift observations were analysed. We found absolute improvement on adequate pain management in the feedback-with-toolbox group (14.8%; 95% CI 14.0% to 15.5%) and the feedback-only group (4.8%; 95% CI 4.2% to 5.5%). Improvement was limited to the two process indicators. The feedback-with-toolbox group achieved larger effects than the feedback-only group both on the composite adequate pain management (p<0.05) and on measuring pain each shift (p<0.001). No important adverse effects have occurred.

Conclusion

Feedback with toolbox improved the number of shifts where patients received adequate pain management compared with feedback alone, but only in process and not outcome indicators.

Trial registration number

NCT02922101.

Use of a maternal newborn audit and feedback system in Ontario: a collective case study

Background

As part of a larger study examining the effectiveness of the Maternal Newborn Dashboard, an electronic audit and feedback system to improve maternal-newborn care practices and outcomes, the purpose of this study was to increase our understanding of factors explaining variability in performance after implementation of the Dashboard in Ontario, Canada.

Methods

A collective case study. A maximum variation sampling approach was used to invite hospitals reflecting different criteria to participate in a 1-day to 2-day site visit by the research team. The visits included: (1) semistructured interviews and focus groups with healthcare providers, leaders and personnel involved in clinical change processes; (2) observations and document review. Interviews and focus groups were audio-recorded and transcribed verbatim. Qualitative content analysis was used to code and categorise the data.

Results

Between June and November 2016, we visited 14 maternal-newborn hospitals. Hospitals were grouped into four quadrants based on their key indicator performance and level of engagement with the Dashboard. Findings revealed four overarching themes that contribute to the varying success of sites in achieving practice change on the Dashboard key performance indicators, namely, interdisciplinary collaboration and accountability, application of formal change strategies, team trust and use of evidence and data, as well as alignment with organisational priorities and support.

Conclusion

The diversity of facilitators and barriers across the 14 hospitals highlights the need to go beyond a ‘one size fits all’ approach when implementing audit and feedback systems. Future work to identify tools to assess barriers to practice change and to evaluate the effects of cointerventions to optimise audit and feedback systems for clinical practice change is needed.

Audit and feedback in mental healthcare: staff experiences

Purpose

Audit and feedback (A&F) often underlie implementation projects, described as a circular process; i.e. an A&F cycle. They are widely used, but effect varies with no apparent explanation. We need to understand how A&F work in real-life situations. The purpose of this paper, therefore, is to describe and explore mental healthcare full A&F cycle experiences.

Design/methodology/approach

This is a naturalistic qualitative study that uses four focus groups and qualitative content analysis.

Findings

Staff accepted the initial A&F stages, perceiving it to enhance awareness and reassure them about good practice. They were willing to participate in the full cycle and implement changes, but experienced poor follow-up and prioritization, not giving them a chance to own to the process. An important finding is the need for an A&F cycle facilitator.

Practical implications

Research teams cannot be expected to be involved in implementing clinical care. Guidelines will keep being produced to improve service quality and will be expected to be practiced. This study gives insights into planning and tailoring A&F cycles.

Originality/value

Tools to ease implementation are not enough, and the key seems to lie with facilitating a process using A&F. This study underscores leadership, designated responsibility and facilitation throughout a full audit cycle.

Audit-and-Feedback and Workflow Changes Improve Emergency Department Care of Critically Ill Children

Supplemental Digital Content is available in the text.

Introduction:

Children with severe infection have improved outcomes when they received antibiotics promptly. Positive cultures help guide physicians in antibiotic selection. In 2011, 30% of children intubated in the emergency department received antibiotics and had respiratory culture collected within 60 minutes of intubation. Knowing the risk of delaying appropriate antibiotics, we charted a quality improvement team to improve compliance with 80% of intubated patients receiving both.

Methods:

The team evaluated all children intubated with concern for infection in the emergency department. Using a multidisciplinary team and employing quality improvement methods, we implemented multiple plan-do-study-act cycles to improve time to antibiotics and respiratory cultures. The team continued to implement successful interventions and restarted interventions directly affecting improvement.

Results:

While multiple interventions had small effects on the baseline of 30% compliance, 2 interventions appeared more influential than others. Workflow changes and audit-and-feedback created the largest, persistent positive changes. The importance of audit-and-feedback became very obvious when the project entered sustain mode. An abrupt decrease in compliance occurred when audit-and-feedback stopped. Complete recovery in compliance to greater than 80% occurred with the resumption of the audit-and-feedback intervention.

Conclusions:

Workflow changes and audit-and-feedback interventions resulted in large improvements. Loss of compliance with cessation of the audit-and-feedback and resumption demonstrated the importance of this intervention. Recovery to >80% compliance with the renewal of the audit-and-feedback program indicates its strength as a positive intervention.

How do hospitals respond to feedback about blood transfusion practice? A multiple case study investigation

National clinical audits play key roles in improving care and driving system-wide change. However, effects of audit and feedback depend upon both reach (e.g. relevant staff receiving the feedback) and response (e.g. staff regulating their behaviour accordingly). This study aimed to investigate which hospital staff initially receive feedback and formulate a response, how feedback is disseminated within hospitals, and how responses are enacted (including barriers and enablers to enactment). Using a multiple case study approach, we purposively sampled four UK hospitals for variation in infrastructure and resources. We conducted semi-structured interviews with staff from transfusion-related roles and observed Hospital Transfusion Committee meetings. Interviews and analysis were based on the Theoretical Domains Framework of behaviour change. We coded interview transcripts into theoretical domains, then inductively identified themes within each domain to identify barriers and enablers. We also analysed data to identify which staff currently receive feedback and how dissemination is managed within the hospital. Members of the hospital’s transfusion team initially received feedback in all cases, and were primarily responsible for disseminating and responding, facilitated through the Hospital Transfusion Committee. At each hospital, key individuals involved in prescribing transfusions reported never having received feedback from a national audit. Whether audits were discussed and actions explicitly agreed in Committee meetings varied between hospitals. Key enablers of action across all cases included clear lines of responsibility and strategies to remind staff about recommendations. Barriers included difficulties disseminating to relevant staff and needing to amend feedback to make it appropriate for local use. Appropriate responses by hospital staff to feedback about blood transfusion practice depend upon supportive infrastructures and role clarity. Hospitals could benefit from support to disseminate feedback systematically, particularly to frontline staff involved in the behaviours being audited, and practical tools to support strategic decision-making (e.g. action-planning around local response to feedback).

Acting on audit & feedback: a qualitative instrumental case study in mental health services in Norway

BACKGROUND

The National Guideline for Assessment, Treatment and Social Rehabilitation of Persons with Concurrent Substance Use and Mental Health Disorders, launched in 2012, is to be implemented in mental health services in Norway. Audit and feedback (A&F) is commonly used as the starting point of an implementation process. It aims to measure the research-practice gap, but its effect varies greatly. Less is known of how audit and feedback is used in natural settings. The aim of this study was to describe and investigate what is discussed and thematised when Quality Improvement (QI) teams in a District Psychiatric Centre (DPC) work to complete an action form as part of an A&F cycle in 2014.

METHODS

This was an instrumental multiple case study involving four units in a DPC in Norway. We used open non-participant observation of QI team meetings in their natural setting, a total of seven teams and eleven meetings.

RESULTS

The discussions provided health professionals with insight into their own and their colleagues' practices. They revealed insufficient knowledge of substance-related disorders and experienced unclear role expectations. We found differences in how professional groups sought answers to questions of clinical practice and that they were concerned about whether new tasks fitted in with their routine ways of working.

CONCLUSION

Acting on A&F provided an opportunity to discuss practice in general, enhancing awareness of good practice. There was a general need for arenas to relate to practice and QI team meetings after A&F may well be a suitable arena for this. Self-assessment audits seem valuable, particular in areas where no benchmarked data exists, and there is a demand for implementation of new guidelines that might change routines and develop new roles. QI teams could benefit from having a unit leader present at meetings. Nurses and social educators and others turn to psychiatrists or psychologists for answers to clinical and organisational questions beyond guidelines, and show less confidence or routine in seeking research-based information. There is a general need to emphasise training in evidence-based practice and information seeking behaviour for all professional groups.

Improving Appropriate Neurologic Prognostication after Cardiac Arrest. A Stepped Wedge Cluster Randomized Controlled Trial

RATIONALE

Predictions about neurologic prognosis that are based on early clinical findings after out-of-hospital cardiac arrest (OHCA) are often inaccurate and may lead to premature decisions to withdraw life-sustaining treatments (LST) in patients who might otherwise survive with good neurologic outcomes.

OBJECTIVES

To improve adherence to recommendations for appropriate neurologic prognostication after OHCA and reduce deaths from premature decisions to withdraw LST.

METHODS

This was a pragmatic stepped wedge cluster randomized controlled trial evaluating a multifaceted quality intervention (education, pathways, local champions, audit-feedback). The primary outcome was appropriate neurologic prognostication, defined as (1a) no early withdrawal of LST (WLST) (within 72 h) based on estimates of poor neurologic prognosis and (1b) no WLST between 72 hours and 7 days in absence of clinical predictors of poor neurologic prognosis or (2) surviving beyond 7 days. Secondary outcomes were deaths from early WLST and survival with good neurologic outcome.

MEASUREMENTS AND MAIN RESULTS

Between June 1, 2011, and June 30, 2014, a total of 905 patients with OHCA were enrolled from ICUs of 18 Ontario hospitals. Rates of appropriate neurologic prognostication increased after the intervention (68% vs. 74% patients; odds ratio [OR], 1.79; 95% confidence interval [CI], 1.01-3.19; P = 0.05). However, rates of survival to hospital discharge (46% vs. 50%; OR, 1.71; 95% CI, 0.97-3.01; P = 0.06) and survival with good neurologic outcome remained similar (38% vs. 43%; OR, 1.43; 95% CI, 0.84-2.86; P = 0.19).

CONCLUSIONS

A multicenter quality intervention improved rates of appropriate neurologic prognostication after OHCA but did not increase survival with good neurologic outcome. Clinical trial registered with www.clinicaltrials.gov (NCT 01472458).

Effect of a web-based audit and feedback intervention with outreach visits on the clinical performance of multidisciplinary teams: a cluster-randomized trial in cardiac rehabilitation

BACKGROUND

The objective of this study was to assess the effect of a web-based audit and feedback (A&F) intervention with outreach visits to support decision-making by multidisciplinary teams.

METHODS

We performed a multicentre cluster-randomized trial within the field of comprehensive cardiac rehabilitation (CR) in the Netherlands. Our participants were multidisciplinary teams in Dutch CR centres who were enrolled in the study between July 2012 and December 2013 and received the intervention for at least 1 year. The intervention included web-based A&F with feedback on clinical performance, facilities for goal setting and action planning, and educational outreach visits. Teams were randomized either to receive feedback that was limited to psychosocial rehabilitation (study group A) or to physical rehabilitation (study group B). The main outcome measure was the difference in performance between study groups in 11 care processes and six patient outcomes, measured at patient level. Secondary outcomes included effects on guideline concordance for the four main CR therapies.

RESULTS

Data from 18 centres (14,847 patients) were analysed, of which 12 centres (9353 patients) were assigned to group A and six (5494 patients) to group B. During the intervention, a total of 233 quality improvement goals was identified by participating teams, of which 49 (21%) were achieved during the study period. Except for a modest improvement in data completeness (4.5% improvement per year; 95% CI 0.65 to 8.36), we found no effect of our intervention on any of our primary or secondary outcome measures.

CONCLUSIONS

Within a multidisciplinary setting, our web-based A&F intervention engaged teams to define local performance improvement goals but failed to support them in actually completing the improvement actions that were needed to achieve those goals. Future research should focus on improving the actionability of feedback on clinical performance and on addressing the socio-technical perspective of the implementation process.

TRIAL REGISTRATION

NTR3251.

Telemedicine with clinical decision support for critical care: a systematic review

BACKGROUND

Telemedicine applications aim to address variance in clinical outcomes and increase access to specialist expertise. Despite widespread implementation, there is little robust evidence about cost-effectiveness, clinical benefits, and impact on quality and safety of critical care telemedicine. The primary objective was to determine the impact of critical care telemedicine (with clinical decision support available 24/7) on intensive care unit (ICU) and hospital mortality and length of stay in adults and children. The secondary objectives included staff and patient experience, costs, protocol adherence, and adverse events.

METHODS

Data sources included MEDLINE, EMBASE, CINAHL, Cochrane Library databases, Health Technology Assessment Database, Web of Science, OpenGrey, OpenDOAR, and the HMIC through to December 2015. Randomised controlled trials and quasi-experimental studies were eligible for inclusion. Eligible studies reported on differences between groups using the telemedicine intervention and standard care. Two review authors screened abstracts and assessed potentially eligible studies using Cochrane guidance.

RESULTS

Two controlled before-after studies met the inclusion criteria. Both were assessed as high risk of bias. Meta-analysis was not possible as we were unable to disaggregate data between the two studies. One study used a non-randomised stepped-wedge design in seven ICUs. Hospital mortality was the primary outcome which showed a reduction from 13.6 % (CI, 11.9-15.4 %) to 11.8 % (CI, 10.9-12.8 %) during the intervention period with an adjusted odds ratio (OR) of 0.40 (95 % CI, 0.31-0.52; p = .005). The second study used a non-randomised, unblinded, pre-/post-assessment of telemedicine interventions in 56 adult ICUs. Hospital mortality (primary outcome) reduced from 11 to 10 % (adjusted hazard ratio (HR) = 0.84; CI, 0.78-0.89; p = <.001).

CONCLUSIONS

This review highlights the poor methodological quality of most studies investigating critical care telemedicine. The results of the two included studies showed a reduction in hospital mortality in patients receiving the intervention. Further multi-site randomised controlled trials or quasi-experimental studies with accompanying process evaluations are urgently needed to determine effectiveness, implementation, and associated costs.

TRIAL REGISTRATION

PROSPERO CRD42014007406.

Factors influencing the effectiveness of audit and feedback: nurses' perceptions

AIM

To explore the perceptions of nurses in an acute care setting on factors influencing the effectiveness of audit and feedback.

BACKGROUND

Audit and feedback is widely used and recommended in nursing to promote evidence-based practice and to improve care quality. Yet the literature has shown a limited to modest effect at most. Audit and feedback will continue to be unreliable until we learn what influences its effectiveness.

METHOD

A qualitative study was conducted using individual, semi-structured interviews with 14 registered nurses in an acute care teaching hospital in Montreal, Canada.

RESULTS

Three themes were identified: the relevance of audit and feedback, particularly understanding the purpose of audit and feedback and the prioritisation of audit criteria; the audit and feedback process, including its timing and feedback characteristics; and individual factors, such as personality and perceived accountability.

CONCLUSION

According to participants, they were likely to have a better response to audit and feedback when they perceived that it was relevant and that the process fitted their preferences.

IMPLICATIONS FOR NURSING MANAGEMENT

This study benefits nursing leaders and managers involved in quality improvement by providing a better understanding of nurses' perceptions on how best to use audit and feedback as a strategy to promote evidence-based practice.