Facilitating action planning within audit and feedback interventions: a mixed-methods process evaluation of an action implementation toolbox in intensive care

A realist synthesis of multicentre comparative audit implementation: exploring what works and in which healthcare contexts

BACKGROUND

Multicentre comparative clinical audits have the potential to improve patient care, allow benchmarking and inform resource allocation. However, implementing effective and sustainable large-scale audit can be difficult within busy and resource-constrained contemporary healthcare settings. There are little data on what facilitates the successful implementation of multicentre audits. As healthcare environments are complex sociocultural organisational environments, implementing multicentre audits within them is likely to be highly context dependent.

OBJECTIVE

We aimed to examine factors that were influential in the implementation process of multicentre comparative audits within healthcare contexts-what worked, why, how and for whom?

METHODS

A realist review was conducted in accordance with the Realist and Meta-narrative Evidence Syntheses: Evolving Standards reporting standards. A preliminary programme theory informed two systematic literature searches of peer-reviewed and grey literature. The main context-mechanism-outcome (CMO) configurations underlying the implementation processes of multicentre audits were identified and formed a final programme theory.

RESULTS

69 original articles were included in the realist synthesis. Four discrete CMO configurations were deduced from this synthesis, which together made up the final programme theory. These were: (1) generating trustworthy data; (2) encouraging audit participation; (3) ensuring audit sustainability; and (4) facilitating audit cycle completion.

CONCLUSIONS

This study elucidated contexts, mechanisms and outcomes influential to the implementation processes of multicentre or national comparative audits in healthcare. The relevance of these contextual factors and generative mechanisms were supported by established theories of behaviour and findings from previous empirical research. These findings highlight the importance of balancing reliability with pragmatism within complex adaptive systems, generating and protecting human capital, ensuring fair and credible leadership and prioritising change facilitation.

National ICU Registries as Enablers of Clinical Research and Quality Improvement

OBJECTIVES

Clinical quality registries (CQRs) have been implemented worldwide by several medical specialties aiming to generate a better characterization of epidemiology, treatments, and outcomes of patients. National ICU registries were created almost 3 decades ago to improve the understanding of case-mix, resource use, and outcomes of critically ill patients. This narrative review describes the challenges, proposed solutions, and evidence generated by National ICU registries as facilitators for research and quality improvement.

DATA SOURCES

English language articles were identified in PubMed using phrases related to ICU registries, CQRs, outcomes, and case-mix.

STUDY SELECTION

Original research, review articles, letters, and commentaries, were considered.

DATA EXTRACTION

Data from relevant literature were identified, reviewed, and integrated into a concise narrative review.

DATA SYNTHESIS

CQRs have been implemented worldwide by several medical specialties aiming to generate a better characterization of epidemiology, treatments, and outcomes of patients. National ICU registries were created almost 3 decades ago to improve the understanding of case-mix, resource use, and outcomes of critically ill patients. The initial experience in European countries and in Oceania ensured that through locally generated data, ICUs could assess their performances by using risk-adjusted measures and compare their results through fair and validated benchmarking metrics with other ICUs contributing to the CQR. The accomplishment of these initiatives, coupled with the increasing adoption of information technology, resulted in a broad geographic expansion of CQRs as well as their use in quality improvement studies, clinical trials as well as international comparisons, and benchmarking for ICUs.

CONCLUSIONS

ICU registries have provided increased knowledge of case-mix and outcomes of ICU patients based on real-world data and contributed to improve care delivery through quality improvement initiatives and trials. Recent increases in adoption of new technologies (i.e., cloud-based structures, artificial intelligence, machine learning) will ensure a broader and better use of data for epidemiology, healthcare policies, quality improvement, and clinical trials.

Feedback on audit and action planning for dental caries control: a qualitative study to investigate the acceptability among interdisciplinary pediatric dental care teams

Introduction

American Indian and Alaska Native children suffer from the poorest oral health of all populational groups in the United States. Evidence-based practices (EBP) for caries control are well established, but systematically implementing such practices have proven difficult. Audit and feedback with goal setting, and action planning to implement these EBPs have not been tested or adapted for Alaska Native healthcare settings. The aim of this study was to investigate acceptability and perceived feasibility of an audit and feedback intervention for pediatric dental caries control among dental providers and patient stakeholders.

Methods

The pilot program was implemented in two dental clinics from a tribal healthcare consortium in Alaska. Key-informant interviews were conducted to investigate the contextual, organizational, and behavioral facilitators and barriers to the implementation and expansion of the program. Interview transcripts were analyzed by two researchers using thematic analysis.

Results

Eight key informants were interviewed twice (during and after the intervention period), and one once, for a total of 17 interviews. Patient stakeholders were not interviewed due to COVID-19 pandemic clinic closures and social isolation mandates. Three principal themes emerged: a positive organizational climate and culture fostered the acceptability of the program, the positive impacts of the program observed in the pediatric dental teams and the organization, and the challenges to implement the program including understanding the data reports, trusting the accuracy of the data, and competing priorities.

Conclusions

The intervention of audit and feedback with goal setting and action planning was well accepted and perceived as feasible by the study participants given the financial and human resources provided by the research project. This qualitative study can inform the design and evaluation of process-oriented implementation strategies geared towards decreasing health inequities and improving health outcomes, such as dental caries in American Indian and Alaska Native children and adolescents.

Thought Leader Perspectives on the Benefits, Barriers, and Enablers for Routinely Collected Electronic Health Data to Support Professional Development: Qualitative Study

BACKGROUND

Hospitals routinely collect large amounts of administrative data such as length of stay, 28-day readmissions, and hospital-acquired complications; yet, these data are underused for continuing professional development (CPD). First, these clinical indicators are rarely reviewed outside of existing quality and safety reporting. Second, many medical specialists view their CPD requirements as time-consuming, having minimal impact on practice change and improving patient outcomes. There is an opportunity to build new user interfaces based on these data, designed to support individual and group reflection. Data-informed reflective practice has the potential to generate new insights about performance, bridging the gap between CPD and clinical practice.

OBJECTIVE

This study aims to understand why routinely collected administrative data have not yet become widely used to support reflective practice and lifelong learning.

METHODS

We conducted semistructured interviews (N=19) with thought leaders from a range of backgrounds, including clinicians, surgeons, chief medical officers, information and communications technology professionals, informaticians, researchers, and leaders from related industries. Interviews were thematically analyzed by 2 independent coders.

RESULTS

Respondents identified visibility of outcomes, peer comparison, group reflective discussions, and practice change as potential benefits. The key barriers included legacy technology, distrust with data quality, privacy, data misinterpretation, and team culture. Respondents suggested recruiting local champions for co-design, presenting data for understanding rather than information, coaching by specialty group leaders, and timely reflection linked to CPD as enablers to successful implementation.

CONCLUSIONS

Overall, there was consensus among thought leaders, bringing together insights from diverse backgrounds and medical jurisdictions. We found that clinicians are interested in repurposing administrative data for professional development despite concerns with underlying data quality, privacy, legacy technology, and visual presentation. They prefer group reflection led by supportive specialty group leaders, rather than individual reflection. Our findings provide novel insights into the specific benefits, barriers, and benefits of potential reflective practice interfaces based on these data sets. They can inform the design of new models of in-hospital reflection linked to the annual CPD planning-recording-reflection cycle.

Supporting care engagement in primary care; the development of a maturity matrix

BACKGROUND

Care engagement or active patient involvement in healthcare contributes to the quality of primary care, but organisational preconditions in routine practice need to be aligned. A Maturity Matrix for Care Engagement to assess and discuss these preconditions in the general practice team was developed and tested on feasibility and acceptability in general practice.

METHODS AND FINDINGS

A systematic user-centred approach was applied, starting with a scoping literature search to describe the domains on the horizontal axis of the maturity matrix. The domains and growing steps (vertical axis) were refined by patients (n = 16) and general practice staff (n = 11) in three focus group discussions and reviewed by six experts (local facilitators and scientists). Seven domains could be distinguished: Personalised Care, Shared Decision Making, Self-Management, Patient as Partner, Supportive Means, Patient Environment, and Teamwork among Healthcare Professionals. The growing steps described three to six activities per domain (n = 32 in total) that contribute to care engagement. Local facilitators implemented the tool in two general practice teams according to a user guide, starting with a two-hour kick-off meeting on care engagement. In the next step, practitioners, nurses and assistants in each practice indicated their score on the domains individually. The scores were discussed in the facilitated practice meeting which was aimed at SMART improvement plans. Feasibility and acceptability were assessed in interviews showing that the tool was well received by the pilot practices, although the practice assistants had difficulties scoring some of the activities as they did not always relate to their daily work. An assessment after three months showed changes in practice organisation towards increased care engagement.

CONCLUSIONS

The maturity matrix on care engagement is a tool to identify the organisational practice maturity for care engagement. Suggested adaptations must be implemented before large-scale testing.

Healthcare professionals' perspectives on implementation of universal tumor DNA testing in ovarian cancer patients: multidisciplinary focus groups

Universal tumor DNA testing in epithelial ovarian cancer patients can function not only as an efficient prescreen for hereditary cancer testing, but may also guide treatment choices. This innovation, introduced as Tumor-First workflow, offers great opportunities, but ensuring optimal multidisciplinary collaboration is a challenge. We investigated factors that were relevant and important for large-scale implementation. In three multidisciplinary online focus groups, healthcare professionals (gynecologic oncologists, pathologists, clinical geneticists, and clinical laboratory specialists) were interviewed on factors critical for the implementation of the Tumor-First workflow. Recordings were transcribed for analysis in Atlas.ti according to the framework of Flottorp that categorizes seven implementation domains. Healthcare professionals from all disciplines endorse implementation of the Tumor-First workflow, but more detailed standardization and advice regarding the logistics of the workflow were needed. Healthcare professionals explored ways to stay informed about the different phases of the workflow and the results. They emphasized the importance of including all epithelial ovarian cancer patients in the workflow and monitoring this inclusion. Overall, healthcare professionals would appreciate supporting material for the implementation of the Tumor-First workflow in the daily work routine. Focus group discussions have revealed factors for developing a tailored implementation strategy for the Tumor-First workflow in order to optimize care for epithelial ovarian cancer patients. Future innovations affecting multidisciplinary oncology teams including clinical geneticists can benefit from the lessons learned.

A Systematic Review of Electronic Medical Record Driven Quality Measurement and Feedback Systems

Historically, quality measurement analyses utilize manual chart abstraction from data collected primarily for administrative purposes. These methods are resource-intensive, time-delayed, and often lack clinical relevance. Electronic Medical Records (EMRs) have increased data availability and opportunities for quality measurement. However, little is known about the effectiveness of Measurement Feedback Systems (MFSs) in utilizing EMR data. This study explores the effectiveness and characteristics of EMR-enabled MFSs in tertiary care. The search strategy guided by the PICO Framework was executed in four databases. Two reviewers screened abstracts and manuscripts. Data on effect and intervention characteristics were extracted using a tailored version of the Cochrane EPOC abstraction tool. Due to study heterogeneity, a narrative synthesis was conducted and reported according to PRISMA guidelines. A total of 14 unique MFS studies were extracted and synthesized, of which 12 had positive effects on outcomes. Findings indicate that quality measurement using EMR data is feasible in certain contexts and successful MFSs often incorporated electronic feedback methods, supported by clinical leadership and action planning. EMR-enabled MFSs have the potential to reduce the burden of data collection for quality measurement but further research is needed to evaluate EMR-enabled MFSs to translate and scale findings to broader implementation contexts.

A randomised controlled trial of an implementation strategy delivered at scale to increase outdoor free play opportunities in early childhood education and care (ECEC) services: a study protocol for the get outside get active (GOGA) trial

BACKGROUND

Increased outdoor play time in young children is associated with many health and developmental benefits. This study aims to evaluate the impact of a multi-strategy implementation strategy delivered at scale, to increase opportunities for outdoor free play in Early Childhood Education and Care (ECEC) services.

METHODS

The study will employ a parallel-group randomised controlled trial design. One hundred ECEC services in the Hunter New England region of New South Wales, Australia, will be recruited and randomised to receive either a 6-month implementation strategy or usual care. The trial will seek to increase the implementation of an indoor-outdoor routine (whereby children are allowed to move freely between indoor and outdoor spaces during periods of free play), to increase their opportunity to engage in outdoor free play. Development of the strategy was informed by the Behaviour Change Wheel to address determinants identified in the Theoretical Domains Framework. ECEC services allocated to the control group will receive 'usual' implementation support delivered as part of state-wide obesity prevention programs. The primary trial outcome is the mean minutes/day (calculated across 5 consecutive days) of outdoor free play opportunities provided in ECEC services measured at baseline, 6-months (primary end point) and 18-months post baseline. Analyses will be performed using an intention-to-treat approach with ECEC services as the unit of analysis, using a linear mixed effects regression model to assess between-group differences. A sensitivity analysis will be undertaken, adjusting for service characteristics that appear imbalanced between groups at baseline, and a subgroup analysis examining potential intervention effect among services with the lowest baseline outdoor free play opportunities.

DISCUSSION

Identifying effective strategies to support the implementation of indoor-outdoor routines in the ECEC setting at scale is essential to improve child population health.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry ( ACTRN12621000987864 ). Prospectively registered 27th July 2021, ANZCTR - Registration.

Process evaluations undertaken alongside randomised controlled trials in the hospital setting: A scoping review

Background

There is increasing recognition of the importance of undertaking process evaluations alongside implementation of health interventions by examining mechanisms of impact and contextual factors. However, a comprehensive synthesis of process evaluations undertaken alongside clinical trials in hospital settings is lacking. We undertook a scoping review to address this gap.

Methods

This review was guided by the methodological framework for scoping studies. Studies were identified using four databases; Ovid Medline, EBSCO CINAHL, EMBASE and Scopus. Two authors independently screened all titles and available abstracts, with a third author available to adjudicate. Studies were eligible for inclusion if they described a process evaluation undertaken alongside a randomised controlled trial in the hospital setting. Data were abstracted by one author and checked by two others and analysed both descriptively and using inductive content analysis.

Results

Data were extracted from 30 articles reporting on 15 trials, most of which were cluster randomised trials (c-RTs) (n = 12). The most common data collection methods used in process evaluations were interviews, questionnaires or surveys, and records or logs. Data analysis revealed three themes relative to how authors: use process data to interpret, understand and explain trial outcomes; evaluate responses to the intervention; and consider the implementation context.

Conclusions

Findings from this review demonstrate the complex nature of intervention implementation in the hospital setting. Overall, there is need for standardised reporting of process evaluations and more explicit descriptions of how authors use frameworks to guide their evaluation.

Understanding how and why audits work in improving the quality of hospital care: A systematic realist review

BACKGROUND

Several types of audits have been used to promote quality improvement (QI) in hospital care. However, in-depth studies into the mechanisms responsible for the effectiveness of audits in a given context is scarce. We sought to understand the mechanisms and contextual factors that determine why audits might, or might not, lead to improved quality of hospital care.

METHODS

A realist review was conducted to systematically search and synthesise the literature on audits. Data from individual papers were synthesised by coding, iteratively testing and supplementing initial programme theories, and refining these theories into a set of context-mechanism-outcome configurations (CMOcs).

RESULTS

From our synthesis of 85 papers, seven CMOcs were identified that explain how audits work: (1) externally initiated audits create QI awareness although their impact on improvement diminishes over time; (2) a sense of urgency felt by healthcare professionals triggers engagement with an audit; (3) champions are vital for an audit to be perceived by healthcare professionals as worth the effort; (4) bottom-up initiated audits are more likely to bring about sustained change; (5) knowledge-sharing within externally mandated audits triggers participation by healthcare professionals; (6) audit data support healthcare professionals in raising issues in their dialogues with those in leadership positions; and (7) audits legitimise the provision of feedback to colleagues, which flattens the perceived hierarchy and encourages constructive collaboration.

CONCLUSIONS

This realist review has identified seven CMOcs that should be taken into account when seeking to optimise the design and usage of audits. These CMOcs can provide policy makers and practice leaders with an adequate conceptual grounding to design contextually sensitive audits in diverse settings and advance the audit research agenda for various contexts.

PROSPERO REGISTRATION

CRD42016039882.

Implementing a digital patient feedback system: an analysis using normalisation process theory

BACKGROUND

Patient feedback in the English NHS is now widespread and digital methods are increasingly used. Adoption of digital methods depends on socio-technical and contextual factors, alongside human agency and lived experience. Moreover, the introduction of these methods may be perceived as disruptive of organisational and clinical routines. The focus of this paper is on the implementation of a particular digital feedback intervention that was co-designed with health professionals and patients (the DEPEND study).

METHODS

The digital feedback intervention was conceptualised as a complex intervention and thus the study focused on the contexts within which it operated, and how the different participants made sense of the intervention and engaged with it (or not). Four health care sites were studied: an acute setting, a mental health setting, and two general practices. Qualitative data was collected through interviews and focus groups with professionals, patients and carers. In total 51 staff, 24 patients and 8 carers were included. Forty-two observations of the use of the digital feedback system were carried out in the four settings. Data analysis was based on modified grounded theory and Normalisation Process Theory (NPT) formed the conceptual framework.

RESULTS

Digital feedback made sense to health care staff as it was seen as attractive, fast to complete and easier to analyse. Patients had a range of views depending on their familiarity with the digital world. Patients mentioned barriers such as kiosk not being visible, privacy, lack of digital know-how, technical hitches with the touchscreen. Collective action in maintaining participation again differed between sites because of workload pressure, perceptions of roles and responsibilities; and in the mental health site major organisational change was taking place. For mental health service users, their relationship with staff and their own health status determined their digital use.

CONCLUSION

The potential of digital feedback was recognised but implementation should take local contexts, different patient groups and organisational leadership into account. Patient involvement in change and adaptation of the intervention was important in enhancing the embedding of digital methods in routine feedback. NPT allowed for a in-depth understanding of actions and interactions of both staff and patients.

Revitalising audit and feedback to improve patient care

Audit and feedback are widely used in quality improvement. Robbie Foy and colleagues argue that their full potential to improve patient care could be realised through a more evidence based and imaginative approach