Advancing the literature on designing audit and feedback interventions: identifying theory-informed hypotheses

A scoping review of therapist behaviors in guided digital mental health interventions

Guided digital mental health interventions are more effective than unguided interventions. While research often emphasizes the frequency and intensity of guidance, less attention has been paid to the behaviors enacted by the therapists supporting clients using these interventions. A scoping review of the literature was conducted to systematically examine the evidence on therapist behaviors (i.e., the actions and feedback provided by supporters to patients). Applying broad eligibility criteria, a systematic search was conducted in PubMed, PsycInfo, the Cochrane Library, and Embase from their inception to January 1st 2024. Sixteen studies met inclusion criteria and were included in the review. Following data extraction, a descriptive analysis and synthesis of the results was performed. Most studies (n = 12; 75 %) focused on therapist behaviors in the context of internet-delivered Cognitive Behavioral Therapy for anxiety and depression. While earlier studies primarily focused on identifying therapist behaviors, later studies shifted towards examining the associations between therapist behaviors and different outcomes, as well as deriving research and clinical applications for improving guided internet-delivered treatments. Identified gaps and recommendations for clinical practice, research, training, and treatment development are discussed.

How can strategies based on performance measurement and feedback support changes in nursing practice? A theoretical reflection drawing on Habermas' social perspective

Strategies based on performance measurement and feedback are commonly used to support quality improvement among nurses. These strategies require practice change, which, for nurses, rely to a large extent on their capacity to coordinate with each other effectively. However, the levers for coordinated action are difficult to mobilize. This discussion paper offers a theoretical reflection on the challenges related to coordinating nurses' actions in the context of practice changes initiated by performance measurement and feedback strategies. We explore how Jürgen Habermas' theory of Communicative Action may shed light on the issues underlying nurses' collective actions and self-determination in practice change and the implications for the design of strategies based on performance measurement and feedback. Based on this theory, we propose differences between communicative and functional coordination according to the nature of the actions and the purposes involved. The domains of action underlying these coordination processes, which Habermas referred to as the lifeworld and the system, are then used to draw a parallel with aspects of nursing practice. Further exploration of these concepts allows us to consider the tensions between the demands of the system and the self-determination of nurses within their practice.

Precision feedback: A conceptual model

Introduction

When performance data are provided as feedback to healthcare professionals, they may use it to significantly improve care quality. However, the question of how to provide effective feedback remains unanswered, as decades of evidence have produced a consistent pattern of effects-with wide variation. From a coaching perspective, feedback is often based on a learner's objectives and goals. Furthermore, when coaches provide feedback, it is ideally informed by their understanding of the learner's needs and motivation. We anticipate that a "coaching"-informed approach to feedback may improve its effectiveness in two ways. First, by aligning feedback with healthcare professionals' chosen goals and objectives, and second, by enabling large-scale feedback systems to use new types of data to learn what kind of performance information is motivating in general. Our objective is to propose a conceptual model of precision feedback to support these anticipated enhancements to feedback interventions.

Methods

We iteratively represented models of feedback's influence from theories of motivation and behavior change, visualization, and human-computer interaction. Through cycles of discussion and reflection, application to clinical examples, and software development, we implemented and refined the models in a software application to generate precision feedback messages from performance data for anesthesia providers.

Results

We propose that precision feedback is feedback that is prioritized according to its motivational potential for a specific recipient. We identified three factors that influence motivational potential: (1) the motivating information in a recipient's performance data, (2) the surprisingness of the motivating information, and (3) a recipient's preferences for motivating information and its visual display.

Conclusions

We propose a model of precision feedback that is aligned with leading theories of feedback interventions to support learning about the success of feedback interventions. We plan to evaluate this model in a randomized controlled trial of a precision feedback system that enhances feedback emails to anesthesia providers.

Deploying an asthma dashboard to support quality improvement across a nationally representative sentinel network of 7.6 million people in England

Every year, there are ~100,000 hospital admissions for asthma in the UK, many of which are potentially preventable. Evidence suggests that carefully conceptualised and implemented audit and feedback (A&F) cycles have the potential to improve clinical outcomes for those with chronic conditions. We wanted to investigate the technical feasibility of developing a near-real time asthma dashboard to support A&F interventions for asthma management in primary care. We extracted cross-sectional data on asthma from 756 participating GP practices in the Oxford-Royal College of General Practitioners Research and Surveillance Centre (RCGP RSC) database in England comprising 7.6 million registered people. Summary indicators for a GP practice were compared to all participating RCGP RSC practices using practice-level data, for the week 6-12th-Mar-2023. A weekly, automated asthma dashboard with features that can support electronic-A&F cycles that compared key asthma indicators for a GP practice to RCGP RSC could be created ( https://tinyurl.com/3ydtrt85 ): 12-weeks-incidence 0.4% vs 0.4%, annual prevalence 6.1% vs 6.7%, inhaled relievers to preventer 1.2 vs 1.1, self-management plan given 83.4% vs 60.8%, annual reviews 36.8% vs 57.3%, prednisolone prescriptions 2.0% vs 3.2%, influenza vaccination 56.6% vs 55.5%, pneumococcal vaccination ever (aged ≥65 years) 90.2% vs 84.1% and current smokers 14.9% vs 14.8%. Across the RCGP RSC, the rate of hospitalisations was 0.024%; comparative data had to be suppressed for the study practice because of small numbers. We have successfully created an automated near real-time asthma dashboard that can be used to support A&F initiatives to improve asthma care and outcomes in primary care.

Paediatric oncologists' perspectives on Strategic solutions to develop Integrated Cancer Palliative Care: feedback intervention theory as an explanatory Framework

BACKGROUND

Globally, children with cancer often experience delays in palliative care referral or are infrequently referred. Therefore, we conducted a qualitative study to gain insight from paediatric oncologists into what enables or deters palliative care referral. Strategic solutions to develop integrated palliative care was a critical study theme. In this paper, we have explained and interpreted these strategic solutions through the lens of feedback intervention theory.

METHODOLOGY

The study findings were interpreted using Kumar's six-step approach that enabled systematic evaluation of a theory's appropriateness and alignment with the researcher's paradigm, methodology, and study findings. It also explained how theory informed analysis and elucidated challenges or the development of new models. The feedback intervention theory appraises the discrepancy between actual and desired goals and provides feedback to improve it.

RESULTS

Strategic solutions generated from the study findings were coherent with the aspects elucidated in theory, like coping mechanisms, levels of feedback hierarchy, and factors determining the effect of the feedback intervention on performance. Paediatric oncologists suggested integrating palliative care providers in the team innocuously, improving communication between teams, relabelling palliative care as symptom control, and working with a skilled and accessible palliative care team. The paper proposes an infinite loop model developed from the study, which has the potential to foster integrated palliative care through excellent collaboration and continuous feedback.

CONCLUSION

Applying feedback intervention theory can bridge the gap between actual and desired practice for integrated cancer palliative care in paediatric oncology.

Evaluating a Physician Audit and Feedback Intervention to Increase Clinical Trial Enrollment in Radiation Oncology in a Multisite Tertiary Cancer Center: A Randomized Study

PURPOSE

Clinical trial participation continues to be low, slowing new cancer therapy development. Few strategies have been prospectively tested to address barriers to enrollment. We investigated the effectiveness of a physician audit and feedback report to improve clinical trial enrollment.

METHODS AND MATERIALS

We conducted a randomized quality improvement study among radiation oncologists at a multisite tertiary cancer network. Physicians in the intervention group received quarterly audit and feedback reports comparing the physician's trial enrollments with those of their peers. The primary outcome was trial enrollments.

RESULTS

Among physicians randomized to receive the feedback report (n = 30), the median proportion of patients enrolled during the study period increased to 6.1% (IQR, 2.6%-9.3%) from 3.2% (IQR, 1.1%-10%) at baseline. Among those not receiving the feedback report (n = 29), the median proportion of patients enrolled increased to 4.1% (IQR, 1.3%-7.6%) from 1.6% (IQR, 0%-4.1%) at baseline. There was a nonsignificant change in the proportion of enrollments associated with receiving the feedback report (-0.6%; 95% CI, -3.0% to 1.8%; P = .6). Notably, there was an interaction between baseline trial accrual and receipt of feedback reports (P = .005), with enrollment declining among high accruers. There was an increase in enrollment throughout the study, regardless of study group (P = .001).

CONCLUSIONS

In this study, a positive effect of physician audit and feedback on clinical trial enrollment was not observed. Future efforts should avoid disincentivizing high accruers and might consider pairing feedback with other patient- or physician-level strategies. The increase in trial enrollment in both groups over time highlights the importance of including a comparison group in quality improvement studies to reduce confounding from secular trends.

Perspectives on non-financial conflicts of interest in health-related journals: A scoping review

The objective of this scoping review was to systematically review the literature on how non-financial conflicts of interest (nfCOI) are defined and evaluated, and the strategies suggested for their management in health-related and biomedical journals. PubMed, Embase, Scopus and Web of Science were searched for peer reviewed studies published in English between 1970 and December 2023 that addressed at least one of the following: the definition, evaluation, or management of non-financial conflicts of interest. From 658 studies, 190 studies were included in the review. nfCOI were discussed most commonly in empirical (22%; 42/190), theoretical (15%; 29/190) and "other" studies (18%; 34/190) - including commentary, perspective, and opinion articles. nfCOI were addressed frequently in the research domain (36%; 68/190), publication domain (29%; 55/190) and clinical practice domain (17%; 32/190). Attitudes toward nfCOI and their management were divided into two distinct groups. The first larger group claimed that nfCOI were problematic and required some form of management, whereas the second group argued that nfCOI were not problematic, and therefore, did not require management. Despite ongoing debates about the nature, definition, and management of nfCOI, many articles included in this review agreed that serious consideration needs to be given to the prevalence, impact and optimal mitigation of non-financial COI.

EASY-NET Program: Effectiveness of an Audit and Feedback Intervention in the Emergency Care for Acute Conditions in the Lazio Region

The EASY-NET network program (NET-2016-02364191)-effectiveness of audit and feedback (A&F) strategies to improve health practice and equity in various clinical and organizational settings), piloted a novel and more structured A&F strategy. This study compared the effectiveness of the novel strategy against the sole periodic dissemination of indicators in enhancing the appropriateness and timeliness of emergency health interventions for patients diagnosed with acute myocardial infarction (AMI) and ischemic stroke in the Lazio Region. The efficacy of the intervention was assessed through a prospective quasi-experimental design employing a pre- and post-intervention (2021-2022) comparison with a control group. Participating hospitals in the Lazio Region, where professional teams voluntarily engaged in the intervention, constituted the exposed group, while the control group exclusively engaged in routine reporting activities. Effectiveness analysis was conducted at the patient level, utilizing regional health information systems to compute process and outcome indicators. The effectiveness of the intervention was evaluated using difference-in-difference models, comparing pre- and post-intervention periods between exposed and control groups. Estimates were calculated in terms of the difference in percentage points (PP) between absolute risks. Sixteen facilities for the AMI pathway and thirteen for the stroke pathway participated in the intervention. The intervention yielded a reduction in the proportion of 30-day readmissions following hospitalization for ischemic stroke by 0.54 pp in the exposed patients demonstrating a significant difference of -3.80 pp (95% CI: -6.57; -1.03; 5453 patients, 63.7% cases) in the exposed group compared to controls. However, no statistically significant differences attributable to the implemented A&F intervention were observed in other indicators considered. These results represent the first evidence in Italy of the impact of A&F interventions in an emergency setting, utilizing aggregated data from hospitals involved in the Lazio Region's emergency network.

Audit and feedback interventions involving pharmacists to influence prescribing behaviour in general practice: a systematic review and meta-analysis

INTRODUCTION

Pharmacists, as experts in medicines, are increasingly employed in general practices and undertake a range of responsibilities. Audit and feedback (A&F) interventions are effective in achieving behaviour change, including prescribing. The extent of pharmacist involvement in A&F interventions to influence prescribing is unknown. This review aimed to assess the effectiveness of A&F interventions involving pharmacists on prescribing in general practice compared with no A&F/usual care and to describe features of A&F interventions and pharmacist characteristics.

METHODS

Electronic databases (MEDLINE, EMBASE, CINAHL, Cochrane Central Register of Controlled Trials, (Social) Science Citation Indexes, ISI Web of Science) were searched (2012, 2019, 2020). Cochrane systematic review methods were applied to trial identification, selection, and risk of bias. Results were summarized descriptively and heterogeneity was assessed. A random-effects meta-analysis was conducted where studies were sufficiently homogenous in design and outcome.

RESULTS

Eleven cluster-randomized studies from 9 countries were included. Risk of bias across most domains was low. Interventions focussed on older patients, specific clinical area(s), or specific medications. Meta-analysis of 6 studies showed improved prescribing outcomes (pooled risk ratio: 0.78, 95% confidence interval: 0.64-0.94). Interventions including both verbal and written feedback or computerized decision support for prescribers were more effective. Pharmacists who received study-specific training, provided ongoing support to prescribers or reviewed prescribing for individual patients, contributed to more effective interventions.

CONCLUSIONS

A&F interventions involving pharmacists can lead to small improvements in evidence-based prescribing in general practice settings. Future implementation of A&F within general practice should compare different ways of involving pharmacists to determine how to optimize effectiveness.PRISMA-compliant abstract included in Supplementary Material 1.

Audit and feedback to reduce unwarranted clinical variation at scale: a realist study of implementation strategy mechanisms

BACKGROUND

Unwarranted clinical variation in hospital care includes the underuse, overuse, or misuse of services. Audit and feedback is a common strategy to reduce unwarranted variation, but its effectiveness varies widely across contexts. We aimed to identify implementation strategies, mechanisms, and contextual circumstances contributing to the impact of audit and feedback on unwarranted clinical variation.

METHODS

Realist study examining a state-wide value-based healthcare program implemented between 2017 and 2021 in New South Wales, Australia. Three initiatives within the program included audit and feedback to reduce unwarranted variation in inpatient care for different conditions. Multiple data sources were used to formulate the initial audit and feedback program theory: a systematic review, realist review, program document review, and informal discussions with key program stakeholders. Semi-structured interviews were then conducted with 56 participants to refute, refine, or confirm the initial program theories. Data were analysed retroductively using a context-mechanism-outcome framework for 11 transcripts which were coded into the audit and feedback program theory. The program theory was validated with three expert panels: senior health leaders (n = 19), Agency for Clinical Innovation (n = 11), and Ministry of Health (n = 21) staff.

RESULTS

The program's audit and feedback implementation strategy operated through eight mechanistic processes. The strategy worked well when clinicians (1) felt ownership and buy-in, (2) could make sense of the information provided, (3) were motivated by social influence, and (4) accepted responsibility and accountability for proposed changes. The success of the strategy was constrained when the audit process led to (5) rationalising current practice instead of creating a learning opportunity, (6) perceptions of unfairness and concerns about data integrity, 7) development of improvement plans that were not followed, and (8) perceived intrusions on professional autonomy.

CONCLUSIONS

Audit and feedback strategies may help reduce unwarranted clinical variation in care where there is engagement between auditors and local clinicians, meaningful audit indicators, clear improvement plans, and respect for clinical expertise. We contribute theoretical development for audit and feedback by proposing a Model for Audit and Feedback Implementation at Scale. Recommendations include limiting the number of audit indicators, involving clinical staff and local leaders in feedback, and providing opportunities for reflection.

Types and effects of feedback for emergency ambulance staff: a systematic mixed studies review and meta-analysis

BACKGROUND

Extensive research has been conducted into the effects of feedback interventions within many areas of healthcare, but prehospital emergency care has been relatively neglected. Exploratory work suggests that enhancing feedback and follow-up to emergency medical service (EMS) staff might provide staff with closure and improve clinical performance. Our aim was to summarise the literature on the types of feedback received by EMS professionals and its effects on the quality and safety of patient care, staff well-being and professional development.

METHODS

A systematic review and meta-analysis, including primary research studies of any method published in peer-reviewed journals. Studies were included if they contained information on systematic feedback to emergency ambulance staff regarding their performance. Databases searched from inception were MEDLINE, Embase, AMED, PsycINFO, HMIC, CINAHL and Web of Science, with searches last updated on 2 August 2022. Study quality was appraised using the Mixed Methods Appraisal Tool. Data analysis followed a convergent integrated design involving simultaneous narrative synthesis and random effects multilevel meta-analyses.

RESULTS

The search strategy yielded 3183 articles, with 48 studies meeting inclusion criteria after title/abstract screening and full-text review. Interventions were categorised as audit and feedback (n=31), peer-to-peer feedback (n=3), postevent debriefing (n=2), incident-prompted feedback (n=1), patient outcome feedback (n=1) or a combination thereof (n=4). Feedback was found to have a moderate positive effect on quality of care and professional development with a pooled effect of d=0.50 (95% CI 0.34, 0.67). Feedback to EMS professionals had large effects in improving documentation (d=0.73 (0.00, 1.45)) and protocol adherence (d=0.68 (0.12, 1.24)), as well as small effects in enhancing cardiac arrest performance (d=0.46 (0.06, 0.86)), clinical decision-making (d=0.47 (0.23, 0.72)), ambulance times (d=0.43 (0.12, 0.74)) and survival rates (d=0.22 (0.11, 0.33)). The between-study heterogeneity variance was estimated at σ2=0.32 (95% CI 0.22, 0.50), with an I2 value of 99% (95% CI 98%, 99%), indicating substantial statistical heterogeneity.

CONCLUSION

This review demonstrated that the evidence base currently does not support a clear single point estimate of the pooled effect of feedback to EMS staff as a single intervention type due to study heterogeneity. Further research is needed to provide guidance and frameworks supporting better design and evaluation of feedback interventions within EMS.

PROSPERO REGISTRATION NUMBER

CRD42020162600.

A cluster randomised controlled trial, process and economic evaluation of quality improvement collaboratives aligned to a national audit to improve the care for people with diabetes (EQUIPD): study protocol

BACKGROUND

People with type 1 diabetes and raised glucose levels are at greater risk of retinopathy, nephropathy, neuropathy, cardiovascular disease, sexual health problems and foot disease. The UK National Institute for Health and Care Excellence (NICE) recommends continuous subcutaneous 'insulin pump' therapy for people with type 1 diabetes whose HbA1c is above 69 mmol/mol. Insulin pump use can improve quality of life, cut cardiovascular risk and increase treatment satisfaction. About 90,000 people in England and Wales meet NICE criteria for insulin pumps but do not use one. Insulin pump use also varies markedly by deprivation, ethnicity, sex and location. Increasing insulin pump use is a key improvement priority. Audit and feedback is a common but variably effective intervention. Limited capabilities of healthcare providers to mount effective responses to feedback from national audits, such as the National Diabetes Audit (NDA), undermines efforts to improve care. We have co-developed a theoretically and empirically informed quality improvement collaborative (QIC) to strengthen local responses to feedback with patients and carers, national audits and healthcare providers. We will evaluate whether the QIC improves the uptake of insulin pumps following NDA feedback.

METHODS

We will undertake an efficient cluster randomised trial using routine data. The QIC will be delivered alongside the NDA to specialist diabetes teams in England and Wales. Our primary outcome will be the proportion of people with type 1 diabetes and an HbA1c above 69 mmol/mol who start and continue insulin pump use during the 18-month intervention period. Secondary outcomes will assess change in glucose control and duration of pump use. Subgroup analyses will explore impacts upon inequalities by ethnicity, sex, age and deprivation. A theory-informed process evaluation will explore diabetes specialist teams' engagement, implementation, fidelity and tailoring through observations, interviews, surveys and documentary analysis. An economic evaluation will micro-cost the QIC, estimate cost-effectiveness of NDA feedback with QIC and estimate the budget impact of NHS-wide QIC roll out.

DISCUSSION

Our study responds to a need for more head-to-head trials of different ways of reinforcing feedback delivery. Our findings will have implications for other large-scale audit and feedback programmes.

TRIAL REGISTRATION

ISRCTN82176651 Registered 18 October 2022.

Using audit and feedback to guide tailored implementations of measurement-based care in community mental health: a multiple case study

BACKGROUND

Audit and feedback (A&F) is an implementation strategy that can facilitate implementation tailoring by identifying gaps between desired and actual clinical care. While there are several theory-based propositions on which A&F components lead to better implementation outcomes, many have not been empirically investigated, and there is limited guidance for stakeholders when applying A&F in practice. The current study aims to illustrate A&F procedures in six community mental health clinics, with an emphasis on reporting A&F components that are relevant to theories of how feedback elicits behavior change.

METHODS

Six clinics from a larger trial using a tailored approach to implement measurement-based care (MBC) were analyzed for feedback content, delivery mechanisms, barriers to feedback, and outcomes of feedback using archival data. Pattern analysis was conducted to examine relations between A&F components and changes in MBC use.

RESULTS

Several sites utilized both aggregate and individualized data summaries, and data accuracy concerns were common. Feedback cycles featuring individual-level clinician data, data relevant to MBC barriers, and information requested by data recipients were related to patterns of increased MBC use.

CONCLUSIONS

These findings support extant theory, such as Feedback Intervention Theory. Mental health professionals wishing to apply A&F should consider establishing reciprocal feedback mechanisms on the quality and amount of data being received and adopting specific roles communicating and addressing data quality concerns.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT02266134.

Feedback for Emergency Ambulance Staff: A National Review of Current Practice Informed by Realist Evaluation Methodology

Research suggests that feedback in Emergency Medical Services (EMS) positively affects quality of care and professional development. However, the mechanisms by which feedback achieves its effects still need to be better understood across healthcare settings. This study aimed to understand how United Kingdom (UK) ambulance services provide feedback for EMS professionals and develop a programme theory of how feedback works within EMS, using a mixed-methods, realist evaluation framework. A national cross-sectional survey was conducted to identify feedback initiatives in UK ambulance services, followed by four in-depth case studies involving qualitative interviews and documentary analysis. We used qualitative content analysis and descriptive statistics to analyse survey responses from 40 prehospital feedback initiatives, alongside retroductive analysis of 17 interviews and six documents from case study sites. Feedback initiatives mainly provided individual patient outcome feedback through "pull" initiatives triggered by staff requests. Challenges related to information governance were identified. Our programme theory of feedback to EMS professionals encompassed context (healthcare professional and organisational characteristics), mechanisms (feedback and implementation characteristics, psychological reasoning) and outcomes (implementation, staff and service outcomes). This study suggests that most UK ambulance services use a range of feedback initiatives and provides 24 empirically based testable hypotheses for future research.

A Quality Improvement Project to Decrease CLABSIs in Non-ICU Settings

BACKGROUND AND OBJECTIVES

Central line-associated bloodstream infections (CLABSIs) are a common, preventable healthcare-associated infection. In our 3-hospital health system, CLABSI rates in non-intensive care unit (ICU) settings were above the internal target rate of zero. A robust quality improvement (QI) project to reduce non-ICU CLABSIs was undertaken by a team of Doctor of Nursing Practice (DNP)-prepared nurse leaders enrolled in a post-DNP Quality Implementation Scholars program and 2 QI experts. Based on a review of the literature and local root cause analyses, the QI team implemented the evidence-based practice of using 2% chlorhexidine gluconate (CHG) cloths for daily bathing for non-ICU patients with a central line.

METHODS

A pre-post-design was used for this QI study. CHG bathing was implemented using multifaceted educational strategies that included an e-learning module, printed educational materials, educational outreach, engagement of unit-based CLABSI champions, and an electronic reminder in the electronic health record. Generalized linear mixed-effects models were used to assess the change in CLABSI rates before and after implementation of CHG bathing. CLABSI rates were also tracked using statistical process control (SPC) charts to monitor stability over time. CHG bathing documentation compliance was audited as a process measure. These audit data were provided to unit-based leadership (nurse managers and clinical team leaders) on a monthly basis. A Qualtrics survey was also disseminated to nursing leadership to evaluate their satisfaction with the CHG bathing implementation processes.

RESULTS

Thirty-four non-ICU settings participated in the QI study, including general medical/surgical units and specialty areas (oncology, neurosciences, cardiac, orthopedic, and pediatrics). While the change in CLABSI rates after the intervention was not statistically significant ( b = -0.35, P = .15), there was a clinically significant CLABSI rate reduction of 22.8%. Monitoring the SPC charts demonstrated that CLABSI rates remained stable after the intervention at all 3 hospitals as well as the health system. CHG bathing documentation compliance increased system-wide from 77% (January 2020) to 94% (February 2021). Overall, nurse leaders were satisfied with the CHG bathing implementation process.

CONCLUSIONS

To sustain this practice change in non-ICU settings, booster sessions will be completed at least on an annual basis. This study provides further support for using CHG cloths for daily patient bathing in the non-ICU setting.

Audit, Feedback, and Education to Improve Quality and Outcomes in Transurethral Resection and Single-Instillation Intravesical Chemotherapy for Nonmuscle Invasive Bladder Cancer Treatment: Protocol for a Multicenter International Observational Study With an Embedded Cluster Randomized Trial

BACKGROUND

Nonmuscle invasive bladder cancer (NMIBC) accounts for 75% of bladder cancers. It is common and costly. Cost and detriment to patient outcomes and quality of life are driven by high recurrence rates and the need for regular invasive surveillance and repeat treatments. There is evidence that the quality of the initial surgical procedure (transurethral resection of bladder tumor [TURBT]) and administration of postoperative bladder chemotherapy significantly reduce cancer recurrence rates and improve outcomes (cancer progression and mortality). There is surgeon-reported evidence that TURBT practice varies significantly across surgeons and sites. There is limited evidence from clinical trials of intravesical chemotherapy that NMIBC recurrence rate varies significantly between sites and that this cannot be accounted for by differences in patient, tumor, or adjuvant treatment factors, suggesting that how the surgery is performed may be a reason for the variation.

OBJECTIVE

This study primarily aims to determine if feedback on and education about surgical quality indicators can improve performance and secondarily if this can reduce cancer recurrence rates. Planned secondary analyses aim to determine what surgeon, operative, perioperative, institutional, and patient factors are associated with better achievement of TURBT quality indicators and NMIBC recurrence rates.

METHODS

This is an observational, international, multicenter study with an embedded cluster randomized trial of audit, feedback, and education. Sites will be included if they perform TURBT for NMIBC. The study has four phases: (1) site registration and usual practice survey; (2) retrospective audit; (3) randomization to audit, feedback, and education intervention or to no intervention; and (4) prospective audit. Local and national ethical and institutional approvals or exemptions will be obtained at each participating site.

RESULTS

The study has 4 coprimary outcomes, which are 4 evidence-based TURBT quality indicators: a surgical performance factor (detrusor muscle resection); an adjuvant treatment factor (intravesical chemotherapy administration); and 2 documentation factors (resection completeness and tumor features). A key secondary outcome is the early cancer recurrence rate. The intervention is a web-based surgical performance feedback dashboard with educational and practical resources for TURBT quality improvement. It will include anonymous site and surgeon-level peer comparison, a performance summary, and targets. The coprimary outcomes will be analyzed at the site level while recurrence rate will be analyzed at the patient level. The study was funded in October 2020 and began data collection in April 2021. As of January 2023, there were 220 hospitals participating and over 15,000 patient records. Projected data collection end date is June 30, 2023.

CONCLUSIONS

This study aims to use a distributed collaborative model to deliver a site-level web-based performance feedback intervention to improve the quality of endoscopic bladder cancer surgery. The study is funded and projects to complete data collection in June 2023.

TRIAL REGISTRATION

ClinicalTrials.org NCT05154084; https://clinicaltrials.gov/ct2/show/NCT05154084.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/42254.

Qualitative study of international key informants' perspectives on the current and future state of healthcare quality measurement and feedback

OBJECTIVES

The aim of this study is to explore the current and future state of quality measurement and feedback and identify factors influencing measurement feedback systems, including the barriers and enablers to their effective design, implementation, use and translation into quality improvement.

DESIGN

This qualitative study used semistructured interviews with key informants. A deductive framework analysis was conducted to code transcripts to the Theoretical Domains Framework (TDF). An inductive analysis was used to produce subthemes and belief statements within each TDF domain.

SETTING

All interviews were conducted by videoconference and audio-recorded.

PARTICIPANTS

Key informants were purposively sampled experts in quality measurement and feedback, including clinical (n=5), government (n=5), research (n=4) and health service leaders (n=3) from Australia (n=7), the USA (n=4), the UK (n=2), Canada (n=2) and Sweden (n=2).

RESULTS

A total of 17 key informants participated in the study. The interview length ranged from 48 to 66 min. 12 theoretical domains populated by 38 subthemes were identified as relevant to measurement feedback systems. The most populous domains included environmental context and resources, memory, attention and decision-making, and social influences. The most populous subthemes included 'quality improvement culture', 'financial and human resource support' and 'patient-centred measurement'. There were minimal conflicting beliefs outside of 'data quality and completeness'. Conflicting beliefs in these subthemes were predominantly between government and clinical leaders.

CONCLUSIONS

Multiple factors were found to influence measurement feedback systems and future considerations are presented within this manuscript. The barriers and enablers that impact these systems are complex. While there are some clear modifiable factors in the design of measurement and feedback processes, influential factors described by key informants were largely socioenvironmental. Evidence-based design and implementation, coupled with a deeper understanding of the implementation context, may lead to enhanced quality measurement feedback systems and ultimately improved care delivery and patient outcomes.

EASY-NET Program: Methods and Preliminary Results of an Audit and Feedback Intervention in the Emergency Care for Acute Myocardial Infarction in the Lazio Region, Italy

Within the EASY-NET network program (NET-2016-02364191), Work Package 1 Lazio evaluates the effectiveness of a structured audit and feedback (A&F) intervention compared with the web-based regional periodic publication of indicators in improving the appropriateness and timeliness of emergency healthcare for acute myocardial infarction (AMI). This work describes the A&F methodology and presents the results of the first feedback delivered. The intervention involves sending periodic reports via e-mail to participating hospitals. The feedback reports include a set of volume and quality (process and outcome) indicators, calculated by facility through the health information system of the Lazio Region and compared with regional mean, target values and values calculated for hospitals with similar volumes of activity. Health managers and clinicians of each participating hospital represent the "feedback recipients". They are invited to organize clinical and organizational audit meetings to identify possible critical issues in the care pathway and define, where necessary, improvement actions. A total of 16 facilities are involved. Twelve facilities present high volumes in all volume indicators, while three facilities present low volumes for each indicator. Concerning the quality indicators, four facilities do not present critical indicators or had average results, three facilities do not present critical indicators but show average results in at least one of the indicators and six facilities present a critical value for at least one of the indicators. The first report highlighted some critical issues in some facilities on several indicators. During the audit meetings, each facility analyzes these issues, defining appropriate improvement actions. The outcome of these actions will be monitored through subsequent reporting to support the continuous care quality improvement process.

Designing audit and feedback dashboards for dentists to monitor their opioid prescribing

BACKGROUND AND OBJECTIVE

Prescription drug abuse is a major factor leading to drug overdose deaths in the US and dentists are one of the leading prescribers of opioid pain medication. Knowing that Audit & Feedback (A&F) dashboards are an effective tool and are used as quality improvement interventions, we aimed to develop such dashboards personalized for dental providers which could allow them to monitor their own opioid prescribing performance.

METHODS

In this paper we report on the process for designing the A&F dashboards for dentists which were developed by using an iterative human-centered design process. The results obtained from each iteration were used to enrich the information needs analyses, provide function testing, and guide the design decisions of the next iteration.

RESULTS

Engaging dentists in the development and refinement of the dashboards while using the think-aloud protocol for user-testing, provided rapid feedback and identified areas that were confusing and needed either a redesign or additional explanatory content. The final version of dashboards consisted of displaying necessary information through easy to interpret visualizations and interactive features. These included providing access to current national and organizational prescribing guidelines, displaying changes in individual prescribing behavior over time, comparing individual prescribing rate to peer group rate and target rate, displaying procedure specific prescribing, integrating patient reported post-operative dental pain experience and providing navigation and interpretation tips for users. The dashboards were easy to learn and understand for the dentists and were deemed as worth using often in dental practice.

CONCLUSION

Our research was able to demonstrate the creation of useful and usable A&F dashboards using data from electronic dental records and patient surveys, for dentists to effectively monitor their opioid prescribing behavior. Efficacy of the dashboards will be tested in future work.

Feedback on clinical team performance: how does it work, in what contexts, for whom, and for what changes? A critical realist qualitative multiple case study

BACKGROUND

Feedback on clinical performance aims to provide teams in health care settings with structured results about their performance in order to improve these results. Two systematic reviews that included 147 randomized studies showed unresolved variability in professional compliance with desired clinical practices. Conventional recommendations for improving feedback on clinical team performance generally appear decontextualized and, in this regard, idealized. Feedback involves a complex and varied arrangement of human and non-human entities and interrelationships. To explore this complexity and improve feedback, we sought to explain how feedback on clinical team performance works, for whom, in what contexts, and for what changes. Our goal in this research was to present a realistic and contextualized explanation of feedback and its outcomes for clinical teams in health care settings.

METHODS

This critical realist qualitative multiple case study included three heterogeneous cases and 98 professionals from a university-affiliated tertiary care hospital. Five data collection methods were used: participant observation, document retrieval, focus groups, semi-structured interviews, and questionnaires. Intra- and inter-case analysis performed during data collection involved thematic analysis, analytical questioning, and systemic modeling. These approaches were supported by critical reflexive dialogue among the research team, collaborators, and an expert panel.

RESULTS

Despite the use of a single implementation model throughout the institution, results differed on contextual decision-making structures, responses to controversy, feedback loop practices, and use of varied technical or hybrid intermediaries. Structures and actions maintain or transform interrelationships and generate changes that are in line with expectations or the emergence of original solutions. Changes are related to the implementation of institutional and local projects or indicator results. However, they do not necessarily reflect a change in clinical practice or patient outcomes.

CONCLUSIONS

This critical realist qualitative multiple case study offers an in-depth explanation of feedback on clinical team performance as a complex and open-ended sociotechnical system in constant transformation. In doing so, it identifies reflexive questions that are levers for the improvement of team feedback.

How do healthcare providers use national audit data for improvement?

BACKGROUND

Substantial resources are invested by Health Departments worldwide in introducing National Clinical Audits (NCAs). Yet, there is variable evidence on the NCAs' effectiveness and little is known on factors underlying the successful use of NCAs to improve local practice. This study will focus on a single NCA (the National Audit of Inpatient Falls -NAIF 2017) to explore: (i) participants' perspectives on the NCA reports, local feedback characteristics and actions undertaken following the feedback underpinning the effective use of the NCA feedback to improve local practice; (ii) reported changes in local practice following the NCA feedback in England and Wales.

METHODS

Front-line staff perspectives were gathered through interviews. An inductive qualitative approach was used. Eighteen participants were purposefully sampled from 7 of the 85 participating hospitals in England and Wales. Analysis was guided by constant comparative techniques.

RESULTS

Regarding the NAIF annual report, interviewees valued performance benchmarking with other hospitals, the use of visual representations and the inclusion of case studies and recommendations. Participants stated that feedback should target front-line healthcare professionals, be straightforward and focused, and be delivered through an encouraging and honest discussion. Interviewees highlighted the value of using other relevant data sources alongside NAIF feedback and the importance of continuous data monitoring. Participants reported that engagement of front-line staff in the NAIF and following improvement activities was critical. Leadership, ownership, management support and communication at different organisational levels were perceived as enablers, while staffing level and turnover, and poor quality improvement (QI) skills, were perceived as barriers to improvement. Reported changes in practice included increased awareness and attention to patient safety issues and greater involvement of patients and staff in falls prevention activities.

CONCLUSIONS

There is scope to improve the use of NCAs by front-line staff. NCAs should not be seen as isolated interventions but should be fully embedded and integrated into the QI strategic and operational plans of NHS trusts. The use of NCAs could be optimised, but knowledge of them is poor and distributed unevenly across different disciplines. More research is needed to provide guidance on key elements to consider throughout the whole improvement process at different organisational levels.

Can a novel constructivist theory-informed feedback intervention reduce prescribing errors ? A pre-post study

CONTEXT

Medical interns (interns) find prescribing challenging and many report lacking readiness when commencing work. Errors in prescribing puts patients' safety at risk. Yet error rates remain high, despite education, supervision and pharmacists' contributions. Feedback on prescribing may improve performance. Yet, work-based prescribing feedback practices focus on rectifying errors. We aimed to explore if prescribing can be improved using a theory-informed feedback intervention.

METHODS

In this pre-post study, we designed and implemented a constructivist-theory informed prescribing feedback intervention, informed by Feedback-Mark 2 Theory. Interns commencing internal medicine terms in two Australian teaching hospitals were invited to engage in the feedback intervention. Their prescribing was evaluated by comparing errors per medication order of at least 30 orders per intern. Pre/baseline (weeks 1-3) were compared with post intervention (weeks 8-9). Interns' baseline prescribing audit findings were analysed and discussed at individualised feedback sessions. These sessions were with a clinical pharmacologist (Site 1) and a pharmacist educator (Site 2).

RESULTS

Eighty eight intern's prescribing over five 10-week terms was analysed from two hospitals. The frequency of prescribing errors significantly reduced at both sites after the intervention, across all five terms (p < 0.001).There were initially 1598 errors in 2750 orders (median [IQR] 0.48 [0.35-0.67] errors per order) and after the intervention 1113 errors in 2694 orders (median [IQR] 0.30 [0.17-0.50] errors per order).

CONCLUSION

Our findings suggest interns' prescribing practices may improve as a result of constructivist -theory learner centred, informed feedback with an agreed plan. This novel intervention, contributed, to a reduction in interns' prescribing errors. This study suggests new strategies for improving prescribing safety should include the design and implementation of theory-informed feedback interventions.

Exploring the Association between Individual-Level Attributes and Fidelity to a Vocational Rehabilitation Intervention within a Randomised Controlled Trial

Understanding what attributes or characteristics of those delivering interventions affect intervention fidelity and patient outcomes is important for contextualising intervention effectiveness. It may also inform implementation of interventions in future research and clinical practice. This study aimed to explore the relationships between attributes of Occupational Therapists (OTs), their faithful delivery of an early stroke specialist vocational rehabilitation intervention (ESSVR), and stroke survivor return-to-work (RTW) outcomes. Thirty-nine OTs were surveyed about their experience and knowledge of stroke and vocational rehabilitation and were trained to deliver ESSVR. ESSVR was delivered across 16 sites in England and Wales between February 2018 and November 2021. OTs received monthly mentoring to support ESSVR delivery. The amount of mentoring each OT received was recorded in OT mentoring records. Fidelity was assessed using an intervention component checklist completed using retrospective case review of one randomly selected participant per OT. Linear and logistic regression analyses explored relationships between OT attributes, fidelity, and stroke survivor RTW outcome. Fidelity scores ranged from 30.8 to 100% (Mean: 78.8%, SD: 19.2%). Only OT engagement in mentoring was significantly associated with fidelity (b = 0.29, 95% CI = 0.05-0.53, p < 0.05). Increased fidelity (OR = 1.06, 95% CI = 1.01-1.1, p = 0.01) and increasing years of stroke rehabilitation experience (OR = 1.17, 95% CI = 1.02-1.35) was significantly associated with positive stroke survivor RTW outcomes. Findings of this study suggest that mentoring OTs may increase fidelity of delivery of ESSVR, which may also be associated with positive stroke survivor return-to-work outcomes. The results also suggest that OTs with more experience of stroke rehabilitation may be able to support stroke survivors to RTW more effectively. Upskilling OTs to deliver complex interventions, such as ESSVR, in clinical trials may require mentoring support in addition to training to ensure fidelity.

INCOG 2.0 Guidelines for Cognitive Rehabilitation Following Traumatic Brain Injury: Methods, Overview, and Principles

INTRODUCTION

Moderate to severe traumatic brain injury (TBI) results in complex cognitive sequelae. Despite hundreds of clinical trials in cognitive rehabilitation, the translation of these findings into clinical practice remains a challenge. Clinical practice guidelines are one solution. The objective of this initiative was to reconvene the international group of cognitive researchers and clinicians (known as INCOG) to develop INCOG 2.0: Guidelines for Cognitive Rehabilitation Following TBI.

METHODS

The guidelines adaptation and development cycle was used to update the recommendations and derive new ones. The team met virtually and reviewed the literature published since the original INCOG (2014) to update the recommendations and decision algorithms. The team then prioritized the recommendations for implementation and modified the audit tool accordingly to allow for the evaluation of adherence to best practices.

RESULTS

In total, the INCOG update contains 80 recommendations (25 level A, 15 level B, and 40 level C) of which 27 are new. Recommendations developed for posttraumatic amnesia, attention, memory, executive function and cognitive-communication are outlined in other articles, whereas this article focuses on the overarching principles of care for which there are 38 recommendations pertaining to: assessment (10 recommendations), principles of cognitive rehabilitation (6 recommendations), medications to enhance cognition (10 recommendations), teleassessment (5 recommendations), and telerehabilitation intervention (7 recommendations). One recommendation was supported by level A evidence, 7 by level B evidence, and all remaining recommendations were level C evidence. New to INCOG are recommendations for telehealth-delivered cognitive assessment and rehabilitation. Evidence-based clinical algorithms and audit tools for evaluating the state of current practice are also provided.

CONCLUSIONS

Evidence-based cognitive rehabilitation guided by these recommendations should be offered to individuals with TBI. Despite the advancements in TBI rehabilitation research, further high-quality studies are needed to better understand the role of cognitive rehabilitation in improving patient outcomes after TBI.

Audit and Feedback in the Hospitals of the Emergency Networks in the Lazio Region, Italy: A Cross-Sectional Evaluation of the State of Implementation

Audit and Feedback (A&F) is an effective multidimensional strategy for improving the quality of care. The optimal methods for its implementation remain unclear. This study aimed to map the state of art of A&F strategies in the hospitals involved in a time-dependent emergency network. For these purposes, a structured questionnaire was defined and discussed within the research group. This consists of 29 questions in three sections: (1) characteristics of the structure, (2) internal feedback systems, and (3) external feedback systems. All structures involved in the network were invited to participate in the e-survey by indicating a Health Management representative and a clinical representative for the Cardiovascular (CaV) and/or for the Cerebrovascular area (CeV). Of 20 structures invited, a total of 13 (65%) responded to the survey, 11 for the CaV area and 8 for the CeV area. A total of 10 of 11 (91%) facilities for the CaV area and 8/11 (75%) for the CeV area reported that they perform A&F activities. All facilities perform at least one of the activities defined as "assimilating A&F procedures." The most frequent is the presentation and discussion of clinical cases (82% CaV and 88% CeV) and the least is the identification of responsible for improvement actions (45% CaV and 38% CeV). In 4/10 (40%) facilities for the CaV area and 4/8 (50%) for the CEV area, corrective actions are suggested or planned when the feedback is returned. These results confirm the need to define, in a synergistic way with the relevant stakeholders, an effective and agreed A&F intervention to improve the level of implementation of A&F strategies.

Characterising performance information use in the primary healthcare systems of El Salvador, Lebanon and Malawi: multiple qualitative case study protocol

INTRODUCTION

Governments in low-income and middle-income countries (LMICs) and official development assistance agencies use a variety of performance measurement and management approaches to improve the performance of healthcare systems. The effectiveness of such approaches is contingent on the extent to which managers and care providers use performance information. To date, major knowledge gaps exist about the contextual factors that contribute, or not, to performance information use by primary healthcare (PHC) decision-makers in LMICs. This study will address three research questions: (1) How do decision-makers use performance information, and for what purposes? (2) What are the contextual factors that influence the use or non-use of performance information? and (3) What are the proximal outcomes reported by PHC decision-makers from performance information use?

METHODS AND ANALYSIS

We present the protocol of a theory-driven, qualitative study with a multiple case study design to be conducted in El Salvador, Lebanon and Malawi.Data sources include semi structured in-depth interviews and document review. Interviews will be conducted with approximately 60 respondents including PHC system decision-makers and providers. We follow an interdisciplinary theoretical framework that draws on health policy and systems research, public administration, organisational science and health service research. Data will be analysed using thematic analysis to explore how respondents use performance information or not, and for what purposes as well as barriers and facilitators of use.

ETHICS AND DISSEMINATION

The ethical boards of the participating universities approved the protocol presented here. Study results will be disseminated through peer-reviewed journals and global health conferences.

The Influence of Health Behavior Theory on Implementation Practice and Science: Brief Review and Commentary

As research defines new treatments and policies to improve the health of patients, an increasing challenge has been to translate these insights into routine clinical practice to benefit patients and society. An important exploration is how theories of human behavior change fit into the science of implementation and quality improvement. In this paper, we begin with a brief review of the intellectual roots of implementation science and quality improvement, followed by a discussion of how theories and principles of behavior change can inform both goals and challenges in using behavior change theories. The insights offered through health behavior change theory have led to changes in how we plan for implementation and select, develop, design and tailor implementation interventions and strategies. While the degree to which organizational and external contexts influence the behavior of providers in these organizations varies widely, some degree of context external to the individual is important and needs adequate consideration. In short, health behavior change theory is essential but not sufficient to integrate in most implementation efforts, where priority must be given to both individual factors and contexts in which individuals operate.

How does nursing-sensitive indicator feedback with nursing or interprofessional teams work and shape nursing performance improvement systems? A rapid realist review

BACKGROUND

Care quality varies between organizations and even units within an organization. Inadequate care can have harmful financial and social consequences, e.g. nosocomial infection, lengthened hospital stays or death. Experts recommend the implementation of nursing performance improvement systems to assess team performance and monitor patient outcomes as well as service efficiency. In practice, these systems provide nursing or interprofessional teams with nursing-sensitive indicator feedback. Feedback is essential since it commits teams to improve their practice, although it appears somewhat haphazard and, at times, overlooked. Research findings suggest that contextual dynamics, initial system performance and feedback modes interact in unknown ways. This rapid review aims to produce a theorization to explain what works in which contexts, and how feedback to nursing or interprofessional teams shape nursing performance improvement systems.

METHODS

Based on theory-driven realist methodology, with reference to an innovative combination of Actor-Network Theory and Critical Realist philosophy principles, this realist rapid review entailed an iterative procedure: 8766 documents in French and English, published between 2010 and 2018, were identified via 5 databases, and 23 were selected and analysed. Two expert panels (scientific and clinical) were consulted to improve the synthesis and systemic modelling of an original feedback theorization.

RESULTS

We identified three hypotheses, subdivided into twelve generative configurations to explain how feedback mobilizes nursing or interprofessional teams. Empirically founded and actionable, these propositions are supported by expert panels. Each configuration specifies contextualized mechanisms that explain feedback and team outcomes. Socially mediated mechanisms are particularly generative of action and agency.

CONCLUSIONS

This rapid realist review provides an informative theoretical proposition to embrace the complexity of nursing-sensitive indicator feedback with nursing or interdisciplinary teams. Building on general explanations previously observed, this review provides insight into a deep explanation of feedback mechanisms.

SYSTEMATIC REVIEW REGISTRATION

Prospero CRD42018110128 .

A commentary on the quality improvement practices in leading an organizational response to audit feedback

ABSTRACT

Clinical audit is commonly used to assess and improve the quality of care. The impact of clinical audit on practice could be improved by supporting the ability of people who receive feedback from audits to mount effective responses. This article, written for people who receive feedback, audit providers and quality improvement leads, describes important practices in the organisational response to feedback. The practices were identified through work to describe the current response to national audit and draw upon theory-informed hypotheses for enhancing audit and feedback. Both the content and implementation of the practices were co-designed with stakeholders and tested alongside different national audits. The identified quality improvement practices provide practical guidance for feedback recipients and enable providers and quality improvement leads to consider the capabilities required for such practices. The approach resonates with organisational readiness to change theory, proposing that informational appraisal and change commitment underpin effective improvement actions.

Routine data-based quality indicators for the treatment of gonarthrosis and coxarthrosis patients in the ambulatory care sector - A study protocol for a cluster-randomised pilot trial to evaluate the MobilE-ARTH study

BACKGROUND

In 2019, Germany had the highest rate of hip replacement surgery and the fourth highest rate of knee replacement surgery among more than 30 OECD countries. The age-standardised rates were estimated at 174 hip joint and 137 knee joint replacements per 100,000 population. Against this background, the contrast between financial incentives for surgery and missing incentives for non-surgical treatment options is repeatedly discussed. Quality indicators (QIs) can serve to measure and transparently present the quality of evidence-based care. Comparing results in the form of audit and feedback has been shown to improve e.g. guideline-compliant ambulatory care. Existing QIs targeting the care of gon- and coxarthrosis mainly focus on discharge management after joint replacement surgery and/or require additional data collection. Therefore, as part of the MobilE-ARTH project, a set of QIs for ambulatory care prior to joint replacement surgery calculable based on routine data is being developed. The present study's aim is to evaluate the impact of this QI set in terms of providing feedback on the quality of care.

METHODS

The MobilE-ARTH project comprises (Phase 1) developing a QI set following the RAND/UCLA Appropriateness Method, (Phase 2) implementing the QIs in established physician networks of a German statutory health insurance (SHI) within a prospective, non-blinded, cluster-randomised pilot study, and (Phase 3) evaluating the QI set's effectiveness. The physicians in the intervention networks will (a) receive feedback reports providing information about the routine data-based QIs of their gon- and/or coxarthrosis patients and aggregated results for their network, and (b) be invited to two voluntary, facilitated network meetings. In these network meetings, the physicians can use the information provided on the feedback reports to discuss multiprofessional care pathways for patients with gon- and/or coxarthrosis. Selected indicators of the QI set will serve as primary and secondary outcome measures. Routine data will be analysed within multi-level models using an intention-to-treat approach.

DISCUSSION

Feedback reports help maintaining clinical standards and closing the gap between evidence and medical practice, thus enabling an overall improvement in health care. Providing physicians with QI-based information on quality of care promotes identifying strengths and weaknesses in medical treatments.

TRIAL REGISTRATION

German Clinical Trials Register, number DRKS00027516 , Registered 25th January 2022 - Prospectively registered.

Evaluating implementation strategies to support documentation of veterans' care preferences

OBJECTIVE

To evaluate the effectiveness of feedback reports and feedback reports + external facilitation on completion of life-sustaining treatment (LST) note the template and durable medical orders. This quality improvement program supported the national roll-out of the Veterans Health Administration (VA) LST Decisions Initiative (LSTDI), which aims to ensure that seriously-ill veterans have care goals and LST decisions elicited and documented.

DATA SOURCES

Primary data from national databases for VA nursing homes (called Community Living Centers [CLCs]) from 2018 to 2020.

STUDY DESIGN

In one project, we distributed monthly feedback reports summarizing LST template completion rates to 12 sites as the sole implementation strategy. In the second involving five sites, we distributed similar feedback reports and provided robust external facilitation, which included coaching, education, and learning collaboratives. For each project, principal component analyses matched intervention to comparison sites, and interrupted time series/segmented regression analyses evaluated the differences in LSTDI template completion rates between intervention and comparison sites.

DATA COLLECTION METHODS

Data were extracted from national databases in addition to interviews and surveys in a mixed-methods process evaluation.

PRINCIPAL FINDINGS

LSTDI template completion rose from 0% to about 80% throughout the study period in both projects' intervention and comparison CLCs. There were small but statistically significant differences for feedback reports alone (comparison sites performed better, coefficient estimate 3.48, standard error 0.99 for the difference between groups in change in trend) and feedback reports + external facilitation (intervention sites performed better, coefficient estimate -2.38, standard error 0.72).

CONCLUSIONS

Feedback reports + external facilitation was associated with a small but statistically significant improvement in outcomes compared with comparison sites. The large increases in completion rates are likely due to the well-planned national roll-out of the LSTDI. This finding suggests that when dissemination and support for widespread implementation are present and system-mandated, significant enhancements in the adoption of evidence-based practices may require more intensive support.

Program Evaluation of Implementation Science Outcomes From an Intervention to Improve Compliance With Chlorhexidine Gluconate Bathing: A Qualitative Study

BACKGROUND AND OBJECTIVES

Evaluation of implementation science research is warranted to better understand and determine the success of translating evidence-based infection prevention practices at the bedside. The purpose of this program evaluation was to evaluate implementation outcomes from the perspectives of nurses and nursing leaders regarding a previously conducted chlorhexidine gluconate (CHG) bathing implementation science study among 14 critical care units.

METHODS

Focus groups and interviews, using semistructured interview questions, were conducted to examine the perceptions of nurses who participated in a CHG bathing implementation science study. A deductive qualitative analysis using Proctor and colleagues' implementation outcomes framework was used. Transcripts were analyzed and categorized using the framework as a predetermined code list to structure the implementation outcomes of acceptability, appropriateness, adoption, feasibility, and sustainability.

FINDINGS

A total of 19 nurses and nurse leaders participated in a focus group or interview. Participants noted that both implementation strategies used in the initial study (educational outreach and audit and feedback) were acceptable and appropriate and expressed that the evidence-based CHG bathing practice was feasible to integrate into practice and was being adopted.

DISCUSSION

The program evaluation identified strengths and opportunities for improvement related to the implementation strategies and evidence-based CHG bathing protocol. Findings can inform future studies that seek to implement CHG bathing protocols in the critical care setting using audit and feedback and educational outreach strategies.

Systems analysis and improvement approach to optimize outpatient mental health treatment cascades in Mozambique (SAIA-MH): study protocol for a cluster randomized trial

BACKGROUND

Significant investments are being made to close the mental health (MH) treatment gap, which often exceeds 90% in many low- and middle-income countries (LMICs). However, limited attention has been paid to patient quality of care in nascent and evolving LMIC MH systems. In system assessments across sub-Saharan Africa, MH loss-to-follow-up often exceeds 50% and sub-optimal medication adherence often exceeds 60%. This study aims to fill a gap of evidence-based implementation strategies targeting the optimization of MH treatment cascades in LMICs by testing a low-cost multicomponent implementation strategy integrated into routine government MH care in Mozambique.

METHODS

Using a cluster-randomized trial design, 16 clinics (8 intervention and 8 control) providing primary MH care will be randomized to the Systems Analysis and Improvement Approach for Mental Health (SAIA-MH) or an attentional placebo control. SAIA-MH is a multicomponent implementation strategy blending external facilitation, clinical consultation, and provider team meetings with system-engineering tools in an overall continuous quality improvement framework. Following a 6-month baseline period, intervention facilities will implement the SAIA-MH strategy for a 2-year intensive implementation period, followed by a 1-year sustainment phase. Primary outcomes will be the proportion of all patients diagnosed with a MH condition and receiving pharmaceutical-based treatment who achieve functional improvement, adherence to medication, and retention in MH care. The Consolidated Framework for Implementation Research (CFIR) will be used to assess determinants of implementation success. Specific Aim 1b will include the evaluation of mechanisms of the SAIA-MH strategy using longitudinal structural equation modeling as well as specific aim 2 estimating cost and cost-effectiveness of scaling-up SAIA-MH in Mozambique to provincial and national levels.

DISCUSSION

This study is innovative in being the first, to our knowledge, to test a multicomponent implementation strategy for MH care cascade optimization in LMICs. By design, SAIA-MH is a low-cost strategy to generate contextually relevant solutions to barriers to effective primary MH care, and thus focuses on system improvements that can be sustained over the long term. Since SAIA-MH is integrated into routine government MH service delivery, this pragmatic trial has the potential to inform potential SAIA-MH scale-up in Mozambique and other similar LMICs.

TRIAL REGISTRATION

ClinicalTrials.gov; NCT05103033 ; 11/2/2021.

Developing and applying synergistic multilevel implementation strategies to promote reach of an evidence-based parenting intervention in primary care

Background: This practical implementation report describes a primary care-based group parenting intervention—Child–Adult Relationship Enhancement in Primary Care (PriCARE)—and the approach taken to understand and strengthen the referral process for PriCARE within a pediatric primary care clinic through the deployment of synergistic implementation strategies to promote physician referrals. PriCARE has evidence of effectiveness for reducing child behavior problems, harsh and permissive parenting, and parent stress from three randomized controlled trials (RCTs). The integration of evidence-based parenting interventions into pediatric primary care is a promising means for widespread dissemination. Yet, even when integrated into this setting, the true reach will depend on parents knowing about and attending the intervention. A key factor in this process is the endorsement of and referral to the intervention by the child's pediatrician. Therefore, identifying strategies to improve physician referrals to parenting interventions embedded in primary care is worthy of investigation. Method: Through lessons learned from the RCTs and key informant interviews with stakeholders, we identified barriers and facilitators to physician referrals of eligible parent–child dyads to PriCARE. Based on this data, we selected and implemented five strategies to increase the PriCARE referral rate. We outline the selection process, the postulated synergistic interactions, and the results of these efforts. Conclusions: The following five discrete strategies were implemented: physician reminders, direct advertising to patients, incentives/public recognition, interpersonal patient narratives, and audit and feedback. These discrete strategies were synergistically combined to create a multifaceted approach to improve physician referrals. Following implementation, referrals increased from 13% to 55%. Continued development, application, and evaluation of implementation strategies to promote the uptake of evidence-based parenting interventions into general use in the primary care setting are discussed.

Plain Language Summary

There is strong evidence that parenting interventions are effective at improving child behavioral health outcomes when delivered in coordination with pediatric primary care. However, there is a lack of focus on the implementation, including the screening and referral process, of parenting interventions in the primary care setting. This is contributing to the delay in the scale-up of parenting interventions and to achieving public health impact. To address this gap, we identified barriers and facilitators to physician screening and referrals to a primary care-based parenting intervention, and selected and piloted five synergistic strategies to improve this critical process. This effort successfully increased physician referrals of eligible patients to the intervention from 13% to 55%. This demonstration project may help advance the implementation of evidence-based interventions by providing an example of how to develop and execute multilevel strategies to improve intervention referrals in a local context.

Improving the Development and Implementation of Audit and Feedback Systems to Support Health Care Workers in Limiting Antimicrobial Resistance in the Hospital: Scoping Review

BACKGROUND

For eHealth technologies in general and audit and feedback (AF) systems specifically, integrating interdisciplinary theoretical underpinnings is essential, as it increases the likelihood of achieving desired outcomes by ensuring a fit among eHealth technology, stakeholders, and their context. In addition, reporting on the development and implementation process of AF systems, including substantiations of choices, enables the identification of best practices and accumulation of knowledge across studies but is often not elaborated on in publications.

OBJECTIVE

This scoping review aims to provide insights into the development and implementation strategies for AF systems for a real-world problem that threatens modern health care-antimicrobial resistance-and provide an interdisciplinary conceptual framework that can serve as a checklist and guidance for making informed choices in the development and implementation of future AF systems.

METHODS

A scoping review was conducted by querying PubMed, Scopus, Web of Science, IEEE Xplore Digital Library, and Embase (≥2010) for studies describing either the development or implementation process, or both, of an AF system for antimicrobial resistance or infections in hospitals. Studies reporting only on effectiveness or impact were excluded. A total of 3 independent reviewers performed the study selection, and 2 reviewers constructed the conceptual framework through the axial and selective coding of often-used theories, models, and frameworks (TMFs) from the literature on AF and eHealth development and implementation. Subsequently, the conceptual framework was used for the systematic extraction and interpretation of the studies' descriptions of AF systems and their development and implementation.

RESULTS

The search resulted in 2125 studies that were screened for eligibility, of which 12 (0.56%); 2012-2020) were included. These studies described the development and implementation processes heterogeneously in terms of study aims, study targets, target groups, methods, and theoretical underpinnings. Few studies have explicitly explained how choices for the development and implementation of AF systems were substantiated by the TMFs. The conceptual framework provided insights into what is reported on the development and implementation process and revealed underreported AF system constructs (eg, AF system design; engagement with the AF system; and comparison, goal setting, and action planning) and development and implementation (eg, champions) constructs.

CONCLUSIONS

This scoping review showed the current heterogeneous reporting of AF systems and their development and implementation processes and exemplified how interdisciplinary TMFs can (and should) be balanced in a conceptual framework to capture relevant AF systems and development and implementation constructs. Thereby, it provides a concrete checklist and overall guidance that supports the professionalization and harmonization of AF system development and implementation. For the development and implementation of future AF systems and other eHealth technologies, researchers and health care workers should be supported in selecting and integrating TMFs into their development and implementation process and encouraged to explicitly report on theoretical underpinnings and the substantiation of choices.

Systems-level audit and feedback interventions to improve oncology care: a scoping review

Audit and feedback is commonly used as a method of both monitoring and attempting to improve the quality of healthcare. No review has examined the literature on systems-level audit and feedback strategies to improve the quality of oncology care. This scoping review examines the number, care focus (technical, nontechnical, or both) and methodological quality of published intervention studies which have used systems-level audit and feedback intended to improve the quality of care delivered in oncology treatment centers. Medline, Embase, PsycINFO, and the Cochrane database were searched, from inception to March 2021, for intervention studies which examined the effectiveness of systems-level audit and feedback in improving care for cancer patients. Studies which met the Effective Practice and Organization of Care (EPOC) minimum design criteria were then assessed using the EPOC risk of bias tool. Study characteristics and outcomes were extracted for those meeting methodological criteria. A narrative approach was used to synthesize the results. A total of 32 intervention studies met the inclusion criteria, of which 53% focused on technical aspects of care, 31% focused on nontechnical and 16% focused on both. Four of the included 32 studies met the EPOC minimum design criteria (13%). Most studies had a before-after study design (75%; n = 24) and methodological quality of the final four studies was moderate. Audit and feedback studies involving oncology treatment centers have primarily focused on technical care aspects. The low number and moderate methodological quality of the studies make it difficult to draw clear inferences about the effectiveness of systems-level audit and feedback. Furthermore, high-quality audit and feedback interventions are required across technical and nontechnical aspects of care to quantify the effectiveness of strategies for improving cancer care and ensure healthcare resources are being optimized.

Implementation and Evaluation of a Digitally Enabled Precision Public Health Intervention to Reduce Inappropriate Gabapentinoid Prescription: Cluster Randomized Controlled Trial

Background

Digital technologies can enable rapid targeted delivery of audit and feedback interventions at scale. Few studies have evaluated how mode of delivery affects clinical professional behavior change and none have assessed the feasibility of such an initiative at a national scale.

Objective

The aim of this study was to develop and evaluate the effect of audit and feedback by digital versus postal (letter) mode of delivery on primary care physician behavior.

Methods

This study was developed as part of the Veterans’ Medicines Advice and Therapeutics Education Services (MATES) program, an intervention funded by the Australian Government Department of Veterans’ Affairs that provides targeted education and patient-specific audit with feedback to Australian general practitioners, as well as educational material to veterans and other health professionals. We performed a cluster randomized controlled trial of a multifaceted intervention to reduce inappropriate gabapentinoid prescription, comparing digital and postal mode of delivery. All veteran patients targeted also received an educational intervention (postal delivery). Efficacy was measured using a linear mixed-effects model as the average number of gabapentinoid prescriptions standardized by defined daily dose (individual level), and number of veterans visiting a psychologist in the 6 and 12 months following the intervention.

Results

The trial involved 2552 general practitioners in Australia and took place in March 2020. Both intervention groups had a significant reduction in total gabapentinoid prescription by the end of the study period (digital: mean reduction of 11.2%, P=.004; postal: mean reduction of 11.2%, P=.001). We found no difference between digital and postal mode of delivery in reduction of gabapentinoid prescriptions at 12 months (digital: –0.058, postal: –0.058, P=.98). Digital delivery increased initiations to psychologists at 12 months (digital: 3.8%, postal: 2.0%, P=.02).

Conclusions

Our digitally delivered professional behavior change intervention was feasible, had comparable effectiveness to the postal intervention with regard to changes in medicine use, and had increased effectiveness with regard to referrals to a psychologist. Given the logistical benefits of digital delivery in nationwide programs, the results encourage exploration of this mode in future interventions.

Rethinking Our Annual Congress-Meeting the Needs of Specialist Physicians by Partnering With Provincial Simulation Centers

Canada's maintenance of certification programs for physicians has evolved to emphasize assessment activities. Our organization recognized the importance of offering more practice assessment opportunities to our members to enhance their practice and help them comply with a regulation from our provincial professional body related to ongoing continuing education. This led us to rethink our annual congress and enrich the program with a curriculum of interdisciplinary simulation sessions tailored to meet the needs of a broad audience of specialists. Our challenges are similar to those of many national specialty societies having limited access to simulation facilities, instructors, and simulation teams that can cover the breadth and scope of perceived and unperceived simulation needs for their specialty. Our innovative solution was to partner with local experts to develop 22 simulation sessions over the past three years. The response was very positive, drawing 867 participants. Over 95% of participants either agreed or strongly agreed that their simulation session (1) met their learning objectives, (2) was relevant for their practice, and (3) encouraged them to modify their practice. Narrative comments from a survey sent to the 2018 participants four months after their activity indicated several self-reported changes in their practice or patient outcomes. We were able to centralize offers from organizations that had previously worked in silo to develop simulation sessions meeting the needs of our members. Proposing simulation sessions allowed our organization to establish long-term partnerships and to expend our "educational toolbox" to address skill gaps not usually addressed during annual meetings.

OUP accepted manuscript

Objectives

(1) Systematically review the literature on computerized audit and feedback (e-A&F) systems in healthcare. (2) Compare features of current systems against e-A&F best practices. (3) Generate hypotheses on how e-A&F systems may impact patient care and outcomes.

Methods

We searched MEDLINE (Ovid), EMBASE (Ovid), and CINAHL (Ebsco) databases to December 31, 2020. Two reviewers independently performed selection, extraction, and quality appraisal (Mixed Methods Appraisal Tool). System features were compared with 18 best practices derived from Clinical Performance Feedback Intervention Theory. We then used realist concepts to generate hypotheses on mechanisms of e-A&F impact. Results are reported in accordance with the PRISMA statement.

Results

Our search yielded 4301 unique articles. We included 88 studies evaluating 65 e-A&F systems, spanning a diverse range of clinical areas, including medical, surgical, general practice, etc. Systems adopted a median of 8 best practices (interquartile range 6–10), with 32 systems providing near real-time feedback data and 20 systems incorporating action planning. High-confidence hypotheses suggested that favorable e-A&F systems prompted specific actions, particularly enabled by timely and role-specific feedback (including patient lists and individual performance data) and embedded action plans, in order to improve system usage, care quality, and patient outcomes.

Conclusions

e-A&F systems continue to be developed for many clinical applications. Yet, several systems still lack basic features recommended by best practice, such as timely feedback and action planning. Systems should focus on actionability, by providing real-time data for feedback that is specific to user roles, with embedded action plans.

Protocol Registration

PROSPERO CRD42016048695.

Using a dark logic model to explore adverse effects in audit and feedback: a qualitative study of gaming in colonoscopy

BACKGROUND

Audit and feedback (A&F) interventions improve patient care but may result in unintended consequences. To evaluate plausible harms and maximise benefits, theorisation using logic models can be useful. We aimed to explore the adverse effects of colonoscopy A&F using a feedback intervention theory (FIT) dark logic model before the National Endoscopy Database Automated Performance Reports to Improve Quality Outcomes Trial study.

METHODS

We undertook a qualitative study exploring A&F practices in colonoscopy. Interviews were undertaken with endoscopists from six English National Health Service endoscopy centres, purposively sampled for professional background and experience. A thematic framework analysis was performed, mapping paradoxical effects and harms using FIT and the theory of planned behaviour.

RESULTS

Data saturation was achieved on the 19th participant, with participants from nursing, surgical and medical backgrounds and a median of 7 years' experience.When performance was below aspirational targets participants were falsely reassured by social comparisons. Participants described confidence as a requirement for colonoscopy. Negative feedback without a plan to improve risked reducing confidence and impeding performance (cognitive interference). Unmet targets increased anxiety and prompted participants to question messages' motives and consider gaming.Participants described inaccurate documentation of subjective measures, including patient comfort, to achieve targets perceived as important. Participants described causing harm from persevering to complete procedures despite patient discomfort and removing insignificant polyps to improve detection rates without benefiting the patient.

CONCLUSION

Our dark logic model highlighted that A&F interventions may create both desired and adverse effects. Without a priori theorisation evaluations may disregard potential harms. In colonoscopy, improved patient experience measures may reduce harm. To address cognitive interference the motivation of feedback to support improvement should always be clear, with plans targeting specific behaviours and offering face-to-face support for confidence.

TRIAL REGISTRATION NUMBER

ISRCTN11126923.

Multi-faceted strategies improve collection compliance and sample acceptance rate for carbapenem-resistant Enterobacteriaceae (CRE) active surveillance testing

BACKGROUND

Active surveillance testing (AST) is one element of a comprehensive Carbapenem-resistant Enterobacteriaceae (CRE) prevention strategy. However, the utility of AST may be impacted by compliance with sample collection and the quality of specimens. Here, we describe strategies used to optimize a CRE AST program at a large academic medical center.

METHODS

Tests ordered, collected, rejected, and processed were tracked weekly for each participating unit. Sample collection compliance and acceptance rates were calculated and tracked weekly. Strategies were implemented to improve collection compliance and sample acceptance rates, including computerized provider order entry, printed educational materials, and audit and feedback. Weekly dedicated Infection Preventionist (IP) time was estimated.

RESULTS

Over 35 months, mean collection compliance increased from 82.7% to 91.2%, and then decreased to 86.2%. Over 27 months, sample acceptance rate increased from 57.7% to 83.6%, and then remained stable at 83.4%. Over 39 months, dedicated weekly IP time decreased 92%.

DISCUSSION

Use of evidence-based quality improvement strategies optimized our CRE AST program. Optimizing the AST process aids in early CRE detection, leading to timely isolation and preventing the spread of CRE to other patients. Other hospitals may benefit from these lessons and enhance local AST programs.

The effect of electronic audits and feedback in primary care and factors that contribute to their effectiveness: a systematic review

PURPOSE

The aim of this systematic review was (i) to assess whether electronic audit and feedback (A&F) is effective in primary care and (ii) to evaluate important features concerning content and delivery of the feedback in primary care, including the use of benchmarks, the frequency of feedback, the cognitive load of feedback and the evidence-based aspects of the feedback.

DATA SOURCES

The MEDLINE, Embase, CINAHL and CENTRAL databases were searched for articles published since 2010 by replicating the search strategy used in the last Cochrane review on A&F.

STUDY SELECTION

Two independent reviewers assessed the records for their eligibility, performed the data extraction and evaluated the risk of bias. Our search resulted in 8744 records, including the 140 randomized controlled trials (RCTs) from the last Cochrane Review. The full texts of 431 articles were assessed to determine their eligibility. Finally, 29 articles were included.

DATA EXTRACTION

Two independent reviewers extracted standard data, data on the effectiveness and outcomes of the interventions, data on the kind of electronic feedback (static versus interactive) and data on the aforementioned feedback features.

RESULTS OF DATA SYNTHESIS

Twenty-two studies (76%) showed that electronic A&F was effective. All interventions targeting medication safety, preventive medicine, cholesterol management and depression showed an effect. Approximately 70% of the included studies used benchmarks and high-quality evidence in the content of the feedback. In almost half of the studies, the cognitive load of feedback was not reported. Due to high heterogeneity in the results, no meta-analysis was performed.

CONCLUSION

This systematic review included 29 articles examining electronic A&F interventions in primary care, and 76% of the interventions were effective. Our findings suggest electronic A&F is effective in primary care for different conditions such as medication safety and preventive medicine. Some of the benefits of electronic A&F include its scalability and the potential to be cost effective. The use of benchmarks as comparators and feedback based on high-quality evidence are widely used and important features of electronic feedback in primary care. However, other important features such as the cognitive load of feedback and the frequency of feedback provision are poorly described in the design of many electronic A&F intervention, indicating that a better description or implementation of these features is needed. Developing a framework or methodology for automated A&F interventions in primary care could be useful for future research.

Collaborative care for depression management in primary care: A randomized roll-out trial using a type 2 hybrid effectiveness-implementation design

Background

The Collaborative Care Model (CoCM) is a well-established treatment for depression in primary care settings. The critical drivers and specific strategies for improving implementation and sustainment are largely unknown. Rigorous pragmatic research is needed to understand CoCM implementation processes and outcomes.

Methods

This study is a hybrid Type 2 randomized roll-out effectiveness-implementation trial of CoCM in 11 primary care practices affiliated with an academic medical center. The Collaborative Behavioral Health Program (CBHP) was developed as a means of improving access to effective mental health services for depression. Implementation strategies are provided to all practices. Using a sequential mixed methods approach, we will assess key stakeholders’ perspectives on barriers and facilitators of implementation and sustainability of CBHP. The speed and quantity of implementation activities completed over a 30-month period for each practice will be assessed. Economic analyses will be conducted to determine the budget impact and cost offset of CBHP in the healthcare system. We hypothesize that CBHP will be effective in reducing depressive symptoms and spillover effects on chronic health conditions. We will also examine differential outcomes among racial/ethnic minority patients.

Discussion

This study will elucidate critical drivers of successful CoCM implementation. It will be among the first to conduct economic analyses on a fee-for-service model utilizing billing codes for CoCM. Data may inform ways to improve implementation efficiency with an optimization approach to successive practices due to the roll-out design. Changes to the protocol and current status of the study are discussed.

Incentives for physician teams: Effectiveness of performance feedback and payment distribution methods

Best practices in team-based incentive design remain underexplored. This study examines under group-based pay-for-performance, how managers incentivize physicians for teamwork through internal feedback and payment distribution methods. In collaboration with Taiwan Association of Family Medicine, authors conducted a national survey of physician groups, with a response rate of 48.3%. Multilevel linear regression was applied to 134 groups, collectively consisting of 1,245 physicians in Taiwan. The outcome variables were two manager-rated scores for group performance on achieving (a) comprehensive, coordinated, continuous care, and (b) patient health improvement. The results indicate that providing each physician feedback on peer performance is superior to not providing it; when providing peer information within a group, concealing identities is superior to revealing them. These findings imply that application of the principle of social comparison can be effective; however, caution should be taken when disclosure of identifiable peer performance may intensify peer competition and undermine care coordination in team-based models. Further, groups that distribute payments equally among physicians perform better than groups that distributed payment proportionally to physicians' patient shares. The findings are germane to small teams, where physicians do not have full control over care processes and outcomes, and need to work cooperatively to maximize group-based payment.

Protocol for a hybrid type 3 cluster randomized trial of a technical assistance system supporting coalitions and evidence-based drug prevention programs

BACKGROUND

Over 5000 community anti-drug coalitions operating in the USA serve as a cornerstone of federal drug prevention. These coalitions, however, have demonstrated effectiveness in preventing substance use only when they use technical assistance (TA) and implement evidence-based programs (EBPs). The absence of TA and EBP implementation by coalitions is a key research-to-practice gap. The Coalition Check-Up TA system is designed to fill this gap by supporting community coalition implementation of EBPs. Existing TA models for evidence-based coalition approaches are resource intensive and coalition model specific. The Coalition Check-Up is a lower cost strategy that works with a variety of types of coalitions to support sustainable implementation of EBPs. This study protocol describes a hybrid type 3 effectiveness-implementation trial applying Wandersman's Interactive Systems Framework to test the effects of the Coalition Check-Up on coalition EBP implementation capacity and outcomes. The Interactive Systems Framework outlines how the prevention support system-especially TA-bolsters EBP dissemination and implementation.

METHODS

Using a cluster randomized controlled design, this trial will test the overall effectiveness of the Coalition Check-Up, including how it contributes to EBP implementation and prevention of youth substance use. The first aim is to estimate the impact of the Coalition Check-Up on coalitions' capacity to do their work. We will recruit 68 anti-drug coalitions for random assignment to the Coalition Check-Up or "TA as usual" condition. We will evaluate whether the Coalition Check-Up improves coalition capacity using measures of coalition member responses about team processes, coalition network composition, and collaborative structure. Our second aim is to estimate the impact of the Coalition Check-Up on implementation of EBPs, and our third aim is to estimate the impact of the Coalition Check-Up on youth substance use.

DISCUSSION

This project will clarify how the Coalition Check-Up, a scalable approach to TA due to its low cost, affects coalition capacity to support EBP implementation. Analyses also provide insight into causal pathways from the prevention support system to the prevention delivery system outlined by the Interactive Systems Framework. Results will build the evidence-base for how to support community coalitions' sustainable implementation of evidence-based prevention programs and policies.

TRIAL REGISTRATION

Clinicaltrials.gov registration number NCT04592120 . Registered on October 19, 2020.

Interactive and Participatory Audit and Feedback (IPAF): theory-based development and multi-site implementation outcomes with specialty clinic staff

BACKGROUND

Theory-based implementation strategies, such as audit and feedback (A&F), can improve the adoption of evidence-based practices. However, few strategies have been developed and tested to meet the needs of specialty clinics. In particular, frontline staff can execute cardiovascular disease (CVD) risk reduction protocols, but A&F strategies to support them are not well examined. Our objective was to develop and evaluate a theory-based approach to A&F, Interactive and Participatory A&F (IPAF).

METHODS

We developed IPAF informed by two complementary theories, self-regulation theory (SRT) and self-determination theory (SDT). IPAF applies concepts from these theories to inform (1) what to address with staff to improve rates of best practices (SRT) and (2) how to interact with staff to improve behaviors aligned with best practices (SDT). We promoted IPAF fidelity by developing a semi-structured guide to facilitate staff discussion of target behaviors, perceived barriers, goals, and action plans. We evaluated IPAF in the context of eight quasi-experimental implementations in specialty clinics across two health systems. Following a hybrid type 2 effectiveness-implementation design, we reported intervention outcomes for CVD risk reduction elsewhere. This paper reports implementation outcomes associated with IPAF, focusing on feasibility, appropriateness, acceptability, fidelity, and adoption. We evaluated implementation outcomes using mixed-methods data including electronic health record (EHR) data, team records, and staff questionnaire responses.

RESULTS

Eighteen staff participated in 99 monthly, individual, synchronous (face-to-face or phone) IPAF sessions during the first 6 months of implementation. Subsequently, we provided over 375 monthly feedback emails. Feasibility data revealed high staff attendance (90-93%) and engagement in IPAF sessions. Staff highly rated questionnaire items about IPAF acceptability. Team records and staff responses demonstrated fidelity of IPAF delivery and receipt. Adoption of target behaviors increased significantly (all P values < 0.05), and adoption or behaviors were maintained for over 24 months.

CONCLUSIONS

We developed and evaluated a theory-based approach to A&F with frontline staff in specialty clinics to improve the implementation of evidence-based interventions. The findings support feasibility, appropriateness, acceptability, and fidelity of IPAF, and staff adoption and maintenance of target behaviors. By evaluating multi-site implementation outcomes, we extended prior research on clinic protocols and A&F beyond primary care settings and providers.

The effect of automated audit and feedback on data completeness in the electronic health record of the general physician: protocol for a cluster randomized controlled trial

Background

The electronic health record (EHR) of the general physician (GP) is an important tool that can be used to assess and improve the quality of healthcare. However, there are some problems when (re) using the data gathered in the EHR for quality assessments. One problem is the lack of data completeness in the EHR. Audit and feedback (A&F) is a well-known quality intervention that can improve the quality of healthcare. We hypothesize that an automated A&F intervention can be adapted to improve the data completeness of the EHR of the GP, more specifically, the number of correctly registered diagnoses of type 2 diabetes and chronic kidney disease.

Methods

This study is a pragmatic cluster randomized controlled trial with an intervention at the level of GP practice. The intervention consists of an audit and extended electronically delivered feedback with multiple components that will be delivered 4 times electronically to general practices over 12 months. The data will be analyzed on an aggregated level (per GP practice). The primary outcome is the percentage of correctly registered diagnoses of type 2 diabetes. The key secondary outcome is the registration of chronic kidney disease. Exploratory secondary outcomes are the registration of heart failure, biometric data and lifestyle habits, and the evolution of 4 different EHR-extractable quality indicators.

Discussion

This cluster randomized controlled trial intends to primarily improve the registration of type 2 diabetes in the EHR of the GP and to secondarily improve the registration of chronic kidney disease. In addition, the registration of heart failure, lifestyle parameters, and biometric data in the EHR of the GP are explored together with 4 EHR-extractable quality indicators. By doing so, this study aims to improve the data completeness of the EHR, paving the way for future quality assessments.

Trial registration

ClinicalTrials.gov NCT04388228. Registered on May 14, 2020.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13063-021-05259-9.

Implementation support for contingency management: preferences of opioid treatment program leaders and staff

BACKGROUND

Contingency management (CM), a behavioral intervention that provides incentives for achieving treatment goals, is an evidence-based adjunct to medication to treat opioid use disorder. Unfortunately, many front-line treatment providers do not utilize CM, likely due to contextual barriers that limit effective training and ongoing support for evidence-based practices. This study applied user-informed approaches to adapt a multi-level implementation strategy called the Science to Service Laboratory (SSL) to support CM implementation.

METHODS

Leaders and treatment providers working in community-based opioid treatment programs (OTPs; N = 43) completed qualitative interviews inquiring about their preferences for training and support implementation strategies (didactic training, performance feedback, and external facilitation). Our team coded interviews using a reflexive team approach to identify common a priori and emergent themes.

RESULTS

Leaders and providers expressed a preference for brief training that included case examples and research data, along with experiential learning strategies. They reported a desire for performance feedback from internal supervisors, patients, and clinical experts. Providers and leaders had mixed feelings about audio-recording sessions but were open to the use of rating sheets to evaluate CM performance. Finally, participants desired both on-call and regularly scheduled external facilitation to support their continued use of CM.

CONCLUSIONS

This study provides an exemplar of a user-informed approach to adapt the SSL implementation support strategies for CM scale-up in community OTPs. Study findings highlight the need for user-informed approaches to training, performance feedback, and facilitation to support sustained CM use in this setting.