Champion and audit and feedback strategy fidelity and their relationship to depression intervention fidelity: A mixed method study

A research agenda to advance the study of implementation mechanisms

BACKGROUND

Implementation science scholars have made significant progress identifying factors that enable or obstruct the implementation of evidence-based interventions, and testing strategies that may modify those factors. However, little research sheds light on how or why strategies work, in what contexts, and for whom. Studying implementation mechanisms-the processes responsible for change-is crucial for advancing the field of implementation science and enhancing its value in facilitating equitable policy and practice change. The Agency for Healthcare Research and Quality funded a conference series to achieve two aims: (1) develop a research agenda on implementation mechanisms, and (2) actively disseminate the research agenda to research, policy, and practice audiences. This article presents the resulting research agenda, including priorities and actions to encourage its execution.

METHOD

Building on prior concept mapping work, in a semi-structured, 3-day, in-person working meeting, 23 US-based researchers used a modified nominal group process to generate priorities and actions for addressing challenges to studying implementation mechanisms. During each of the three 120-min sessions, small groups responded to the prompt: "What actions need to be taken to move this research forward?" The groups brainstormed actions, which were then shared with the full group and discussed with the support of facilitators trained in structured group processes. Facilitators grouped critical and novel ideas into themes. Attendees voted on six themes they prioritized to discuss in a fourth, 120-min session, during which small groups operationalized prioritized actions. Subsequently, all ideas were collated, combined, and revised for clarity by a subset of the authorship team.

RESULTS

From this multistep process, 150 actions emerged across 10 priority areas, which together constitute the research agenda. Actions included discrete activities, projects, or products, and ways to shift how research is conducted to strengthen the study of implementation mechanisms.

CONCLUSIONS

This research agenda elevates actions to guide the selection, design, and evaluation of implementation mechanisms. By delineating recommended actions to address the challenges of studying implementation mechanisms, this research agenda facilitates expanding the field of implementation science, beyond studying what works to how and why strategies work, in what contexts, for whom, and with which interventions.

Utility of the Leadership and Organizational Change for Implementation-Systems Level (LOCI-SL) strategy for a statewide substance use treatment implementation effort

INTRODUCTION

Multi-level and cross-context implementation strategies are needed to support health systems, healthcare delivery organizations, and providers to adopt evidence-based practice (EBP) for substance use disorder (SUD) treatment. However, misalignment between state oversight agencies and healthcare organizations about which services to prioritize and which outcomes are reasonable to expect can hinder implementation success and widespread access to high-quality care. This study investigated the utility of the Leadership and Organizational Change for Implementation-System Level (LOCI-SL) strategy for supporting statewide EBP implementation for SUD treatment.

METHODS

Nine community mental health centers (CMHCs) contracted by a state agency participated in a combined motivational-enhancement therapy and cognitive behavioral therapy (MET/CBT) implementation effort. Five of the CMHCs also received the LOCI-SL strategy to obtain ongoing implementation support. We conducted 21 individual interviews and three small group interviews with 30 participants across CMHCs and state health agencies to investigate the utility of LOCI-SL in supporting their EBP implementation efforts. Deductive thematic analysis was guided by the Exploration, Preparation, Implementation, Sustainment Framework.

RESULTS

Five themes described CMHCs' LOCI-SL and broader contextual experiences implementing EBPs: (1) LOCI-SL supported executives in Preparation phase activities that holistically considered organizational needs and capacity to implement and sustain EBPs; (2) LOCI-SL facilitated trust and communication processes across Preparation, Implementation, and Sustainment phases to improve EBP uptake; (3) LOCI-SL increased CMHCs' use of implementation climate strengthening activities throughout the Implementation phase; (4) state contracts did not emphasize quality and thus were not sufficient bridging factors to enforce EBP fidelity during Implementation; and, (5) limited funding and low Medicaid reimbursement rates hindered EBP use throughout the Implementation and Sustainment phases.

CONCLUSIONS

LOCI-SL was viewed as a favorable and useful implementation strategy for supporting statewide adoption of EBPs. However, outer context barriers, including limited financial investments in the treatment system, impeded implementation and sustainment efforts. While previous research suggests that contracts are viable alignment-promoting bridging factors, this study demonstrates the importance of articulating implementation outcome expectations to aid state-contracted organizations in achieving EBP implementation success. This study also highlights the need for multi-level implementation strategies to effectively align implementation expectations between outer- and inner-context entities.

Fidelity and acceptability of implementation strategies developed for adherence to a clinical pathway for screening, assessment and management of anxiety and depression in adults with cancer

BACKGROUND

Implementation strategies are crucial to facilitate implementation success. To prepare and support implementation of a clinical pathway for screening, assessment and management of anxiety and depression in cancer patients (the ADAPT CP), six broad categories of implementation strategies; (1) Awareness campaigns, (2) Champions, (3) Education, (4) Academic Detailing and Support, (5) Reporting, (6) Technological Support, were developed. The aim of this paper is to describe the fidelity and acceptability of six categories of implementation strategies and any subsequent changes/adaptations made to those strategies.

METHODS

The ADAPT CP was implemented in twelve cancer services in NSW, Australia, as part of a cluster randomised controlled trial of core versus enhanced implementation strategies. Fidelity to and any subsequent changes to the delivery of the planned six categories of implementation strategies were captured using the ADAPT contact log, which recorded the contacts made between the ADAPT research team and services, engagement meetings and monthly meetings. To explore acceptability and awareness/engagement with the implementation strategies, interviews with a purposively selected staff sample across both study arms were held prior to implementation (T0), six months into implementation (T1) and at the end of the 12-month implementation period (T2). Interviews were thematically analysed across the six categories of strategies.

RESULTS

Delivery of all six categories of implementation strategies as planned was moderated by service context and resources and staff engagement. As such, for some implementation strategies, subsequent changes or adaptations to the content, mode of delivery, frequency and duration such as abbreviated training sessions, were made to optimise fidelity to and engagement with the strategies. Most strategies were perceived to be acceptable by service staff. Use of strategies prior to implementation of the ADAPT CP such as the engagement meetings and training sessions, positively impacted on ownership and preparedness to implement the ADAPT CP. Furthermore, ongoing support such as provision of additional training or monthly meetings facilitated increased awareness and engagement with the ADAPT program.

CONCLUSION

Flexibility in delivering implementation strategies, and ensuring staff engagement with, and acceptability of those strategies, can support implementation of interventions within healthcare settings.

TRIAL REGISTRATION

The ADAPT CRCT was registered prospectively with the ANZCTR on 22/3/2017. Trial ID ACTRN12617000411347.  https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=372486&isReview=true.

Two implementation strategies to support the integration of depression screening and treatment into hypertension and diabetes care in Malawi (SHARP): parallel, cluster-randomised, controlled, implementation trial

BACKGROUND

Although evidence-based treatments for depression in low-resource settings are established, implementation strategies to scale up these treatments remain poorly defined. We aimed to compare two implementation strategies in achieving high-quality integration of depression care into chronic medical care and improving mental health outcomes in patients with hypertension and diabetes.

METHODS

We conducted a parallel, cluster-randomised, controlled, implementation trial in ten health facilities across Malawi. Facilities were randomised (1:1) by covariate-constrained randomisation to either an internal champion alone (ie, basic strategy group) or an internal champion plus external supervision with audit and feedback (ie, enhanced strategy group). Champions integrated a three-element, evidence-based intervention into clinical care: universal depression screening; peer-delivered psychosocial counselling; and algorithm-guided, non-specialist antidepressant management. External supervision involved structured facility visits by Ministry officials and clinical experts to assess quality of care and provide supportive feedback approximately every 4 months. Eligible participants were adults (aged 18-65 years) seeking hypertension and diabetes care with signs of depression (Patient Health Questionnaire-9 score ≥5). Primary implementation outcomes were depression screening fidelity, treatment initiation fidelity, and follow-up treatment fidelity over the first 3 months of treatment, analysed by intention to treat. This trial is registered with ClinicalTrials.gov, NCT03711786, and is complete.

FINDINGS

Five (50%) facilities were randomised to the basic strategy and five (50%) to the enhanced strategy. Between Oct 1, 2019, and Nov 30, 2021, in the basic group, 587 patients were assessed for eligibility, of whom 301 were enrolled; in the enhanced group, 539 patients were assessed, of whom 288 were enrolled. All clinics integrated the evidence-based intervention and were included in the analyses. Of 60 774 screening-eligible visits, screening fidelity was moderate (58% in the enhanced group vs 53% in the basic group; probability difference 5% [95% CI -38% to 47%]; p=0·84) and treatment initiation fidelity was high (99% vs 98%; 0% [-3% to 3%]; p=0·89) in both groups. However, treatment follow-up fidelity was substantially higher in the enhanced group than in the basic group (82% vs 20%; 62% [36% to 89%]; p=0·0020). Depression remission was higher in the enhanced group than in the basic group (55% vs 36%; 19% [3% to 34%]; p=0·045). Serious adverse events were nine deaths (five in the basic group and four in the enhanced group) and 26 hospitalisations (20 in the basic group and six in the enhanced group); none were treatment-related.

INTERPRETATION

The enhanced implementation strategy led to an increase in fidelity in providers' follow-up treatment actions and in rates of depression remission, consistent with the literature that follow-up decisions are crucial to improving depression outcomes in integrated care models. These findings suggest that external supervision combined with an internal champion could offer an important advance in integrating depression treatment into general medical care in low-resource settings.

FUNDING

The National Institute of Mental Health.

Implementation strategies to build mental health-care capacity in Malawi: a health-economic evaluation

BACKGROUND

Depression is a major contributor to morbidity and mortality in sub-Saharan Africa. Due to low system capacity, three in four patients with depression in sub-Saharan Africa go untreated. Despite this, little attention has been paid to the cost-effectiveness of implementation strategies to scale up evidence-based depression treatment in the region. In this study, we investigate the cost-effectiveness of two different implementation strategies to integrate the Friendship Bench approach and measurement-based care in non-communicable disease clinics in Malawi.

METHODS

The two implementation strategies tested in this study are part of a trial, in which ten clinics were randomly assigned (1:1) to a basic implementation package consisting of an internal coordinator acting as a champion (IC-only group) or to an enhanced package that complemented the basic package with quarterly external supervision, and audit and feedback of intervention delivery (IC + ES group). We included material costs, training costs, costs related to project-wide meetings, transportation and medication costs, time costs related to internal champion activities and depression screening or treatment, and costs of external supervision visits if applicable. Outcomes included the number of patients screened with the patient health questionnaire 2 (PHQ-2), cases of remitted depression at 3 and 12 months, and disability-adjusted life-years (DALYs) averted. We compared the cost-effectiveness of both packages to the status quo (ie, no intervention) using a micro-costing-informed decision-tree model.

FINDINGS

Relative to the status quo, IC + ES would be on average US$10 387 ($1349-$17 365) more expensive than IC-only but more effective in achieving remission and averting DALYs. The cost per additional remission would also be lower with IC + ES than IC-only at 3 months ($119 vs $223) and 12 months ($210 for IC + ES; IC-only dominated by the status quo at 12 months). Neither package would be cost-effective under the willingness-to-pay threshold of $65 per DALY averted currently used by the Malawian Ministry of Health. However, the IC + ES package would be cost-effective in relation to the commonly used threshold of three times per-capita gross domestic product per DALY averted.

INTERPRETATION

Investing in supporting champions might be an appropriate use of resources. Although not currently cost-effective by Malawian willingness-to-pay standards compared with the status quo, the IC + ES package would probably be a cost-effective way to build mental health-care capacity in resource-constrained settings in which decision makers use higher willingness-to-pay thresholds.

FUNDING

National Institute of Mental Health.

Provider Use of, Attitudes Towards, and Self-efficacy with Key Measurement-based Care Practices in Youth Mental Health Treatment: A Multi-site Examination

Measurement-based care (MBC) is an underutilized evidence-based practice, and current implementation efforts demonstrate limited success in increasing MBC use. A better understanding of MBC implementation determinants is needed to improve these efforts, particularly from studies examining the full range of MBC practices and that span multiple samples of diverse providers using different MBC systems. This study addressed these limitations by conducting a multi-site survey examining MBC predictors and use in youth treatment. Participants were 159 clinicians and care coordinators working in youth mental health care settings across the United States. Participants were drawn from three program evaluations of MBC implementation. Providers completed measures assessing use of five MBC practices (administering measures, viewing feedback, reviewing feedback in supervision, sharing feedback with clients in session, and using feedback to plan treatment), MBC self-efficacy, and MBC attitudes. Despite expectations that MBC should be standard care for all clients, providers reported only administering measures to 40-60% of clients on average, with practices related to the use of feedback falling in the 1-39% range. Higher MBC self-efficacy and more positive views of MBC practicality predicted higher MBC use, although other attitude measures were not significant predictors. Effects of predictors were not moderated by site, suggesting consistent predictors across implementation settings. Implications of study findings for future research and for the implementation of MBC are discussed.

Improving integrated depression and non-communicable disease care in Malawi through engaged leadership and supportive implementation climate

BACKGROUND

Low- and middle-income countries often lack access to mental health services, leading to calls for integration within other primary care systems. In sub-Saharan Africa, integration of depression treatment in non-communicable disease (NCD) settings is feasible, acceptable, and effective. However, leadership and implementation climate challenges often hinder effective integration and quality of services. The aim of this study was to identify discrete leadership strategies that facilitate overcoming barriers to the integration of depression care in NCD clinics in Malawi and to understand how clinic leadership shapes the implementation climate.

METHODS

We conducted 39 in-depth interviews with the District Medical Officer, the NCD coordinator, one NCD provider, and the research assistant from each of the ten Malawian NCD clinics (note one District Medical Officer served two clinics). Based on semi-structured interview guides, participants were asked their perspectives on the impact of leadership and implementation climate on overcoming barriers to integrating depression care into existing NCD services. Thematic analysis used both inductive and deductive approaches to identify emerging themes and compare among participant type.

RESULTS

The results revealed how engaged leadership can fuel a positive implementation climate where clinics had heightened capacity to overcome implementation barriers. Effective leaders were approachable and engaged in daily operations of the clinic and problem-solving. They held direct involvement with and mentorship during the intervention, providing assistance in patient screening and consultation with treatment plans. Different levels of leadership utilized their respective standings and power dynamics to influence provider attitudes and perceptions surrounding the intervention. Leaders acted by informing providers about the intervention source and educating them on the importance of mental healthcare, as it was often undervalued. Lastly, they prioritized teamwork and collective ownership for the intervention, increasing provider responsibility.

CONCLUSION

Training that prioritizes leadership visibility and open communication will facilitate ongoing Malawi Ministry of Health efforts to scale up evidence-based depression treatment within NCD clinics. This proves useful where extensive and external monitoring may be limited. Ultimately, these results can inform successful strategies to close implementation gaps to achieve integration of mental health services in low-resource settings through improved leadership and implementation climate.

TRIAL REGISTRATION

These findings are reported from ClinicalTrials.gov, NCT03711786. Registered on 18/10/2018. https://clinicaltrials.gov/ct2/show/NCT03711786 .