Skills for Developing and Maintaining Community-Partnerships for Dissemination and Implementation Research in Children's Behavioral Health: Implications for Research Infrastructure and Training of Early Career Investigators

Mis-implementation of evidence-based behavioural health practices in primary care: lessons from randomised trials in Federally Qualified Health Centers

Background

Implementing evidence-based practices (EBPs) within service systems is critical to population-level health improvements - but also challenging, especially for complex behavioral health interventions in low-resource settings. "Mis-implementation" refers to poor outcomes from an EBP implementation effort; mis-implementation outcomes are an important, but largely untapped, source of information about how to improve knowledge exchange.

Aims and objectives

We present mis-implementation cases from three pragmatic trials of behavioral health EBPs in U.S. Federally Qualified Health Centers (FQHCs).

Methods

We adapted the Consolidated Framework for Implementation Research and its Outcomes Addendum into a framework for mis-implementation and used it to structure the case summaries with information about the EBP and trial, mis-implementation outcomes, and associated determinants (barriers and facilitators). We compared the three cases to identify shared and unique mis-implementation factors.

Findings

Across cases, there was limited adoption and fidelity to the interventions, which led to eventual discontinuation. Barriers contributing to mis-implementation included intervention complexity, low buy-in from overburdened providers, lack of alignment between providers and leadership, and COVID-19-related stressors. Mis-implementation occurred earlier in cases that experienced both patient- and provider-level barriers, and that were conducted during the COVID-19 pandemic.

Discussion and conclusion

Multi-level determinants contributed to EBP mis-implementation in FQHCs, limiting the ability of these health systems to benefit from knowledge exchange. To minimize mis-implementation, knowledge exchange strategies should be designed around common, core barriers but also flexible enough to address a variety of site-specific contextual factors and should be tailored to relevant audiences such as providers, patients, and/or leadership.

Community Engagement in Implementation Science: the Impact of Community Engagement Activities in the DECIPHeR Alliance

Background

The translation of evidence-based interventions into practice settings remains challenging. Implementation science aims to bridge the evidence-to-practice gap by understanding multilevel contexts and tailoring evidence-based interventions accordingly. Engaging community partners who possess timely, local knowledge is crucial for this process to be successful. The Disparities Elimination through Coordinated Interventions to Prevent and Control Heart and Lung Disease Risk (DECIPHeR) Alliance aims to address cardiopulmonary health disparities by engaging diverse community partners to improve the implementation of evidence-based interventions. The goal of the Community Engagement Subcommittee is to strengthen community engagement practice across DECIPHeR. This paper presents the subcommittee's "Why We Engage Communities" statement that outlines why community engagement is critical for implementation science. The paper also provides case examples of DECIPHeR community engagement activities.

Methods

To develop the "Why We Engage Communities" statement, we conducted a literature review, surveyed subcommittee members to assess the importance of community engagement in their work, and integrated community partner feedback. We synthesize the findings into three key themes and present examples of community engagement activities and their impact across DECIPHeR projects.

Results

The statement presents three themes that illustrate why community engagement increases the impact of implementation and health equity research. Community engagement (1) engages local knowledge and expertise, (2) promotes authentic relationships, and (3) builds community and researcher capacity. The statement provides a guiding framework for strengthening DECIPHeR research and enhancing community partnerships.

Conclusion

Community engagement can improve the implementation of evidence-based interventions across diverse settings, improving intervention effectiveness in underserved communities and furthering health equity.

Implementation and Mental Health Outcomes of a Service Cascade Linking Child Welfare and Children's Mental Health Systems: A Case Study of the Gateway CALL Demonstration

The mental health needs of children and youth involved in the child welfare system remain largely unmet. Service cascades are an emerging approach to systematizing mental health screening, assessment, and treatment referral processes. However, evidence is minimal and inconsistent regarding the effectiveness of such approaches for improving mental health service access and outcomes. In an effort to address this gap, this study presents a case-study of the implementation fidelity and treatment outcomes of the Gateway CALL service cascade. Study analyses involved longitudinal data collected as part of a larger evaluation of Gateway CALL. Specifically, descriptive and linear mixed model analyses were conducted to assess the implementation of service cascade components, and changes in mental health outcomes (behavior problems) among 175 children placed out-of-home during the study. Study analyses found that although fidelity was strong early in the service cascade, implementation began to break down once components involved more than one service system (child welfare, mental health). However, results also indicated that parent-reported child behavior problems decreased significantly over time, despite later cascade components being implemented with poor fidelity to the Gateway CALL service model. For children and youth involved in child welfare systems, service cascades like Gateway CALL have the potential to significantly improve both mental health service receipt and outcomes. To maximize the effectiveness of such approaches, later phases of implementation may require increased attention and support, particularly regarding processes and outcomes that cross child welfare and mental health service systems.

Application of community - engaged dissemination and implementation science to improve health equity

Community engagement is critical to accelerate and improve implementation of evidence-based interventions to reduce health inequities. Community-engaged dissemination and implementation research (CEDI) emphasizes engaging stakeholders (e.g., community members, practitioners, community organizations, etc.) with diverse perspectives, experience, and expertise to provide tacit community knowledge regarding the local context, priorities, needs, and assets. Importantly, CEDI can help improve health inequities through incorporating unique perspectives from communities experiencing health inequities that have historically been left out of the research process. The community-engagement process that exists in practice can be highly variable, and characteristics of the process are often underreported, making it difficult to discern how engagement of community partners was used to improve implementation. This paper describes the community-engagement process for a multilevel, pragmatic randomized trial to increase the reach and impact of evidence-based tobacco cessation treatment among Community Health Center patients; describes how engagement activities and the resulting partnership informed the development of implementation strategies and improved the research process; and presents lessons learned to inform future CEDI research.

Does mental health screening and assessment in child welfare improve mental health service receipt, child safety, and permanence for children in out-of-home care? An evaluation of the Gateway CALL demonstration

BACKGROUND

Unmet mental health service needs among children in out-of-home care are sometimes attributed to poor assessments and referrals in child welfare. The Gateway CALL project implemented mental health screening, diagnostic assessment, and referral to treatment practices.

OBJECTIVE

We examined the effectiveness of Gateway CALL for improving children's mental health service receipt, safety, and permanency outcomes.

PARTICIPANTS AND SETTING

Participants included 538 children (birth to 18 years) in out-of-home placements through a county-based child welfare agency over a 17-month period.

METHODS

We compared the mental health service receipt, safety, and permanency outcomes for 175 children who received Gateway CALL with 175 children who received "services as usual" identified through propensity score matching. Participant demographics, safety, and permanency outcomes were drawn from child welfare administrative records, and mental health service visits and diagnoses were drawn from Medicaid billing records.

RESULTS

Gateway CALL appeared to increase the number of mental health service visits children received (z = 2.14, p = 0.032), although not the likelihood of receiving services. In terms of child safety, children in Gateway CALL had a greater number of screened-in calls after the intervention than those in the comparison group [t(348) = -1.92, p = 0.03]; there were no differences in substantiations. There were also no observed effects on permanency.

CONCLUSIONS

Despite systematic efforts to identify, assess, and refer children to mental health services through the Gateway CALL intervention, substantial unmet mental health service needs among children persisted. Results have implications for designing interventions that promote cross-system service access.

Festschrift for Leonard Bickman: Introduction to The Future of Children's Mental Health Services Special Issue

This introductory article describes the genesis of the Festschrift for Leonard Bickman and of this Festschrift special issue entitled, The Future of Children’s Mental Health Services. The special issue includes a collection of 11 original children’s mental health services research articles, broadly organized in accordance with three themes (i.e., Improving Precision and Use of Service Data to Guide Policy and Practice, Implementation and Dissemination, and Preparing for Innovation), followed by an interview-style article with Bickman. Then follows a featured manuscript by Bickman himself, three invited commentaries, and a compilation of letters and notes in which colleagues reflect on his career and on their experiences of him. The introduction concludes with a few thoughts about the future of children’s mental health services portended by the extraordinary scholarly contributions of Bickman and those who have been inspired by him.

From Research Training to Scientific Advancement-Contributions from the Implementation Research Institute: An Introduction to the Special Issue

The special series is designed to provide examples of funded implementation research conducted by alumni of the first four cohorts of the Implementation Research Institute (IRI). The introduction links the six substantive papers to the conceptual and methodological challenges laid out in a 2009 publication in this journal which led to the IRI training program in the emerging science of implementation with a special focus on behavior health settings. The 7th paper in the series illustrates an innovative evaluative approach to design and measurement of IRI fellow publications and grants informed by the training program such as bibliometrics. The introduction also notes some elements identified in the 2009 foundational paper not represented in these papers such as costs as well as important developments and foci in the decade since 2009 such as de-implementation, sustainability, dynamic adaptation processes, and hybrid designs that need to be an integral part of training programs in implementation research.

Commentary: Establishing Scientific Rigor and Excellence in Implementation Science Training to Improve the Deployment of Evidence-Based Mental Health Services

Real-world challenges continue to impede the dissemination, implementation and sustainment of high-quality, evidence-based practices, resulting in too many individuals not receiving the effective mental health care that they urgently need. The field of implementation science is poised to generate new solutions to address this important public health problem. Training a new generation of researchers in implementation science is one solution. The Implementation Research Institute (IRI) offers one approach to such training. The papers in this series, authored by some of the IRI graduates, illustrate the depth and breadth of the intellectual scope of IRI graduates' contributions to the implementation science field, chronicles important lessons learned, and underscores the IRI's training capacity. This commentary reflects upon the series in terms of a framework and themes that relate to core implementation science principles and to future research that corresponds to the National Institute of Mental Health research priorities and strategic plan.

Elucidating the influence of supervisors' roles on implementation climate

Background

Supervisors play an essential role in implementation by diffusing and synthesizing information, selling implementation, and translating top management’s project plans to frontline workers. Theory and emerging evidence suggest that through these roles, supervisors shape implementation climate—i.e., the degree to which innovations are expected, supported, and rewarded. However, it is unclear exactly how supervisors carry out each of these roles in ways that contribute to implementation climate—this represents a gap in the understanding of the causal mechanisms that link supervisors’ behavior with implementation climate. This study examined how supervisors’ performance of each of these roles influences three core implementation climate domains (expectations, supports, and rewards).

Materials and methods

A sequenced behavioral health screening, assessment, and referral intervention was implemented within a county-based child welfare agency. We conducted 6 focus groups with supervisors and frontline workers from implementing work units 6 months post-implementation (n = 51) and 1 year later (n = 40) (12 groups total). Participants were asked about implementation determinants, including supervision and implementation context. We audio-recorded, transcribed, and analyzed focus groups using an open coding process during which the importance of the supervisors’ roles emerged as a major theme. We further analyzed this code using concepts and definitions related to middle managers’ roles and implementation climate.

Results

In this work setting, supervisors (1) diffused information about the intervention proactively, and in response to workers’ questions, (2) synthesized information by tailoring it to workers’ individual needs, (3) translated top managements’ project plans into day-to-day tasks through close monitoring and reminders, and (4) justified implementation. All four of these roles appeared to shape the implementation climate by conveying strong expectations for implementation. Three roles (diffusing, synthesizing, and mediating) influenced climate by supporting workers during implementation. Only one role (diffusing) influenced climate by conveying rewards.

Conclusions

Supervisors shaped implementation climate by carrying out four roles (diffusing, synthesizing, mediating, and selling). Findings suggest that the interaction of these roles convey expectations and support for implementation (two implementation climate domains). Our study advances the causal theory explaining how supervisors’ behavior shapes the implementation climate, which can inform implementation practice.