Exploring Provider Use of a Digital Implementation Support System for School Mental Health: A Pilot Study

The Opioid Response Project: An Effective Learning Collaborative for Local Communities?

The Opioid Response Project (ORP) was designed as an intensive 2-year health promotion learning collaborative grounded in the Collective Impact Model (CIM) to prepare ten local communities to address the opioid crisis. The purpose of this evaluation was to describe the ORP implementation, provide a summary of the evaluation results, share insights, and discuss implications. Results were informed by multiple data sources, including project documents, surveys, and interviews with members of the ORP and community teams. Based on process evaluation results, 100% of community teams reported being satisfied with the ORP and recommended this experience to others. ORP participation outputs ranged from new opioid response programs, to strengthened community teams, to receipt of additional funding. Based on the outcome evaluation, the ORP was effective at increasing community knowledge and capacity, promoting collaboration, and facilitating sustainability. This initiative is an example of an effective learning collaborative to curb the opioid epidemic at the community level. Participating communities found great value in working together as part of a larger cohort and reported benefits from the peer learning and support provided by the ORP. In particular, access to technical assistance, identification of engagement strategies within and across community teams, and a focus on sustainability are key practice components to be included in learning collaboratives designed to address large-scale public health issues.

Strategies for Enhancing the Implementation of Universal Mental Health Prevention Programs in Schools: A Systematic Review

A number of school-based mental health prevention programs have been found to be effective in research trials, but little is known about how to support implementation in real-life settings. To address this translational problem, this systematic review aims to identify effective strategies for enhancing the implementation of mental health prevention programs for children in schools. Four electronic databases were searched for empirical, peer-reviewed articles in English from January 2000 to October 2021 reporting the effects of implementation strategies for school-based universal mental health programs. Twenty-one articles were included in the narrative synthesis and assessed for quality using the Mixed Methods Appraisal Tool. Twenty-two strategies were found to be effective at improving program fidelity or adoption. The strategies with the strongest positive evidence base were those that involved monitoring and provision of feedback, engaging principals as program leaders, improving teachers' buy-in and organising school personnel implementation meetings. We recommend school-based practitioners trial strategies with positive findings from this review as part of their continuous quality improvement. This review highlights the pressing need for large-scale, randomised controlled trials to develop and trial more robust strategies to enhance adoption, as the five implementation studies found to measure adoption used qualitative methods limited by small samples sizes and case study designs.

A scoping review of the evaluation and effectiveness of technical assistance

Background

Although the benefits of evidence-based practices (EBPs) for advancing community outcomes are well-recognized, challenges with the uptake of EBPs are considerable. Technical assistance (TA) is a core capacity building strategy that has been widely used to support EBP implementation and other community development and improvement efforts. Yet despite growing reliance on TA, no reviews have systematically examined the evaluation of TA across varying implementation contexts and capacity building aims. This study draws on two decades of peer-reviewed publications to summarize the evidence on the evaluation and effectiveness of TA.

Methods

Guided by Arksey and O’Malley’s six-stage methodological framework, we used a scoping review methodology to map research on TA evaluation. We included peer-reviewed articles published in English between 2000 and 2020. Our search involved five databases: Business Source Complete, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Education Resources Information Center (ERIC), PsycInfo, and PubMed.

Results

A total of 125 evaluation research studies met the study criteria. Findings indicate that publications have increased over the last two decades, signaling a growth in the recognition and reporting of TA. Technical assistance is being implemented across diverse settings, often serving socially vulnerable and under-resourced populations. Most evaluation research studies involved summative evaluations, with TA outcomes mostly reported at the organizational level. Only 5% of the studies examined sustainability of TA outcomes. This review also demonstrates that there is a lack of consistent standards regarding the definition of TA and the level of reporting across relevant TA evaluation categories (e.g., cadence of contact, and directionality).

Conclusions

Advances in the science and practice of TA hinge on understanding what aspects of TA are effective and when, how, and for whom these aspects of TA are effective. Addressing these core questions requires (i) a standard definition for TA; (ii) more robust and rigorous evaluation research designs that involve comparison groups and assessment of direct, indirect, and longitudinal outcomes; (iii) increased use of reliable and objective TA measures; and (iv) development of reporting standards. We view this scoping review as a foundation for improving the state of the science and practice of evaluating TA.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00314-1.

Readiness as a precursor of early implementation outcomes: an exploratory study in specialty clinics

Background

Readiness has been identified as an essential precursor of successful implementation. However, evidence supporting its value is sparse. Empirical studies exploring the relationship between the application of readiness interventions, readiness levels, and implementation outcomes are lacking. The purpose of this study was twofold: (1) to evaluate the effectiveness of a readiness intervention (based on increases in readiness levels, changes in early implementation outcomes (i.e., acceptability, appropriateness, feasibility, and intent to adopt), and qualitative insights into the types of perceived outcomes) and (2) to assess the role of readiness as a predictor of these early implementation outcomes.

Methods

Seven healthcare specialty clinics engaged in a structured process to assess and build readiness for implementing a comprehensive medication management (CMM) service over a 10-month period. A mixed methods approach, which included surveys with healthcare stakeholders at each clinic (N = 27) and interviews with the lead pharmacists (N = 7), was used to evaluate the effectiveness of the readiness intervention (aim 1). Survey data were also used to conduct multiple regression models to assess the role of readiness as a predictor of early acceptability, appropriateness, feasibility, and intent to adopt CMM (aim 2).

Results

Significantly higher readiness levels, as well as higher scores on acceptability, appropriateness, feasibility, and intent to adopt, were reported as a result of engaging in the readiness intervention. However, upon closer examination, the direction of this association seemed to be dependent on the type of clinic. Qualitative data on the types of perceived outcomes resulting from engaging in the readiness intervention provided further insights into the potential reasons for these findings. Furthermore, post-readiness levels predicted between 44 and 68% of the variance in the early implementation outcomes. When accounting for clinic membership, readiness remained critical for service acceptability, feasibility, and intent to adopt but not for appropriateness.

Conclusion

These findings provide insights into the relationship between use of a readiness intervention, readiness levels, and early implementation outcomes. Engaging healthcare settings in a readiness intervention was beneficial in ways more complex than a simple positive linear relationship, highlighting the opportunity to broaden its purpose and expand definitions of readiness success. In addition, the importance of readiness levels in predicting early implementation outcomes, while critical, also seems to be highly dependent on context, particularly for appropriateness (fit).

Use of research evidence varied in efforts to expand specific pharmacist autonomous prescriptive authority: an evaluation and recommendations to increase research utilization

BACKGROUND

An expanding body of literature shows that pharmacists' interventions improve health outcomes and are cost-saving. However, diverse state regulations of pharmacists' scope of practice create a discrepancy between what pharmacists are trained to do and what they legally can do. This study investigated how stakeholders utilized research evidence when developing expanded scope of practice policies in their respective states.

METHODS

Using autonomous pharmacist prescriptive authority as a surrogate for general pharmacist scope of practice, a general policy document analysis was performed to understand the scope of practice landscape for pharmacists across the United States. Next, semi-structured interviews with policy-makers and pharmacy advocates were conducted to explore how the identified states in the policy document analysis utilized evidence during the policy-making process. Investigators analysed findings from the transcribed interviews through application of the SPIRIT Action Framework. Resulting codes were summarized across themes, and recommendations to researchers about increasing utilization of research evidence were crafted.

RESULTS

Sixteen states with 27 autonomous pharmacist prescriptive authority policies were identified. Public health need and safety considerations motivated evidence engagement, while key considerations dictating utilization of research included perceptions of research, access to resources and experts, and the successful implementation of similar policy. Research evidence helped to advocate for and set terms for pharmacist prescribing. Barriers to research utilization include stakeholder opposition to pharmacist prescribing, inability to interpret research, and a lack of relevant evidence. Recommendations for researchers include investigating specific metrics to evaluate scope of practice policy, developing relationships between policy-makers and researchers, and leveraging pharmacy practice stakeholders.

CONCLUSIONS

Overall, alignment of researcher goals and legislative priorities, coupled with timely communication, may help to increase research evidence engagement in pharmacist scope of practice policy. By addressing these factors regarding research engagement identified in this study, researchers can increase evidence-based scope of practice, which can help to improve patient outcomes, contain costs, and provide pharmacists with the legal infrastructure to practise at the top of their license.

Adapting a Technology-Based Implementation Support Tool for Community Mental Health: Challenges and Lessons Learned

With mental health services shifting to community-based settings, community mental health (CMH) organizations are under increasing pressure to deliver effective services. Despite availability of evidence-based interventions, there is a gap between effective mental health practices and the care that is routinely delivered. Bridging this gap requires availability of easily tailorable implementation support tools to assist providers in implementing evidence-based intervention with quality, thereby increasing the likelihood of achieving the desired client outcomes. This study documents the process and lessons learned from exploring the feasibility of adapting such a technology-based tool, Centervention, as the example innovation, for use in CMH settings. Mixed-methods data on core features, innovation-provider fit, and organizational capacity were collected from 44 CMH providers. Lessons learned included the need to augment delivery through technology with more personal interactions, the importance of customizing and integrating the tool with existing technologies, and the need to incorporate a number of strategies to assist with adoption and use of Centervention-like tools in CMH contexts. This study adds to the current body of literature on the adaptation process for technology-based tools and provides information that can guide additional innovations for CMH settings.