A Systematic Review of Strategies for Implementing Empirically Supported Mental Health Interventions

Qualitative Evaluation of a Novel Security Role to Reduce Occupational Violence in Inpatient Hospital Settings

Background: Occupational violence (OV) is a priority issue that significantly affects the safety of nurses, leading to staff burnout and poor retention issues. Security personnel are common in inpatient settings, yet there is limited research on their role, function, and impacts. The study aims to qualitatively evaluate a novel security role to reduce OV in inpatient settings. Methods: This qualitative study was conducted in a tertiary hospital in Brisbane, Queensland. A novel security role (Ambassador) was piloted in three inpatient wards over 6 months in 2020 to mitigate OV risk. Semi-structured in-depth interviews were conducted. All interviews were audio recorded. Interview transcripts were transcribed. Deductive analysis based on the Consolidated Framework for Implementation Research (CFIR) was used to identify the experiences and perceptions of the participants. Findings: 17 participants were interviewed. Five themes were identified including program characteristics, internal drivers, external drivers, individual experience, and implementation process. An Ambassador proactively engages with patients and visitors and employs behavioral strategies to de-escalate or redirect persons of concern. Participants considered Ambassadors to be important members of the health care team who supported the provision of patient and family-centered care. Successful implementation was said to require collaboration between clinical and security services and a small agile project team with authority and autonomy. Conclusion/Application to practice: This study provides many insights into the successful implementation of a novel security role in acute hospital settings. More research is needed on the effectiveness, appropriateness, feasibility, and cost of different security models.

Youth not engaged in education, employment, or training: a discrete choice experiment of service preferences in Canada

BACKGROUND

Prior research has showed the importance of providing integrated support services to prevent and reduce youth not in education, employment, or training (NEET) related challenges. There is limited evidence on NEET youth's perspectives and preferences for employment, education, and training services. The objective of this study was to identify employment, education and training service preferences of NEET youth. We acknowledge the deficit-based lens associated with the term NEET and use 'upcoming youth' to refer to this population group.

METHODS

Canadian youth (14-29 years) who reported Upcoming status or at-risk of Upcoming status were recruited to the study. We used a discrete choice experiment (DCE) survey, which included ten attributes with three levels each indicating service characteristics. Sawtooth software was used to design and administer the DCE. Participants also provided demographic information and completed the Global Appraisal of Individual Needs-Short Screener. We analyzed the data using hierarchical Bayesian methods to determine service attribute importance and latent class analyses to identify groups of participants with similar service preferences.

RESULTS

A total of n=503 youth participated in the study. 51% of participants were 24-29 years of age; 18.7% identified as having Upcoming status; 41.1% were from rural areas; and 36.0% of youth stated that they met basic needs with a little left. Participants strongly preferred services that promoted life skills, mentorship, basic income, and securing a work or educational placement. Three latent classes were identified and included: (i) job and educational services (38.9%), or services that include career counseling and securing a work or educational placement; (ii) mental health and wellness services (34.9%), or services that offer support for mental health and wellness in the workplace and free mental health and substance use services; and (iii) holistic skills building services (26.1%), or services that endorsed skills for school and job success, and life skills.

CONCLUSIONS

This study identified employment, education, and training service preferences among Upcoming youth. The findings indicate a need to create a service model that supports holistic skills building, mental health and wellness, and long-term school and job opportunities.

Implementation Feasibility and Hidden Costs of Statewide Scaling of Evidence-Based Therapies for Children and Adolescents

OBJECTIVE

State mental health systems are retraining their workforces to deliver services supported by research. Knowledge about evidence-based therapies (EBTs) for child and adolescent disorders is robust, but the feasibility of their statewide scaling has not been examined. The authors reviewed implementation feasibility for 12 commonly used EBTs, defining feasibility for statewide scaling as an EBT having at least one study documenting acceptability, facilitators and barriers, or fidelity; at least one study with a racially and ethnically diverse sample; an entity for training, certification, or licensing; and fiscal data reflecting the costs of implementation.

METHODS

The authors reviewed materials for 12 EBTs being scaled in New York State and conducted a literature review with search terms relevant to their implementation. Costs and certification information were supplemented by discussions with treatment developers and implementers.

RESULTS

All 12 EBTs had been examined for implementation feasibility, but only three had been examined for statewide scaling. Eleven had been studied in populations reflecting racial-ethnic diversity, but few had sufficient power for subgroup analyses to demonstrate effectiveness with these samples. All had certifying or licensing entities. The per-clinician costs of implementation ranged from $500 to $3,500, with overall ongoing costs ranging from $100 to $6,000. A fiscal analysis of three EBTs revealed hidden costs ranging from $5,000 to $24,000 per clinician, potentially limiting sustainability.

CONCLUSIONS

The evidence necessary for embedding EBTs in state systems has notable gaps that may hinder sustainability. Research-funding agencies should prioritize studies that focus on the practical aspects of scaling to assist states as they retrain their workforces.

Application of 'Readiness for Change' concept within implementation of evidence-based mental health interventions globally: protocol for a scoping review

Background

Concerning the growing burden of mental illnesses globally, there has been an increased investment into the implementation of evidence-based mental health interventions (EBmhIs) in routine care settings. However, the uptake and implementation of these EBmhIs has faced challenges in the real-world context. Among the many barriers and facilitators of implementation of EBmhIs identified by implementation science frameworks, evidence on the role of readiness for change (RFC) remains sparse. RFC constitutes the willingness and perceived capacity of stakeholders across an organization to implement a new practice. Theoretically, RFC has been defined at organizational, group, and individual levels, however, its conceptualization and operationalization across all these levels have differed in studies on the implementation of EBmhIs. By conducting a scoping review, we aim to examine the literature on RFC within the implementation of EBmhIs.

Methods

This scoping review will be conducted following the PRISMA-ScR guidelines. Iterative review stages will include a systematic and comprehensive search through four electronic databases (PubMed, Web of Science, Embase, and PsycINFO), selecting studies, charting data, and synthesizing the results. English-language studies meeting the inclusion criteria will be screened independently by two reviewers.

Discussion

This review will synthesize knowledge on the conceptualization of RFC across organizational, group, and individual levels within the implementation of EBmhIs. In addition, it will identify how RFC has been measured in these studies and summarize the reported evidence on its impact on the implementation of EBmhIs.

Conclusions

This review will assist mental health researchers, implementation scientists, and mental health care providers to gain a better understanding of the state of research on RFC within the implementation of EBmhIs.

Registration

The final protocol was registered with the Open Science Framework on October 21, 2022 ( https://osf.io/rs5n7).

Barriers and facilitators to a task-shifted stroke prevention program for children with sickle cell anemia in a community hospital: a qualitative study

BACKGROUND

Children with sickle cell anemia (SCA) are at high risk for stroke. Protocols for stroke prevention including blood transfusions, screening for abnormal non-imaging transcranial Doppler (TCD) measurements, and hydroxyurea therapy are difficult to implement in low-resource environments like Nigeria. This study aimed to examine the contextual factors around TCD screening in a community hospital in Nigeria using qualitative interviews and focus groups.

METHODS

We conducted a descriptive qualitative study in a community hospital in Kaduna, Nigeria, using focus groups and interviews. Interview guides and analysis were informed by the Consolidated Framework for Implementation Research (CFIR) framework and the Theory of Planned Behavior. Transcripts were coded and analyzed using an iterative deductive (CFIR)/Inductive (transcribed quotes) qualitative methodology.

RESULTS

We conducted two focus groups and five interviews with health care workers (nurses and doctors) and hospital administrators, respectively. Themes identified key elements of the inner setting (clinic characteristics, resource availability, implementation climate, and tension for change), characteristics of individuals (normative, control, and behavioral beliefs), and the implementation process (engage, implement, and adopt), as well as factors that were influenced by external context, caregiver needs, team function, and intervention characteristics. Task shifting, which is already being used, was viewed by providers and administrators as a necessary strategy to implement TCD screening in a clinic environment that is overstressed and under-resourced, a community stressed by poverty, and a nation with an underperforming health system.

CONCLUSION

Task shifting provides a viable option to improve health care by making more efficient use of already available human resources while rapidly expanding the human resource pool and building capacity for TCD screening of children with SCD that is more sustainable.

TRIAL REGISTRATION

NCT05434000.

An Implementation Approach to Translating Assessment Data into Treatment for Disorders of Addiction

Effective translation of data to inform real-time patient care is lacking in addiction inpatient settings. The current study presents the optimization of an assessment report that is used by clinicians to individualize treatment. A multi-aim, iterative approach was taken, utilizing an implementation science perspective to arrive at a final version of the assessment report. This occurred at a small inpatient addiction treatment facility. Participants were all available clinical staff (N = 7; female = 71%). A quantitative survey was used for aims 1 and 2 to, respectively, assess motives and context around the report as well as evaluate its design. Aim 3 focused on optimization via semi-structured interviews. Descriptive and modified content analyses were utilized appropriately across aims. This resulted in five versions of the assessment report being created between February 2021 and August 2022, the most recent of which was adapted into patients' electronic medical records. We discuss each version of the report in depth, including clinicians' iterative feedback and researchers' perceived barriers to this translational process. The response rate was 64.3%. The current study highlights a replicable approach for optimizing the translation of assessment data into treatment for patients with disorders of addiction as well as an assessment report that could be utilized by similar facilities with a naturally low sample size.

Providing a scaffold for considering theoretical frameworks in evidence implementation projects: the JBI approach to evidence implementation

ABSTRACT

There are many theories, models, and frameworks that have been proposed in the field of implementation science. Despite this, many evidence implementation or practice improvement projects do not consider these theories, models, or frameworks in their improvement efforts. The JBI approach is one example of an implementation theory, model, or framework. This approach has been developed particularly with health care professionals in mind and is designed to clearly guide pragmatic evidence implementation efforts based on the best available evidence. In this paper, we discuss how the JBI approach to evidence implementation can interact with and support theory-informed, pragmatic evidence implementation projects.

Using the exploration, preparation, implementation, sustainment (EPIS) framework to assess the cooperative re-engagement controlled trial (CoRECT)

Background

"Data to Care" (D2C) is a strategy which relies on a combination of public health surveillance data supplemented by clinic data to support continuity of HIV care. The Cooperative Re-Engagement Controlled Trial (CoRECT) was a CDC-sponsored randomized controlled trial of a D2C model, which provided an opportunity to examine the process of implementing an intervention for people with HIV (PWH) who are out-of-care across three public health department jurisdictions. Using the EPIS (Exploration, Preparation, Implementation, Sustainment) framework, we aimed to retrospectively describe the implementation process for each site to provide insights and guidance to inform future D2C activities implemented by public health agencies and their clinical and community partners.

Methods

After completion of CoRECT, the three (Connecticut, Massachusetts, Philadelphia) trial sites reviewed study protocols and held iterative discussions to describe and compare their processes regarding case identification, interactions with partnering clinics and patients, and sustainability. The EPIS framework provided a structure for comparing key organizational and operational practices and was applied to the entire implementation process.

Results

The trial sites varied in their implementation processes and the specific elements of the intervention. Factors including prior D2C experience, data management and analytic infrastructure, staff capacity, and relationships with clinic partners informed intervention development and implementation. Additionally, this review identified key lessons learned including to: (1) explore new supplemental sources for public health surveillance data; (2) work with stakeholders representing core functions/components in the early stages of the intervention design process; (3) build flexibility into all components of the follow-up activities; and (4) integrate data sharing, project management, and follow-up activities within existing DPH organizational structure.

Conclusion

The CoRECT study provides a general blueprint and lessons learned for implementing a D2C intervention for re-engagement in HIV care. Interventions should be tailored to local operational and structural factors, and responsive to evolving clinical and public health practices.

Partnering with social service staff to implement pragmatic clinical trials: an interim analysis of implementation strategies

BACKGROUND

With recent growth in the conduct of pragmatic clinical trials, the reliance on frontline staff to contribute to trial-related activities has grown as well. Active partnerships with staff members are often critical to pragmatic trial implementation, but rarely do research teams track and evaluate the specific "implementation strategies" used to support staff's involvement in trial procedures (e.g., participant recruitment). Accordingly, we adapted implementation science methodologies and conducted an interim analysis of the strategies deployed with social service staff involved in one multi-site pragmatic clinical trial.

METHODS

We used a naturalistic, observational study design to characterize strategies our research team deployed with staff during monthly, virtual meetings. Data were drawn from meeting notes and recordings from the trial's 4-month Preparation phase and 8-month Implementation phase. Strategies were mapped to the Expert Recommendations for Implementing Change taxonomy and categorized into nine implementation clusters. Survey data were also collected from staff to identify the most useful strategies the research team should deploy when onboarding new staff members in the trial's second year.

RESULTS

A total of 287 strategies were deployed. Strategies in the develop stakeholder interrelationships cluster predominated in both the Preparation (35%) and Implementation (31%) phases, followed by strategies in the use iterative and evaluative approaches cluster, though these were more prevalent during trial Preparation (24%) as compared to trial Implementation (18%). When surveyed on strategy usefulness, strategies in the provide interactive assistance, use financial approaches, and support staff clusters were most useful, per staff responses.

CONCLUSIONS

While strategies to develop stakeholder interrelationships were used most frequently during trial Preparation and Implementation, program staff perceived strategies that provided technical assistance, supported clinicians, and used financial approaches to be most useful and should be deployed when onboarding new staff members. Research teams are encouraged to adapt and apply implementation strategy tracking methods when partnering with social service staff and deploy practical strategies that support pragmatic trial success given staff needs and preferences.

TRIAL REGISTRATION

NCT05357261. May 2, 2022.

Multifaceted training and readiness to respond to family violence: A prospective cohort evaluation

BACKGROUND

As frontline healthcare workers, there is a growing expectation that nurses should be able to respond to disclosures of family violence. However, the profession and hospital systems have been slow to respond with clear skills, knowledge and confidence deficits identified in existing research. There is limited research which has robustly evaluated the effectiveness of in-depth, multifaceted training on readiness to respond among nurses.

AIM

To longitudinally evaluate the effectiveness of an in-depth family violence training intervention on confidence, knowledge and clinical skills of nurses working in a large tertiary adult hospital.

DESIGN

Single-centre, longitudinal intervention study. Strengthening the Reporting of Observational studies in Epidemiology (STROBE) cross-sectional guidelines were used.

METHODS

One hundred and ten nursing clinicians participated in this study to complete a comprehensive evidence-based model of healthcare workers response for assisting patients experiencing family violence. A mixed methods survey was used to assess change in knowledge, confidence and clinical skills among participants. Outcome assessment was electronically undertaken at baseline, 6-9 months and 12-15 months following intervention.

RESULTS

Statistically significant improvement was seen in self-reported knowledge, confidence and frequency of screening for family violence. Relative to baseline estimates, these improvements were identified 6-9 months and 12-15 months following intervention; albeit with consideration to the visually observed trend of skill reversion at follow-up. Quantitative findings were paralleled by qualitatively identified improvements in the recognition of the intersectional nature of violence, need for patient collaboration in screening and depth in considerations around how family violence is screened for.

CONCLUSIONS

Findings provide tentative support for the utility of a multidimensional training approach to improving nurses' readiness to respond to disclosures of family violence.

RELEVANCE TO CLINICAL PRACTICE

This study provides preliminary support for multidimensional, evidence-based training to effectively improve nurses' confidence, knowledge and clinical skills required for responding to family violence.

REPORTING METHOD

The study complies with the Strengthening the Reporting of Observational Studies in Epidemiology (SQUIRE) Statement: guidelines for reporting observational studies (see Table S1).

PATIENT CONTRIBUTION

Patients were involved in the initial design of the survey tool. This involvement came through the Allied Health Consumer panel and included input on the design and question wording of the survey items.

The effect of adverse childhood experience training, screening, and response in primary care: a systematic review

Background

Adverse childhood experiences (ACEs) can have harmful, long-term health effects. Although primary care providers (PCPs) could help mitigate these effects, no studies have reviewed the impacts of ACE training, screening, and response in primary care.

Methods

This systematic review searched four electronic databases (PubMed, Web of Science, APA PsycInfo, CINAHL) for peer-reviewed articles on ACE training, screening, and/or response in primary care published between Jan 1, 1998, and May 31, 2023. Searches were limited to primary research articles in the primary care setting that reported provider-related outcomes (knowledge, confidence, screening behavior, clinical care) and/or patient-related outcomes (satisfaction, referral engagement, health outcomes). Summary data were extracted from published reports.

Findings

Of 6532 records, 58 met inclusion criteria. Fifty-two reported provider-related outcomes; 21 reported patient-related outcomes. 50 included pediatric populations, 12 included adults. A majority discussed screening interventions (n = 40). Equal numbers (n = 25) discussed training and clinical response interventions. Strength of evidence (SOE) was generally low, especially for adult studies. This was due to reliance on observational evidence, small samples, and self-report measures for heterogeneous outcomes. Exceptions with moderate SOE included the effect of training interventions on provider confidence/self-efficacy and the effect of screening interventions on screening uptake and patient satisfaction.

Interpretation

Primary care represents a potentially strategic setting for addressing ACEs, but evidence on patient- and provider-related outcomes remains scarce.

Funding

The California Department of Health Care Services and the Office of the California Surgeon General.

Application of the Expert Recommendations for Implementing Change (ERIC) compilation of strategies to health intervention implementation in low- and middle-income countries: a systematic review

BACKGROUND

The Expert Recommendations for Implementing Change (ERIC) project developed a compilation of implementation strategies that are intended to standardize reporting and evaluation. Little is known about the application of ERIC in low- and middle-income countries (LMICs). We systematically reviewed the literature on the use and specification of ERIC strategies for health intervention implementation in LMICs to identify gaps and inform future research.

METHODS

We searched peer-reviewed articles published through March 2023 in any language that (1) were conducted in an LMIC and (2) cited seminal ERIC articles or (3) mentioned ERIC in the title or abstract. Two co-authors independently screened all titles, abstracts, and full-text articles, then abstracted study, intervention, and implementation strategy characteristics of included studies.

RESULTS

The final sample included 60 studies describing research from all world regions, with over 30% published in the final year of our review period. Most studies took place in healthcare settings (n = 52, 86.7%), while 11 (18.2%) took place in community settings and four (6.7%) at the policy level. Across studies, 548 distinct implementation strategies were identified with a median of six strategies (range 1-46 strategies) included in each study. Most studies (n = 32, 53.3%) explicitly matched implementation strategies used for the ERIC compilation. Among those that did, 64 (87.3%) of the 73 ERIC strategies were represented. Many of the strategies not cited included those that target systems- or policy-level barriers. Nearly 85% of strategies included some component of strategy specification, though most only included specification of their action (75.2%), actor (57.3%), and action target (60.8%). A minority of studies employed randomized trials or high-quality quasi-experimental designs; only one study evaluated implementation strategy effectiveness.

CONCLUSIONS

While ERIC use in LMICs is rapidly growing, its application has not been consistent nor commonly used to test strategy effectiveness. Research in LMICs must better specify strategies and evaluate their impact on outcomes. Moreover, strategies that are tested need to be better specified, so they may be compared across contexts. Finally, strategies targeting policy-, systems-, and community-level determinants should be further explored.

TRIAL REGISTRATION

PROSPERO, CRD42021268374.

A Qualitative Study on Using Telemedicine for Precepting and Teaching in the Academic Setting

PURPOSE

To examine the impact of telemedicine use on precepting and teaching among preceptors and patients during the COVID-19 pandemic.

METHOD

The authors conducted a secondary analysis of a qualitative study focusing on providers' and patients' experiences with and attitudes toward telemedicine at 4 academic health centers. Teaching and precepting were emergent codes from the data and organized into themes. Themes were mapped to domains from the 2009 Consolidated Framework for Implementation Research (CFIR), a framework that assists with effective implementation and consists of 5 domains: intervention characteristics, outer settings, inner settings, characteristics of individuals, and process.

RESULTS

In total, 86 interviews were conducted with 65 patients and 21 providers. Nine providers and 3 patients recounted descriptions related to teaching and precepting with telemedicine. Eight themes were identified, mapping across all 5 CFIR domains, with the majority of themes (n = 6) within the domains of characteristics of individuals, processes, and intervention characteristics. Providers and patients described how a lack of prepandemic telemedicine experience and inadequate processes in place to precept and teach with telemedicine affected the learning environment and perceived quality of care. They also discussed how telemedicine exacerbated existing difficulties in maintaining resident continuity. Providers described ways communication changed with telemedicine use during the pandemic, including having to wear masks while in the same room as the trainee and sitting closely to remain within range of the camera, as well as the benefit of observing trainees with the attending's camera off. Providers expressed a lack of protected structure and time for teaching and supervising with telemedicine, and a general view that telemedicine is here to stay.

CONCLUSIONS

Efforts should focus on increasing knowledge of telemedicine skills and improving processes to implement telemedicine in the teaching setting in order to best integrate it into undergraduate and graduate medical education.

Assessing the implementation of nurse practitioner-led huddles in long-term care using the Consolidated Framework for Implementation Research (CFIR)

BACKGROUND

The COVID-19 pandemic created major challenges in long-term care (LTC) homes across Canada and globally. A nurse practitioner-led interdisciplinary huddle intervention was developed to support staff wellbeing in two LTC homes in Ontario, Canada. The objective of this study was to identify the constructs strongly influencing the process of implementation of huddles across both sites, capturing the overall barriers and facilitators and the intervention's intrinsic properties.

METHODS

Nineteen participants were interviewed about their experiences, pre-, post-, and during huddle implementation. The Consolidated Framework for Implementation Research (CFIR) was used to guide data collection and analysis. CFIR rating rules and a cross-comparison analysis was used to identify differentiating factors between sites. A novel extension to the CFIR analysis process was designed to assess commonly influential factors across both sites.

RESULTS

Nineteen of twenty selected CFIR constructs were coded in interviews from both sites. Five constructs were determined to be strongly influential across both implementation sites and a detailed description is provided: evidence strength and quality; needs and resources of those served by the organization; leadership engagement; relative priority; and champions. A summary of ratings and an illustrative quote are provided for each construct.

CONCLUSION

Successful huddles require long-term care leaders to consider their involvement, the inclusion all team members to help build relationships and foster cohesion, and the integration of nurse practitioners as full-time staff members within LTC homes to support staff and facilitate initiatives for wellbeing. This research provides an example of a novel approach using the CFIR methodology, extending its use to identify significant factors for implementation when it is not possible to compare differences in success.

Addictions treatment mechanisms of change science and implementation science: A critical review

This manuscript aims to contribute to the next phase of mechanisms of behavior change (MOBC) science on alcohol or other drug use. Specifically, we encourage the transition from a basic science orientation (i.e., knowledge generation) to a translational science orientation (i.e., knowledge application or Translational MOBC Science). To inform that transition, we examine MOBC science and implementation science and consider how these two research areas can intersect to capitalize on the goals, strengths, and key methodologies of each. First, we define MOBC science and implementation science and offer a brief historical rationale for these two areas of clinical research. Second, we summarize similarities in rationale and discuss two scenarios where one draws from the other-MOBC science on implementation strategy outcomes and implementation science on MOBC. We then focus on the latter scenario, and briefly review the MOBC knowledge base to consider its readiness for knowledge translation. Finally, we provide a series of research recommendations to facilitate the translation of MOBC science. These recommendations include: (1) identifying and targeting MOBC that are well suited for implementation, (2) use of MOBC research results to inform broader health behavior change theory, and (3) triangulation of a more diverse set of research methodologies to build a translational MOBC knowledge base. Ultimately, it is important for gains borne from MOBC science to affect direct patient care, while basic MOBC research continues to be developed and refined over time. Potential implications of these developments include greater clinical significance for MOBC science, an efficient feedback loop between clinical research methodologies, a multi-level approach to understanding behavioral change, and reduced or eliminated siloes between MOBC science and implementation science.

Effectiveness of Self-guided Tailored Implementation Strategies in Integrating and Embedding Internet-Based Cognitive Behavioral Therapy in Routine Mental Health Care: Results of a Multicenter Stepped-Wedge Cluster Randomized Trial

BACKGROUND

Internet-based cognitive behavioral therapy (iCBT) services for common mental health disorders have been found to be effective. There is a need for strategies that improve implementation in routine practice. One-size-fits-all strategies are likely to be ineffective. Tailored implementation is considered as a promising approach. The self-guided integrated theory-based Framework for intervention tailoring strategies toolkit (ItFits-toolkit) supports local implementers in developing tailored implementation strategies. Tailoring involves identifying local barriers; matching selected barriers to implementation strategies; developing an actionable work plan; and applying, monitoring, and adapting where necessary.

OBJECTIVE

This study aimed to compare the effectiveness of the ItFits-toolkit with implementation-as-usual (IAU) in implementing iCBT services in 12 routine mental health care organizations in 9 countries in Europe and Australia.

METHODS

A stepped-wedge cluster randomized trial design with repeated measures was applied. The trial period lasted 30 months. The primary outcome was the normalization of iCBT delivery by service providers (therapists, referrers, IT developers, and administrators), which was measured with the Normalization Measure Development as a proxy for implementation success. A 3-level linear mixed-effects modeling was applied to estimate the effects. iCBT service uptake (referral and treatment completion rates) and implementation effort (hours) were used as secondary outcomes. The perceived satisfaction (Client Satisfaction Questionnaire), usability (System Usability Scale), and impact of the ItFits-toolkit by implementers were used to assess the acceptability of the ItFits-toolkit.

RESULTS

In total, 456 mental health service providers were included in this study. Compared with IAU, the ItFits-toolkit had a small positive statistically significant effect on normalization levels in service providers (mean 0.09, SD 0.04; P=.02; Cohen d=0.12). The uptake of iCBT by patients was similar to that of IAU. Implementers did not spend more time on implementation work when using the ItFits-toolkit and generally regarded the ItFits-toolkit as usable and were satisfied with it.

CONCLUSIONS

The ItFits-toolkit performed better than the usual implementation activities in implementing iCBT services in routine practice. There is practical utility in the ItFits-toolkit for supporting implementers in developing and applying effective tailored implementation strategies. However, the effect on normalization levels among mental health service providers was small. These findings warrant modesty regarding the effectiveness of self-guided tailored implementation of iCBT services in routine practice.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03652883; https://clinicaltrials.gov/ct2/show/NCT03652883.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.1186/s13063-020-04686-4.

Strategies for Enhancing the Implementation of Universal Mental Health Prevention Programs in Schools: A Systematic Review

A number of school-based mental health prevention programs have been found to be effective in research trials, but little is known about how to support implementation in real-life settings. To address this translational problem, this systematic review aims to identify effective strategies for enhancing the implementation of mental health prevention programs for children in schools. Four electronic databases were searched for empirical, peer-reviewed articles in English from January 2000 to October 2021 reporting the effects of implementation strategies for school-based universal mental health programs. Twenty-one articles were included in the narrative synthesis and assessed for quality using the Mixed Methods Appraisal Tool. Twenty-two strategies were found to be effective at improving program fidelity or adoption. The strategies with the strongest positive evidence base were those that involved monitoring and provision of feedback, engaging principals as program leaders, improving teachers' buy-in and organising school personnel implementation meetings. We recommend school-based practitioners trial strategies with positive findings from this review as part of their continuous quality improvement. This review highlights the pressing need for large-scale, randomised controlled trials to develop and trial more robust strategies to enhance adoption, as the five implementation studies found to measure adoption used qualitative methods limited by small samples sizes and case study designs.

Rethinking Context in Continuing Professional Development: From Identifying Barriers to Understanding Social Dynamics

INTRODUCTION

For continuing professional development (CPD) to reach its potential to improve outcomes requires an understanding of the role of context and the influencing conditions that enable interventions to succeed. We argue that the heuristic use of frameworks to design and implement interventions tends to conceptualize context as defined lists of barriers, which may obscure consideration of how different contextual factors interact with and intersect with each other.

METHODS

We suggest a framework approach that would benefit from postmodernist theory that explores how ideologies, meanings, and social structures in health care settings shape social practices. As an illustrative example, we conducted a Foucauldian discourse analysis of diabetes care to make visible how the social, historical, and political conditions in which clinicians experience, practice, and shape possibilities for behavior change.

RESULTS

The discursive construction of continuing education as a knowledge translation mechanism assumes and is contingent on family physicians to implement guidelines. However, they enact other discursively constituted roles that may run in opposition. This paradoxical position creates a tension that must be navigated by family physicians, who may perceive it possible to provide good care without necessarily implementing guidelines.

DISCUSSION

We suggest marrying "framework" thinking with postmodernist theory that explores how ideologies, meanings, and social structures shape practice behavior change. Such a proposed reconceptualization of context in the design of continuing professional development interventions could provide a more robust and nuanced understanding of how the dynamic relationships and interactions between clinicians, patients, and their work environments shape educational effectiveness.

Learning Collaboratives: a Strategy for Quality Improvement and Implementation in Behavioral Health

Learning collaboratives are increasingly used in behavioral health. They generally involve bringing together teams from different organizations and using experts to educate and coach the teams in quality improvement, implementing evidence-based practices, and measuring the effects. Although learning collaboratives have demonstrated some effectiveness in general health care, the evidence is less clear in behavioral health and more rigorous studies are needed. Learning collaboratives may contain a range of elements, and which elements are included in any one learning collaborative varies widely; the unique contribution of each element has not been established. This commentary seeks to clarify the concept of a learning collaborative, highlight its common elements, review evidence of its effectiveness, identify its application in behavioral health, and highlight recommendations to guide technical assistance purveyors and behavioral health providers as they employ learning collaboratives to improve behavioral health access and quality.

Strategies for supporting the implementation of a task-shared psychological intervention in South Africa's chronic disease services: qualitative insights from health managers' experiences of project MIND

BACKGROUND

Although evidence indicates that task-shared psychological interventions can reduce mental health treatment gaps in resource-constrained settings, systemic barriers have limited their widespread implementation. Evidence on how to sustain and scale such approaches is scant. This study responds to this gap by examining the experiences of South African health managers involved in the implementation of a task-shared counselling service for Project MIND.

OBJECTIVES

To qualitatively describe managers' experiences of implementing the MIND programme and their insights into potential strategies for supporting sustained implementation.

METHODS

Two focus group discussions (FGDs) and eight in-depth interviews (IDIs) were conducted with managers of urban and rural primary care facilities in the Western Cape province. All managers were female and 30-50 years old. FGDs and IDIs used an identical semi-structured topic guide to explore the experiences of the MIND programme and perceived barriers to sustained implementation. Normalisation process theory (NPT) guided the thematic analysis.

RESULTS

Four themes emerged that mapped onto the NPT constructs. First, managers noted that their relational work with staff to promote support for the intervention and reduce resistance was key to facilitating implementation. Second, managers emphasised the need for staff reorientation and upskilling to foster openness to mental health practice and for adequate time for quality counselling. Third, managers underscored the importance of strengthening linkages between the health and social service sectors to facilitate delivery of comprehensive mental health services. Finally, managers recommended ongoing monitoring of the service and communication about its impacts as strategies for supporting integration into routine practice.

CONCLUSIONS

Findings contribute to the emerging literature on strategies to support implementation of task-shared interventions in low- and middle-income countries. The findings highlight the leadership role of managers in identifying and actioning these strategies. Investing in managers' capacity to support implementation of psychological interventions is critical for scale-up of these mental health innovations.

Implementing the language comprehension test C-BiLLT: a qualitative description study using the COM-B model of behaviour change

Background

It is challenging to reliably assess the language comprehension of children with severe motor and speech impairments using traditional assessment tools. The Computer Based instrument for Low motor Language Testing (C-BiLLT) aims to reduce barriers to evidence-based assessment for this population by allowing children to access the test using non-traditional methods such as eye gaze so they can independently respond to test items. The purpose of this study is to develop a contextualized understanding of the factors that influenced clinicians’ implementation of the C-BILLT in practice in the Netherlands and Norway.

Materials and methods

A qualitative approach including semi-structured individual interviews with 15 clinicians (speech-language pathologists, neuropsychologists, and one teacher, counsellor, and vision specialist) was used. Data analysis was conducted in two rounds. First, a deductive approach including a codebook was used to code data within the COM-B components describing clinicians’ capability, opportunity, and motivation for behaviour change. Then, an abductive approach applying thematic analysis was used to identify meaningful patterns within the COM-B components.

Results

Several meaningful barriers and facilitators were identified across the data. Clinicians used the C-BiLLT with two distinct groups of clients: (1) the population it was originally developed for, and (2) clients that could have also been assessed using a traditional language test. Clinicians working with the first group experienced more, and more complex barriers across all COM-B components, to successful C-BiLLT use than the latter.

Conclusion

This study provides timely insights into the capability, opportunity, and motivation factors important for creating and sustaining assessment behaviour change in clinicians who used or attempted to use the C-BiLLT. Potential tailored intervention strategies aimed at improving implementation of novel assessment tools are discussed and may be helpful for others working to improve service delivery for children with complex needs.

Demands on Health Information and Clinical Practice Guidelines for Patients from the Perspective of Adults with Mental Illness and Family Members: A Qualitative Study with In-Depth Interviews

(1) Background: "Patient health information" promote health literacy. "Patient guidelines" as a sub group reflect the current evidence about illnesses and treatment options adapted to the needs of laypersons. Little is known about factors promoting and hindering their use by people affected by mental illness and their relatives. (2) Methods: Telephone interviews (N = 15; n = 4 adults affected by mental illness, n = 5 relatives, n = 6 both applicable) were conducted according to the Sørensen model of health literacy. Data were recorded, transcribed and content-analyzed following Mayring. (3) Results: Health information is used regularly by individuals affected by mental illness and their relatives, but "patient guidelines" are largely unknown. Yet, there is a great willingness to use them. Main barriers are a lack of statistical knowledge, the complexity of health-related topics and cognitive impairment sometimes accompanying mental illnesses. Target group-oriented adaptation as well as transparent and even-handed presentation of (dis-)advantages of treatment options can increase trust. (4) Conclusions: Health information and guidelines can help affected persons and relatives to make treatment decisions by conveying unbiased, up-to-date knowledge. Target group-specific adaptations should be made for psychiatric illnesses and features specific to mental illnesses compared to physical illnesses should be included. Clinical practice guidelines must be distributed more widely to increase their impact.

"We start where we are": a qualitative study of barriers and pragmatic solutions to the assessment and reporting of implementation strategy fidelity

BACKGROUND

Fidelity measurement of implementation strategies is underdeveloped and underreported, and the level of reporting is decreasing over time. Failing to properly measure the factors that affect the delivery of an implementation strategy may obscure the link between a strategy and its outcomes. Barriers to assessing and reporting implementation strategy fidelity among researchers are not well understood. The aims of this qualitative study were to identify barriers to fidelity measurement and pragmatic pathways towards improvement.

METHODS

We conducted in-depth interviews among researchers conducting implementation trials. We utilized a theory-informed interview approach to elicit the barriers and possible solutions to implementation strategy fidelity assessment and reporting. Reflexive-thematic analysis guided coding and memo-writing to determine key themes regarding barriers and solutions.

RESULTS

Twenty-two implementation researchers were interviewed. Participants agreed that implementation strategy fidelity was an essential element of implementation trials and that its assessment and reporting should improve. Key thematic barriers focused on (1) a current lack of validated fidelity tools with the need to assess fidelity in the short term, (2) the complex nature of some implementation strategies, (3) conceptual complications when assessing fidelity within mechanisms-focused implementation research, and (4) structural issues related to funding and publishing. Researchers also suggested pragmatic solutions to overcome each barrier. Respondents reported using specification and tracking data in the short term until validated tools become available. Participants suggested that researchers with strategy-specific content expertise lead the way in identifying core components and setting fidelity requirements for them. Addressing the third barrier, participants provided examples of what pragmatic prospective and retrospective fidelity assessments might look like along a mechanistic pathway. Finally, researchers described approaches to minimize costs of data collection, as well as more structural accountability like adopting and enforcing reporting guidelines or changing the structure of funding opportunities.

DISCUSSION

We propose short- and long-term priorities for improving the assessment and reporting of implementation strategy fidelity and the quality of implementation research.

CONCLUSIONS

A better understanding of the barriers to implementation strategy fidelity assessment may pave the way towards pragmatic solutions.

Ho'ouna Pono implementation: applying concept mapping to a culturally grounded substance use prevention curriculum in rural Hawai'i schools

BACKGROUND

Despite their potential to ameliorate health disparities and address youth substance use, prevention programs have been poorly disseminated and implemented across Hawai'i, which begs the question: Why are effective prevention programs not being used in communities most in need of them? Implementing and sustaining culturally grounded prevention programs is critical to address equitable healthcare and minimize health disparities in communities. The field of implementation science provides frameworks, theories, and methods to examine factors associated with community adoption of these programs.

METHOD

Our project applies concept mapping methods to a culturally grounded youth drug prevention program with state level educational leadership in rural Hawai'i schools. The goal is to integrate barrier and facilitator salience collected through teacher and school staff surveys and specific implementation strategies to regionally tailored implementation plans on Hawai'i island. This protocol paper describes the concept mapping steps and how they will be applied in public and public-charter schools.

DISCUSSION

Improving prevention program implementation in rural schools can result in sustained support for populations that need it most. The project will integrate implementation science and culturally grounded methods in rural Hawai'i, where most youth are of Native Hawaiian and Pacific Islander descent. This project addresses health disparities among Native Hawaiian and Pacific Islander youth and provides actionable plans for rural Hawai'i communities to implement effective prevention programming.

Required sample size to detect mediation in 3-level implementation studies

Background

Statistical tests of mediation are important for advancing implementation science; however, little research has examined the sample sizes needed to detect mediation in 3-level designs (e.g., organization, provider, patient) that are common in implementation research. Using a generalizable Monte Carlo simulation method, this paper examines the sample sizes required to detect mediation in 3-level designs under a range of conditions plausible for implementation studies.

Method

Statistical power was estimated for 17,496 3-level mediation designs in which the independent variable (X) resided at the highest cluster level (e.g., organization), the mediator (M) resided at the intermediate nested level (e.g., provider), and the outcome (Y) resided at the lowest nested level (e.g., patient). Designs varied by sample size per level, intraclass correlation coefficients of M and Y, effect sizes of the two paths constituting the indirect (mediation) effect (i.e., X→M and M→Y), and size of the direct effect. Power estimates were generated for all designs using two statistical models—conventional linear multilevel modeling of manifest variables (MVM) and multilevel structural equation modeling (MSEM)—for both 1- and 2-sided hypothesis tests.

Results

For 2-sided tests, statistical power to detect mediation was sufficient (≥0.8) in only 463 designs (2.6%) estimated using MVM and 228 designs (1.3%) estimated using MSEM; the minimum number of highest-level units needed to achieve adequate power was 40; the minimum total sample size was 900 observations. For 1-sided tests, 808 designs (4.6%) estimated using MVM and 369 designs (2.1%) estimated using MSEM had adequate power; the minimum number of highest-level units was 20; the minimum total sample was 600. At least one large effect size for either the X→M or M→Y path was necessary to achieve adequate power across all conditions.

Conclusions

While our analysis has important limitations, results suggest many of the 3-level mediation designs that can realistically be conducted in implementation research lack statistical power to detect mediation of highest-level independent variables unless effect sizes are large and 40 or more highest-level units are enrolled. We suggest strategies to increase statistical power for multilevel mediation designs and innovations to improve the feasibility of mediation tests in implementation research.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-022-01235-2.

Methodologies to Advance a "Science of How": Identifying and Engaging Intervention Targets and Outcomes

OBJECTIVE

Although implementation science has taken hold in many areas of psychiatric services research, a need remains for developing effective, low-cost interventions for specific subpopulations with mental health conditions. The experimental therapeutics approach has gained momentum as a framework for developing effective interventions. However, few studies have taken steps to rigorously apply experimental therapeutics. This article provides a blueprint for applying this approach.

METHODS

A focused literature review was conducted to document the frequency of the application of experimental therapeutics among articles published between 2011 and 2021 in some of the American Psychiatric Association's journals. Independently of the review, the authors delineated a four-component approach for applying experimental therapeutics in research and present practical, innovative strategies to advance psychiatric services research.

RESULTS

The four-component approach includes outlining prerequisites, identifying target mechanisms, proposing intervention strategies to address target mechanisms, and using advanced analytic methods. The strategies described for each component are not exhaustive; rather, they suggest promising avenues for research that can lead to more effective interventions and deeper understanding of how, and for whom, an intervention works.

CONCLUSIONS

The application of experimental therapeutics in psychiatric services research can lead to increased development, refinement, and implementation of effective interventions for specific populations or conditions.

A Systematic Review of Evidence-Based Family Interventions for Trauma-Affected Refugees

Family connections are crucial for trauma-affected refugees from collectivistic cultures. Evidence-based family interventions are consistently promoted to support a host of mental and relational health needs of families exposed to traumatic stressors; however, there is still limited research focused on cultural adaptation and the testing of the effectiveness of these interventions on some of the most disenfranchised populations in the aftermath of forced displacement. This systematic review was conducted to examine the reach of existing evidence-based family interventions implemented with newly resettled refugees globally. Studies included in this review include those testing the effectiveness of a systemic treatment with pre and post intervention evaluation, studies with or without control groups, and studies that include at least one family member in addition to the target participants. Twelve studies met the inclusion criteria. Barriers to conducting randomized control trials with displaced refugee populations are discussed. Recommendations are made for future studies to include a focus on scientifically rigorous multi-method designs, specific cultural adaptation frameworks, and the integration of relational aspects rather than focusing only on individual adjustment. Global displacement continues to rise; therefore, it is imperative that the mental health and wellbeing of displaced populations be treated with a comprehensive, multi-level framework.

Use of implementation science to plan adoption of a tobacco-free policy on a historically black college/university (HBCU) campus

Objective: Implementation science tools were used to identify facilitators and barriers to adopting a tobacco-free policy on a historically Black college/university (HBCU). Participants: Study participants included 13 University staff, two students, and two public health educators/school alumni. Data were collected in 2015-2016. Methods: The Consolidated Framework for Implementation Research (CFIR) was used to develop questions for focus groups and individual interviews. The CFIR was also used by two research team members to code transcripts independently, using an iterative deduction process to achieve ≥85% agreement on CFIR domains. Coded quotations were evaluated using the Qualitative Implementation Framework (QIF) to ascertain next adoption planning steps. Results: Qualitative data suggested that building capacity for tobacco control, linking policy to HBCU identity and priorities such as addressing marijuana use might facilitate adoption of a tobacco-free campus policy. Conclusions: Implementation science tools may be useful for planning adoption of comprehensive tobacco-free policies at HBCUs.

Systems analysis and improvement approach to optimize outpatient mental health treatment cascades in Mozambique (SAIA-MH): study protocol for a cluster randomized trial

BACKGROUND

Significant investments are being made to close the mental health (MH) treatment gap, which often exceeds 90% in many low- and middle-income countries (LMICs). However, limited attention has been paid to patient quality of care in nascent and evolving LMIC MH systems. In system assessments across sub-Saharan Africa, MH loss-to-follow-up often exceeds 50% and sub-optimal medication adherence often exceeds 60%. This study aims to fill a gap of evidence-based implementation strategies targeting the optimization of MH treatment cascades in LMICs by testing a low-cost multicomponent implementation strategy integrated into routine government MH care in Mozambique.

METHODS

Using a cluster-randomized trial design, 16 clinics (8 intervention and 8 control) providing primary MH care will be randomized to the Systems Analysis and Improvement Approach for Mental Health (SAIA-MH) or an attentional placebo control. SAIA-MH is a multicomponent implementation strategy blending external facilitation, clinical consultation, and provider team meetings with system-engineering tools in an overall continuous quality improvement framework. Following a 6-month baseline period, intervention facilities will implement the SAIA-MH strategy for a 2-year intensive implementation period, followed by a 1-year sustainment phase. Primary outcomes will be the proportion of all patients diagnosed with a MH condition and receiving pharmaceutical-based treatment who achieve functional improvement, adherence to medication, and retention in MH care. The Consolidated Framework for Implementation Research (CFIR) will be used to assess determinants of implementation success. Specific Aim 1b will include the evaluation of mechanisms of the SAIA-MH strategy using longitudinal structural equation modeling as well as specific aim 2 estimating cost and cost-effectiveness of scaling-up SAIA-MH in Mozambique to provincial and national levels.

DISCUSSION

This study is innovative in being the first, to our knowledge, to test a multicomponent implementation strategy for MH care cascade optimization in LMICs. By design, SAIA-MH is a low-cost strategy to generate contextually relevant solutions to barriers to effective primary MH care, and thus focuses on system improvements that can be sustained over the long term. Since SAIA-MH is integrated into routine government MH service delivery, this pragmatic trial has the potential to inform potential SAIA-MH scale-up in Mozambique and other similar LMICs.

TRIAL REGISTRATION

ClinicalTrials.gov; NCT05103033 ; 11/2/2021.

Implementation strategies for knowledge products in primary healthcare: a systematic review of systematic reviews (Preprint)

Background

The underuse or overuse of knowledge products leads to waste in health care, and primary care is no exception.

Objective

This study aimed to characterize which knowledge products are frequently implemented, the implementation strategies used in primary care, and the implementation outcomes that are measured.

Methods

We performed a systematic review (SR) of SRs using the Cochrane systematic approach to include eligible SRs. The inclusion criteria were any primary care contexts, health care professionals and patients, any Effective Practice and Organization of Care implementation strategies of specified knowledge products, any comparators, and any implementation outcomes based on the Proctor framework. We searched the MEDLINE, EMBASE, CINAHL, Ovid PsycINFO, Web of Science, and Cochrane Library databases from their inception to October 2019 without any restrictions. We searched the references of the included SRs. Pairs of reviewers independently performed selection, data extraction, and methodological quality assessment by using A Measurement Tool to Assess Systematic Reviews 2. Data extraction was informed by the Effective Practice and Organization of Care taxonomy for implementation strategies and the Proctor framework for implementation outcomes. We performed a descriptive analysis and summarized the results by using a narrative synthesis.

Results

Of the 11,101 records identified, 81 (0.73%) SRs were included. Of these 81, a total of 47 (58%) SRs involved health care professionals alone. Moreover, 15 SRs had a high or moderate methodological quality. Most of them addressed 1 type of knowledge product (56/81, 69%), common clinical practice guidelines (26/56, 46%) or management, and behavioral or pharmacological health interventions (24/56, 43%). Mixed strategies were used for implementation (67/81, 83%), predominantly education-based (meetings in 60/81, 74%; materials distribution in 59/81, 73%; and academic detailing in 45/81, 56%), reminder (53/81, 36%), and audit and feedback (40/81, 49%) strategies. Education meetings (P=.13) and academic detailing (P=.11) seemed to be used more when the population was composed of health care professionals alone. Improvements in the adoption of knowledge products were the most commonly measured outcome (72/81, 89%). The evidence level was reported in 12% (10/81) of SRs on 62 outcomes (including 48 improvements in adoption), of which 16 (26%) outcomes were of moderate or high level.

Conclusions

Clinical practice guidelines and management and behavioral or pharmacological health interventions are the most commonly implemented knowledge products and are implemented through the mixed use of educational, reminder, and audit and feedback strategies. There is a need for a strong methodology for the SR of randomized controlled trials to explore their effectiveness and the entire cascade of implementation outcomes.

Investigating strategies to increase general education teachers' adherence to evidence-based social-emotional behavior practices: A meta-analysis of the single-case literature

Educational researchers have produced a variety of evidence-based practices (EBP) to address social, emotional, and behavioral (SEB) needs among students. Yet, these practices are often insufficiently adopted and implemented with fidelity by teachers to produce the beneficial outcomes associated with the EBP, leaving students at risk for developing SEB problems. If ignored, SEB problems can lead to other negative outcomes, such as academic failure. Therefore, implementation strategies (i.e., methods and procedures designed to promote implementation outcomes) are needed to improve teachers' uptake and delivery of EBPs with fidelity. This meta-analysis sought to examine the types and magnitude of effect of implementation strategies that have been designed and tested to improve teacher adherence to SEB EBPs. Included studies (a) used single case experimental designs, (b) employed at least one implementation strategy, (c) targeted general education teachers, and (d) evaluated adherence as a core dimension of fidelity related to the delivery of EBPs. In total, this study included 28 articles and evaluated 122 effect sizes. A total of 15 unique implementation strategies were categorized. Results indicated that, on average, implementation strategies were associated with increases in teacher adherence to EBPs above baseline and group-based pre-implementation trainings alone (g = 2.32, tau = 0.77). Moderator analysis also indicated that larger effects were associated with implementation strategies that used a greater number of unique behavior change techniques (p < .001). Implications and future directions for research and practice regarding use of implementation strategies for general education teachers are discussed.

Clinician adoption of Parent–Child Interaction Therapy: A systematic review of implementation interventions

Background

Parent–Child Interaction Therapy (PCIT) is a parent training intervention for childhood conduct problems, distinctive in its use of live clinician coaching of the parent–child dyad via a one-way mirror and discrete earpiece. However, despite a compelling evidence base, uptake of evidence-based parent training programmes such as PCIT by clinicians in routine care settings remains poor. This systematic review aimed to identify and synthesise implementation interventions that have sought to increase clinician adoption of PCIT in usual care settings.

Methods

We searched MEDLINE (Ovid), Embase (Ovid), PsycInfo (Ovid), CINAHL (EBSCO), Science Citation Index and Social Sciences Citation Index, and Web of Science Core Collection from inception to October 2020. Articles were included if they tested (by way of randomised controlled trials, controlled clinical trials, interrupted time series and controlled before and after trials) implementation interventions across any and all of the patient, clinician, clinic, system or policy domains. Two independent reviewers screened and selected studies, assessed risk of bias and extracted data – summarising implementation intervention components according to items from the Template for Intervention Description and Replication (TIDieR) checklist ( Hoffmann et al., 2014 ).

Results

Of the 769 articles identified once duplicates were removed, 13 papers relating to three studies met the inclusion criteria – all were quantitative or mixed-methods examinations of the effectiveness of different PCIT clinician training or training-related consultation methods. A narrative description of interventions was provided, as quantitative synthesis was not possible.

Conclusions

Research attention has to date been focussed on the establishment of an evidence-base for PCIT's effectiveness, with relatively little attention to the dissemination, implementation and sustainment of this treatment. Those studies that do exist have focused on training methods and training-related expert consultation. Research attention could usefully turn to both adoption and sustainment of this effective treatment in usual care settings.

Plain language summary

In this review, we aimed to summarise what is already known about how to implement PCIT in community settings after clinicians have received training in the approach. While research relating to the implementation of other parent training programmes is interesting and informative, implementation efforts are most effective when tailored to a specific programme in a specific context. As such, it was important to review published studies relating to PCIT specifically. We identified three relevant studies, one of which is yet to publish its main implementation findings. The three studies have focused on how best to train clinicians in PCIT, including how best to provide post-training support from expert trainers. We concluded that a fruitful line for future research would be to focus on the post-training period, particularly how best to support clinicians to adopt and sustain PCIT in their practice.

Systematic review registration

The study was prospectively registered with the International Prospective Register of Systematic Reviews (PROSPERO) on 01/10/2020 (CRD42020207118).

Building organizational readiness: initial field testing of an expert-informed typology of implementation strategies

BACKGROUND

Organizational readiness is a known barrier to implementing evidence-based practices (EBPs) in community mental health services. A robust methodology for enhancing organizational readiness for implementation (ORI) has the potential to improve implementation outcomes of EBPs and ensure better services for people with a psychiatric disability. Prior work established a framework of implementation strategies targeting ORI enhancement by asking a group of implementation experts from various fields to categorize strategies from the "Expert Recommendations for Implementing Change" (ERIC) Project into three readiness stages, consistent with the pre-action stages of the Transtheoretical Model of behavioral change: Pre-contemplation, Contemplation, and Preparation. The current study provides initial confirmation and refinement to this expert-driven typology based on community mental health field experiences.

METHODS

We conducted in-depth interviews with stakeholders involved in a recent EBP implementation project. Participants included staff (n=9) from four community mental health agencies and the implementation team who facilitated the project (n=3). Their pre-implementation experiences were compared with the experts' typology to identify consistencies and discrepancies.

RESULTS

The participants' experiences were congruent with two thirds of the strategies identified by the experts for specific ORI stages. The refinements included 12 strategies used in additional stages beyond the experts' classification, four strategies from the ERIC list that were not included in the ORI typology, and five new strategies.

CONCLUSIONS

This study provides initial confirmation and refinements to the previously published ORI typology. The results offer guidance as to how ORI could be enhanced in the community mental health field.

Applying implementation science in mental health services: Technical assistance cases from the Mental Health Technology Transfer Center (MHTTC) network

OBJECTIVE

Critical gaps exist between implementation of effective interventions and the actual services delivered to people living with mental disorders. Many technical assistance (TA) efforts rely on one-time trainings of clinical staff and printed guidelines that alone are not effective in changing clinical practice. The Mental Health Technology Transfer Center (MHTTC) Network uses implementation science to accelerate the use of evidence-based practices (EBPs), improve performance, and bring about systems-level change.

METHOD

Four case examples illustrate how MHTTCs employ the Exploration-Preparation-Implementation-Sustainment (EPIS) implementation framework and intensive implementation strategies to educate clinicians, manage change, and improve processes. These examples include implementing motivational interviewing, cognitive-behavioral therapy for people with psychosis, strategies to decrease the no show rate for virtual appointments, and school mental health systems development.

RESULTS

From Preparation through Sustainment, MHTTCs successfully employed implementation strategies including learning communities, audit and feedback, and coaching to bring about change. Each project attended to inner and outer contexts to eliminate barriers. The examples also show the benefit of integrating process improvement alongside implementation.

CONCLUSIONS

The MHTTCs are a model for using implementation science to design technical assistance that leads to more successful practical execution of EBPs; thus reducing the gap between research and practice.

The case for prioritizing implementation strategy fidelity measurement: benefits and challenges

Implementation strategies are systematic approaches to improve the uptake and sustainability of evidence-based interventions. They frequently focus on changing provider behavior through the provision of interventions such as training, coaching, and audit-and-feedback. Implementation strategies often impact intermediate behavioral outcomes like provider guideline adherence, in turn improving patient outcomes. Fidelity of implementation strategy delivery is defined as the extent to which an implementation strategy is carried out as it was designed. Implementation strategy fidelity measurement is under-developed and under-reported, with the quality of reporting decreasing over time. Benefits of fidelity measurement include the exploration of the extent to which observed effects are moderated by fidelity, and critical information about Type-III research errors, or the likelihood that null findings result from implementation strategy fidelity failure. Reviews of implementation strategy efficacy often report wide variation across studies, commonly calling for increased implementation strategy fidelity measurement to help explain variations. Despite the methodological benefits of rigorous fidelity measurement, implementation researchers face multi-level challenges and complexities. Challenges include the measurement of a complex variable, multiple data collection modalities with varying precision and costs, and the need for fidelity measurement to change in-step with adaptations. In this position paper, we weigh these costs and benefits and ultimately contend that implementation strategy fidelity measurement and reporting should be improved in trials of implementation strategies. We offer pragmatic solutions for researchers to make immediate improvements like the use of mixed methods or innovative data collection and analysis techniques, the inclusion of implementation strategy fidelity assessment in reporting guidelines, and the staged development of fidelity tools across the evolution of an implementation strategy. We also call for additional research into the barriers and facilitators of implementation strategy fidelity measurement to further clarify the best path forward.

Scaling Evidence-Based Interventions to Improve the Cardiovascular Health of People With Serious Mental Illness

People with serious mental illnesses (SMIs) experience excess mortality, driven in large part by high rates of cardiovascular disease (CVD), with all cardiovascular disease risk factors elevated. Interventions designed to improve the cardiovascular health of people with SMI have been shown to lead to clinically significant improvements in clinical trials; however, the uptake of these interventions into real-life clinical settings remains limited. Implementation strategies, which constitute the "how to" component of changing healthcare practice, are critical to supporting the scale-up of evidence-based interventions that can improve the cardiovascular health of people with SMI. And yet, implementation strategies are often poorly described and rarely justified theoretically in the literature, limiting the ability of researchers and practitioners to tease apart why, what, how, and when implementation strategies lead to improvement. In this Perspective, we describe the implementation strategies that the Johns Hopkins ALACRITY Center for Health and Longevity in Mental Illness is using to scale-up three evidenced-based interventions related to: (1) weight loss; (2) tobacco smoking cessation treatment; and (3) hypertension, dyslipidemia, and diabetes care for people with SMI. Building on concepts from the literature on complex health interventions, we focus on considerations related to the core function of an intervention (i.e., or basic purposes of the change process that the health intervention seeks to facilitate) vs. the form (i.e., implementation strategies or specific activities taken to carry out core functions that are customized to local contexts). By clearly delineating how implementation strategies are operationalized to support the interventions' core functions across these three studies, we aim to build and improve the future evidence base of how to adapt, implement, and evaluate interventions to improve the cardiovascular health of people with SMI.

The Implementation Process of Two Evidence-Based Protocols: A Spatial Neglect Network Initiative

Introduction

Spatial neglect, a neurocognitive disorder of lateralized spatial attention, is prevalent among stroke survivors especially in inpatient rehabilitation facilities (IRFs). The ultimate goal of the project was to improve spatial neglect care in inpatient rehabilitation and trained as many OTs as possible using both tools in their regular practices as the means to achieve our overall objective. Therefore, we conducted a project aimed at implementing two evidence-based protocols, one for assessment (KF-NAP®) and the other for treatment (KF-PAT®), and share the implementation process, which included barriers and facilitators identified during and after the process, and implementation outcomes.

Methods

Sixteen IRFs were involved. The Knowledge-To-Action Cycle was used to describe the process of knowledge inquiry (training), translating knowledge (implementation) and evaluating the use of knowledge in clinical practice (outcomes). Barriers and strategies were reported using the Consolidated Framework for Implementation Research and identified through a survey, after the study concluded.

Results

Thirty-two therapists at the participating sites were trained to some level of the KF-NAP and KF-PAT. Throughout the project and also once after it finished, different barriers were identified by researchers and clinicians, who then determined together actions to eliminate or minimize the barriers. For example, multiple sites reported: "not having time to train other staff at their hospital due to high patient volume and other responsibilities."

Discussion

The project shared our implementation process which demonstrated the importance of using implementation methods and incorporating a researcher-clinician partnership, not only for knowledge generation but also knowledge translation. Frequent communications and exchanging information with stakeholders at different levels, may be determinant to the success of each implementation phase. Further research is needed.

The barriers and facilitators of radical innovation implementation in secondary healthcare: a systematic review

PURPOSE

This study aimed to identify the barriers and facilitators related to the implementation of radical innovations in secondary healthcare.

DESIGN/METHODOLOGY/APPROACH

A systematic review was conducted and presented in accordance with a PRISMA flowchart. The databases PubMed and Web of Science were searched for original publications in English between the 1st of January 2010 and 6th of November 2020. The level of radicalness was determined based on five characteristics of radical innovations. The level of evidence was classified according to the level of evidence scale of the University of Oxford. The Consolidated Framework for Implementation Research was used as a framework to classify the barriers and facilitators.

FINDINGS

Based on the inclusion and exclusion criteria, nine publications were included, concerning six technological, two organizational and one treatment innovation. The main barriers for radical innovation implementation in secondary healthcare were lack of human, material and financial resources, and lack of integration and organizational readiness. The main facilitators included a supportive culture, sufficient training, education and knowledge, and recognition of the expected added value.

ORIGINALITY/VALUE

To our knowledge, this is the first systematic review examining the barriers and facilitators of radical innovation implementation in secondary healthcare. To ease radical innovation implementation, alternative performance systems may be helpful, including the following prerequisites: (1) Money, (2) Added value, (3) Timely knowledge and integration, (4) Culture, and (5) Human resources (MATCH). This study highlights the need for more high-level evidence studies in this area.

Specifying and Reporting Implementation Strategies Used in a School-Based Prevention Efficacy Trial

Background

Implementation strategies used to enhance the implementation of interventions during efficacy and effectiveness studies are rarely reported. Tracking and reporting implementation strategies during these phases has potential to improve future research studies and real-world implementation. We present an exemplar of how this might be executed by specifying and reporting the implementation strategies that were used during a school-based efficacy trial, Project POWER, which tested a trauma-informed prevention program delivered by a university research team, community members, and school staff facilitators in 29 schools.

Methods

Following the conclusion of the 4-year trial, core Project POWER research team members identified the implementation strategies that supported intervention delivery during the trial using an established taxonomy of school-based implementation strategies. The actors, actions, action targets, temporality, dose, and implementation outcomes were specified using established implementation strategies reporting guidelines.

Results

The research team identified 37 implementation strategies that were used during the Project POWER trial. Most strategies fell within the categories of Train and Educate Stakeholders, Use Evaluative and Iterative Strategies, and Develop Stakeholder Interrelationships. Actors included members of the research team and partner schools. Strategies were used multiple times during the preparation and implementation phases. Action targets were most often characteristics of individuals, implementation process, and characteristics of the inner setting. Strategies predominantly targeted the implementation outcomes of fidelity, acceptability, feasibility, and adoption.

Conclusions

This study provided evidence that implementation strategies are used and can be identified in efficacy research using a retrospective approach. Identifying and specifying implementation strategies used during the initial phases of the translational research pipeline can inform the implementation strategies that are carried forward, adapted, or discontinued in future trials and routine practice to improve implementation and effectiveness outcomes.

Medication adherence interventions in transplantation lack information on how to implement findings from randomized controlled trials in real-world settings: A systematic review

BACKGROUND

Growing numbers of randomized controlled trials (RCTs) are showing the effectiveness of interventions to improve medication adherence in transplantation recipients. However, real-world implementation is still a major challenge. This systematic review assesses the range of information available in RCTs supporting these interventions' clinical adoption in adult transplant populations.

METHODS

We included RCTs of interventions that a) targeted any phase of medication adherence in solid organ or allogeneic stem cell transplantation recipients and b) were published between January 2015 and November 2020. We excluded study protocols, conference abstracts and studies focusing only on pediatric populations. We identified relevant database and trial registries as well as traced references backward and citations forward. Implementation-relevant information was evaluated using adapted versions of Peters' ten criteria: 1. healthcare/organizational context; 2. social/economic/policy context; 3. patient involvement; 4. other stakeholder involvement; 5. sample representativeness; 6. trial conducted in a real-world-setting; 7. presence of feasibility study; 8. implementation strategy; 9. process evaluation; 10. implementation outcomes, using a stoplight color-rating system.

RESULTS

Screening 17'004 titles/abstracts resulted in 23 eligible RCTs, including 2'339 patients (n = 19-209/study). All included studies focused on the implementation phase of medication adherence. The best-reported criteria were feasibility study (43%), representative sample (17%) and conducted in a real-world-setting (17%). Least reported were context (9%), implementation strategies (4%), process evaluation (4%).

CONCLUSIONS

RCTs testing medication adherence interventions tend to report limited implementation-relevant information. This hinders their translation to real-world transplant settings. Integrating implementation science principles early in the conceptualization of RCTs would fuel real-world-translation, reducing research waste.

A review of workplace mental health interventions and their implementation in public safety organizations

OBJECTIVE

Workplace mental health is relevant to public safety organizations due to the exposure that many public safety personnel (PSP) have to psychological trauma in the course of their daily work. While the importance of attending to PSP mental health has been established, the implementation of workplace mental health interventions is not as well understood. This scoping review describes workplace mental health interventions and their implementation in public safety organizations.

METHODS

English published primary studies with any publication date up to July 3, 2020 were considered. JBI methodology and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews was followed.

RESULTS

89 citations met inclusion criteria out of the 62,299 found. Articles and reports found were largely published within the last decade, most frequently from Western nations, and most often applied to police, followed by firefighters. The focus of interventions was commonly stress management and resilience, and a frequent implementation strategy was multi-session group training. Comprehensive quality improvement initiatives, a focus on supervisors and managers, and interventions across primary, secondary, and tertiary prevention, were infrequent.

CONCLUSION

Public safety organizations are frequently reporting on stress management and resilience interventions for police and firefighters, implemented through multi-session group training. A focus across a range of PSP, including paramedics, corrections officers, and emergency dispatchers, using implementation strategies beyond group training, is suggested. This area of research is currently expanding, with many studies published within the past decade; ongoing evaluation of the quality of interventions and implementation strategies is recommended.

Implementation of pharmacist-led services in primary care: A mixed-methods exploration of pharmacists' perceptions of a national educational resource package

Background:

To help alleviate the global pressure on primary care, there has been an increase in the number of clinical pharmacists within primary care. Educational resources are necessary to support this workforce and their development within this role. An educational resource package was developed in Scotland to support the General Practice Clinical Pharmacists (GPCPs), containing a hard copy Competency and Capability Framework (CCF), an online platform (TURAS) and both clinical and educational supervisors in 2016.

Objective:

To examine the implementation of a competency-based educational resource package through the exploration of pharmacists’ perceptions of its adoption, acceptability, appropriateness, and feasibility.

Methods:

Participants were GPCPs who had been part of a national training event between 2016 and 2018. The participants were given the opportunity to complete an online questionnaire or a semi-structured telephone interview. Both data collection tools were based on Proctor’s model of implementation outcomes: adoption, acceptability, appropriateness and feasibility. Areas covered included GPCPs’ perceptions and level of adoption of the educational resource package developed to support them in their role.

Results:

Of a potential 164 participants, 52 (31.7%) completed the questionnaire and 12 (7.3%) completed the interview. GPCPs indicated widespread adoption and were accepting of the resources; however, it was suggested that its value was undermined, as it was not associated with a qualification. The appropriateness and feasibility of the resources depended on GPCPs’ individual situation (including current role, previous job experience, time available, support received from peers and supervisors, and perceptions of resources available).

Conclusions:

The suitability of the CCF was evidenced by participants’ adoption and acceptance of the resource, indicating the necessity of a competence-based framework to support the GPCPs’ role. However, its suitability was hindered in terms of varied perceptions of appropriateness and feasibility. Despite the limited sample size, the results indicate that the value of these resources should be promoted across primary care; nevertheless further facilitation is required to allow GPCPs to fully engage with the resources.

A Systematic Review of Mental Health Interventions for ASD: Characterizing Interventions, Intervention Adaptations, and Implementation Outcomes

Youth with autism spectrum disorder (ASD) have high rates of co-occurring mental health needs that necessitate mental health interventions. Given the unique clinical characteristics of youth with ASD, there have been significant efforts to adapt and test mental health interventions for this population. Yet, characterization of the nature and types of interventions adaptations is limited, especially across the wide range of interventions tested for youth with ASD with a focus on implementation factors. Additionally, understanding how these interventions may be implemented in community services is limited. The aims of this systematic review are to characterize the (1) types of interventions tested for co-occurring mental health conditions for youth with ASD; (2) adaptations to mental health interventions for use with youth with ASD; and (3) implementation strategies, outcomes, and determinants of mental health interventions to inform their translation to community service settings. Eighty-three articles testing interventions targeting mental health symptoms in youth with ASD that included implementation factors in analyses were reviewed. The Stirman et al. (2013; 2019) FRAME adaptation, Powell et al. (2012;2015) implementation strategies, and Proctor et al. (2011) implementation outcomes taxonomies were applied to characterize the nature and types of adaptations for use with youth with ASD and types of implementation strategies, outcomes, and determinants used, when available, respectively. Of the interventions examined, the majority (64.1%) were originally designed to target youth mental health concerns and were then adapted to be used with ASD. The most common adaptations included those to the intervention content, particularly adding elements with tailoring or refining aspects of the intervention while maintaining core functions. Half of the articles described at least one implementation strategy used during intervention testing. Fidelity and acceptability were the most frequently examined implementation outcomes, with some examination of appropriateness and feasibility. Nineteen percent of articles described implementation determinants (i.e. barriers/facilitators) of these implementation outcomes. The common adaptations for ASD provide direction for future intervention development and for training community therapists. Further examination, specification, and reporting of implementation strategies and outcomes within ongoing efforts to adapt and interventions to meet the co-occurring mental health needs of youth ASD are needed to facilitate their translation to community settings. Areas for future research as well as clinical implications are discussed.

Implementing lung cancer screening in primary care: needs assessment and implementation strategy design

Lung cancer screening with low-dose computed tomography (CT) could help avert thousands of deaths each year. Since the implementation of screening is complex and underspecified, there is a need for systematic and theory-based strategies. Explore the implementation of lung cancer screening in primary care, in the context of integrating a decision aid into the electronic health record. Design implementation strategies that target hypothesized mechanisms of change and context-specific barriers. The study had two phases. The Qualitative Analysis phase included semi-structured interviews with primary care physicians to elicit key task behaviors (e.g., ordering a low-dose CT) and understand the underlying behavioral determinants (e.g., social influence). The Implementation Strategy Design phase consisted of defining implementation strategies and hypothesizing causal pathways to improve screening with a decision aid. Three key task behaviors and four behavioral determinants emerged from 14 interviews. Implementation strategies were designed to target multiple levels of influence. Strategies included increasing provider self-efficacy toward performing shared decision making and using the decision aid, improving provider performance expectancy toward ordering a low-dose CT, increasing social influence toward performing shared decision making and using the decision aid, and addressing key facilitators to using the decision aid. This study contributes knowledge about theoretical determinants of key task behaviors associated with lung cancer screening. We designed implementation strategies according to causal pathways that can be replicated and tested at other institutions. Future research is needed to evaluate the effectiveness of these strategies and to determine the contexts in which they can be effectively applied.

Outer-context determinants in the sustainment phase of a reimbursement-driven implementation of evidence-based practices in children's mental health services

BACKGROUND

Although there is increasing investment to implement evidence-based practices (EBPs) in public systems across the USA, continued or sustained use of EBPs after initial implementation remains a challenge. The low integration of EBPs in routine practice severely limits their public health impact, highlighting the need to understand factors that affect the return on costly investments in EBP implementation. This study aims to (1) characterize trajectories of EBP delivery volume through a reimbursement-driven implementation and (2) examine impacts of system-level policy regulatory activity and state-level mental health services funding on the implementation reimbursement strategy.

METHODS

This study involved secondary data analyses. Psychotherapy administrative claims and regulatory site visit data from the Los Angeles County Department of Mental Health and California state mental health expenditures were extracted from 2010 to 2017. Multilevel regression examined EBP claims volume over time with state expenditures and regulatory compliance as predictors.

RESULTS

EBP claims volume trajectories demonstrated a rapid initial increase, followed by a period of decrease, and a small increase in the final year. State mental health expenditures increased across time reflecting increased funding availability. State mental health expenditures and system regulatory compliance were inversely related to EBP claims volume.

CONCLUSIONS

The impact of reimbursement-driven EBP implementation strategy is sensitive to multiple outer-context determinants. At the system level, commitment to fidelity of implementation regulations resulted in reduced use of the reimbursement strategy. Alternative reimbursement streams not tied to EBPs coupled with an expanded array of reimbursable services also impacted the use of the reimbursement strategy to implement EBPs.

The Practice-Based Implementation (PBI) Network: Technology (Tech) into Care pilot

Technology can support the use of evidence-based behavioral health treatments, as well as serve as a link to care to improve patient engagement and adherence to treatment. While research suggests that mobile applications (apps) specifically have the potential to enhance the delivery of behavioral health treatment, little guidance exists on how clinicians can effectively and safely integrate mobile apps into their clinical practice. The purpose of this paper is to describe the Department of Defense Practice-Based Implementation (PBI) Network's approach to implementation and present the results of the Technology (Tech) into Care pilot. The PBI Network conducted this pilot to address providers' knowledge gaps regarding the integration of mobile apps into behavioral health care within the Military Health System (MHS) and examine the feasibility of widespread dissemination of the practice change in the MHS. A mixed-method design guided by the Promoting Action on Research Implementation in Health Services framework was used to evaluate the approach and identify facilitators and barriers to practice change. Positive preliminary support for the feasibility of the Tech into Care approach was demonstrated in improvements in providers' knowledge, comfort, and perceived level of preparation to integrate mobile apps into care. Organizational challenges including time constraints and staff turnover negatively impacted implementation success. The PBI Network Tech into Care implementation approach is a feasible method to address knowledge gaps related to the integration of mobile apps into clinical practice within the MHS. Contextual factors, such as competing time demands, often impede providers' ability to effectively integrate mobile apps into care.

Three Flavorings for a Soup to Cure what Ails Mental Health Services

With new tools from artificial intelligence and new perspectives on personalizing interventions, we could revolutionize the way mental health services are delivered and achieve major gains in improving the public's mental health. We examine Dr. Bickman's vision around these technological and paradigm changes that would usher in major scientific, workforce training, and societal cultural changes. We argue that additional efforts in research evaluations in implementation have the potential to scale up and adapt existing interventions and scale them out to diverse populations and service systems. The next stage of this work involves testing the effectiveness of personalized interventions that are preferred by the public and integrating these choices into sustainable service systems. We note cautions on the delivery of these programs as automated algorithmic recommendations are heretofore foreign to humans.