Systematic review of the Exploration, Preparation, Implementation, Sustainment (EPIS) framework

Characterizing Shared and Unique Implementation Influences in Two Community Services Systems for Autism: Applying the EPIS Framework to Two Large-Scale Autism Intervention Community Effectiveness Trials

The purpose of this study was to examine common and unique factors influencing implementation process for two evidence-based interventions for children with autism spectrum disorder (ASD) in mental health and education service contexts. This study prospectively collected qualitative data from intervention developers and research staff on the implementation process within the context of two separate ASD intervention effectiveness trials. Results reveal common and unique factors influencing implementation in both study contexts. Implementation leadership and provider attitudes and motivation emerge as key influences on implementation across systems. These findings provide promising targets for modular implementation interventions that can be leveraged within growing, large-scale translation efforts in usual care.

Research Community Collaboration in Observational Implementation Research: Complementary Motivations and Concerns in Engaging in the Study of Implementation as Usual

Implementation research is dominated by studies of investigator-driven implementation of evidence-based practices (EBPs) in community settings. However, systems of care have increasingly driven the scale-up of EBPs through policy and fiscal interventions. Research community partnerships (RCPs) are essential to generating knowledge from these efforts. Interviews were conducted with community stakeholders (system leaders, program managers, therapists) involved in a study of a system-driven implementation of multiple EBPs in children's mental health services. Findings suggest novel considerations in initial engagement phases of an RCP, given the unique set of potentially competing and complementary interests of different stakeholder groups in implementation as usual.

Shifting probation culture and advancing juvenile probation reform through a community-based, participatory action research-informed training

Many jurisdictions are currently transforming their juvenile probation systems to reflect research-informed, developmentally based case management approaches. Training, developed through a community-based, participatory action research (CBPAR) framework and guided by a community-academic partnership, may promote probation staff's readiness for such change. This paper examined whether such training could serve as an effective method to impact organizational culture by exploring shifts in probation staff's knowledge, attitudes, and beliefs about graduated response - a structured juvenile probation case management system that aligns with research on adolescent development and youth behavior change. Data came from 559 juvenile probation staff trained in graduated response theory and practice between 2016 and 2019. Results demonstrated that at the end of a 1-day, CBPAR-grounded training program, participating staff reported significantly more buy-in to graduated response, greater knowledge of this new system, and attitudes more strongly in favor of graduated response and intended behaviors. Study findings revealed important shifts indicative of readiness for change within juvenile probation departments. Additional implementation strategies are needed to extend training gains and overcome organizational- and system-level barriers to change.

Concordance between Community Supervisor and Provider Ratings of Fidelity: Examination of Multi-Level Predictors and Outcomes

Fidelity is conceptualized as a key mechanism impacting treatment outcomes, yet community examination of fidelity is limited, partly due to limited feasible training methods and tools. Provider-report tools represent a feasible method for improving routine fidelity monitoring. Additional work examining the effectiveness of these methods and concordance with other fidelity sources is needed. Further, provider and client factors impact both intervention implementation and concordance, but their impact on fidelity measurement is poorly understood. Using a train-the-trainer methodology, we examine concordance between three methods of assessing fidelity (trained independent coders, supervisor evaluation and provider self-report) using a fidelity assessment tool adapted for community use. Multilevel models examined predictors (provider- and child- level factors) and outcomes (provider average fidelity) of concordance. Results suggest supervisors and providers are able to use the fidelity tool, but demonstrated variable concordance, with higher concordance with trained coders for supervisors than providers. Provider credentials and child language were marginally predictive of rater concordance. Higher supervisor accuracy was associated with higher provider fidelity. Findings inform broader efforts to improve the development and implementation of feasible fidelity tools for routine use in community settings and highlight multi-level factors impacting the adoption and implementation of such tools.

Sustainability planning in the US response to the opioid crisis: An examination using expert and text mining approaches

Between January 2016 and June 2020, the Substance Abuse and Mental Health Services Administration rapidly distributed $7.5 billion in response to the U.S. opioid crisis. These funds are designed to increase access to medications for addiction treatment, reduce unmet treatment need, reduce overdose death rates, and provide and sustain effective prevention, treatment and recovery activities. It is unclear whether or not the services developed using these funds will be sustained beyond the start-up period. Based on 34 (64%) State Opioid Response (SOR) applications, we assessed the states' sustainability plans focusing on potential funding sources, policies, and quality monitoring. We found variable commitment to sustainability across response plans with less than half the states adequately describing sustainability plans. States with higher proportions of opioid prescribing, opioid misuse, and poverty had somewhat higher scores on sustainment. A text mining/machine learning approach automatically rated sustainability in SOR applications with an 82% accuracy compared to human ratings. Because life saving evidence-based programs and services may be lost, intentional commitment to sustainment beyond the bolus of start-up funding is essential.

Identifying the "Active Ingredients" of a School-Based, Workplace Safety and Health Training Intervention

Young workers in many industrialized countries experience a higher rate of largely preventable occupation-related injuries compared with adults. Safety education and training are considered critical to the prevention of these incidents. This can be promoted by the dissemination and scale-out of an evidence-based, safety training programs in vocational education. The aim of this study was to identify the intervention core components that comprise the "active ingredients" of a safety training intervention for young workers and assess the impact on student outcomes of interest. Fidelity of implementation was operationalized using measures of adherence and quality of intervention delivery. For this study, data were collected through a school-based, cluster randomized trial conducted in 2015 in eight Finnish upper secondary-level vocational schools (n = 229 students in 22 groups, each with one teacher). Results indicate that the intervention core components (safety skills training, safety inoculation training, a positive atmosphere for safety learning, and active learning techniques) had differing associations with student outcomes. Adherence related to the acquisition of safety skills training was the strongest active ingredient in terms of positive effects. Furthermore, quality of delivery in terms of fostering positive learning atmosphere and utilizing active learning methods was associated especially with motivational outcomes. These findings indicate that different active ingredients complemented each other. Contrary to expectations, we found no statistically significant relationship between any of the core components and risk-taking attitudes. The current study advances prevention science by identifying the active ingredients of an evidence-based intervention, implemented in Finnish vocational school settings, that helps protect young workers from work-related morbidity and mortality.

Stepped-wedge randomized controlled trial of a novel opioid court to improve identification of need and linkage to medications for opioid use disorder treatment for court-involved adults

In response to the opioid crisis in New York State (NYS), the Unified Court System developed a new treatment court model-the opioid intervention court-designed around 10 Essential Elements of practice to address the flaws of existing drug courts in handling those with opioid addiction via broader inclusion criteria, rapid screening, and linkage to medications to treat opioid use disorder (MOUD). The new court model is now being rolled out statewide yet, given the innovation of the opioid court, the exact barriers to implementation in different counties with a range of resources are largely unknown. We describe a study protocol for the development and efficacy-test of a new implementation intervention (Opioid Court REACH; Research on Evidence-Based Approaches to Court Health) that will allow the opioid court, as framed by the 10 Essential Elements, to be scaled-up across 10 counties in NYS. Using a cluster-randomized stepped-wedge type-2 hybrid effectiveness-implementation design, we will test: (a) the implementation impact of Opioid Court REACH in improving implementation outcomes along the opioid cascade of care (screening, referral, treatment enrollment, MOUD initiation), and (b) the clinical and cost effectiveness of Opioid Court REACH in improving public health (treatment retention/court graduation) and public safety (recidivism) outcomes. Opioid Court REACH has the potential to improve management of individuals with opioid addiction in the court system via widespread scale-up of the opioid court model across the U.S., should this study find it to be effective.

Train-to-Sustain: Predictors of Sustainment in a Large-Scale Implementation of Parent-Child Interaction Therapy

Sustainment of evidence-based practices is necessary to ensure their public health impact. The current study examined predictors of sustainment of Parent-Child Interaction Therapy (PCIT) within a large-scale system-driven implementation effort in Los Angeles County. Data were drawn from PCIT training data and county administrative claims between January 2013 and March 2018. Participants included 241 therapists from 61 programs. Two sustainment outcomes were examined at the therapist- and program-levels: 1) PCIT claim volume and 2) PCIT claim discontinuation (discontinuation of claims during study period; survival time of claiming in months). Predictors included therapist- and program-level caseload, training, and workforce characteristics. On average, therapists and programs continued claiming to PCIT for 17.7 and 32.3 months, respectively. Across the sustainment outcomes, there were both shared and unshared significant predictors. For therapists, case-mix fit (higher proportions of young child clients with externalizing disorders) and participation in additional PCIT training activities significantly predicted claims volume. Furthermore, additional training activity participation was associated with lower likelihood of therapist PCIT claim discontinuation in the follow-up period. Programs with therapists eligible to be internal trainers were significantly less likely to discontinue PCIT claiming. Findings suggest that PCIT sustainment may be facilitated by implementation strategies including targeted outreach to ensure eligible families in therapist caseloads, facilitating therapist engagement in advanced trainings, and building internal infrastructure through train-the-trainer programs.

Massachusetts Justice Community Opioid Innovation Network (MassJCOIN)

A major driver of the U.S. opioid crisis is limited access to effective medications for opioid use disorder (MOUD) that reduce overdose risks. Traditionally, jails and prisons in the U.S. have not initiated or maintained MOUD for incarcerated individuals with OUD prior to their return to the community, which places them at high risk for fatal overdose. A 2018 law (Chapter 208) made Massachusetts (MA) the first state to mandate that five county jails deliver all FDA-approved MOUDs (naltrexone [NTX], buprenorphine [BUP], and methadone). Chapter 208 established a 4-year pilot program to expand access to all FDA-approved forms of MOUD at five jails, with two more MA jails voluntarily joining this initiative. The law stipulates that MOUD be continued for individuals receiving it prior to detention and be initiated prior to release among sentenced individuals where appropriate. The jails must also facilitate continuation of MOUD in the community on release. The Massachusetts Justice Community Opioid Innovation Network (MassJCOIN) partnered with these seven diverse jails, the MA Department of Public Health, and community treatment providers to conduct a Type 1 hybrid effectiveness-implementation study of Chapter 208. We will: (1) Perform a longitudinal treatment outcome study among incarcerated individuals with OUD who receive NTX, BUP, methadone, or no MOUD in jail to examine postrelease MOUD initiation, engagement, and retention, as well as fatal and nonfatal opioid overdose and recidivism; (2) Conduct an implementation study to understand systemic and contextual factors that facilitate and impede delivery of MOUDs in jail and community care coordination, and strategies that optimize MOUD delivery in jail and for coordinating care with community partners; (3) Calculate the cost to the correctional system of implementing MOUD in jail, and conduct an economic evaluation from state policy-maker and societal perspectives to compare the value of MOUD prior to release from jail to no MOUD among matched controls. MassJCOIN made significant progress during its first six months until the COVID-19 pandemic began in March 2020. Participating jail sites restricted access for nonessential personnel, established other COVID-19 mitigation policies, and modified MOUD programming. MassJCOIN adapted research activities to this new reality in an effort to document and account for the impacts of COVID-19 in relation to each aim. The goal remains to produce findings with direct implications for policy and practice for OUD in criminal justice settings.

What Happens When Training Goes Virtual? Adapting Training and Technical Assistance for the School Mental Health Workforce in Response to COVID-19

The Northwest Mental Health Technology Transfer Center (MHTTC) provides workforce training and technical assistance (TA) to support evidence-based school mental health practices. Historically, this support targeted school professionals through in-person and online trainings, workshops, and coaching. However, in response to COVID-19 restrictions, all support moved to online formats, and the Center introduced trainings for families and caregivers. The purpose of this article is to present preliminary process and outcome data that compare the reach and impact of support before and following COVID-19-related restrictions. Results suggest that transition to online support resulted in a wider reach and a more diverse audience, with no decrease in trainee satisfaction and perceived impact. Furthermore, families and caregivers reported positive gains in knowledge and behaviors following participation in a virtual youth suicide prevention training. Together, these findings suggest that online training and TA can provide tangible benefits to professionals and family members who support student mental health.

Barriers to Implementing Motivational Interviewing in Addiction Treatment: A Nominal Group Technique Process Evaluation

Background

The successful implementation of evidence-based practices (EBPs) in real-world settings requires an adaptive approach and ongoing process evaluation and tailoring. Although conducting a needs assessment during the preparation phase of implementation is beneficial, it is challenging to predict all barriers to EBP implementation that may arise over the course of implementation and sustainment. This article describes a process evaluation that identified emergent and persistent barriers that impacted the implementation of an EBP across multiple behavioral health organizations and clinics.

Methods

This study was conducted during the first cohort of a cluster randomized controlled trial testing the effectiveness of the Leadership and Organizational Change for Implementation (LOCI) strategy to implement motivational interviewing (MI) in substance use disorder treatment agencies and clinics. We used a modified nominal group technique (NGT) in which clinic leaders identified barriers faced during the implementation process. Barriers were categorized, then ranked and rated according to leaders' perceptions of each barrier's influence on implementation. The barriers were then contextualized through individual qualitative interviews.

Results

Fifteen barriers were identified, grouped into staff-level barriers, management-level barriers, and implementation program barriers. Time and resistance to MI were rated as the most influential staff-level barriers. Among management-level barriers, time was also rated highest, followed by turnover and external contractual constraints. The most influential implementation barrier was client apprehension of recording for fidelity assessment and feedback. Individual interviews supported these findings and provided suggested adaptations for future implementation efforts.

Conclusion

EBP implementation is an ongoing process whereby implementation strategies must be proactively and strategically tailored to address emergent barriers. This research described a process evaluation that was used to identify 15 emergent and/or persistent barriers related to staff, management, and the implementation program. Using implementation strategies that can be tailored and/or adapted to such emergent barriers is critical to implementation effectiveness.

Community-based implementation strategy use and satisfaction: A mixed-methods approach to using the ERIC compilation for organizations serving children on the autism spectrum

Background

Despite efforts to use standardized taxonomy and research reporting, documenting implementation strategies utilized in community settings remains challenging. This case study demonstrates a practical approach to gather use of and satisfaction with implementation strategies utilized within community-based sites to understand community providers' perspectives of implementing an early intensive behavioral intervention (EIBI) for children on the autism spectrum across different settings.

Methods

Using a sequential explanatory mixed-methods design, survey and interview data were collected from directors/supervisors and direct providers (n = 26) across three sites (one university and two community-based replication sites). The Implementation Strategies and Satisfaction Survey (ISSS) was administered to identify staff-reported implementation strategy use and satisfaction. Informed by quantitative results, follow-up semi-structured interviews were conducted with a subsample (n = 13) to further understand staff experiences with endorsed implementation strategies and elicit recommendations for future efforts.

Results

Survey results were used to demonstrate frequencies of implementation strategies endorsed by site and role. Overall, staff felt satisfied with implementation strategies used within their agencies. Content analysis of qualitative data revealed three salient themes related to implementation strategy use-context, communication, and successes and challenges-providing in-depth detail on how strategies were utilized, and strategy effectiveness based on community providers' experiences. Recommendations were also elicited to improve strategy use within "broader" community settings.

Conclusions

The project demonstrated a practical approach to identifying and evaluating implementation strategies used within sites delivering autism services. Reporting implementation strategies using the ISSS can provide insight into community providers' perspectives and satisfaction with agency implementation strategy use that can generate more relevant and responsive strategies to address barriers in community settings. Plain language abstract Examining community providers' preferences and experiences with implementation strategies used to facilitate evidence-based practice uptake can broaden our understanding of what, how, and why implementation strategies work in community-based settings ( Chaudoir et al., 2013; Leeman et al., 2017; Proctor et al., 2013). Such efforts have great potential to tailor implementation strategies to address barriers/facilitators typically found in community-based settings.This case study demonstrates a practical approach using mixed methodology to: (a) gather self-reported use of and satisfaction with implementation strategies to understand community providers' perspectives of implementation strategy success. Using a new survey, the Implementation Strategies and Satisfaction Survey (ISSS) conjoined with interviews, the study demonstrated a practical approach using standardized language to report strategies used in one university-based site and two community-based replication sites that deliver an early intensive behavioral intervention (EIBI) for children on the autism spectrum. This paper contributes to one of the five priorities to enhance public health impact-improve tracking and reporting of implementation strategies utilized when translating research into practice ( Dingfelder & Mandell, 2011; Powell et al., 2019; Stahmer et al., 2019). This approach emphasizes the importance of understanding context (e.g., community organizations providing services to children on the autism spectrum) to develop strategies that work better for EIBI implementation and scale-up. Understanding community provider's preferences and experiences with implementation strategies can support use of implementation strategies that better fit usual care contexts, with the ultimate goal of improving implementation practice in community-based settings.

Prospective predictors of sustainment of multiple EBPs in a system-driven implementation context: Examining sustained delivery based on administrative claims

Background

System-driven scale-up of multiple evidence-based practices (EBPs) is an increasingly common method used in public mental health to improve care. However, there are little data on the long-term sustained delivery of EBPs within these efforts, and previous studies have relied on retrospective self-report within cross-sectional studies. This study identified prospective predictors of sustained EBP delivery at the EBP-, therapist-, and organizational-levels using survey and administrative claims data within a large-scale system-driven implementation effort.

Methods

777 therapists and 162 program leaders delivering at least one of six EBPs of interest completed surveys assessing perceptions of EBPs and organizational context. These surveys were linked to administrative data to examine prospective predictors of therapists' EBP delivery over 33 months.

Results

Five of the six EBPs implemented showed sustained delivery in the system, with volume varying by EBP. Although total EBP claim volume per therapist decreased over time, the volume ratio (ratio of EBP-specific claims to total EBP and non-EBP claims) stayed relatively stable. Multilevel models revealed that EBPs that required consultation, had unstructured content, higher therapist self-efficacy with the EBP, and more positive program leader perceptions of the EBP were associated with greater sustained volume and volume ratio of the EBP. Therapists who were trained in fewer EBPs, who were unlicensed, and who worked in agencies rated by program leaders as lower on organizational staff autonomy and stress showed greater sustained EBP volume and volume ratio. Finally, more direct service hours per week provided by therapist predicted greater sustained EBP volume, but lower volume ratio.

Conclusions

The results point to the importance of EBP, therapist, and organizational factors that may be targeted in implementation strategies to promote the sustainment of EBPs.

Scale up mHealth HIV interventions: site and public health perspectives and lessons learned from P3

A number of mobile health (mHealth) interventions have been shown to be effective and highly acceptable tools for improving human immunodeficiency virus (HIV) prevention and care for youth. Scale-up of efficacious technology-based interventions is challenging and best practices for scale-up have not been clearly established. Developers of mHealth interventions should have plans in mind for wide scale implementation throughout all stages of development including planning, during trials and during analysis and dissemination. We discuss an approach of focus on researchers, funders and potential implementers including members of the community, public health practitioners and policymakers during initial planning, trials, analysis and dissemination, and planning for scale-up. Development of the P3 (Prepared, Protected, emPowered) mobile application (app), an intervention built to encourage and increase pre-exposure prophylaxis (PrEP) adherence among young men who have sex with men (YMSM) and young transgender women who have sex with men (YTWSM), is discussed in terms of designing for scale-up and lessons learned.

Implementing a complex psychosocial intervention for unstably housed Veterans: A realist-informed evaluation case study

Background

Only 7% of individuals with co-occurring mental health and substance use disorder (COD) receive services for both conditions. We implemented and evaluated maintaining independence and sobriety through systems integration, outreach and networking-Veteran's edition (MISSION-Vet), an evidence-based manualized psychosocial intervention for Veterans with CODs. This paper identifies the generative mechanisms that explain "how, for whom, and under what conditions" MISSION-Vet adoption, implementation, and fidelity work when applied in a complex adaptive system with facilitation support.

Methods

Within two VA healthcare systems (Sites A and B), a hybrid III trial tested facilitation to implement MISSION-Vet. We conducted a two-site case study based on 42 semi-structured consolidated framework for implementation research (CFIR) guided interviews with site leadership, implementers (social workers, peer specialists), and team members (facilitators, site leads). Interviews were coded and CFIR constructs used to generate "Context  +  Mechanism  =  Outcome" configurations to understand the conditions of MISSION-Vet adoption, implementation, and fidelity.

Results

Site A was low, and Site B was high in adoption, implementation, and fidelity. Adoption hesitancy/eagerness (outcome) resulted from the interaction of "external policy" (context) dampening/encouraging a "tension for change" (mechanism). Implementation intensity (outcome) was based on how "peer pressure" or practice culture (context) activated staff "self-efficacy" (mechanism) to engage with MISSION-Vet and appraise its "relative advantage" over current practices (mechanism). Fidelity relied on how "staffing structure and availability" (context) activated/muted "facilitation" (mechanism) to result in strategy and intervention adaptation (outcome).

Conclusions

We delineated how specific contexts activated certain mechanisms to drive the different stages of implementation of a multi-faceted COD treatment intervention.

Trial registration

ClinicalTrials.gov, NCT02942979.

Plain language abstract

Implementation is inherently dynamic and influenced by interdependent factors operating at the individual, organizational, and system levels. This is especially true for complex interventions addressing co-occurring mental health and substance use disorders because such interventions involve multiple treatment modalities delivered simultaneously, in busy practice settings, with challenging populations. This paper pairs consolidated framework for implementation research (CFIR) constructs with a realist evaluation approached to generate configurations important to the adoption, implementation, and adaptation stages of a highly complex intervention addressing the behavioural health and housing needs of a vulnerable population. Each configuration describes how contextual factors trigger mechanisms to generate implementation outcomes and answers "what works for whom, in what circumstances and in what respects, and how?" These findings further our understanding of possible mechanisms of change and push us to be more precise about identifying causal relationships among constructs that contribute to the success of implementing complex interventions. This work also moves us to think theoretically and methodologically in a more dynamic fashion, thereby leading to more responsive implementation practice.

A Taxonomy for External Support for Practice Transformation

BACKGROUND

There is no commonly accepted comprehensive framework for describing the practical specifics of external support for practice change. Our goal was to develop such a taxonomy that could be used by both external groups or researchers and health care leaders.

METHODS

The leaders of 8 grants from Agency for Research and Quality for the EvidenceNOW study of improving cardiovascular preventive services in over 1500 primary care practices nationwide worked collaboratively over 18 months to develop descriptions of key domains that might comprehensively characterize any external support intervention. Combining literature reviews with our practical experiences in this initiative and past work, we aimed to define these domains and recommend measures for them.

RESULTS

The taxonomy includes 1 domain to specify the conceptual model(s) on which an intervention is built and another to specify the types of support strategies used. Another 5 domains provide specifics about the dose/mode of that support, the types of change process and care process changes that are encouraged, and the degree to which the strategies are prescriptive and standardized. A model was created to illustrate how the domains fit together and how they would respond to practice needs and reactions.

CONCLUSIONS

This taxonomy and its use in more consistently documenting and characterizing external support interventions should facilitate communication and synergies between 3 areas (quality improvement, practice change research, and implementation science) that have historically tended to work independently. The taxonomy was designed to be as useful for practices or health systems managing change as it is for research.

Engaging community voices to assess Kenya's strengths and limitations to support a child maltreatment prevention program

BACKGROUND

Preventing child maltreatment is a global mission of numerous international organizations, with parent support programs as the critical prevention strategy. In Kenya, 70 % of children are at risk of experiencing abuse and neglect, most often by their parents. Yet, there is a lack of evidence-based parent support programs, and a limited understanding of Kenya's capacity and infrastructures (e.g., policies, funding, service agencies) to support and sustain such programs.

OBJECTIVE

The purpose of this study was to assess systematically Kenya's strengths and limitations to implement a parent support program using a mixed-methods study design.

PARTICIPANTS AND METHODS

Twenty-one community stakeholders from Kenya completed the World Health Organization's (WHO) Readiness Assessment for the Prevention of Child Maltreatment to understand Kenya's preparedness to undertake a prevention program. In addition, 91 participants (e.g., parents, community health workers, community leaders) took part in focus group discussions or individual interviews to understand existing support networks around parenting programs.

RESULTS

Kenya's overall 'readiness' score was comparable to the other countries that completed the WHO survey. The survey results revealed Kenya's strengths and limitations across the ten readiness dimensions. Several themes emerged from the focus groups and interviews, including the diverse sources of support for parents, specific programs available for parents, and gaps in services offered.

CONCLUSIONS

The results document ways to build upon Kenyan's existing strengths to facilitate implementation of an evidence-based prevention program. These results also highlight the significant need to understand local context when adapting parenting programs for low/middle income countries (LMICs).

Evaluating the dissemination and scale-up of two evidence-based parenting interventions to reduce violence against children: study protocol

Background

Eliminating violence against children is a prominent policy goal, codified in the Sustainable Development Goals, and parenting programs are one approach to preventing and reducing violence. However, we know relatively little about dissemination and scale-up of parenting programs, particularly in low- and middle-income countries (LMICs). The scale-up of two parenting programs, Parenting for Lifelong Health (PLH) for Young Children and PLH for Parents and Teens, developed under Creative Commons licensing and tested in randomized trials, provides a unique opportunity to study their dissemination in 25 LMICs.

Methods

The Scale-Up of Parenting Evaluation Research (SUPER) study uses a range of methods to study the dissemination of these two programs. The study will examine (1) process and extent of dissemination and scale-up, (2) how the programs are implemented and factors associated with variation in implementation, (3) violence against children and family outcomes before and after program implementation, (4) barriers and facilitators to sustained program delivery, and (5) costs and resources needed for implementation.Primary data collection, focused on three case study projects, will include interviews and focus groups with program facilitators, coordinators, funders, and other stakeholders, and a summary of key organizational characteristics. Program reports and budgets will be reviewed as part of relevant contextual information. Secondary data analysis of routine data collected within ongoing implementation and existing research studies will explore family enrolment and attendance, as well as family reports of parenting practices, violence against children, child behavior, and child and caregiver wellbeing before and after program participation. We will also examine data on staff sociodemographic and professional background, and their competent adherence to the program, collected as part of staff training and certification.

Discussion

This project will be the first study of its kind to draw on multiple data sources and methods to examine the dissemination and scale-up of a parenting program across multiple LMIC contexts. While this study reports on the implementation of two specific parenting programs, we anticipate that our findings will be of relevance across the field of parenting, as well as other violence prevention and social programs.

Liver Allografts From Older Donors With or Without Recovery of Thoracic Organs and Their Impact on Hepatic Graft and Patient Survival

OBJECTIVES

Multiorgan procurement involving thoracic organs prolongs the liver recovery cross-clamp time. This may impact the outcome of hepatic allograft, more so in older donors (age > 60 years). We compared the outcomes of liver allografts from older donors with and without recovery of thoracic organs.

MATERIALS AND METHODS

Using the Scientific Registry of Transplant Recipients database, we compared survival outcomes of 258 adults who received a liver allograft from older donors with thoracic organ recovery (group A) with 6006 patients who received liver allografts from older donor without thoracic organ recovery (group B). Furthermore, we performed a subgroup analysis matched for recipient and donor risk factors including presence of hypertension, diabetes mellitus, renal function, donor age, and use of inotropes. For the final analyses, there were 159 patients in group A and 468 in group B.

RESULTS

The 1-month, 1-year, 3-year, and 5-year patient survival rates in group A were 95%, 91.6%, 70.1%, and 65.5% compared with 95%, 92%, 70%, and 57.7% in group B, respectively (P = .695). Graft survival rates for group A at the same time points were 91.5%, 81.0%, 71.7%, and 57.4% compared with 91.3%, 81.1%, 61.9%, and 50.4% in group B, respectively (P = .791). In the matched population, patient survival rates at 1 month, 1 year, 3 years, and 5 years were 95%, 83.1%, 77.1%, and 68.8% compared with 94.4%, 81.6%, 72.2%, and 66.8% in group B, respectively (P = .69). Graft survival rates at the same time points were 88.7%, 76.8%, 71.5%, and 63.1% in group A and 90.0%, 77.5%, 70.4%, and 62.5% in group B, respectively (P = .956).

CONCLUSIONS

Liver procurement with or without recovery of thoracic organs from donors > 60 years old does not affect liver grafts and recipient outcomes in the short-term or long-term and should be encouraged.

Implementing culturally competent transplant care and implications for reducing health disparities: A prospective qualitative study

BACKGROUND

Despite available evidence-based interventions that decrease health disparities, these interventions are often not implemented. Northwestern Medicine's® Hispanic Kidney Transplant Program (HKTP) is a culturally and linguistically competent intervention designed to reduce disparities in living donor kidney transplantation (LDKT) among Hispanics/Latinos. The HKTP was introduced in two transplant programs in 2016 to evaluate its effectiveness.

OBJECTIVE

This study assessed barriers and facilitators to HKTP implementation preparation.

METHODS

Interviews and group discussions were conducted with transplant stakeholders (ie administrators, nurses, physicians) during implementation preparation. The Consolidated Framework for Implementation Research (CFIR) guided interview design and qualitative analysis.

RESULTS

Forty-four stakeholders participated in 24 interviews and/or 27 group discussions. New factors, not found in previous implementation preparation research in health-care settings, emerged as facilitators and barriers to the implementation of culturally competent care. Implementation facilitators included: stakeholders' focus on a moral imperative to implement the HKTP, personal motivations related to their Hispanic heritage, and perceptions of Hispanic patients' transplant education needs. Implementation barriers included: stakeholders' perceptions that Hispanics' health insurance payer mix would negatively impact revenue, a lack of knowledge about LDKT disparities and patient data disaggregated by ethnicity/race, and a perception that the family discussion component was immoral because of the possibility of coercion.

DISCUSSION AND CONCLUSIONS

Our study identified novel barriers and facilitators to the implementation preparation of a culturally competent care intervention. Healthcare administrators can facilitate organizations' implementation of culturally competent care interventions by understanding factors challenging care delivery processes and raising clinical team awareness of disparities in LDKT.

Supplemental Nutrition Assistance Program (SNAP)-Authorized Retailers' Perceived Costs to Use Behavioral Economic Strategies to Encourage Healthy Product Sales

SNAP-authorized retailers could use marketing-mix and choice-architecture (MMCA) strategies to improve SNAP purchases, but associated costs are unknown. Perceived cost and inconvenience to implement eight MMCA strategies were assessed among 29 U.S. retailers. Differences in perspective were explored (owners vs. managers, corporate vs. independent retailers, and by format). Place changes (e.g., added refrigeration) were perceived more costly and prompting (e.g., shelf labeling) less costly. Managers rated the perceived inconvenience to make proximity changes higher than owners (3.78 ± 1.4 and 2.33 ± 1.2, respectively) (p < .05). Results can inform strategies to improve the adoption and implementation of healthy food retail programs.

The prospects for sustaining evidence-based responses to the US opioid epidemic: state leadership perspectives

BACKGROUND

The US 21st Century Cures Act provided $7.5 billion in grant funding to states and territories for evidence-based responses to the opioid epidemic. Currently, little is known about optimal strategies for sustaining these programs beyond this start-up funding.

METHODS

Using an inductive, conventional content analysis, we conducted key informant interviews with former and current state leaders (n = 16) about barriers/facilitators to sustainment and strategies for sustaining time-limited grants.

RESULTS

Financing and reimbursement, service integration, and workforce capacity were the most cited barriers to sustainment. Status in state government structure, public support, and spending flexibility were noted as key facilitators. Effective levers to increase chances for sustainment included strong partnerships with other state agencies, workforce and credentialing changes, and marshalling advocacy through public awareness campaigns.

CONCLUSIONS

Understanding the strategies that leaders have successfully used to sustain programs in the past can inform how to continue future time-limited, grant-funded initiatives.

Implementation strategies for digital mental health interventions in health care settings

U.S. health care systems are tasked with alleviating the burden of mental health, but are frequently underprepared and lack workforce and resource capacity to deliver services to all in need. Digital mental health interventions (DMHIs) can increase access to evidence-based mental health care. However, DMHIs commonly do not fit into the day-to-day activities of the people who engage with them, resulting in a research-to-practice gap for DMHI implementation. For health care settings, differences between digital and traditional mental health services make alignment and integration challenging. Specialized attention is needed to improve the implementation of DMHIs in health care settings so that these services yield high uptake, engagement, and sustainment. The purpose of this article is to enhance efforts to integrate DMHIs in health care settings by proposing implementation strategies, selected and operationalized based on the discrete strategies established in the Expert Recommendations for Implementing Change project, that align to DMHI-specific barriers in these settings. Guidance is offered in how these strategies can be applied to DMHI implementation across four phases commonly distinguished in implementation science using the Exploration, Preparation, Implementation, Sustainment Framework. Next steps to advance research in this area and improve the research-to-practice gap for implementing DMHIs are recommended. Applying implementation strategies to DMHI implementation will enable psychologists to systematically evaluate this process, which can yield an enhanced understanding of the factors that facilitate implementation success and improve the translation of DMHIs from controlled trials to real-world settings. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

Psychometric Properties of the General Organizational Index (GOI): A Measure of Individualization and Quality Improvement to Complement Program Fidelity

To assess the implementation of effective practices, mental health programs need standardized measures. The General Organizational Index (GOI), although widely used for this purpose, has received minimal psychometric research. For this study, we assessed psychometric properties of the GOI scale administered four times over 18 months during the implementation of a new program in 11 sites. The GOI scale demonstrated high levels of interrater reliability (.97), agreement between assessors on item ratings (86% overall), internal consistency (.77-.80 at three time points), sensitivity to change, and feasibility. We conclude that the GOI scale has acceptable psychometric properties, and its use may enhance implementation and research on evidence-based mental health practices.Trial registration: REK2015/2169. ClinicalTrials.gov Identifier: NCT03271242.

Preliminary Findings from Three Models of Motivational Interviewing Training in Jamaica

Introduction: We assessed satisfaction, fidelity, retention, and implementation considerations across three models of motivational interviewing training in Jamaica to identify a promising model for resource-poor settings.

Methods: We conducted t-tests to assess differences in fidelity and examined qualitative data for barriers and facilitators (n=52).

Results: Only 50−75% of all models' trainees completed coaching. Model 1 trainees' mean fidelity was 2.83/4.00 compared with Model 3 trainees' at 2.94/4.00 (t=−0.710, confidence interval=−0.427 to 0.207, p=0.483). Key barriers to completion and fidelity were lack of funding and time.

Conclusion: We found support for continuing workshop-only trainings; we did not find that higher contact hours produced improved trainee fidelity.

Lessons learned about the effective operationalization of champions as an implementation strategy: results from a qualitative process evaluation of a pragmatic trial

BACKGROUND

Though the knowledge base on implementation strategies is growing, much remains unknown about how to most effectively operationalize these strategies in diverse contexts. For example, while evidence shows that champions can effectively support implementation efforts in some circumstances, little has been reported on how to operationalize this role optimally in different settings, or on the specific pathways through which champions enact change.

METHODS

This is a secondary analysis of data from a pragmatic trial comparing implementation strategies supporting the adoption of guideline-concordant cardioprotective prescribing in community health centers in the USA. Quantitative data came from the community health centers' shared electronic health record; qualitative data sources included community health center staff interviews over 3 years. Using a convergent mixed-methods design, data were collected concurrently and merged for interpretation to identify factors associated with improved outcomes. Qualitative analysis was guided by the constant comparative method. As results from the quantitative and initial qualitative analyses indicated the essential role that champions played in promoting guideline-concordant prescribing, we conducted multiple immersion-crystallization cycles to better understand this finding.

RESULTS

Five community health centers demonstrated statistically significant increases in guideline-concordant cardioprotective prescribing. A combination of factors appeared key to their successful practice change: (1) A clinician champion who demonstrated a sustained commitment to implementation activities and exhibited engagement, influence, credibility, and capacity; and (2) organizational support for the intervention. In contrast, the seven community health centers that did not show improved outcomes lacked a champion with the necessary characteristics, and/or organizational support. Case studies illustrate the diverse, context-specific pathways that enabled or prevented study implementers from advancing practice change.

CONCLUSION

This analysis confirms the important role of champions in implementation efforts and offers insight into the context-specific mechanisms through which champions enact practice change. The results also highlight the potential impact of misaligned implementation support and key modifiable barriers and facilitators on implementation outcomes. Here, unexamined assumptions and a lack of evidence-based guidance on how best to identify and prepare effective champions led to implementation support that failed to address important barriers to intervention success.

TRIAL REGISTRATION

ClinicalTrials.gov , NCT02325531 . Registered 15 December 2014.

Bridging Systems to Implement HIV/STI Programming Within the Juvenile Justice System: Strengths and Challenges of a Local Change Team Approach

Justice-involved youth are at risk for HIV/STIs but do not access services. The complex challenges of improving the delivery of health-related services within juvenile justice (JJ) settings warrant exploration of strategies to close this service gap. This study describes the successes and challenges of utilizing a local change team (LCT) strategy comprising JJ and health agency staff to implement HIV/STI programming in JJ settings, across six counties in six states in the U.S. Five focus groups comprising n = 28 JJ and health agency staff who served as LCT members were conducted. Results demonstrated the structured nature of the collaborative process and strength of commitment among LCT members were necessary for successful implementation of HIV/STI programming. The use of LCTs comprising membership of JJ and (behavioral) health systems has broader applicability to other health and behavioral health issues faced by youth on probation that JJ staff may feel ill equipped to address.

Statewide Interagency Collaboration to Support Evidence-based Practice Scale Up: The California Autism Professional Training and Information Network (CAPTAIN)

The California Autism Professional Training and Information Network (CAPTAIN) is a statewide interagency collaboration with the goal of scaling up use of evidence-based practices (EBPs) for individuals with autism spectrum disorder (ASD). CAPTAIN began as a clinical initiative then further developed under the influence of implementation science methodology. The Exploration, Preparation, Implementation and Sustainment framework (EPIS) has impacted targeted strategy use for this statewide scale up of EBPs by informing the development of key partnerships, implementation goals, and collaborative processes within CAPTAIN. Currently, CAPTAIN has over 407 members representing 140 school and community agencies who provide training and coaching in EBP and meet regularly with regional teams. Outcome data indicate 51.9% of members provide training and coaching to more than three direct service providers/programs per year. Primary barriers to implementation of EBP were time for training (25.6%), lack of substitute teachers (16.5%), and staff lacking foundational skills (11.5%). Facilitators of implementation and sustainment of the CAPTAIN model include active participation in effective dissemination practices, creative funding and leveraging of local resources, development of the regional collaboratives with active membership, member commitment to EBP for ASD, and use of implementation science to identify and overcome barriers. The purpose of this paper is to highlight CAPTAIN as a model for statewide scale up of EBP in schools as well as other community agencies. Although these efforts have focused on EBP for ASD, the concepts, partnerships and procedures will likely be transferable to other focal issues and may be generalized across service sectors.

Implementation Science, Behavior Analysis, and Supporting Evidence-based Practices for Individuals with Autism

Although applied behavior analysis researchers have created efficacious treatment and intervention practices for children and youth with autism spectrum disorder (ASD), there is a gap between research and practice. Implementation Science (IS) and Organizational Behavior Management (OBM), based with Applied Behavior Analysis, are two parallel fields that could close this gap. This paper provides descriptions of both IS and OBM, highlighting their commonalities and unique featuers. The paper concludes with examples of how researchers have used IS and OBM to promote practitioners' use of evidence-based practices and services for children and youth with ASD.

Opportunities and challenges to integrating mental health into HIV programs in a low- and middle-income country: insights from the Nigeria implementation science Alliance

Background

In Nigeria, there is an estimated 1.9 million people living with HIV (PLHIV), 53% of whom utilize HIV care and services. With decreasing HIV-related deaths and increasing new infections, HIV with its associated comorbidities continue to be a key public health challenge in Nigeria. Untreated, comorbid mental disorders are a critical but potentially modifiable determinant of optimal HIV treatment outcomes.

This study aimed to identify the challenges and opportunities related to integrating mental health care into existing HIV programs in Nigeria.

Method

Attendees at the Nigeria Implementation Science Alliance (NISA)‘s 2019 conference participated in nominal group technique (NGT) exercise informed by the “Exploration, Preparation, Implementation, and Sustainment (EPIS)” framework. The NGT process was conducted among the nominal groups in two major sessions of 30-min phases followed by a 30-min plenary session. Data analysis proceeded in four steps: transcription, collation, theming and content analysis.

Results

The two major theoretical themes from the study were – opportunities and challenges of integrating mental health treatment into HIV services. Three sub-themes emerged on opportunities: building on health care facilities for HIV services (screening, counseling, task-sharing monitoring and evaluation frameworks), utilizing existing human resources or workforce in HIV programs (in-service training and including mental health in education curriculum) and the role of social and cultural structures (leveraging existing community, traditional and faith-based infrastructures). Four sub-themes emerged for challenges: double burden of stigma and the problems of early detection (HIV and mental health stigma, lack of awareness), existing policy gaps and structural challenges (fragmented health system), limited human resources for mental health care in Nigeria (knowledge gap and burnout) and dearth of data/evidence for planning and action (research gaps).

Conclusions

Potential for integrating treatments for mental disorders into HIV programs and services exist in Nigeria. These include opportunities for clinicians’ training and capacity building as well as community partnerships. Multiple barriers and challenges such as stigma, policy and research gaps would need to be addressed to leverage these opportunities. Our findings serve as a useful guide for government agencies, policy makers and research organizations to address co-morbid mental disorders among PLHIV in Nigeria.

Comparing, Contrasting, and Integrating Dissemination and Implementation Outcomes Included in the RE-AIM and Implementation Outcomes Frameworks

As the field of dissemination and implementation science matures, there are a myriad of outcomes, identified in numerous frameworks, that can be considered across individual, organizational, and population levels. This can lead to difficulty in summarizing literature, comparing across studies, and advancing translational science. This manuscript sought to (1) compare, contrast, and integrate the outcomes included in the RE-AIM and Implementation Outcomes Frameworks (IOF) and (2) expand RE-AIM indicators to include relevant IOF dissemination and implementation outcomes. Cross tabular comparisons were made between the constitutive definitions of each construct, across frameworks, to reconcile apparent discrepancies between approaches and to distinguish between implementation outcomes and implementation antecedents. A great deal of consistency was identified across approaches, including adoption (the intention, initial decision, or action to employ an evidence-based intervention), fidelity/implementation (the degree to which an intervention was delivered as intended), organizational maintenance/sustainability (extent to which a newly implemented treatment is maintained or institutionalized), and cost. The IOF construct of penetration was defined as a higher-order construct that may encompass the reach, adoption, and organizational maintenance outcomes within RE-AIM. Within the IOF approach acceptability, appropriateness, and feasibility did not match constitutive definitions of dissemination or implementation but rather reflected theoretical antecedents of implementation outcomes. Integration of the IOF approach across RE-AIM indicators was successfully achieved by expanding the operational definitions of RE-AIM to include antecedents to reach, adoption, implementation, and organizational maintenance. Additional combined metrics were also introduced including penetration, individual level utility, service provider utility, organizational utility, and systemic utility. The expanded RE-AIM indicators move beyond the current approaches described within both the RE-AIM framework and IOF and provides additional planning and evaluation targets that can contribute to the scientific field and increase the translation of evidence into practice.

The Implementation Research Logic Model: a method for planning, executing, reporting, and synthesizing implementation projects

BACKGROUND

Numerous models, frameworks, and theories exist for specific aspects of implementation research, including for determinants, strategies, and outcomes. However, implementation research projects often fail to provide a coherent rationale or justification for how these aspects are selected and tested in relation to one another. Despite this need to better specify the conceptual linkages between the core elements involved in projects, few tools or methods have been developed to aid in this task. The Implementation Research Logic Model (IRLM) was created for this purpose and to enhance the rigor and transparency of describing the often-complex processes of improving the adoption of evidence-based interventions in healthcare delivery systems.

METHODS

The IRLM structure and guiding principles were developed through a series of preliminary activities with multiple investigators representing diverse implementation research projects in terms of contexts, research designs, and implementation strategies being evaluated. The utility of the IRLM was evaluated in the course of a 2-day training to over 130 implementation researchers and healthcare delivery system partners.

RESULTS

Preliminary work with the IRLM produced a core structure and multiple variations for common implementation research designs and situations, as well as guiding principles and suggestions for use. Results of the survey indicated a high utility of the IRLM for multiple purposes, such as improving rigor and reproducibility of projects; serving as a "roadmap" for how the project is to be carried out; clearly reporting and specifying how the project is to be conducted; and understanding the connections between determinants, strategies, mechanisms, and outcomes for their project.

CONCLUSIONS

The IRLM is a semi-structured, principle-guided tool designed to improve the specification, rigor, reproducibility, and testable causal pathways involved in implementation research projects. The IRLM can also aid implementation researchers and implementation partners in the planning and execution of practice change initiatives. Adaptation and refinement of the IRLM are ongoing, as is the development of resources for use and applications to diverse projects, to address the challenges of this complex scientific field.

Barriers and Facilitators in the Strengthening Families Program (SFP 10-14) Implementation Process in Northeast Brazil: A Retrospective Qualitative Study

This study analyzed contextual barriers and facilitators in the implementation of Strengthening Families Program (SFP 10–14), Brazilian version, a family-based preventive program focused on the prevention of risk behaviors for adolescent health. SFP 10–14 was implemented between 2016 and 2017 for socioeconomically vulnerable families in four Northeast Brazilian states as a tool of the National Drug Policy. A retrospective qualitative study was carried out in which 26 implementation agents participated. Data from 16 individual interviews and two group interviews were analyzed through content analysis. The most recurrent barriers were the group facilitators’ working conditions, weak municipal administration, precarious infrastructure, inadequate group facilitator training methodologies, low adherence of managers and professionals, and funding scarcity. The conditions highlighted as favorable to the implementation were proper intersectoral coordination, engagement of involved actors, awareness of public agency administrators, municipal management efficacy, and efficient family recruitment strategies. Favorable political contexts, engagement of implementation agents, and intersectoral implementation strategies were identified as central to the success of the implementation of SFP 10–14, especially in the adoption of the intervention, community mobilization, and intervention delivery stages. Further studies should combine contexts, mechanisms, and results for a broad understanding of the effectiveness of this intervention in the public sector.

Advancing the pragmatic measurement of sustainment: a narrative review of measures

BACKGROUND

Sustainment, an outcome indicating an intervention continues to be implemented over time, has been comparatively less studied than other phases of the implementation process. This may be because of methodological difficulties, funding cycles, and minimal attention to theories and measurement of sustainment. This review synthesizes the literature on sustainment measures, evaluates the qualities of each measure, and highlights the strengths and gaps in existing sustainment measures. Results of the review will inform recommendations for the development of a pragmatic, valid, and reliable measure of sustainment.

METHODS

A narrative review of published sustainment outcome and sustainability measures (i.e., factors that influence sustainment) was conducted, including appraising measures in the Society of Implementation Research Collaboration (SIRC) instrument review project (IRP) and the Dissemination and Implementation Grid-Enabled Measures database initiative (GEM-D&I). The narrative review used a snowballing strategy by searching the reference sections of literature reviews and definitions of sustainability and sustainment. Measures used frequently and judged to be comprehensive and/or validated by a team of implementation scientists were extracted for analysis.

RESULTS

Eleven measures were evaluated. Three of the included measures were found in the SIRC-IRP, three in the GEM-D&I database, (one measure was in both databases) and six were identified in our additional searches. Thirteen constructs relating to sustainment were coded from selected measures. Measures covered a range of determinants for sustainment (i.e., construct of sustainability) as well as constructs of sustainment as an outcome. Strengths of the measures included, development by expert panels knowledgeable about particular interventions, fields or contexts, and utility in specific scenarios. A number of limitations were found in the measures analyzed including inadequate assessment of psychometric characteristics, being overly intervention or context specific, being lengthy and/or complex, and focusing on outer context factors.

CONCLUSION

There is a lack of pragmatic and psychometrically sound measures of sustainment that can be completed by implementation stakeholders within inner context settings (e.g., frontline providers, supervisors).

Incorporators, Early Investors, and Learners: a longitudinal study of organizational adaptation during EBP implementation and sustainment

BACKGROUND

The majority of literature on evidence-based practice (EBP) adaptation focuses on changes to clinical practices without explicitly addressing how organizations must adapt to accommodate a new EBP. This study explores the process of organizational-rather than EBP-adaptation during implementation and sustainment. To the authors' knowledge, there are no previous implementation studies that focus on organizational adaptation in this way.

METHODS

This analysis utilizes a case study approach to examine longitudinal qualitative data from 17 community-based organizations (CBOs) in one state and seven county-based child welfare systems. The CBOs had sustained a child-neglect intervention EBP (SafeCare®) for 2 to 10 years. The unit of analysis was the organization, and each CBO represented a case. Organizational-level profiles were created to describe the organizational adaptation process.

RESULTS

Three organizational-level adaptation profiles were identified as follows: incorporators (n = 7), early investors (n = 6), and learners (n = 4). Incorporators adapted by integrating SafeCare into existing operations to meet contractual or EBP fidelity requirements. Early Investors made substantial organizational adaptations during the early implementation period, then operated relatively consistently as the EBP became embedded in the organization and service system. Learners were characterized by steady and continuous attention to new ways that the organization could adapt to support the EBP.

CONCLUSION

The profiles demonstrated that there can be multiple effective paths to EBP sustainment. Organizational adaptation was calibrated to fit a CBO's operations (e.g., size of the program) and immediate environmental constraints (e.g., funding levels). Additionally, organizations fulfilled different functional roles in the network of entities involved in EBP implementation. Knowing organizational roles and adaptation profiles can guide implementation planning and help to structure contract designs that bridge the outer (system) and inner (organizational) contexts. Adaptation profiles can also inform the intensity of the implementation strategy tailoring process and the way that strategies are marketed to organizations.

Sustaining Treatment for Youth with Problematic Sexual Behavior: Administrator and Stakeholder Perspectives Following Implementation

This study examines administrator and stakeholder perspectives on factors influencing the sustainability of Problematic Sexual Behavior Cognitive Behavioral Therapy (PSB-CBT). After initial implementation of PSB-CBT, qualitative interviews (N = 42) on the sustainability of services were conducted with treatment program administrators (n = 10) and community stakeholders (n = 32) from six sites across the USA. Interviewees discussed key facilitators and barriers for sustainability and identified the following factors as impacting the sustainability of PSB-CBT: public relations, policy, inter-agency collaboration, funding, and workforce issues. Results were overall consistent with the EPIS (Exploration, Preparation, Implementation, Sustainment) model for implementation in public service sectors, but proposed modifications are also noted. Findings underscore the value of qualitative research in identifying best practices for sustaining valuable therapeutic interventions. The authors argue that future efforts to sustain evidence-based interventions, especially those dealing with sensitive topics around which misinformation and stigma are associated, should prioritize public relations (i.e., outreach, education) in addition to the intervention.

Ten recommendations for using implementation frameworks in research and practice

Background

Recent reviews of the use and application of implementation frameworks in implementation efforts highlight the limited use of frameworks, despite the value in doing so. As such, this article aims to provide recommendations to enhance the application of implementation frameworks, for implementation researchers, intermediaries, and practitioners.

Discussion

Ideally, an implementation framework, or multiple frameworks should be used prior to and throughout an implementation effort. This includes both in implementation science research studies and in real-world implementation projects. To guide this application, outlined are ten recommendations for using implementation frameworks across the implementation process. The recommendations have been written in the rough chronological order of an implementation effort; however, we understand these may vary depending on the project or context: (1) select a suitable framework(s), (2) establish and maintain community stakeholder engagement and partnerships, (3) define issue and develop research or evaluation questions and hypotheses, (4) develop an implementation mechanistic process model or logic model, (5) select research and evaluation methods (6) determine implementation factors/determinants, (7) select and tailor, or develop, implementation strategy(s), (8) specify implementation outcomes and evaluate implementation, (9) use a framework(s) at micro level to conduct and tailor implementation, and (10) write the proposal and report. Ideally, a framework(s) would be applied to each of the recommendations. For this article, we begin by discussing each recommendation within the context of frameworks broadly, followed by specific examples using the Exploration, Preparation, Implementation, Sustainment (EPIS) framework.

Summary

The use of conceptual and theoretical frameworks provides a foundation from which generalizable implementation knowledge can be advanced. On the contrary, superficial use of frameworks hinders being able to use, learn from, and work sequentially to progress the field. Following the provided ten recommendations, we hope to assist researchers, intermediaries, and practitioners to improve the use of implementation science frameworks.

Determinants of Implementing Evidence-Based Trauma-Focused Interventions for Children and Youth: A Systematic Review

A systematic review was conducted to identify determinants (barriers and facilitators) of implementing evidence-based psychosocial interventions for children and youth who experience emotional or behavioral difficulties due to trauma exposure. Determinants were coded, abstracted, and synthesized using the Exploration, Preparation, Implementation, and Sustainment framework. Twenty-three articles were included, all of which examined implementation of Trauma-Focused Cognitive Behavioral Therapy or Cognitive-Behavioral Intervention for Trauma in Schools. This review identified multilevel and multiphase determinants that can be addressed by implementation strategies to improve implementation and clinical outcomes, and suggests how future studies might address gaps in the evidence base.

Determinants of Evidence-based Practice Uptake in Rural Intensive Care Units. A Mixed Methods Study

Rationale: Evidence-based practices promote quality care for intensive care unit patients but chronic evidence-to-practice gaps limit their reach.Objectives: To characterize key determinants of evidence-based practice uptake in the rural intensive care setting.Methods: A parallel convergent mixed methods design was used with six hospitals receiving a quality improvement intervention. Guided by implementation science principles, we identified barriers and facilitators to uptake using clinician surveys (N = 90), key informant interviews (N = 14), and an implementation tracking log. Uptake was defined as completion of eight practice change steps within 12 months. After completing qualitative and quantitative data analyses for each hospital, site, staff, and program delivery factors were summarized within and across hospitals to identify patterns by uptake status.Results: At the site level, although structural characteristics (hospital size, intensivist staffing) did not vary by uptake status, interviews highlighted variability in staffing patterns and culture that differed by uptake status. At the clinician team level, readiness and self-efficacy were consistently high across sites at baseline with time and financial resources endorsed as primary barriers. However, interviews highlighted that as initiatives progressed, differences across sites in attitudes and ownership of change were key uptake influences. At the program delivery level, mixed methods data highlighted program engagement and leadership variability by uptake status. Higher uptake sites had better training attendance; more program activities completed; and a stable, engaged, collaborative nurse and physician champion team.Conclusions: Results provide an understanding of the multiple dynamic influences on different patterns of evidence-based practice uptake and the importance of implementation support strategies to accelerate uptake in the intensive care setting.

Acquisition of Information About Innovative Practices in Outpatient Mental Health Clinics

This study uses qualitative interviews with leaders of 34 mental health clinics in the context of a statewide rollout of clinical and business innovations to explore how clinics first learn about innovations and which external sources of information they access. Clinic leaders reported accessing information about innovations mainly from government agencies, professional associations, peer organizations, and research literature. Leaders mentioned an average of two external sources of information. There was evidence of variation in how leaders accessed information and how information about innovations was communicated within clinics. Findings have implications for improving dissemination of information about innovations in mental health systems.

COVID-19 and routine childhood immunization in Africa: Leveraging systems thinking and implementation science to improve immunization system performance

One of the routine health services that is being disrupted by coronavirus disease 2019 (COVID-19) in Africa is childhood immunization. This is because the immunization system relies on functioning health facilities and stable communities to be effective. Its disruption increases the risk of epidemics of vaccine-preventable diseases, which could increase child mortality. Therefore, policymakers must quickly identify robust and context-specific strategies to rapidly scale-up routine immunization in order to mitigate the impact of COVID-19 on their national immunization performance. To achieve this, we propose a paradigm shift towards systems thinking and use of implementation science in immunization decision-making. Systems thinking can inform a more nuanced and holistic understanding of the interrelationship between COVID-19, its control strategies, and childhood immunization. Tools like causal loop diagrams can be used to explicitly illustrate the systems structure by identifying feedback loops. Once mapped and leverage points for interventions have been identified, implementation science can be used to guide the rapid uptake and utilization of multifaceted evidence-based innovations in complex practice settings. As Africa re-strategizes for the post-2020 era, these emerging fields could contribute significantly in accelerating progress towards universal access to vaccines for all children on the continent despite COVID-19.

What Is Dissemination and Implementation Science?: An Introduction and Opportunities to Advance Behavioral Medicine and Public Health Globally

There has been a well-documented gap between research (e.g., evidence-based programs, interventions, practices, policies, guidelines) and practice (e.g., what is routinely delivered in real-world community and clinical settings). Dissemination and implementation (D&I) science has emerged to address this research-to-practice gap and accelerate the speed with which translation and real-world uptake and impact occur. In recent years, there has been tremendous development in the field and a growing global interest, but much of the introductory literature has been U.S.-centric. This piece provides an introduction to D&I science and summarizes key concepts and progress of the field for a global audience, provides two case studies that highlight examples of D&I research globally, and identifies opportunities and innovations for advancing the field of D&I research globally.

Establishing cross-systems collaborations for implementation: protocol for a longitudinal mixed methods study

Background

Cross-system interventions can help integrate services across different service delivery systems but require organizations to establish strong collaborative relationships for implementation. Contingency theory suggests that the effectiveness of different collaborative strategies (i.e. specific ways organizations align operations and services) varies by context. This paper describes a study of different strategies for fostering collaboration between child welfare and substance abuse treatment agencies and the conditions under which they are effective for implementation. We also describe the development and piloting of the Collaborating Across Systems for Program Implementation (CASPI) tool—a decision-making guide intended to help researchers and organizational leaders identify and use appropriate collaborative strategies for their context.

Methods/design

This multisite longitudinal, mixed methods study, leverages a naturally occurring implementation initiative -- in up to 17 Ohio counties -- to implement Ohio START (Sobriety Treatment and Reducing Trauma). START is a child welfare model that requires strong collaboration with local substance use treatment organizations to promote integrated services. During the first two years, we will identify collaborative strategies associated with improved START implementation (penetration and fidelity) and service delivery outcomes (timeliness), given system, and organizational features. We will conduct a convergent mixed methods study drawing on worker surveys, agency documents, administrative data, formal partner agreements, and group interviews. Data will be integrated and analyzed using Qualitative Comparative Analysis (QCA). To develop the CASPI, an expert panel comprised of implementation experts, and community stakeholders will convene to synthesize our findings and develop contents (including a decision tree). During the final year of the study, we will assess the acceptability, appropriateness, and feasibility of the CASPI in a randomized vignette experiment, and a pilot-test with 3 child welfare agencies that have not yet implemented START.

Discussion

Our results will lay the groundwork for a larger controlled trial that will test the CASPI’s effectiveness for supporting effective and efficient implementation of cross-system interventions like START. The CASPI is expected to help leaders and researchers select and use collaboration strategies tailored to their context and be applicable in a wide range of settings including rural communities. Our work also advances system-level implementation strategies.

Trial registration

NCT03931005, Registered April 29, 2019.

Evaluation of an Education Strategy versus Usual Care to Implement the STEADI Algorithm in Primary Care Clinics in an Academic Medical Center

Background

Although falls are the leading cause of morbidity and mortality in the US in the older adult population, there is little information regarding implementation of evidence-based fall prevention guidelines within primary care settings. The objective of this study was to address this gap in the literature by determining the effectiveness of the use of education and written materials as implementation strategies.

Methods

Using a prospective, mixed methods, controlled before-and-after study design, we studied the effect of the Stopping Elderly Accidents, Deaths, and Injuries (STEADI) education and written materials on knowledge and intention to use in primary care clinics as well as test the screening, assessment, and intervention behaviors. This manuscript details the quantitative findings of the study, using STEADI Knowledge Test, Continuing Professional Development (CPD) Reaction Questionnaire, and EMR Reports. We compared data between the study arms (usual implementation versus education implementation) using descriptive statistics, paired t-tests, and factorial ANOVAs.

Results

In total, data from 29 primary care staff, including physicians, APRNs, RNs, and medical assistants, were analyzed. Although we found a statistically significant difference within the education arm between immediate pretests and posttests/surveys mean scores, there was no statistically significant difference between the study arms' knowledge, intent to use STEADI, or use behaviors. The pre/immediate post education mean knowledge score increased by 1.19 (p= 0.02) and the pre/immediate post education intent to use mean increased by 0.64 (p 0.01). There was no statistically significant change between the study arms over time.

Conclusion

Educational strategies, particularly written materials and an online module, did not increase the long-term use of the STEADI toolkit. Implementation research is needed to identify the strategies that are most effective for promoting the adoption of STEADI in primary care.

Stakeholder perspectives on transition planning, implementation, and outcomes for students with autism spectrum disorder

LAY ABSTRACT

Little is known about factors impacting poor post-school outcomes for transition-age students with autism spectrum disorder. Guided by an implementation science framework that takes into account the multiple factors that influence transition outcomes, we sought to better understand the interdependent impacts of policy, organizational, provider, and individual factors that shape the transition planning process in schools, and the subsequent process through which transition plans are implemented as youth with autism spectrum disorder access services and gain employment after school. We conducted focus groups with individuals with autism spectrum disorder, parents, classroom teachers, school administrators, adult service providers, and state policymakers (10 groups, N = 40). Participants described how core tenets of the individualized education planning process were not reliably implemented: planning was described by inappropriate goal-setting, ineffective communication, and inadequate involvement of all decision-makers needed to inform planning. After school, youth struggled to access the services specified in their transition plans due to inadequate planning, overburdened services, and insufficient accountability for adult service providers. Finally, a failure to include appropriate skill-building and insufficient interagency and community relationships limited efforts to gain and maintain employment. Diverse stakeholder perspectives illuminate the need for implementation efforts to target the provider, organizational, and policy levels to improve transition outcomes for individuals with autism spectrum disorder.

Engaging stakeholders in the adaptation of the Connect for Health pediatric weight management program for national implementation

Background

Connect for Health is an evidence-based weight management program with clinical- and family-facing components for delivery in pediatric primary care for families of children ages 2 to 12 years. We used the Consolidated Framework for Implementation Research (CFIR) to guide formative work prior to national implementation. The purpose of this study was to describe the process and results of stakeholder engagement and program adaptation.

Methods

We used mixed qualitative and quantitative methods to iteratively adapt and optimize the program by assessing needs and perspectives of clinicians and parents, as well as contextual barriers, facilitators, and organizational readiness for the uptake of the proposed program tools and implementation strategies. We conducted interviews with primary care clinicians from four health care organizations in Boston, MA; Denver, CO; and Greenville, SC, and used principles of immersion-crystallization for qualitative analyses. We also conducted surveys of parents of children with a body mass index ≥ 85th percentile.

Results

We reached thematic saturation after 52 clinician interviews. Emergent themes representing the CFIR domains of intervention characteristics, outer and inner setting, and process included (1) importance of evidence-based clinical decision support tools that integrate into the workflow and do not extend visit time, (2) developing resources that respond to family’s needs, (3) using multimodal delivery options for family resources, (4) addressing childhood obesity while balancing competing demands, (5) emphasizing patient care rather than documentation and establishing sustainability plans, and (6) offering multiple training methods that incorporate performance feedback. Of the parents surveyed (n = 400), approximately 50% were Spanish-speaking and over 75% reported an annual income < $50,000. Parents affirmed the importance of addressing weight management during well-child visits, being provided with referrals and resources, and offering multiple methods for resource delivery. Decisions about program modifications were made at the program and healthcare-system level and based on stakeholder engagement findings. Modifications included cultural, geographic, and target audience adaptations, as well as varied resource delivery options.

Conclusions

To ensure the fit between the Connect for Health program and national implementation settings, adaptations were systematically made through engagement of clinician and parent stakeholders to support adoption, sustainability, and health outcomes.

Trial registration

NCT04042493

Contracting as a bridging factor linking outer and inner contexts during EBP implementation and sustainment: a prospective study across multiple U.S. public sector service systems

BACKGROUND

Bridging factors are relational ties (e.g. partnerships), formal arrangements (e.g. contracts or polices) and processes (e.g. data sharing agreements) linking outer and inner contexts and are a recent evolution of the Exploration-Preparation-Implementation-Sustainment (EPIS) framework. Bridging factor research can elucidate ways that service systems may influence and/or be influenced by organizations providing health services. This study used the EPIS framework and open systems and resource dependence theoretical approaches to examine contracting arrangements in U.S. public sector systems. Contracting arrangements function as bridging factors through which systems communicate, interact, and exchange resources with the organizations operating within them.

METHODS

The sample included 17 community-based organizations in eight service systems. Longitudinal data is derived from 113 contract documents and 88 qualitative interviews and focus groups involving system and organizational stakeholders. Analyses consisted of a document review using content analysis and focused coding of transcripts from the interviews and focus groups. A multiple case study analysis was conducted to identify patterns across service systems and organizations. The dataset represented service systems that had sustained the same EBP for between 2 and 10 years, which allowed for observation of bridging factors and outer-inner context interactions over time.

RESULTS

Service systems and organizations influenced each other in a number of ways through contracting arrangements. Service systems influenced organizations when contracting arrangements resulted in changes to organizational functioning, required organizational responses to insufficient funding, and altered interorganizational network relationships. Organizations influenced service systems when contract arrangements prompted organization-driven contract negotiation/tailoring, changes to system-level processes, and interorganizational collaboration. Service systems and organizations were dependent on each other as implementation progressed. Resources beyond funding emerged, including adequate numbers of eligible clients, expertise in the evidence-based practice, and training and coaching capacity.

CONCLUSION

This study advances implementation science by expanding the range and definition of bridging factors and illustrating specific bi-directional influences between outer context service systems and inner context organizations. This study also identifies bi-directional dependencies over the course of implementation and sustainment. An analysis of influence, dependencies, and resources exchanged through bridging factors has direct implications for selecting and tailoring implementation strategies, especially those that require system-level coordination and change.

Applying the Exploration Preparation Implementation Sustainment (EPIS) Framework to the Kigali Imbereheza Project for Rwandan Adolescents Living With HIV

BACKGROUND

Sub-Saharan African adolescents living with HIV face challenges to antiretroviral therapy (ART) adherence. Poor mental health drives nonadherence but can be improved with cognitive behavioral therapy (CBT). CBT delivered by peers may strengthen effects while building capacity for sustainment in low-income countries. This case study retrospectively applied the Exploration Preparation Implementation Sustainment framework to characterize the execution of the Kigali Imbereheza Project, a 2-arm individually randomized group controlled trial of Trauma-Informed Adherence-Enhanced CBT (TI-CBTe) delivered by Rwandan youth leaders (YLs) to adolescents living with HIV.

METHODS

YL (n = 14, 43% female, M = 22.71 years) had confirmed HIV and self-reported ART adherence >95%. Participants (n = 356, 51% female, M = 16.78 years) living with HIV were randomized to TI-CBTe or usual care. Two YLs co-led TI-CBTe sessions over 2 months for a total of 12 hours, while other YL observed and rated fidelity. Participants reported on YL competence. Additional data evaluated feasibility, acceptability, uptake, and fidelity.

RESULTS

In the Exploration phase, focus groups, stakeholder meetings, and individual interviews revealed strong consensus for delivering TI-CBT to reduce adolescent depression and trauma and improve ART adherence. In the Preparation phase, curriculum revisions were made, YLs were successfully trained, and a cascading supervision model was established. In the Implementation phase, YL delivered TI-CBTe with close monitoring and supervision. Findings revealed strong feasibility, acceptability, uptake, and fidelity, increasing the likelihood of Sustainment.

CONCLUSIONS

Exploration Preparation Implementation Sustainment can guide implementation planning and delivery and evaluate implementation outcomes.