Cluster randomized adaptive implementation trial comparing a standard versus enhanced implementation intervention to improve uptake of an effective re-engagement program for patients with serious mental illness

"So Many Other Things Improve" with Transdiagnostic Treatment for Sleep and Circadian Problems: Interviews with Community Providers on Treating Clients with Serious Mental Illness

Community mental health centers (CMHCs) offer invaluable, publicly-funded treatment for serious mental illness (SMI). Unfortunately, evidence-based psychological treatments are often not delivered at CMHCs, in part due to implementation barriers, such as limited time, high caseloads, and complex clinical presentations. Transdiagnostic treatments may help address these barriers, because they allow providers to treat symptoms across multiple disorders concurrently. However, little research has investigated CMHC providers' experiences of delivering transdiagnostic treatments "on the ground," particularly for adults with SMI. Thus, the aim of the present study was to assess CMHC providers' perspectives on delivering a transdiagnostic treatment - the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) - to adults diagnosed with SMI. In the context of a larger parent trial, providers were randomized to deliver a standard version of TranS-C (Standard TranS-C) or a version adapted to the CMHC context (Adapted TranS-C). Twenty-five providers from the parent trial participated in a semi-structured interview (n = 10 Standard TranS-C; n = 15 from Adapted TranS-C). Responses were deductively and inductively coded to identify themes related to Proctor's taxonomy of implementation outcomes. Four novel "transdiagnostic take homes" were identified: (1) transdiagnostic targets, such as sleep, can be perceived as motivating and appropriate when treating SMI, (2) strategies to bolster client motivation/adherence and address a wider range of symptom severity may improve transdiagnostic treatments, (3) balancing feasibility with offering in-depth resources is an important challenge for transdiagnostic treatment development, and (4) adapting transdiagnostic treatments to the CMHC context may improve provider perceptions of implementation outcomes.

Building on Strong Foundations: Deploying Enhanced Replicating Effective Programs for evidence-based prevention curriculum adaptation

Schools frequently adopt new interventions for each new public health issue, but this is both time- and resource-intensive. Adversity exposure is an example of a pervasive public health issue that emerged during Coronavirus Disease 2019 (COVID-19) with notable consequences, including an elevated risk of developing substance use disorders and mental illnesses. Adapting existing, universal, evidence-based interventions, such as the Michigan Model for HealthTM (MMH), by incorporating trauma-sensitive content is a promising approach to meet this need. We examined critical steps in promoting MMH adaptability as part of the Enhanced REP (Replicating Effective Programs) implementation strategy during the COVID-19 pandemic. We share usability testing from the 2020 to 2021 school year and describe how we apply the results to inform the group randomized trial pilot study. We applied key steps from implementation adaptation frameworks to integrate trauma-sensitive content as COVID-19 unfolded, documenting the process through field notes. We conducted initial usability testing with two teachers via interviews and used a rapid qualitative analysis approach. We conducted member checking by sharing the information with two health coordinators to validate results and inform additional curriculum refinement. We developed an adapted MMH curriculum to include trauma-sensitive content, with adaptations primarily centered on adding content, tailoring content, substituting content, and repeating/reinforcing elements across units. We designed adaptations to retain the core functional elements of MMH. Building foundational relationships and infrastructure supports opportunities to user-test intervention materials for Enhanced REP that enhance utility and relevance for populations that would most benefit. Enhanced REP is a promising strategy to use an existing evidence-based intervention to meet better the needs of youth exposed to adversity. Building on the foundations of existing evidence-based interventions, is vital to implementation success and achieving desired public health outcomes.

Pilot Implementation of Guiando Buenas Decisiones, an Evidence-Based Parenting Program for Spanish-Speaking Families, in Pediatric Primary Care in a Large, U.S. Health System: A Qualitative Interview Study

Adolescent substance use is a significant public health problem in the United States and Hispanic youth engage in substance use services at lower rates than other groups. For this under-served group, prevention services delivered in non-stigmatized, non-specialty care settings may increase access to the services. We describe findings from a feasibility pilot of the implementation of a virtual version of Guiando Buenas Decisiones (GBD), a universal, group-based substance use prevention program for parents. It was conducted with Spanish-speaking families and delivered, virtually, in pediatric primary care in a large healthcare system in the U.S. Through qualitative interviews with pediatricians (n =7)  and parents (n = 26), we explored potential barriers and facilitators of GBD enrollment and engagement. Parents and pediatricians alike noted the dearth of universal prevention programming in Spanish and that GBD could help address the need for linguistically appropriate programming. Parents liked the curriculum content, materials and videos; they felt the focus on strengthening family bonds, setting clear expectations and guidelines, the use of family meetings, and the positive tools provided for navigating family conflict were well-aligned with their cultural and family values. Feedback from parents was helpful for informing more personalized and attentive approaches to program outreach and recruitment methods, and for adaptation of recruitment fliers and letters. In this pediatric primary care context serving an underserved population, we found virtual GBD feasible to implement, acceptable and appealing to parents, and judged by pediatricians as a promising, much-needed addition to their prevention armamentarium.

Getting to Fidelity: Consensus Development Process to Identify Core Activities of Implementation Facilitation

Transferring successful implementation strategies from research to practice requires approaches for assessing fidelity to the strategy's core components. Implementation facilitation (IF) is a strategy involving an interactive process of problem-solving, enabling, and supporting individuals in efforts to implement clinical innovations that occurs in the context of a recognized need for improvement and supportive interpersonal relationships. Because IF is a dynamic strategy involving numerous activities, our objective was to conduct a rigorous consensus development process to identify core activities for monitoring fidelity to IF when applied in clinical settings. We first conducted a scoping literature review to identify the range of activities used when IF has been applied in clinical settings, searching multiple citation databases for English-language articles including "facilitation" or other commonly-used terms for the strategy published from 1996-2015. Through multi-stage screening, 135 articles (from 94 studies) were identified for data extraction on IF activities, frequency with which IF activities were identified as 'core' by study authors, and study outcomes. From the literature review, we identified 32 distinct IF activities and developed definitions/examples for each. Next, we conducted a 3-stage, modified-Delphi expert panel consensus development process to identify core IF activities across three implementation phases (i.e., Pre-Implementation, Implementation, Sustainment). The expert panel identified 8 core activities for the Pre-Implementation Phase, 8 core activities for the Implementation Phase, and 4 core activities for the Sustainment Phase. This work provides an important foundation for developing measures/tools to assess use of core IF activities to ensure the strategy is delivered with fidelity.

Scaling Interventions to Manage Chronic Disease: Innovative Methods at the Intersection of Health Policy Research and Implementation Science

Policy implementation is a key component of scaling effective chronic disease prevention and management interventions. Policy can support scale-up by mandating or incentivizing intervention adoption, but enacting a policy is only the first step. Fully implementing a policy designed to facilitate implementation of health interventions often requires a range of accompanying implementation structures, like health IT systems, and implementation strategies, like training. Decision makers need to know what policies can support intervention adoption and how to implement those policies, but to date research on policy implementation is limited and innovative methodological approaches are needed. In December 2021, the Johns Hopkins ALACRITY Center for Health and Longevity in Mental Illness and the Johns Hopkins Center for Mental Health and Addiction Policy convened a forum of research experts to discuss approaches for studying policy implementation. In this report, we summarize the ideas that came out of the forum. First, we describe a motivating example focused on an Affordable Care Act Medicaid health home waiver policy used by some US states to support scale-up of an evidence-based integrated care model shown in clinical trials to improve cardiovascular care for people with serious mental illness. Second, we define key policy implementation components including structures, strategies, and outcomes. Third, we provide an overview of descriptive, predictive and associational, and causal approaches that can be used to study policy implementation. We conclude with discussion of priorities for methodological innovations in policy implementation research, with three key areas identified by forum experts: effect modification methods for making causal inferences about how policies' effects on outcomes vary based on implementation structures/strategies; causal mediation approaches for studying policy implementation mechanisms; and characterizing uncertainty in systems science models. We conclude with discussion of overarching methods considerations for studying policy implementation, including measurement of policy implementation, strategies for studying the role of context in policy implementation, and the importance of considering when establishing causality is the goal of policy implementation research.

Organize and mobilize for implementation effectiveness to improve overdose education and naloxone distribution from syringe services programs: a randomized controlled trial

BACKGROUND

The United States (US) continues to face decades-long increases in opioid overdose fatalities. As an opioid overdose reversal medication, naloxone can dramatically reduce opioid overdose mortality rates when distributed to people likely to experience or witness an opioid overdose and packaged with education on its use, known as overdose education and naloxone distribution (OEND). Syringe services programs (SSPs) are ideal venues for OEND with staff who are culturally competent in providing services for people who are at risk of experiencing or observing an opioid overdose. We carried out a randomized controlled trial of SSPs to understand the effectiveness of the organize and mobilize for implementation effectiveness (OMIE) approach at improving OEND implementation effectiveness within SSPs.

METHODS

Using simple randomization, 105 SSPs were enrolled into the trial and assigned to one of two study arms - (1) dissemination of OEND best practice recommendations (Control SSPs) or the OMIE approach along with dissemination of the OEND best practice recommendations (i.e., OMIE SSPs). OMIE SSPs could participate in 60-min OMIE sessions once a month for up to 12 months. At 12-month post-baseline, 102 of 105 SSPs (97%) responded to the follow-up survey.

RESULTS

The median number of sessions completed by OMIE SSPs was 10. Comparing OMIE SSPs to control SSPs, we observed significant increases in the number of participants receiving naloxone (incidence rate ratio: 2.15; 95% CI: 1.42, 3.25; p < 0.01) and the rate of naloxone doses distributed per SSP participant (adjusted incidence rate ratio: 1.97; 95% CI: 1.18, 3.30; p = 0.01). We observed no statistically significant difference in the number of adopted best practices between conditions (difference in means 0.2, 95% CI: - 0.7, 1.0; p = 0.68). We also observed a threshold effect where SSPs receiving a higher OMIE dose had greater effect sizes with regard to the number of people given naloxone and the number of naloxone doses distributed.

CONCLUSIONS

In conclusion, the multifaceted OMIE approach was effective at increasing naloxone distribution from SSPs, despite substantial external shocks during the trial. These findings have major implications for addressing the overdose crisis, which has continued unabated for decades.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT03924505 . Registered 19 April 2019.

The use of implementation science theories, models, and frameworks in implementation research for medicinal products: A scoping review

BACKGROUND

The uptake, adoption and integration of new medicines and treatment regimens within healthcare delivery can take a decade or more. Increasingly, implementation science (IS) research is being used to bridge this gap between the availability of new therapeutic evidence and its actual application in clinical practice. Little is known, however, about the quality of IS research in this area, including the degree to which theories, models and frameworks (TMFs) are being used. The objective of this study was to conduct a scoping review of the use of TMFs in implementation research involving medicinal products.

METHODS

A search was conducted for English language abstracts and manuscripts describing the application of TMFs in IS studies for medicinal products. Eligible publications were those published between 1 January 1974 and 12 December 2022. All records were screened at the title and abstract stage; included full-text papers were abstracted using data extraction tables designed for the study. Study quality was appraised using the Implementation Research Development Tool.

RESULTS

The initial scoping search identified 2697 publications, of which 9 were ultimately eligible for inclusion in the review. Most studies were published after 2020 and varied in their objectives, design and therapeutic area. Most studies had sample sizes of fewer than 50 participants, and all focused on the post-marketing phase of drug development. The TMF most frequently used was the Consolidated Framework for Implementation Research (CFIR). Although most studies applied all TMF domains, TMF use was limited to instrument development and/or qualitative analysis. Quality appraisals indicated the need for engaging patients and other stakeholders in the implementation research, reporting on the cost of implementation strategies, and evaluating the unintended consequences of implementation efforts.

CONCLUSIONS

We found that few IS studies involving medicinal products reported using TMFs. Those that did encompassed a wide variety of therapeutic indications and medicinal products; all were in the post-marketing phase and involved limited application of the TMFs. Researchers should consider conducting IS in earlier phases of drug development and integrating the TMFs throughout the research process. More consistent and in-depth use of TMFs may help advance research in this area.

The Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for serious mental illness in community mental health part 3: study protocol to evaluate sustainment in a hybrid type 2 effectiveness-implementation cluster-randomized trial

BACKGROUND

Although research on the implementation of evidence-based psychological treatments (EBPTs) has advanced rapidly, research on the sustainment of implemented EBPTs remains limited. This is concerning, given that EBPT activities and benefits regularly decline post-implementation. To advance research on sustainment, the present protocol focuses on the third and final phase-the Sustainment Phase-of a hybrid type 2 cluster-randomized controlled trial investigating the implementation and sustainment of the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for patients with serious mental illness and sleep and circadian problems in community mental health centers (CMHCs). Prior to the first two phases of the trial-the Implementation Phase and Train-the-Trainer Phase-TranS-C was adapted to fit the CMHC context. Then, 10 CMHCs were cluster-randomized to implement Standard or Adapted TranS-C via facilitation and train-the-trainer. The primary goal of the Sustainment Phase is to investigate whether adapting TranS-C to fit the CMHC context predicts improved sustainment outcomes.

METHODS

Data collection for the Sustainment Phase will commence at least three months after implementation efforts in partnering CMHCs have ended and may continue for up to one year. CMHC providers will be recruited to complete surveys (N = 154) and a semi-structured interview (N = 40) on sustainment outcomes and mechanisms. Aim 1 is to report the sustainment outcomes of TranS-C. Aim 2 is to evaluate whether manipulating EBPT fit to context (i.e., Standard versus Adapted TranS-C) predicts sustainment outcomes. Aim 3 is to test whether provider perceptions of fit mediate the relation between treatment condition (i.e., Standard versus Adapted TranS-C) and sustainment outcomes. Mixed methods will be used to analyze the data.

DISCUSSION

The present study seeks to advance our understanding of sustainment predictors, mechanisms, and outcomes by investigating (a) whether the implementation strategy of adapting an EBPT (i.e., TranS-C) to the CMHC context predicts improved sustainment outcomes and (b) whether this relation is mediated by improved provider perceptions of treatment fit. Together, the findings may help inform more precise implementation efforts that contribute to lasting change.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT05956678 . Registered on July 21, 2023.

Evaluation of strategies to support implementation of a hospital walking program: protocol for a type III effectiveness-implementation hybrid trial

BACKGROUND

STRIDE is a supervised walking program designed to address the negative consequences of immobility during hospitalization for older adults. In an 8-hospital stepped wedge randomized controlled trial, STRIDE was associated with reduced odds of hospital discharge to skilled nursing facility. STRIDE has the potential to become a system-wide approach to address hospital-associated disability in Veteran's Affairs; however, critical questions remain about how best to scale and sustain the program. The overall study goal is to compare the impact of two strategies on STRIDE program penetration (primary), fidelity, and adoption implementation outcomes.

METHODS

Replicating Effective Programs will be used as a framework underlying all implementation support activities. In a parallel, cluster randomized trial, we will use stratified blocked randomization to assign hospitals (n = 32) to either foundational support, comprised of standard, low-touch activities, or enhanced support, which includes the addition of tailored, high-touch activities if hospitals do not meet STRIDE program benchmarks at 6 and 8 months following start date. All hospitals begin with foundational support for 6 months until randomization occurs. The primary outcome is implementation penetration defined as the proportion of eligible hospitalizations with ≥ 1 STRIDE walks at 10 months. Secondary outcomes are fidelity and adoption with all implementation outcomes additionally examined at 13 and 16 months. Fidelity will be assessed for STRIDE hospitalizations as the percentage of eligible hospital days with "full dose" of the program, defined as two or more documented walks or one walk for more than 5 min. Program adoption is a binary outcome defined as ≥ 5 patients with a STRIDE walk or not. Analyses will also include patient-level effectiveness outcomes (e.g., discharge to nursing home, length of stay) and staffing and labor costs. We will employ a convergent mixed-methods approach to explore and understand pre-implementation contextual factors related to differences in hospital-level adoption.

DISCUSSION

Our study results will dually inform best practices for promoting successful implementation of an evidence-based hospital-based walking program. This information may support other programs by advancing our understanding of how to apply and scale-up national implementation strategies.

TRIAL REGISTRATION

This study was registered on June 1, 2021, at ClinicalTrials.gov (identifier NCT04868656 ).

Cluster-randomized implementation trial of two facilitation strategies to implement a novel information and communications technology at the Veterans Health Administration

BACKGROUND

Information and communication technologies (ICTs) improve quality and efficiency of healthcare, but effective practices for implementing new ICTs are unknown. From 2019 to 2021, the Veterans Health Administration (VHA) implemented FLOW3, an ICT that facilitates prosthetic limb care. The goal of this study was to compare the impact of two facilitation strategies on FLOW3 adoption, implementation, and sustainment.

METHODS

FLOW3 is a computerized workflow management system comprised of three applications that facilitate the three steps for prosthesis authorization. During VHA's implementation of FLOW3, we randomized 60 VHA sites to basic or enhanced facilitation groups. Basic facilitation included a manualized training toolkit and office hours. Enhanced facilitation included basic facilitation plus monthly learning collaboratives and site-specific performance reports. Outcomes included time to adoption of FLOW3 and complete FLOW3 utilization rates during implementation and sustainment periods. We compared outcomes between sites assigned to basic versus enhanced facilitation groups. Results were calculated using both intent-to-treat (ITT) and dose-response analyses. The dose-response analysis used a per-protocol approach and required sites in the enhanced facilitation group to join two of six learning collaboratives; sites that attended fewer were reassigned to the basic group.

RESULTS

Randomization assigned 30 sites to enhanced facilitation and 30 to basic. Eighteen of 30 randomized sites were included in the enhanced facilitation group for dose-response analysis. During the implementation period, enhanced facilitation sites were significantly more likely to completely utilize FLOW3 than basic facilitation sites (HR: 0.17; 95% CI: 1.18, 4.53, p = 0.02) based on ITT analysis. In the dose-response analysis, the enhanced group was 2.32 (95% CI: 1.18, 4.53) times more likely to adopt FLOW3 than basic group (p = 0.014).

CONCLUSIONS

Enhanced facilitation including a learning collaborative and customized feedback demonstrated greater likelihood for sites to complete a prosthetics consult using FLOW3 throughout our study. We identified statistically significant differences in likelihood of adoption using the dose-response analysis and complete utilization rate using ITT analysis during the implementation period. All sites that implemented FLOW3 demonstrated improvement in completion rate during the sustainment period, but the difference between facilitation groups was not statistically significant. Further study to understand sustainability is warranted.

A mixed methods partner-focused cost and budget impact analysis to deploy implementation strategies for school-based prevention

BACKGROUND

Obtaining information on implementation strategy costs and local budget impacts from multiple perspectives is essential to data-driven decision-making about resource allocation for successful evidence-based intervention delivery. This mixed methods study determines the costs and priorities of deploying Enhanced Replicating Effective Programs (REP) to implement the Michigan Model for Health™, a universal school-based prevention intervention, from key shareholder perspectives.

METHODS

Our study included teachers in 8 high schools across 3 Michigan counties as part of a pilot cluster randomized trial. We used activity-based costing, mapping key Enhanced REP activities across implementation phases. We included multiple perspectives, including state agencies, regional education service agencies, lead organization, and implementers. We also conducted a budget impact analysis (BIA, assessing the potential financial impact of adopting Enhanced REP) and a scenario analysis to estimate replication and account for cost variability. We used an experimental embedded mixed methods approach, conducting semi-structured interviews and collecting field notes during the trial to expand and explain the cost data and the implications of costs across relevant perspectives.

RESULTS

Based on trial results, we estimate costs for deploying Enhanced REP are $11,903/school, with an estimated range between $8263/school and $15,201/school. We estimate that adding four additional schools, consistent with the pilot, would cost $8659/school. Qualitative results indicated misalignment in school and teacher priorities in some cases. Implementation activities, including training and implementation facilitation with the health coordinator, were sometimes in addition to regular teaching responsibilities. The extent to which this occurred was partly due to leadership priorities (e.g., sticking to the district PD schedule) and organizational priorities (e.g., budget).

CONCLUSIONS

Previous research findings indicate that, from a societal perspective, universal prevention is an excellent return on investment. However, notable misalignment in cost burden and priorities exists across shareholder groups. Our results indicate significant personal time costs by teachers when engaging in implementation strategy activities that impose an opportunity cost. Additional strategies are needed to improve the alignment of costs and benefits to enhance the success and sustainability of implementation. We focus on those perspectives informed by the analysis and discuss opportunities to expand a multi-level focus and create greater alignment across perspectives.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04752189. Registered on 12 February 2021.

The Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for serious mental illness in community mental health part 3: Study protocol to evaluate sustainment in a hybrid type 2 effectiveness-implementation cluster-randomized trial

treatments (EBPTs) has advanced rapidly, research on the sustainment of implemented EBPTs remains limited. This is concerning, given that EBPT activities and benefits regularly decline post-implementation. To advance research on sustainment, the present protocol focuses on the third and final phase - the Sustainment Phase - of a hybrid type 2 cluster-randomized controlled trial investigating the implementation and sustainment of the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for patients with serious mental illness and sleep and circadian problems in community mental health centers (CMHCs). Prior to the first two phases of the trial - the Implementation Phase and Train-the-Trainer Phase - TranS-C was adapted to fit the CMHC context. Then, 10 CMHCs were cluster-randomized to implement Standard or Adapted TranS-C via facilitation and train-the-trainer. The primary goal of the Sustainment Phase is to investigate whether adapting TranS-C to fit the CMHC context predicts improved sustainment outcomes.

Methods

Data collection for the Sustainment Phase will commence at least three months after implementation efforts in partnering CMHCs have ended and may continue for up to one year. CMHC providers will be recruited to complete surveys (N = 154) and a semi-structured interview (N = 40) on sustainment outcomes and mechanisms. Aim 1 is to report the sustainment outcomes of TranS-C. Aim 2 is to evaluate whether manipulating EBPT fit to context (i.e., Standard versus Adapted TranS-C) predicts sustainment outcomes. Aim 3 is to test whether provider perceptions of fit mediate the relation between treatment condition (i.e., Standard versus Adapted TranS-C) and sustainment outcomes. Mixed methods will be used to analyze the data.

Discussion

The present study seeks to advance our understanding of sustainment predictors, mechanisms, and outcomes by investigating (a) whether the implementation strategy of adapting an EBPT (i.e., TranS-C) to the CMHC context predicts improved sustainment outcomes and (b) whether this relation is mediated by improved provider perceptions of treatment fit. Together, the findings may help inform more precise implementation efforts that contribute to lasting change.

Trial Registration

ClinicalTrials.gov identifier: NCT05956678. Registered on July 21, 2023. https://classic.clinicaltrials.gov/ct2/show/NCT05956678?term=NCT05956678&draw=2&rank=1.

Group physical therapy for knee osteoarthritis: protocol for a hybrid type III effectiveness-implementation trial

BACKGROUND

Knee osteoarthritis (OA) is a leading cause of chronic pain and disability and one of the most common conditions treated in outpatient physical therapy (PT). Because of the high and growing prevalence of knee OA, there is a need for efficient approaches for delivering exercise-based PT to patients with knee OA. A prior randomized controlled trial (RCT) showed that a 6-session Group Physical Therapy Program for Knee OA (Group PT) yields equivalent or greater improvements in pain and functional outcomes compared with traditional individual PT, while requiring fewer clinician hours per patient to deliver. This manuscript describes the protocol for a hybrid type III effectiveness-implementation trial comparing two implementation packages to support delivery of Group PT.

METHODS

In this 12-month embedded trial, a minimum of 16 Veterans Affairs Medical Centers (VAMCs) will be randomized to receive one of two implementation support packages for their Group PT programs: a standard, low-touch support based on Replicating Effective Programs (REP) versus enhanced REP (enREP), which adds tailored, high-touch support if sites do not meet Group PT adoption and sustainment benchmarks at 6 and 9 months following launch. Implementation outcomes, including penetration (primary), adoption, and fidelity, will be assessed at 6 and 12 months (primary assessment time point). Additional analyses will include patient-level effectiveness outcomes (pain, function, satisfaction) and staffing and labor costs. A robust qualitative evaluation of site implementation context and experience, as well as site-led adaptations to the Group PT program, will be conducted.

DISCUSSION

To our knowledge, this study is the first to evaluate the impact of tailored, high-touch implementation support on implementation outcomes when compared to standardized, low-touch support for delivering a PT-based intervention. The Group PT program has strong potential to become a standard offering for PT, improving function and pain-related outcomes for patients with knee OA. Results will provide information regarding the effectiveness and value of this implementation approach and a deeper understanding of how healthcare systems can support wide-scale adoption of Group PT.

TRIAL REGISTRATION

This study was registered on March 7, 2022 at ClinicalTrials.gov (identifier NCT05282927 ).

The Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for serious mental illness in community mental health part 2: study protocol for a hybrid type 2 effectiveness-implementation cluster-randomized trial using train-the-trainer

BACKGROUND

Train-the-trainer (TTT) is a promising method for implementing evidence-based psychological treatments (EBPTs) in community mental health centers (CMHCs). In TTT, expert trainers train locally embedded individuals (i.e., Generation 1 providers) to deliver an EBPT, who then train others (i.e., Generation 2 providers). The present study will evaluate implementation and effectiveness outcomes of an EBPT for sleep and circadian dysfunction-the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C)-delivered to CMHC patients with serious mental illness by Generation 2 providers (i.e., trained and supervised within CMHCs via TTT). Specifically, we will investigate whether adapting TranS-C to fit CMHC contexts improves Generation 2 (a) patient outcomes and (b) providers' perceptions of fit.

METHODS

TTT will be implemented in nine CMHCs in California, USA (N = 60 providers; N = 130 patients) via facilitation. CMHCs are cluster-randomized by county to Adapted TranS-C or Standard TranS-C. Within each CMHC, patients are randomized to immediate TranS-C or usual care followed by delayed treatment with TranS-C (UC-DT). Aim 1 will assess the effectiveness of TranS-C (combined Adapted and Standard), compared to UC-DT, on improvements in sleep and circadian problems, functional impairment, and psychiatric symptoms for Generation 2 patients. Aim 2 will evaluate whether Adapted TranS-C is superior to Standard TranS-C with respect to Generation 2 providers' perceptions of fit. Aim 3 will evaluate whether Generation 2 providers' perceived fit mediates the relation between TranS-C treatment condition and patient outcomes. Exploratory analyses will (1) evaluate whether the effectiveness of TranS-C for patient outcomes is moderated by generation, (2) compare Adapted and Standard TranS-C on patient perceptions of credibility/improvement and PhenX Toolkit outcomes (e.g., substance use, suicidality), and (3) evaluate other possible moderators.

DISCUSSION

This trial has potential to (a) inform the process of embedding local trainers and supervisors to expand delivery of a promising transdiagnostic treatment for sleep and circadian dysfunction, (b) add to the growing body of TTT literature by evaluating TTT outcomes with a novel treatment and population, and (c) advance our understanding of providers' perceptions of EBPT "fit" across TTT generations.

TRIAL REGISTRATION

ClinicalTrials.gov identifier NCT05805657 . Registered on April 10, 2023.

Ready, set, go! The role of organizational readiness to predict adoption of a family caregiver training program using the Rogers' diffusion of innovation theory

BACKGROUND

Caregivers FIRST is an evidence-based program addressing gaps in caregivers' skills. In 2020, the Veterans Health Administration Caregiver Support Program (CSP) nationally endorsed Caregivers FIRST, offering credit in leadership performance plans to encourage all VA medical centers (VAMCs) to implement locally. This study examines the association of organizational readiness with VAMC adoption of Caregivers FIRST.

METHODS

In a cohort observational study, we surveyed CSP managers about their facilities' readiness to implement using the Organizational Readiness for Implementing Change (ORIC) instrument and compared change commitment and change efficacy domains among VAMCs "adopters" defined as delivering Caregivers FIRST within 1 year of the national announcement to those that did not ("non-adopters"). Within "adopters," we categorized time to adoption based on Rogers' diffusion of innovation theory including "innovators," "early adopters," "early majority," "late adopters," and "laggards." Organizational readiness and site characteristics (facility complexity, staffing levels, volume of applications for caregiver assistance services) were compared between "adopters," "non-adopters," and between time to adoption subcategories. Separate logistic regression models were used to assess whether ORIC and site characteristics were associated with early adoption among "adopters."

RESULTS

Fifty-one of 63 (81%) VAMCs with CSP manager survey respondents adopted Caregivers FIRST during the first year. ORIC change commitment and efficacy were similar for "adopters" and "non-adopters." However, sites that adopted earlier (innovators and early adopters) had higher ORIC change commitment and efficacy scores than the rest of the "adopters." Logistic regression results indicated that higher ORIC change commitment (odds ratio [OR] = 2.57; 95% confidence interval [CI], 1.11-5.95) and ORIC change efficacy (OR = 2.60; 95% CI, 1.12-6.03) scores were associated with increased odds that a VAMC was an early adopter (categorized as an "innovator," "early adopter", or "early majority"). Site-level characteristics were not associated with Caregivers FIRST early adoption.

CONCLUSIONS

To our knowledge, this study is the first to prospectively assess organizational readiness and the timing of subsequent program adoption. Early adoption was associated with higher ORIC change commitment and change efficacy and not site-level characteristics. These findings yield insights into the role of organizational readiness to accelerate program adoption.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT03474380. Registered on March 22, 2018.

The Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for serious mental illness in community mental health part 2: Study protocol for a hybrid type 2 effectiveness-implementation cluster- randomized trial using train-the-trainer

Background

Train-the-trainer (TTT) is a promising method for implementing evidence-based psychological treatments (EBPTs) in community mental health centers (CMHCs). In TTT, expert trainers train locally embedded individuals (i.e., Generation 1 providers) to deliver an EBPT, who then train others (i.e., Generation 2 providers). The present study will evaluate implementation and effectiveness outcomes of an EBPT for sleep and circadian dysfunction-the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C)-delivered to CMHC patients with serious mental illness by Generation 2 providers (i.e., trained and supervised within CMHCs via TTT). Specifically, we will investigate whether adapting TranS-C to fit CMHC contexts improves Generation 2 (a) patient outcomes (b) providers' perceptions of fit.

Methods

TTT will be implemented in nine CMHCs in California, United States (N= 60 providers; N= 130 patients) via facilitation. CMHCs are cluster-randomized by county to Adapted TranS-C or Standard TranS-C. Within each CMHC, patients are randomized to immediate TranS-C or usual care followed by delayed treatment with TranS-C (UC-DT). Aim 1 will assess the effectiveness of TranS-C (combined Adapted and Standard), compared to UC-DT, on improvements in sleep and circadian problems, functional impairment, and psychiatric symptoms for Generation 2 patients. Aim 2 will evaluate whether Adapted TranS-C is superior to Standard TranS-C with respect to Generation 2 providers' perceptions of fit. Aim 3 will evaluate whether Generation 2 providers' perceived fit mediates the relation between TranS-C treatment condition and patient outcomes. Exploratory analyses will: (1) evaluate whether the effectiveness of TranS-C for patient outcomes is moderated by generation, (2) compare Adapted and Standard TranS-C on patient perceptions of credibility/improvement and PhenX Toolkit outcomes (e.g., substance use, suicidality); and (3) evaluate other possible moderators.

Discussion

This trial has potential to inform the process of (a) embedding local trainers and supervisors to expand delivery of a promising transdiagnostic treatment for sleep and circadian dysfunction, (b) adding to the growing body of TTT literature by evaluating TTT outcomes with a novel treatment and population, and (c) advancing our understanding of providers' perceptions of EBPT 'fit' across TTT generations.

Trial registration

Clinicaltrials.gov identifier: NCT05805657. Registered on April 10, 2023. https://clinicaltrials.gov/ct2/show/NCT05805657.

Team-focused implementation strategies to improve implementation of mental health screening and referral in rural Children's Advocacy Centers: study protocol for a pilot cluster randomized hybrid type 2 trial

BACKGROUND

Children's Advocacy Centers (CACs) use multidisciplinary teams to investigate and respond to maltreatment allegations. CACs play a critical role in connecting children with mental health needs to evidence-based mental health treatment, especially in low-resourced rural areas. Standardized mental health screening and referral protocols can improve CACs' capacity to identify children with mental health needs and encourage treatment engagement. In the team-based context of CACs, teamwork quality is likely to influence implementation processes and outcomes. Implementation strategies that target teams and apply the science of team effectiveness may enhance implementation outcomes in team-based settings.

METHODS

We will use Implementation Mapping to develop team-focused implementation strategies to support the implementation of the Care Process Model for Pediatric Traumatic Stress (CPM-PTS), a standardized screening and referral protocol. Team-focused strategies will integrate activities from effective team development interventions. We will pilot team-focused implementation in a cluster-randomized hybrid type 2 effectiveness-implementation trial. Four rural CACs will implement the CPM-PTS after being randomized to either team-focused implementation (n = 2 CACs) or standard implementation (n = 2 CACs). We will assess the feasibility of team-focused implementation and explore between-group differences in hypothesized team-level mechanisms of change and implementation outcomes (implementation aim). We will use a within-group pre-post design to test the effectiveness of the CPM-PTS in increasing caregivers' understanding of their child's mental health needs and caregivers' intentions to initiate mental health services (effectiveness aim).

CONCLUSIONS

Targeting multidisciplinary teams is an innovative approach to improving implementation outcomes. This study will be one of the first to test team-focused implementation strategies that integrate effective team development interventions. Results will inform efforts to implement evidence-based practices in team-based service settings.

TRIAL REGISTRATION

Clinicaltrials.gov, NCT05679154 . Registered on January 10, 2023.

Adapting adaptive design methods to accelerate adoption of a digital asthma management intervention

Investigators conducting translational research in real-world settings may experience changes that create challenges to the successful completion of the trial as well as post-trial adoption and implementation. Adaptive designs support translational research by systematically adapting content and methods to meet the needs of target populations, settings and contexts. This manuscript describes an adaptive implementation research model that provides strategies for changing content, delivery processes, and research methods to correct course when anticipated and unanticipated circumstances occur during a pragmatic trial. The Breathewell Program included two large pragmatic trials of the effectiveness of a digital communication technology intervention to improve symptom management and medication adherence in asthma care. The first trial targeted parents of children with asthma; the second targeted adults with asthma. Adaptations were made iteratively to adjust to dynamic conditions within the healthcare setting, informed by prospectively collected stakeholder input, and were categorized retrospectively by the authors as proactive or reactive. Study outcomes demonstrated improved treatment adherence and clinical efficiency. Kaiser Permanente Colorado, the setting for both studies, adopted the speech recognition intervention into routine care, however, both interventions required numerous adaptations, including changes to target population, intervention content, and internal workflows. Proactive and reactive adaptations assured that both trials were successfully completed. Adaptive research designs will continue to provide an important pathway to move healthcare delivery research into practice while conducting ongoing effectiveness evaluation.

The Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) for serious mental illness in community mental health part 1: study protocol for a hybrid type 2 effectiveness-implementation cluster-randomized trial

BACKGROUND

Serious mental illness (SMI) can have devastating consequences. Unfortunately, many patients with SMI do not receive evidence-based psychological treatment (EBPTs) in routine practice settings. One barrier is poor "fit" between EBPTs and contexts in which they are implemented. The present study will evaluate implementation and effectiveness outcomes of the Transdiagnostic Intervention for Sleep and Circadian Dysfunction (TranS-C) implemented in community mental health centers (CMHCs). TranS-C was designed to target a range of SMI diagnoses by addressing a probable mechanism and predictor of SMI: sleep and circadian problems. We will investigate whether adapting TranS-C to fit CMHC contexts improves providers' perceptions of fit and patient outcomes.

METHODS

TranS-C will be implemented in at least ten counties in California, USA (N = 96 providers; N = 576 clients), via facilitation. CMHC sites are cluster-randomized by county to Adapted TranS-C or Standard TranS-C. Within each county, patients are randomized to immediate TranS-C or usual care followed by delayed treatment with TranS-C (UC-DT). Aim 1 will compare TranS-C (combined Adapted and Standard) with UC-DT on improvements in sleep and circadian problems, functional impairment, and psychiatric symptoms. Sleep and circadian problems will also be tested as a mediator between treatment condition (combined TranS-C versus UC-DT) and functional impairment/psychiatric symptoms. Aim 2 will evaluate whether Adapted TranS-C is superior to Standard TranS-C with respect to provider perceptions of fit. Aim 3 will evaluate whether the relation between TranS-C treatment condition (Adapted versus Standard) and patient outcomes is mediated by better provider perceptions of fit in the Adapted condition. Exploratory analyses will (1) compare Adapted versus Standard TranS-C on patient perceptions of credibility/improvement and select PhenX Toolkit outcomes and (2) evaluate possible moderators.

DISCUSSION

This trial has the potential to (a) expand support for TranS-C, a promising transdiagnostic treatment delivered to patients with SMI in CMHCs; (b) take steps toward addressing challenges faced by providers in delivering EBPTs (i.e., high caseloads, complex patients, poor fit); and (c) advance evidence on causal strategies (i.e., adapting treatments to fit context) in implementation science.

TRIAL REGISTRATION

Clinicaltrials.gov NCT04154631. Registered on 6 November 2019. https://clinicaltrials.gov/ct2/show/NCT04154631.

Association between the number of adopted implementation strategies and contextual determinants: a mixed-methods study

BACKGROUND

The successful implementation of evidence-based innovations to improve healthcare delivery often requires a well-planned strategy to support their use. With a greater recognition of the importance of an implementation process, researchers have turned their attention to implementation strategies and their customization to target specific organizational barriers and facilitators. Further, there is a paucity of empirical evidence demonstrating the link between implementation determinants and the number of selected implementation strategies. The purpose of this mixed methods analysis is to examine how formatively assessed barriers and facilitators to implementation relate to the number and type of implementation strategies adopted to address context-specific factors.

METHODS

A mixed methods evaluation that included 15 rheumatology clinics throughout the United States that were planning for implementation of an evidence-based shared decision-making aid for patients with lupus. Quantitative data consisted of a count of the number of implementation strategies used by a clinic. Qualitative data collection was guided by the Consolidated Framework for Implementation Research (CFIR) and relied upon semi-structured interviews with 90 clinic members between November 2018 and August 2019.

RESULTS

Using the CFIR, we found that local clinic factors (Inner Setting Domain) resulted in different perceptions about Planning and Executing the DA (Process Domain); these domains were most likely to distinguish between the number and type of implementation strategies adopted by the clinics. In contrast, Intervention characteristics, Individual Characteristics, and the Outer Setting did not differentiate between the groups with different numbers of implementation strategies. The number and type of chosen strategies were not those associated with the context-specific factors.

CONCLUSIONS

Findings show that, despite recognition of the value of customizing implementation strategies for the contexts in which they are applied, they are too often chosen in a manner that fail to adequately reflect the diverse settings that may present unique factors associated with implementation. Our findings also highlight the importance of the inner context - both in terms of structural characteristics and existing work processes - as a driving factor for why some organizations select different numbers and types of implementation strategies.

Primary aim results of a clustered SMART for developing a school-level, adaptive implementation strategy to support CBT delivery at high schools in Michigan

BACKGROUND

Schools increasingly provide mental health services to students, but often lack access to implementation strategies to support school-based (and school professional [SP]) delivery of evidence-based practices. Given substantial heterogeneity in implementation barriers across schools, development of adaptive implementation strategies that guide which implementation strategies to provide to which schools and when may be necessary to support scale-up.

METHODS

A clustered, sequential, multiple-assignment randomized trial (SMART) of high schools across Michigan was used to inform the development of a school-level adaptive implementation strategy for supporting SP-delivered cognitive behavioral therapy (CBT). All schools were first provided with implementation support informed by Replicating Effective Programs (REP) and then were randomized to add in-person Coaching or not (phase 1). After 8 weeks, schools were assessed for response based on SP-reported frequency of CBT delivered to students and/or barriers reported. Responder schools continued with phase 1 implementation strategies. Slower-responder schools (not providing ≥ 3 CBT components to ≥10 students or >2 organizational barriers identified) were re-randomized to add Facilitation to current support or not (phase 2). The primary aim hypothesis was that SPs at schools receiving the REP + Coaching + Facilitation adaptive implementation strategy would deliver more CBT sessions than SPs at schools receiving REP alone. Secondary aims compared four implementation strategies (Coaching vs no Coaching × Facilitation vs no Facilitation) on CBT sessions delivered, including by type (group, brief and full individual). Analyses used a marginal, weighted least squares approach developed for clustered SMARTs.

RESULTS

SPs (n = 169) at 94 high schools entered the study. N = 83 schools (88%) were slower-responders after phase 1. Contrary to the primary aim hypothesis, there was no evidence of a significant difference in CBT sessions delivered between REP + Coaching + Facilitation and REP alone (111.4 vs. 121.1 average total CBT sessions; p = 0.63). In secondary analyses, the adaptive strategy that offered REP + Facilitation resulted in the highest average CBT delivery (154.1 sessions) and the non-adaptive strategy offering REP + Coaching the lowest (94.5 sessions).

CONCLUSIONS

The most effective strategy in terms of average SP-reported CBT delivery is the adaptive implementation strategy that (i) begins with REP, (ii) augments with Facilitation for slower-responder schools (schools where SPs identified organizational barriers or struggled to deliver CBT), and (iii) stays the course with REP for responder schools.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT03541317 , May 30, 2018.

Care team and practice-level implementation strategies to optimize pediatric collaborative care: study protocol for a cluster-randomized hybrid type III trial

BACKGROUND

Implementation facilitation is an effective strategy to support the implementation of evidence-based practices (EBPs), but our understanding of multilevel strategies and the mechanisms of change within the "black box" of implementation facilitation is limited. This implementation trial seeks to disentangle and evaluate the effects of facilitation strategies that separately target the care team and leadership levels on implementation of a collaborative care model in pediatric primary care. Strategies targeting the provider care team (TEAM) should engage team-level mechanisms, and strategies targeting leaders (LEAD) should engage organizational mechanisms.

METHODS

We will conduct a hybrid type 3 effectiveness-implementation trial in a 2 × 2 factorial design to evaluate the main and interactive effects of TEAM and LEAD and test for mediation and moderation of effects. Twenty-four pediatric primary care practices will receive standard REP training to implement Doctor-Office Collaborative Care (DOCC) and then be randomized to (1) Standard REP only, (2) TEAM, (3) LEAD, or (4) TEAM + LEAD. Implementation outcomes are DOCC service delivery and change in practice-level care management competencies. Clinical outcomes are child symptom severity and quality of life.

DISCUSSION

This statewide trial is one of the first to test the unique and synergistic effects of implementation strategies targeting care teams and practice leadership. It will advance our knowledge of effective care team and practice-level implementation strategies and mechanisms of change. Findings will support efforts to improve common child behavioral health conditions by optimizing scale-up and sustainment of CCMs in a pediatric patient-centered medical home.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT04946253 . Registered June 30, 2021.

Together in care: Lessons learned at the intersection of integrated care, quality improvement, and implementation practice in opioid treatment programs

Background

Integrated care programs that systematically and comprehensively address both behavioral and physical health may improve patient outcomes. However, there are few examples of such programs in addiction treatment settings. This article is a practical implementation report describing the implementation of an integrated care program into two opioid treatment programs (OTPs).

Method

Strategies used to implement integrated care into two OTPs included external facilitation, quality improvement (QI) processes, staff training, and an integrated organizational structure. Service, implementation, and client outcomes were examined using qualitative interviews with program staff (n  =  16), program enrollment data, and client outcome data (n  =  593) on mental health (MH), physical health, and functional indicators.

Results

Staff found the program to generally be acceptable and appropriate, but also noted that the new services added to already busy workflows and more staffing were needed to fully reach the program's potential. The program had a high level of penetration (∼60%-70%), enrolling over 1,200 clients. Staff noted difficulties in connecting clients with some services. Client general functioning and MH symptoms improved, and heavy smoking decreased. The organizational structure and QI activities provided a strong foundation for interactive problem-solving and adaptations that were needed during implementation.

Conclusions

This article highlights an example of the intersection of QI and implementation practice. Simplified QI processes, consistent post-implementation meetings, and change teams and champions facilitated implementation; however, ongoing training and support, especially related to data are needed. The OTP setting provided a strong foundation to build integrated care, but careful consideration of new workflows and changes in philosophy for staff is necessary.Plain Language Summary: Providing medical and behavioral health treatment services in the same clinic using coordinated treatment teams, also known as integrated care, improves outcomes among those with chronic physical and behavioral health conditions. However, there are few practical examples of implementation of such programs in addiction treatment settings, which are promising, yet underutilized settings for integrated care programs. A multi-sectoral team used quality improvement (QI) and implementation strategies to implement integrated care into two opioid treatment programs (OTPs). The program enrolled over 1,200 clients and client general functioning and mental health (MH) symptoms improved, and heavy smoking decreased. Qualitative interviews provided important information about the barriers, facilitators, and context around implementation of this program. The OTP setting provided a strong foundation to build integrated care, but careful consideration of new workflows and changes in philosophy for staff, as well as ongoing training and supports for staff, are necessary. This project may help to advance the implementation of integrated care in OTPs by identifying barriers and facilitators to implementation, lessons learned, as well as providing a practical example of potentially useful QI and implementation strategies.

Implementing clinical decision support for reducing women Veterans' cardiovascular risk in VA: A mixed-method, longitudinal study of context, adaptation, and uptake

Evaluations of clinical decision support (CDS) implementation often struggle to measure and explain heterogeneity in uptake over time and across settings, and to account for the impact of context and adaptation on implementation success. In 2017-2020, the EMPOWER QUERI implemented a cardiovascular toolkit using a computerized template aimed at reducing women Veterans' cardiovascular risk across five Veterans Healthcare Administration (VA) sites, using an enhanced Replicating Effective Programs (REP) implementation approach. In this study, we used longitudinal joint displays of qualitative and quantitative findings to explore (1) how contextual factors emerged across sites, (2) how the template and implementation strategies were adapted in response to contextual factors, and (3) how contextual factors and adaptations coincided with template uptake across sites and over time. We identified site structure, staffing changes, relational authority of champions, and external leadership as important contextual factors. These factors gave rise to adaptations such as splitting the template into multiple parts, pairing the template with a computerized reminder, conducting academic detailing, creating cheat sheets, and using small-scale pilot testing. All five sites exhibited variability in utilization over the months of implementation, though later sites exhibited higher template utilization immediately post-launch, possibly reflecting a "preloading" of adaptations from previous sites. These findings underscore the importance of adaptive approaches to implementation, with intentional shifts in intervention and strategy to meet the needs of individual sites, as well as the value of integrating mixed-method data sources in conducting longitudinal evaluation of implementation efforts.

Using Human-Centered Design to Develop, Launch, and Evaluate a National Digital Health Platform to Improve Reproductive Health for Rwandan Youth

BACKGROUND

A lack of access to evidence-based, unbiased, and youth-friendly family planning and reproductive health (FP/RH) information and care limit young people's ability to prevent unplanned pregnancies and HIV and sexually transmitted infections. This threat-ens their health and is a significant cause of school drop-out, limiting young peoples' well-being, future potential, and employment opportunities. To address these challenges facing youth, YLabs used an end-to-end human-centered design (HCD) approach to create CyberRwanda, a digital platform aiming to improve the health and livelihoods of adolescents (aged 12-19 years) in Rwanda.

DESIGNING FOR DIGITAL WITH YOUTH

From 2016 to 2020, CyberRwanda was designed and piloted using an HCD approach in partnership with more than 1,000 youth, parents, teachers, and public and private health care providers. During the problem recognition phase, HCD revealed participants' beliefs, behavioral preferences, and experiences as they relate to FP/RH specifically and their broader life experiences, motivations, and challenges. Several phases of analog, digital, and live prototyping with youth and key stakeholders were used to codesign, test, and refine the intervention for implementation.

RESULTS

CyberRwanda is a direct-to-consumer platform where adolescents can learn integrated, age-appropriate health, and skills-building information through edutainment behavior change stories and a robust frequently asked questions library, order health products online, and be linked to CyberRwanda's network of private and public health care providers who have been trained to provide adolescent-friendly care.

IMPLICATIONS FOR FUTURE RESEARCH

The HCD process resulted in significant pivots to the design of the digital platform and the implementation model. Using HCD provided a structured methodology to combine technical FP/RH expertise and visual and product design expertise to codesign and iteratively develop a digital health intervention with and for Rwandan youth.

Optimization Methods and Implementation Science: An Opportunity for Behavioral and Biobehavioral Interventions

This editorial introduces the multiphase optimization strategy (MOST), a principled framework for the development, optimization and evaluation of multicomponent interventions, to the field of implementation science. We suggest that MOST may be integrated with implementation science to advance the field, moving closer towards the ultimate goal of disseminating effective interventions to those in need. We offer three potential ways MOST may advance implementation science: (1) development of an effective and immediately scalable intervention; (2) adaptation of interventions to local contexts; and (3) optimization of the implementation of an intervention itself. Our goal is to inspire the integration of MOST with implementation science across a number of public health contexts.

The Sub-Saharan Africa Regional Partnership (SHARP) for Mental Health Capacity-Building Scale-Up Trial: Study Design and Protocol

BACKGROUND

Depression is a leading cause of death and disability worldwide, including in low- and middle-income countries (LMICs). Depression often coexists with chronic medical conditions and is associated with worse clinical outcomes. This confluence has led to calls to integrate mental health treatment with chronic disease care systems in LMICs. This article describes the rationale and protocol for a trial comparing the clinical effectiveness and cost-effectiveness of two different intervention packages to implement evidence-based antidepressant management and psychotherapy into chronic noncommunicable disease (NCD) clinics in Malawi.

METHODS

Using constrained randomization, the Sub-Saharan Africa Regional Partnership (SHARP) for mental health capacity building will assign five Malawian NCD clinics to a basic implementation strategy via an internal coordinator, a provider within the chronic care clinic who champions depression services by providing training, supervision, operations, and reporting. Another five clinics will be assigned to depression services implementation via an internal coordinator along with an external quality assurance committee, which will provide a quarterly audit of intervention component delivery with feedback to providers and the health management team.

RESULTS

The authors will compare key implementation outcomes (fidelity to intervention), clinical effectiveness outcomes (patient health), and cost-effectiveness and will assess characteristics of clinics that may influence uptake and fidelity.

NEXT STEPS

This trial will provide key information to guide the Malawi Ministry of Health in scaling up depression management in existing NCD settings. The SHARP trial is anticipated to substantially contribute to enhancing both mental health treatment and implementation science research capacity in Malawi and the wider region.

Comparative effectiveness of external vs blended facilitation on collaborative care model implementation in slow-implementer community practices

OBJECTIVE

To evaluate the comparative effectiveness of external facilitation (EF) vs external + internal facilitation (EF/IF), on uptake of a collaborative chronic care model (CCM) in community practices that were slower to implement under low-level implementation support.

STUDY SETTING

Primary data were collected from 43 community practices in Michigan and Colorado at baseline and for 12 months following randomization.

STUDY DESIGN

Sites that failed to meet a pre-established implementation benchmark after six months of low-level implementation support were randomized to add either EF or EF/IF support for up to 12 months. Key outcomes were change in number of patients receiving the CCM and number of patients receiving a clinically significant dose of the CCM. Moderators' analyses further examined whether comparative effectiveness was dependent on prerandomization adoption, number of providers trained or practice size. Facilitation log data were used for exploratory follow-up analyses.

DATA COLLECTION

Sites reported monthly on number of patients that had received the CCM. Facilitation logs were completed by study EF and site IFs and shared with the study team.

PRINCIPAL FINDINGS

N = 21 sites were randomized to EF and 22 to EF/IF. Overall, EF/IF practices saw more uptake than EF sites after 12 months (Δ_EF/IF-EF  = 4.4 patients, 95% CI = 1.87-6.87). Moderators' analyses, however, revealed that it was only sites with no prerandomization uptake of the CCM (nonadopter sites) that saw significantly more benefit from EF/IF (Δ_EF/IF-EF  = 9.2 patients, 95% CI: 5.72, 12.63). For sites with prerandomization uptake (adopter sites), EF/IF offered no additional benefit (Δ_EF/IF-EF  = -0.9; 95% CI: -4.40, 2.60). Number of providers trained and practice size were not significant moderators.

CONCLUSIONS

Although stepping up to the more intensive EF/IF did outperform EF overall, its benefit was limited to sites that failed to deliver any CCM under the low-level strategy. Once one or more providers were delivering the CCM, additional on-site personnel did not appear to add value to the implementation effort.

Adaptation of an exercise intervention for pregnant women to community-based delivery: a study protocol

INTRODUCTION

Despite well-established guidelines and benefits to exercise, the majority of pregnant women in the USA fail to meet recommended activity levels. Studies need to determine feasible ways to translate clinical interventions to community settings by engaging pregnant women in widely accessible locations to ensure benefits to more women. The aim of this study is to adapt and determine feasibility, acceptability and fidelity of the research clinic-based Expecting intervention (NCT02125149) with pregnant women with obesity in community settings.

METHODS AND ANALYSIS

We will use the Replicating Effective Programs (REP) to guide the adaptation and implementation of the research clinic-based intervention into the community. REP provides a four-phase process for implementing evidence-based interventions including collection of feedback from community stakeholders, iterative piloting of the intervention in the community and a process for standardising the intervention across community settings. Following adaptation, the updated intervention will be piloted. The pilot study will include 60 expecting women. We will randomise half to receive the community-adapted Expecting intervention (intervention, N=30) and half to receive standard of care (control, N=30). Feasibility and Acceptability of Intervention Measures are primary outcomes as key indicators of feasibility. Secondary outcomes will include the number of intervention sessions completed, the change in the number of minutes of physical activity as measured by accelerometer, as well as change in health indicators from enrolment to time of delivery and 6 months post-delivery (ie, body mass index, blood pressure and total cholesterol).

ETHICS AND DISSEMINATION

This study has been approved by the Institutional Review Board (#260132). Findings will be shared with study participants and stakeholder advisors through written summaries and in-person presentations; results will also be shared through presentations at scientific conferences and publications in peer-reviewed journals.

TRIAL REGISTRATION NUMBER

NCT04298125; Pre-results.

The use of the PARIHS framework in implementation research and practice-a citation analysis of the literature

BACKGROUND

The Promoting Action on Research Implementation in Health Services (PARIHS) framework was developed two decades ago and conceptualizes successful implementation (SI) as a function (f) of the evidence (E) nature and type, context (C) quality, and the facilitation (F), [SI = f (E,C,F)]. Despite a growing number of citations of theoretical frameworks including PARIHS, details of how theoretical frameworks are used remains largely unknown. This review aimed to enhance the understanding of the breadth and depth of the use of the PARIHS framework.

METHODS

This citation analysis commenced from four core articles representing the key stages of the framework's development. The citation search was performed in Web of Science and Scopus. After exclusion, we undertook an initial assessment aimed to identify articles using PARIHS and not only referencing any of the core articles. To assess this, all articles were read in full. Further data extraction included capturing information about where (country/countries and setting/s) PARIHS had been used, as well as categorizing how the framework was applied. Also, strengths and weaknesses, as well as efforts to validate the framework, were explored in detail.

RESULTS

The citation search yielded 1613 articles. After applying exclusion criteria, 1475 articles were read in full, and the initial assessment yielded a total of 367 articles reported to have used the PARIHS framework. These articles were included for data extraction. The framework had been used in a variety of settings and in both high-, middle-, and low-income countries. With regard to types of use, 32% used PARIHS in planning and delivering an intervention, 50% in data analysis, 55% in the evaluation of study findings, and/or 37% in any other way. Further analysis showed that its actual application was frequently partial and generally not well elaborated.

CONCLUSIONS

In line with previous citation analysis of the use of theoretical frameworks in implementation science, we also found a rather superficial description of the use of PARIHS. Thus, we propose the development and adoption of reporting guidelines on how framework(s) are used in implementation studies, with the expectation that this will enhance the maturity of implementation science.

Home care workers caring for adults with heart failure need better access to training and technology: A role for implementation science

Although highly involved in heart failure (HF) patients' care, home care workers (HCWs) lack HF training and are poorly integrated into the healthcare team. For its potential to address these challenges, we examined the role of technology among HCWs caring for HF patients. We conducted 38 interviews with key stakeholders. Overall, four themes emerged. Participants reported that technology is critical for HF care, but existing systems are outdated and ineffective. HCWs also have limited access to electronic resources. Technology, training, and principles of implementation science can be leveraged to improve HCWs' experience in caring for HF patients and home healthcare delivery.

A type II implementation-effectiveness hybrid quasi-experimental pilot study of a clinical intervention to re-engage people living with HIV into care, 'Lost & Found': an implementation science protocol

Background

At the McGill University Health Centre (MUHC), 10% of patients living with HIV do not return for care annually. Currently, no formal system exists to re-engage out-of-care (OOC) patients. Lost & Found, developed using an implementation science approach, is an intervention to re-engage OOC patients. It is based on existing evidence-based interventions and will be adapted for use by nurses at the MUHC. The aims of this study are to simultaneously assess both implementation and effectiveness of Lost & Found in order to determine the viability of a future multisite stepped-wedge cluster randomised trial.

Methods

Lost & Found consists of two core elements: identifying and contacting OOC patients. Based on formative work involving MUHC nurses, and the use of a combined implementation framework (enhanced Replicating Effective Programs, Tailored Implementation for Chronic Diseases, and Proctor et al.’s implementation outcomes), we will adapt the intervention to our clinic. Adaptations include the creation of an OOC risk prediction tool, an automated real-time OOC list, and prioritization of high-risk OOC patients for re-engagement. Delivery and ongoing adaptation of the intervention will follow a three-pronged implementation strategy consisting of (1) promoting adaptability; (2) planning, engaging, executing, evaluating, and reflecting cycles; and (3) internal facilitation. This 15-month quasi-experimental pilot study adopts a type II implementation-effectiveness hybrid design. To evaluate implementation, a convergent parallel mixed-methods approach will guide the mixing of qualitative and quantitative data at time points throughout the study. In addition, descriptive and pre-post analyses, for each of the implementation and sustainability phases, will inform evaluations of the cumulative effectiveness and sustainability of the Lost & Found intervention.

Discussion

This study will provide preliminary evidence for (1) the utility of our chosen implementation strategies and (2) the effectiveness of the intervention. Ultimately, this information may be used to inform future re-engagement efforts using implementation science in other HIV care centres. In addition, the procedures and measurement tools developed for this study will be foundational to the development of a multi-site, randomised stepped wedge study that would provide more robust evidence in support of the Lost & Found intervention.

Comprehensive approaches to addressing mental health needs and enhancing school security: a hybrid type II cluster randomized trial

BACKGROUND

School safety is fundamental to fostering positive outcomes for children. Violence remains a critical public health issue with 8.1% of elementary and 21.8% of middle school students reporting daily or weekly bullying in 2015-16. Similarly, over half of lifetime mental health concerns become evident before age 14. Thus, elementary school is a key time for comprehensive school safety interventions. Yet, interventions are rarely delivered with fidelity in community settings. Evidence-based interventions must be complemented by implementation strategies to achieve desired public health outcomes.

METHODS

We develop and test an intervention focused on promoting a positive school climate guided by a school-based 3-person leadership team (3-PLT) using a hybrid Type II design. The 3-PLT includes a School Resource Officer, (SRO), administrator and mental health services professional as a newly appointed climate specialist (CS). The interventions to be delivered include 1) Restorative justice, 2) Mental Health First Aid and 3) Crime Prevention Through Environmental Design. The CS will lead the team and coordinate implementation through a process of interactive problem solving and supports, consistent with the implementation facilitation strategy. We will conduct a cluster randomized controlled trial with staged entry over two school years in Genesee County, Michigan (n = 20 elementary schools, with 10 participating per school year). We will use a combination of data sources including data collected by schools (e.g., discipline data), a student survey, and a teacher survey. We will also conduct a process evaluation and assess implementation and sustainability through focus groups with key stakeholders, teachers, and students. Finally, we will conduct a cost-benefit analysis.

DISCUSSION

Results from both the behavioral outcome and implementation strategy evaluations are expected to have significant implications for school safety and student well-being. This study adopts a unique approach by integrating three evidence-based programs and incorporating implementation facilitation led by the CS as part of the 3-PLT to support intervention delivery and enhance public health impact among schools in disadvantaged communities with students at risk of poor health outcomes. This study aims to create a comprehensive, well-integrated model intervention that is sustainable and can be translated to similar high-risk settings.

TRIAL REGISTRATION

Trial was retrospectively registered, registration ISRCTN1226421, May 16, 2019.

Adapting and implementing Caring Contacts in a Department of Veterans Affairs emergency department: a pilot study protocol

Background

Suicide among veterans is a problem nationally, and suicide prevention remains a high priority for the Department of Veterans Affairs (VA). Focusing suicide prevention initiatives in the emergency department setting provides reach to veterans who may not be seen in mental health and targets a critical risk period, transitions in care following discharge. Caring Contacts is a simple and efficacious suicide prevention approach that could be used to target this risk period. The purpose of this study is to (1) adapt Caring Contacts for use in a VA emergency department, (2) conduct a pilot program at a single VA emergency department, and (3) create an implementation toolkit to facilitate spread of Caring Contacts to other VA facilities.

Methods

This project includes planning activities and a pilot at a VA emergency department. Planning activities will include determining available data sources, determining logistics for identifying and sending Caring Contacts, and creating an implementation toolkit. We will conduct qualitative interviews with emergency department staff and other key stakeholders to gather data on what is needed to adapt and implement Caring Contacts in a VA emergency department setting and possible barriers to and facilitators of implementation. An advisory board of key stakeholders in the facility will be created. Qualitative findings from interviews will be presented to the advisory board for discussion, and the board will use these data to inform decision making regarding implementation of the pilot. Once the pilot is underway, the advisory board will convene again to discuss ongoing progress and determine if any changes are needed to the implementation of the Caring Contacts intervention.

Discussion

Findings from the current project will inform future scale-up and spread of this innovation to other VA medical center emergency departments across the network and other networks. The current pilot will adapt Caring Contacts, create an implementation toolkit and implementation guide, evaluate the feasibility of gathering outcome measures, and provide information about what is needed to implement this evidence-based suicide prevention intervention in a VA emergency department.

Sample size considerations for comparing dynamic treatment regimens in a sequential multiple-assignment randomized trial with a continuous longitudinal outcome

Clinicians and researchers alike are increasingly interested in how best to personalize interventions. A dynamic treatment regimen is a sequence of prespecified decision rules which can be used to guide the delivery of a sequence of treatments or interventions that is tailored to the changing needs of the individual. The sequential multiple-assignment randomized trial is a research tool which allows for the construction of effective dynamic treatment regimens. We derive easy-to-use formulae for computing the total sample size for three common two-stage sequential multiple-assignment randomized trial designs in which the primary aim is to compare mean end-of-study outcomes for two embedded dynamic treatment regimens which recommend different first-stage treatments. The formulae are derived in the context of a regression model which leverages information from a longitudinal outcome collected over the entire study. We show that the sample size formula for a sequential multiple-assignment randomized trial can be written as the product of the sample size formula for a standard two-arm randomized trial, a deflation factor that accounts for the increased statistical efficiency resulting from a longitudinal analysis, and an inflation factor that accounts for the design of a sequential multiple-assignment randomized trial. The sequential multiple-assignment randomized trial design inflation factor is typically a function of the anticipated probability of response to first-stage treatment. We review modeling and estimation for dynamic treatment regimen effect analyses using a longitudinal outcome from a sequential multiple-assignment randomized trial, as well as the estimation of standard errors. We also present estimators for the covariance matrix for a variety of common working correlation structures. Methods are motivated using the ENGAGE study, a sequential multiple-assignment randomized trial aimed at developing a dynamic treatment regimen for increasing motivation to attend treatments among alcohol- and cocaine-dependent patients.

Adaptation and Implementation of a Family Caregiver Skills Training Program: From Single Site RCT to Multisite Pragmatic Intervention

PURPOSE

We describe an approach to rapidly adapt and implement an education and skills improvement intervention to address the needs of family caregivers of functionally impaired veterans-Helping Invested Families Improve Veterans' Experience Study (HI-FIVES).

DESIGN

Prior to implementation in eight sites, a multidisciplinary study team made systematic adaptations to the curriculum content and delivery process using input from the original randomized controlled trial (RCT); a stakeholder advisory board comprised of national experts in caregiver education, nursing, and implementation; and a veteran/caregiver engagement panel. To address site-specific implementation barriers in diverse settings, we applied the Replicating Effective Programs implementation framework.

FINDINGS

Adaptations to HI-FIVES content and delivery included identifying core/noncore curriculum components, reducing instruction time, and simplifying caregiver recruitment for clinical settings. To enhance curriculum flexibility and potential uptake, site personnel were able to choose which staff would deliver the intervention and whether to offer class sessions in person or remotely. Curriculum materials were standardized and packaged to reduce the time required for implementation and to promote fidelity to the intervention.

CONCLUSIONS

The emphasis on flexible intervention delivery and standardized materials has been identified as strengths of the adaptation process. Two key challenges have been identifying feasible impact measures and reaching eligible caregivers for intervention recruitment.

CLINICAL RELEVANCE

This systematic implementation process can be used to rapidly adapt an intervention to diverse clinical sites and contexts. Nursing professionals play a significant role in educating and supporting caregivers and care recipients and can take a leading role to implement interventions that address skills and unmet needs for caregivers.

Change in Patient Outcomes After Augmenting a Low-level Implementation Strategy in Community Practices That Are Slow to Adopt a Collaborative Chronic Care Model: A Cluster Randomized Implementation Trial

BACKGROUND

Implementation strategies are essential for promoting the uptake of evidence-based practices and for patients to receive optimal care. Yet strategies differ substantially in their intensity and feasibility. Lower-intensity strategies (eg, training and technical support) are commonly used but may be insufficient for all clinics. Limited research has examined the comparative effectiveness of augmentations to low-level implementation strategies for nonresponding clinics.

OBJECTIVES

To compare 2 augmentation strategies for improving uptake of an evidence-based collaborative chronic care model (CCM) on 18-month outcomes for patients with depression at community-based clinics nonresponsive to lower-level implementation support.

RESEARCH DESIGN

Providers initially received support using a low-level implementation strategy, Replicating Effective Programs (REP). After 6 months, nonresponsive clinics were randomized to add either external facilitation (REP+EF) or external and internal facilitation (REP+EF/IF).

MEASURES

The primary outcome was patient 12-item short form survey (SF-12) mental health score at month 18. Secondary outcomes were patient health questionnaire (PHQ-9) depression score at month 18 and receipt of the CCM during months 6 through 18.

RESULTS

Twenty-seven clinics were nonresponsive after 6 months of REP. Thirteen clinics (N=77 patients) were randomized to REP+EF and 14 (N=92) to REP+EF/IF. At 18 months, patients in the REP+EF/IF arm had worse SF-12 [diff, 8.38; 95% confidence interval (CI), 3.59-13.18] and PHQ-9 scores (diff, 1.82; 95% CI, -0.14 to 3.79), and lower odds of CCM receipt (odds ratio, 0.67; 95% CI, 0.30-1.49) than REP+EF patients.

CONCLUSIONS

Patients at sites receiving the more intensive REP+EF/IF saw less improvement in mood symptoms at 18 months than those receiving REP+EF and were no more likely to receive the CCM. For community-based clinics, EF augmentation may be more feasible than EF/IF for implementing CCMs.

Developing a Comprehensive Evidence-Based Service Package for Toddlers with Autism in a Low Resource Setting: Early Detection, Early Intervention, and Care Coordination

Objective: The number of children with autism, who have many unmet needs, is increasing dramatically. However, the existing evidence shows that early identification and intervention are effective in reducing the later costs and burdens of autism spectrum disorder (ASD). Thus, the present study aimed to develop evidence-based services for children with autism in Iran to reduce its impacts on the affected children and their families and to decrease its burden on the society. Method : A 3-step study was conducted based on a modification of the Replicating Effective Programs (REP) framework (step 1: need assessment and situation analysis; step 2: identifying current evidence-based services; step 3: designing the first draft of the package and its core elements). Each step was conducted by a specific methodology. Results: By considering the obtained data, it was found that a package of services with 4 core components to respond to the perceived needs in Iran was needed: (1) early detection of at-risk children; (2) care coordination and facilitation of access to current services; (3) implementation of an evidence-based early intervention program; and (4) training interventionists using an effective educational framework based on evidence-based material. Conclusion: REP framework was used in the present study, which has been shown to be effective in adapting and implementing health care services. By considering the preconditions of REP, a comprehensive package of services, with 4 components was designed for toddlers with autism in Iran. The next step will be to study this package using a multicenter hybrid effectiveness-implementation randomized control trial.

Overcoming Challenges to Evidence-Based Policy Development in a Large, Integrated Delivery System

OBJECTIVE

To describe a new Veterans Health Administration (VHA) program to foster the learning health system paradigm by rigorously evaluating health care initiatives and to report key lessons learned in designing those evaluations.

PRINCIPAL FINDINGS

The VHA's Quality Enhancement Research Initiative and its Health Services Research and Development Service are cooperating on several large, randomized program evaluations aimed at improving the care veterans receive and the efficiency with which it is delivered. The evaluations we describe involve collaborative design, outcomes assessment, and implementation science through partnerships between VHA operations and researchers. We review key factors to assess before committing to an evaluation. In addition to traditional design issues (such as ensuring adequate power and availability of data), these include others that are easily overlooked: the stability of intervention financing, means of controlling and commitment to adhering to randomized roll-out, degree of buy-in from key implementation staff, and feasibility of managing multiple veto points for interventions that span several programs, among others.

CONCLUSIONS

Successful program implementation and rigorous evaluation require resources, specialized expertise, and careful planning. If the learning health system model is to be sustained, organizations will need dedicated programs to prioritize resources and continuously adapt evaluation designs.

Periodic reflections: a method of guided discussions for documenting implementation phenomena

BACKGROUND

Ethnography has been proposed as a valuable method for understanding how implementation occurs within dynamic healthcare contexts, yet this method can be time-intensive and challenging to operationalize in pragmatic implementation. The current study describes an ethnographically-informed method of guided discussions developed for use by a multi-project national implementation program.

METHODS

The EMPOWER QUERI is conducting three projects to implement innovative care models in VA women's health for high-priority health concerns - prediabetes, cardiovascular risk, and mental health - utilizing the Replicating Effective Programs (REP) implementation strategy enhanced with stakeholder engagement and complexity science. Drawing on tenets of ethnographic research, we developed a lightly-structured method of guided "periodic reflections" to aid in documenting implementation phenomena over time. Reflections are completed as 30-60 min telephone discussions with implementation team members at monthly or bi-monthly intervals, led by a member of the implementation core. Discussion notes are coded to reflect key domains of interest and emergent themes, and can be analyzed singly or in triangulation with other qualitative and quantitative assessments to inform evaluation and implementation activities.

RESULTS

Thirty structured reflections were completed across the three projects during a 15-month period spanning pre-implementation, implementation, and sustainment activities. Reflections provide detailed, near-real-time information on projects' dynamic implementation context, including characteristics of implementation settings and changes in the local or national environment, adaptations to the intervention and implementation plan, and implementation team sensemaking and learning. Reflections also provide an opportunity for implementation teams to engage in recurring reflection and problem-solving.

CONCLUSIONS

To implement new, complex interventions into dynamic organizations, we must better understand the implementation process as it unfolds in real time. Ethnography is well suited to this task, but few approaches exist to aid in integrating ethnographic insights into implementation research. Periodic reflections show potential as a straightforward and low-burden method for documenting events across the life cycle of an implementation effort. They offer an effective means for capturing information on context, unfolding process and sensemaking, unexpected events, and diverse viewpoints, illustrating their value for use as part of an ethnographically-minded implementation approach.

TRIAL REGISTRATION

The two implementation research studies described in this article have been registered as required: Facilitating Cardiovascular Risk Screening and Risk Reduction in Women Veterans (NCT02991534); and Implementation of Tailored Collaborative Care for Women Veterans (NCT02950961).

Adaptive School-based Implementation of CBT (ASIC): clustered-SMART for building an optimized adaptive implementation intervention to improve uptake of mental health interventions in schools

Background

Depressive and anxiety disorders affect 20–30% of school-age youth, most of whom do not receive adequate services, contributing to poor developmental and academic outcomes. Evidence-based practices (EBPs) such as cognitive behavioral therapy (CBT) can improve outcomes, but numerous barriers limit access among affected youth. Many youth try to access mental health services in schools, but school professionals (SPs: counselors, psychologists, social workers) are rarely trained adequately in CBT methods. Further, SPs face organizational barriers to providing CBT, such as lack of administrative support. Three promising implementation strategies to address barriers to school-based CBT delivery include (1) Replicating Effective Programs (REP), which deploys customized CBT packaging, didactic training in CBT, and technical assistance; (2) coaching, which extends training via live supervision to improve SP competence in CBT delivery; and (3) facilitation, which employs an organizational expert who mentors SPs in strategic thinking to promote self-efficacy in garnering administrative support. REP is a relatively low-intensity/low-cost strategy, whereas coaching and facilitation require additional resources. However, not all schools will require all three strategies. The primary aim of this study is to compare the effectiveness of a school-level adaptive implementation intervention involving REP, coaching, and facilitation versus REP alone on the frequency of CBT delivered to students by SPs and student mental health outcomes. Secondary and exploratory aims examine cost-effectiveness, moderators, and mechanisms of implementation strategies.

Methods

Using a clustered, sequential multiple-assignment, randomized trial (SMART) design, ≥ 200 SPs from 100 schools across Michigan will be randomized initially to receive REP vs. REP+coaching. After 8 weeks, schools that do not meet a pre-specified implementation benchmark are re-randomized to continue with the initial strategy or to augment with facilitation.

Discussion

EBPs need to be implemented successfully and efficiently in settings where individuals are most likely to seek care in order to gain large-scale impact on public health. Adaptive implementation interventions hold the promise of providing cost-effective implementation support. This is the first study to test an adaptive implementation of CBT for school-age youth, at a statewide level, delivered by school staff, taking an EBP to large populations with limited mental health care access.

Trial registration

NCT03541317—Registered on 29 May 2018 on ClinicalTrials.gov PRS

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0808-8) contains supplementary material, which is available to authorized users.

Protocol for the ROSE sustainment (ROSES) study, a sequential multiple assignment randomized trial to determine the minimum necessary intervention to maintain a postpartum depression prevention program in prenatal clinics serving low-income women

BACKGROUND

More research on sustainment of interventions is needed, especially return on investment (ROI) studies to determine cost-benefit trade-offs for effort required to sustain and how much is gained when effective programs are sustained. The ROSE sustainment (ROSES) study uses a sequential multiple assignment randomized (SMART) design to evaluate the effectiveness and cost-effectiveness of a stepwise approach to sustainment of the ROSE postpartum depression prevention program in 90 outpatient clinics providing prenatal care to pregnant women on public assistance. Postpartum depression (PPD) is common and can have lasting consequences. Outpatient clinics offering prenatal care are an opportune place to provide PPD prevention because most women visit while pregnant. The ROSE (Reach Out, Stay Strong, Essentials for mothers of newborns) program is a group educational intervention to prevent PPD, delivered during pregnancy. ROSE has been found to reduce cases of PPD in community prenatal settings serving low-income pregnant women.

METHODS

All 90 prenatal clinics will receive enhanced implementation as usual (EIAU; initial training + tools for sustainment). At the first time at which a clinic is determined to be at risk for failure to sustain (i.e., at 3, 6, 9, 12, and 15 months), that clinic will be randomized to receive either (1) no additional implementation support (i.e., EIAU only), or (2) low-intensity coaching and feedback (LICF). If clinics receiving LICF are still at risk at subsequent assessments, they will be randomized to either (1) EIAU + LICF only, or (2) high-intensity coaching and feedback (HICF). Additional follow-up interviews will occur at 18, 24, and 30 months, but no implementation intervention will occur after 18 months. Outcomes include (1) percent sustainment of core program elements at each time point, (2) health impact (PPD rates over time at each clinic) and reach, and (3) ROI (costs and cost-effectiveness) of each sustainment step. Hypothesized mechanisms include sustainment of capacity to deliver core elements and engagement/ownership.

DISCUSSION

This study is the first randomized trial evaluating the ROI of a stepped approach to sustainment, a critical unanswered question in implementation science. It will also advance knowledge of implementation mechanisms and clinical care for an at-risk population.

TRIAL REGISTRATION

Clinicaltrials.gov, NCT03267563 . Registered June 14, 2018.

Evaluating broad-scale system change using the Consolidated Framework for Implementation Research: challenges and strategies to overcome them

OBJECTIVE

The objective of this paper is to demonstrate the utility of the CFIR framework for evaluating broad-scale change by discussing the challenges to be addressed when planning the assessment of broad-scale change and the solutions developed by the evaluation team to address those challenges. The evaluation of implementation of Patient-centered Care and Cultural Transformation (PCC&CT) within the Department of Veterans Affairs (VA) will be used as a demonstrative example. Patient-Centered Care (PCC) is personalized health care that considers a patient's circumstances and goals. The Department of Veterans Affairs (VA) is working towards implementing PCC throughout its healthcare system, comprised of multiple interventions with a singular long-term goal of cultural transformation, however little is known about the factors influencing its implementation. This paper discusses the issues that arose using CFIR to qualitatively assess the factors influencing implementation of cultural transformation.

RESULTS

Application of CFIR to this broad-scale evaluation revealed three strategies recommended for use in evaluating implementation of broad-scale change: (1) the need for adapted definitions for CFIR constructs (especially due to new application to broad-scale change), (2) the use of a mixed deductive-inductive approach with thematic coding to capture emergent themes not encompassed by CFIR, and (3) its use for expedited analysis and synthesis for rapid delivery of findings to operational partners. This paper is among the first to describe use of CFIR to guide the evaluation of a broad-scale transformation, as opposed to discrete interventions. The processes and strategies described in this paper provide a detailed example and structured approach that can be utilized and expanded upon by others evaluating implementation of broad-scale evaluations. Although CFIR was the framework selected for this evaluation, the strategies described in this paper including: use of adapted definitions, use of mixed deductive-inductive approach, and the approach for expedited analysis and synthesis can be transferred and tested with other frameworks.

Organizational culture and climate as moderators of enhanced outreach for persons with serious mental illness: results from a cluster-randomized trial of adaptive implementation strategies

Background

Organizational culture and climate are considered key factors in implementation efforts but have not been examined as moderators of implementation strategy comparative effectiveness. We investigated organizational culture and climate as moderators of comparative effectiveness of two sequences of implementation strategies (Immediate vs. Delayed Enhanced Replicating Effective Programs [REP]) combining Standard REP and REP enhanced with facilitation on implementation of an outreach program for Veterans with serious mental illness lost to care at Veterans Health Administration (VA) facilities nationwide.

Methods

This study is a secondary analysis of the cluster-randomized Re-Engage implementation trial that assigned 3075 patients at 89 VA facilities to either the Immediate or Delayed Enhanced REP sequences. We hypothesized that sites with stronger entrepreneurial culture, task, or relational climate would benefit more from Enhanced REP than Standard REP. Veteran- and site-level data from the Re-Engage trial were combined with site-aggregated measures of entrepreneurial culture and task and relational climate from the 2012 VA All Employee Survey. Longitudinal mixed-effects logistic models examined whether the comparative effectiveness of the Immediate vs. Delayed Enhanced REP sequences were moderated by culture or climate measures at 6 and 12 months post-randomization. Three Veteran-level outcomes related to the engagement with the VA system were assessed: updated documentation, attempted contact by coordinator, and completed contact.

Results

For updated documentation and attempted contact, Veterans at sites with higher entrepreneurial culture and task climate scores benefitted more from Enhanced REP compared to Standard REP than Veterans at sites with lower scores. Few culture or climate moderation effects were detected for the comparative effectiveness of the full sequences of implementation strategies.

Conclusions

Implementation strategy effectiveness is highly intertwined with contextual factors, and implementation practitioners may use knowledge of contextual moderation to tailor strategy deployment. We found that facilitation strategies provided with Enhanced REP were more effective at improving uptake of a mental health outreach program at sites with stronger entrepreneurial culture and task climate; Veterans at sites with lower levels of these measures saw more similar improvement under Standard and Enhanced REP. Within resource-constrained systems, practitioners may choose to target more intensive implementation strategies to sites that will most benefit from them.

Trial registration

ISRCTN: ISRCTN21059161. Date registered: April 11, 2013.

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0787-9) contains supplementary material, which is available to authorized users.

Supporting teams to optimize function and independence in Veterans: a multi-study program and mixed methods protocol

Background

Successful implementation of new clinical programs depends on effectively establishing, reorganizing, or enhancing team structures and processes to coordinate the work of individuals who are interdependent in their tasks, manage relationships, and share responsibility for outcomes. However, a one-size-fits-all approach is rarely effective. In partnership with VA national clinical leaders and local clinical champions, the Optimizing Function and Independence VA Quality Enhancement Research Initiative program (Function QUERI) will evaluate efforts to implement team-based clinical programs for Veterans at risk for functional decline and disability.

Methods

Function QUERI will implement and evaluate three innovative, evidence-based clinical programs in VA medical centers: (1) a group physical therapy program for knee osteoarthritis (Group PT); (2) assisted early mobility for hospitalized older veterans (STRIDE), a supervised walking program for hospitalized older veterans; and (3) implementation of helping invested family members improve veteran experiences study (iHI-FIVES), a skills training program for caregivers of disabled Veterans. A common reason for clinical care gaps in these populations is poor communication and coordination among the many interdisciplinary providers involved in their care. To facilitate the implementation of the clinical programs, Function QUERI will evaluate the impact of complexity science-based implementation intervention to promote team readiness (CONNECT), an implementation intervention designed as a bundle of interaction-oriented activities to promote team function and readiness for change, on the implementation of clinical programs across multiple sites. The evaluation will use a mixed methods design. Group PT is a local, single-site quality improvement project where a modified CONNECT intervention will be tested to inform the remaining program implementation projects. For STRIDE and iHI-FIVES projects, we will randomize participating sites to implement the clinical program, with the CONNECT intervention or not, and will use a stepped-wedge cluster randomized trial design.

Discussion

Function QUERI will translate its findings across its projects to identify the contextual factors and components from CONNECT that improve team processes and function to optimize effective implementation for future rollout of VA clinical programs. Synthesizing findings within and across projects, we will specify dimensions of team characteristics and function that enhance capacity for clinical innovation and uptake of evidence-based programs.

Trial registration

NCT03300336 Registered September 28, 2017, NCT03474380 Registered March 15, 2018.

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0748-3) contains supplementary material, which is available to authorized users.

A Structured Process Description of a Pragmatic Implementation Project: Improving Integrated Care for Older Persons in Residential Care Homes

Evaluation of the implementation of integrated care can differ from trial-based research due to complexity. Therefore, we examined whether a theory-based method for process description of implementation can contribute to improvement of evidence-based care. MOVIT, a Dutch project aimed at implementing integrated care for older vulnerable persons in residential care homes, was used as a case study. The project activities were defined according to implementation taxonomy and mapped in a matrix of theoretical levels and domains. Project activities mainly targeted professionals (both individual and group). A few activities targeted the organizational level, whereas none targeted the policy level, or the patient, or the “social, political, and legal” domains. However, the resulting changes in care delivery arrangement had consequences for professionals, patients, organizations, and the social, political, and legal domains. A structured process description of a pragmatic implementation project can help assess the fidelity and quality of the implementation, and identify relevant contextual factors for immediate adaptation and future research. The description showed that, in the MOVIT project, there was a discrepancy between the levels and domains targeted by the implementation activities and those influenced by the resulting changes in delivery arrangement. This could have influenced, in particular, the adoption and sustainability of the project.

Testing the implementation and sustainment facilitation (ISF) strategy as an effective adjunct to the Addiction Technology Transfer Center (ATTC) strategy: study protocol for a cluster randomized trial

BACKGROUND

Improving the extent to which evidence-based practices (EBPs)-treatments that have been empirically shown to be efficacious or effective-are integrated within routine practice is a well-documented challenge across numerous areas of health. In 2014, the National Institute on Drug Abuse funded a type 2 effectiveness-implementation hybrid trial titled the substance abuse treatment to HIV Care (SAT2HIV) Project. Aim 1 of the SAT2HIV Project tests the effectiveness of a motivational interviewing-based brief intervention (MIBI) for substance use as an adjunct to usual care within AIDS service organizations (ASOs) as part of its MIBI Experiment. Aim 2 of the SAT2HIV Project tests the effectiveness of implementation and sustainment facilitation (ISF) as an adjunct to the Addiction Technology Transfer Center (ATTC) model for training staff in motivational interviewing as part of its ISF Experiment. The current paper describes the study protocol for the ISF Experiment.

METHODS

Using a cluster randomized design, case management and leadership staff from 39 ASOs across the United States were randomized to receive either the ATTC strategy (control condition) or the ATTC + ISF strategy (experimental condition). The ATTC strategy is staff-focused and includes 10 discrete strategies (e.g., provide centralized technical assistance, conduct educational meetings, provide ongoing consultation). The ISF strategy is organization-focused and includes seven discrete strategies (e.g., use an implementation advisor, organize implementation team meetings, conduct cyclical small tests of change). Building upon the exploration-preparation-implementation-sustainment (EPIS) framework, the effectiveness of the ISF strategy is examined via three staff-level measures: (1) time-to-proficiency (i.e., preparation phase outcome), (2) implementation effectiveness (i.e., implementation phase outcome), and (3) level of sustainment (i.e., sustainment phase outcome).

DISCUSSION

Although not without limitations, the ISF experiment has several strengths: a highly rigorous design (randomized, hypothesis-driven), high-need setting (ASOs), large sample size (39 ASOs), large geographic representation (23 states and the District of Columbia), and testing along multiple phases of the EPIS continuum (preparation, implementation, and sustainment). Thus, study findings will significantly improve generalizable knowledge regarding the best preparation, implementation, and sustainment strategies for advancing EBPs along the EPIS continuum. Moreover, increasing ASO's capacity to address substance use may improve the HIV Care Continuum. Trial registration ClinicalTrials.gov: NCT03120598.

Enhancing Mental and Physical Health of Women through Engagement and Retention (EMPOWER): a protocol for a program of research

BACKGROUND

The Enhancing Mental and Physical health of Women through Engagement and Retention or EMPOWER program represents a partnership with the US Department of Veterans Health Administration (VA) Health Service Research and Development investigators and the VA Office of Women's Health, National Center for Disease Prevention and Health Promotion, Primary Care-Mental Health Integration Program Office, Women's Mental Health Services, and the Office of Patient Centered Care and Cultural Transformation. EMPOWER includes three projects designed to improve women Veterans' engagement and retention in evidence-based care for high-priority health conditions, i.e., prediabetes, cardiovascular, and mental health.

METHODS/DESIGN

The three proposed projects will be conducted in VA primary care clinics that serve women Veterans including general primary care and women's health clinics. The first project is a 1-year quality improvement project targeting diabetes prevention. Two multi-site research implementation studies will focus on cardiovascular risk prevention and collaborative care to address women Veterans' mental health treatment needs respectively. All projects will use the evidence-based Replicating Effective Programs (REP) implementation strategy, enhanced with multi-stakeholder engagement and complexity theory. Mixed methods implementation evaluations will focus on investigating primary implementation outcomes of adoption, acceptability, feasibility, and reach. Program-wide organizational-, provider-, and patient-level measures and tools will be utilized to enhance synergy, productivity, and impact. Both implementation research studies will use a non-randomized stepped wedge design.

DISCUSSION

EMPOWER represents a coherent program of women's health implementation research and quality improvement that utilizes cross-project implementation strategies and evaluation methodology. The EMPOWER Quality Enhancement Research Initiative (QUERI) will constitute a major milestone for realizing women Veterans' engagement and empowerment in the VA system. EMPOWER QUERI will be conducted in close partnership with key VA operations partners, such as the VA Office of Women's Health, to disseminate and spread the programs nationally.

TRIAL REGISTRATION

The two implementation research studies described in this protocol have been registered as required: Facilitating Cardiovascular Risk Screening and Risk Reduction in Women Veterans: Trial registration NCT02991534 , registered 9 December 2016. Implementation of Tailored Collaborative Care for Women Veterans: Trial registration NCT02950961 , registered 21 October 2016.

Comparing cluster-level dynamic treatment regimens using sequential, multiple assignment, randomized trials: Regression estimation and sample size considerations

Cluster-level dynamic treatment regimens can be used to guide sequential treatment decision-making at the cluster level in order to improve outcomes at the individual or patient-level. In a cluster-level dynamic treatment regimen, the treatment is potentially adapted and re-adapted over time based on changes in the cluster that could be impacted by prior intervention, including aggregate measures of the individuals or patients that compose it. Cluster-randomized sequential multiple assignment randomized trials can be used to answer multiple open questions preventing scientists from developing high-quality cluster-level dynamic treatment regimens. In a cluster-randomized sequential multiple assignment randomized trial, sequential randomizations occur at the cluster level and outcomes are observed at the individual level. This manuscript makes two contributions to the design and analysis of cluster-randomized sequential multiple assignment randomized trials. First, a weighted least squares regression approach is proposed for comparing the mean of a patient-level outcome between the cluster-level dynamic treatment regimens embedded in a sequential multiple assignment randomized trial. The regression approach facilitates the use of baseline covariates which is often critical in the analysis of cluster-level trials. Second, sample size calculators are derived for two common cluster-randomized sequential multiple assignment randomized trial designs for use when the primary aim is a between-dynamic treatment regimen comparison of the mean of a continuous patient-level outcome. The methods are motivated by the Adaptive Implementation of Effective Programs Trial which is, to our knowledge, the first-ever cluster-randomized sequential multiple assignment randomized trial in psychiatry.