The stages of implementation completion for evidence-based practice: protocol for a mixed methods study

Understanding pathways from implementation to sustainment: a longitudinal, mixed methods analysis of promising practices implemented in the Veterans Health Administration

BACKGROUND

The Veterans Health Administration (VHA) is the United States largest learning health system. The Diffusion of Excellence (DoE) program is a large-scale model of diffusion that identifies and diffuses evidence-informed practices across VHA. During the period of 2016-2021, 57 evidence-informed practices were implemented across 82 VHA facilities. This setting provides a unique opportunity to understand sustainment determinants and pathways. Our objective was to characterize the longitudinal pathways of practices as they transition from initial implementation to long-term sustainment at each facility.

METHODS

A longitudinal, mixed-methods evaluation of 82 VHA facilities. Eighty-two facility representatives, chosen by leadership as points-of-contact for 57 DoE practices, were eligible for post-implementation interviews and annual sustainment surveys. Primary outcomes (implementation, sustainment), and secondary outcomes (institutionalization, effectiveness, anticipated sustainment) at four time-points were collected. We performed descriptive statistics and directed content analysis using Hailemariam et al.'s factors influencing sustainment.

RESULTS

After approximately five years post-implementation (e.g., 2021 sustainment outcomes), of the 82 facilities, about one-third fully sustained their practice compared to one-third that did not fully sustain their practice because it was in a "liminal" stage (neither sustained nor discontinued) or permanently discontinued. The remaining one-third of facilities had missing 2021 sustainment outcomes. A higher percentage of facilities (70%) had inconsistent primary outcomes (changing over time) compared to facilities (30%) with consistent primary outcomes (same over time). Thirty-four percent of facilities with sustained practices reported resilience since they overcame implementation and sustainment barriers. Facilities with sustained practices reported more positive secondary outcomes compared to those that did not sustain their practice. Key factors facilitating practice sustainment included: demonstrating practice effectiveness/benefit, sufficient organizational leadership, sufficient workforce, and adaptation/alignment with local context. Key factors hindering practice sustainment included: insufficient workforce, not able to maintain practice fidelity/integrity, critical incidents related to the COVID-19 pandemic, organizational leadership did not support sustainment of practice, and no ongoing support.

CONCLUSIONS

We identified diverse pathways from implementation to sustainment, and our data underscore that initial implementation outcomes may not determine long-term sustainment outcomes. This longitudinal evaluation contributes to understanding impacts of the DoE program, including return on investment, achieving learning health system goals, and insights into achieving high-quality healthcare in VHA.

Sepsis screening protocol implementation: a clinician-validated rapid realist review

INTRODUCTION

The failed or partial implementation of clinical practices negatively impacts patient safety and increases systemic inefficiencies. Implementation of sepsis screening guidelines has been undertaken in many settings with mixed results. Without a theoretical understanding of what leads to successful implementation, improving implementation will continue to be ad hoc or intuitive. This study proposes a programme theory for how and why the successful implementation of sepsis screening guidelines can occur.

METHODS

A rapid realist review was conducted to develop a focused programme theory for the implementation of sepsis screening guidelines. An independent two-reviewer approach was used to iteratively extract and synthesise context and mechanism data. Theoretical context-mechanism-outcome propositions were refined and validated by clinicians using a focus group and individual realist interviews. Implementation resources and clinical reasoning were differentiated in articulating mechanisms.

RESULTS

Eighteen articles were included in the rapid review. The theoretical domains framework was identified as the salient substantive theory informing the programme theory. The theory consisted of five main middle-range propositions. Three promoting mechanisms included positive belief about the benefits of the protocol, belief in the legitimacy of using the protocol and trust within the clinical team. Two inhibiting mechanisms included pessimism about the protocol being beneficial and pessimism about the team. Successful implementation was defined as achieving fidelity and sustained use of the intervention. Two intermediate outcomes, acceptability and feasibility of the intervention, and adoption, were necessary to achieve before successful implementation.

CONCLUSION

This rapid realist review synthesised key information from the literature and clinician feedback to develop a theory-based approach to clinical implementation of sepsis screening. The programme theory presents knowledge users with an outline of how and why clinical interventions lead to successful implementation and could be applied in other clinical areas to improve quality and safety.

Implementing a pharmacist-integrated collaborative model of medication treatment for opioid use disorder in primary care: study design and methodological considerations

BACKGROUND

Pharmacists remain an underutilized resource in the treatment of opioid use disorder (OUD). Although studies have engaged pharmacists in dispensing medications for OUD (MOUD), few studies have evaluated collaborative care models in which pharmacists are an active, integrated part of a primary care team offering OUD care.

METHODS

This study seeks to implement a pharmacist integrated MOUD clinical model (called PrIMO) and evaluate its feasibility, acceptability, and impact across four diverse primary care sites. The Consolidated Framework for Implementation Research is used as an organizing framework for study development and interpretation of findings. Implementation Facilitation is used to support PrIMO adoption. We assess the primary outcome, the feasibility of implementing PrIMO, using the Stages of Implementation Completion (SIC). We evaluate the acceptability and impact of the PrIMO model at the sites using mixed-methods and combine survey and interview data from providers, pharmacists, pharmacy technicians, administrators, and patients receiving MOUD at the primary care sites with patient electronic health record data. We hypothesize that it is feasible to launch delivery of the PrIMO model (reach SIC Stage 6), and that it is acceptable, will positively impact patient outcomes 1 year post model launch (e.g., increased MOUD treatment retention, medication regimen adherence, service utilization for co-morbid conditions, and decreased substance use), and will increase each site's capacity to care for patients with MOUD (e.g., increased number of patients, number of prescribers, and rate of patients per prescriber).

DISCUSSION

This study will provide data on a pharmacist-integrated collaborative model of care for the treatment of OUD that may be feasible, acceptable to both site staff and patients and may favorably impact patients' access to MOUD and treatment outcomes.

TRIAL REGISTRATION

The study was registered on Clinicaltrials.gov (NCT05310786) on April 5, 2022, https://www.

CLINICALTRIALS

gov/study/NCT05310786?id=NCT05310786&rank=1.

Core Components and Implementation Determinants of Multilevel Service Delivery Frameworks Across Child Mental Health Service Settings

Multilevel service delivery frameworks are approaches to structuring and organizing a spectrum of evidence-based services and supports, focused on assessment, prevention, and intervention designed for the local context. Exemplar frameworks in child mental health include positive behavioral interventions and supports in education, collaborative care in primary care, and systems of care in community mental health settings. Yet, their high-quality implementation has lagged. This work proposes a conceptual foundation for multilevel service delivery frameworks spanning diverse mental health service settings that can inform development of strategic implementation supports. We draw upon the existing literature for three exemplar multilevel service delivery frameworks in different child mental health service settings to (1) identify core components common to each framework, and (2) to highlight prominent implementation determinants that interface with each core component. Six interrelated components of multilevel service delivery frameworks were identified, including, (1) a systems-level approach, (2) data-driven problem solving and decision-making, (3) multiple levels of service intensity using evidence-based practices, (4) cross-linking service sectors, (5) multiple providers working together, including in teams, and (6) built-in implementation strategies that facilitate delivery of the overall model. Implementation determinants that interface with core components were identified at each contextual level. The conceptual foundation provided in this paper has the potential to facilitate cross-sector knowledge sharing, promote generalization across service settings, and provide direction for researchers, system leaders, and implementation intermediaries/practitioners working to strategically support the high-quality implementation of these frameworks.

Implementation of Direct-to-Patient Mobile Teledermatology in VA

BACKGROUND

Innovative technology can enhance patient access to healthcare but must be successfully implemented to be effective.

OBJECTIVE

We evaluated Department of Veterans Affairs' (VA's) implementation of My VA Images, a direct-to-patient asynchronous teledermatology mobile application enabling established dermatology patients to receive follow-up care remotely instead of in-person.

DESIGN /PARTICIPANTS/APPROACH

Following pilot testing at 3 facilities, the app was introduced to 28 facilities (4 groups of 7) every 3 months using a stepped-wedge cluster-randomized design. Using the Organizational Theory of Implementation Effectiveness, we examined the app's implementation using qualitative and quantitative data consisting of encounter data from VA's corporate data warehouse; app usage from VA's Mobile Health database; bi-monthly reports from facility representatives; phone interviews with clinicians; and documented communications between the operational partner and facility staff.

KEY RESULTS

Implementation policies and practices included VA's vision to expand home telehealth and marketing/communication strategies. The COVID-19 pandemic dominated the implementation climate by stressing staffing, introducing competing demands, and influencing stakeholder attitudes to the app, including its fit to their values. These factors were associated with mixed implementation effectiveness, defined as high quality consistent use. Nineteen of 31 exposed facilities prepared to use the app; 10 facilities used it for actual patient care, 7 as originally intended. Residents, nurse practitioners, and physician assistants were more likely than attendings to use the app. Facilities exposed to the app pre-pandemic were more likely to use and sustain the new process.

CONCLUSIONS

Considerable heterogeneity existed in implementing mobile teledermatology, despite VA's common mission, integrated healthcare system, and stakeholders' broad interest. Identifying opportunities to target favorable facilities and user groups (such as teaching facilities and physician extenders, respectively) while addressing internal implementation barriers including incomplete integration with the electronic health record as well as inadequate staffing may help optimize the initial impact of direct-to-patient telehealth. The COVID pandemic was a notable extrinsic barrier.

CLINICAL TRIALS REGISTRATION

NCT03241589.

Evaluating the Implementation of the Connect for Health Pediatric Weight Management Program

Importance

Adoption of primary care interventions to reduce childhood obesity is limited. Progress in reducing obesity prevalence and eliminating disparities can be achieved by implementing effective childhood obesity management interventions in primary care settings.

Objective

To examine the extent to which implementation strategies supported the uptake of research evidence and implementation of the Connect for Health pediatric weight management program.

Design, Setting, and Participants

This quality improvement study took place at 3 geographically and demographically diverse health care organizations with substantially high numbers of children living in low-income communities in Denver, Colorado; Boston, Massachusetts; and Greenville, South Carolina, from November 2019 to April 2022. Participants included pediatric primary care clinicians and staff and families with children aged 2 to 12 years with a body mass index (BMI) in the 85th percentile or higher.

Exposures

Pediatric weight management program with clinician-facing tools (ie, clinical decision support tools) and family-facing tools (ie, educational handouts, text messaging program, community resource guide) along with implementation strategies (ie, training and feedback, technical assistance, virtual learning community, aligning with hospital performance metrics) to support the uptake.

Main Outcomes and Measures

Primary outcomes were constructs from the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) Framework examined through parent, clinician, and leadership surveys and electronic health record data to understand the number of children screened and identified, use of the clinical decision support tools, program acceptability, fidelity to the intervention and implementation strategies, and program sustainability.

Results

The program screened and identified 18 333 children across 3 organizations (Denver Health, 8480 children [46.3%]; mean [SD] age, 7.97 [3.31] years; 3863 [45.5%] female; Massachusetts General Hospital (MGH), 6190 children [33.8%]; mean [SD] age, 7.49 [3.19] years; 2920 [47.2%] female; Prisma Health, 3663 children [20.0%]; mean [SD] age, 7.33 [3.15] years; 1692 [46.2%] female) as having an elevated BMI. The actionable flagging system was used for 8718 children (48%). The reach was equitable, with 7843 children (92.4%) from Denver Health, 4071 children (65.8%) from MGH, and 1720 children (47%) from Prisma Health being from racially and ethnically minoritized groups. The sites had high fidelity to the program and 6 implementation strategies, with 4 strategies (67%) used consistently at Denver Health, 6 (100%) at MGH, and 5 (83%) at Prisma Health. A high program acceptability was found across the 3 health care organizations; for example, the mean (SD) Acceptability of Intervention Measure score was 3.72 (0.84) at Denver Health, 3.82 (0.86) at MGH, and 4.28 (0.68) at Prisma Health. The implementation strategies were associated with 7091 (39%) uses of the clinical decision support tool. The mean (SD) program sustainability scores were 4.46 (1.61) at Denver Health, 5.63 (1.28) at MGH, and 5.54 (0.92) at Prisma Health.

Conclusions and Relevance

These findings suggest that by understanding what strategies enable the adoption of scalable and implementation-ready programs by other health care organizations, it is feasible to improve the screening, identification, and management of children with overweight or obesity and mitigate existing disparities.

Exploring the implementation potential of physical activity assessment and prescription tools in physical therapy practice: a mixed-method study

BACKGROUND

Despite the benefits of physical activity (PA), especially related to aging, physical therapists do not perform regular PA prescriptions secondary to various barriers, including lack of tools. Therefore, we developed the Inventory of Physical Activity Barriers (IPAB).

OBJECTIVE

Explore potential solutions that could address the current lack of PA prescription among United States-based physical therapists treating patients 50 years and older.

METHOD

A convergent parallel mixed-method design consisting of focus groups and self-report questionnaires. Descriptive statistics were used for all quantitative variables. Focus groups were thematically coded.

RESULTS

The 26 participants had 8.6 years (SD = 6.4) of clinical experience, 88.4% (n = 23) reported they regularly have PA conversations with patients, 65.4% (n = 17) regularly assess PA levels, and 19.2% (n = 5) regularly provide PA prescriptions. We identified three themes: 1) opportunities and challenges related to PA prescriptions; 2) lack of standardization in PA assessments and interventions; and 3) implementation potential for innovative solutions that address the current informal PA assessments and interventions.

CONCLUSION

Physical therapists are amenable to incorporating innovative solutions that support physical activity prescription behavior. Therefore, we recommend the continued development and implementation of PA assessment and prescription tools.

Costs of Implementing Electronic Context Factor Assessments and Patient-reported Outcomes in Pain Clinic Settings

BACKGROUND

The Healing Encounters and Attitudes Lists (HEALs) patient-reported measures, consisting of 6 separate context factor questionnaires, predict patients' pain improvements. Our Patient-centered Outcomes Research Initiative-funded implementation project demonstrated success in using HEAL data during clinic consultations to enhance patient engagement, improve patient outcomes, and reduce opioid prescribing.

OBJECTIVE

We aimed to determine the resources needed for additional sites to implement HEAL to improve pain care treatment.

RESEARCH DESIGN

An observational study from March 1 to November 30, 2021, assessing implementation cost data from invoices, time and salary requirements for clinic personnel training, estimates of non-site-based costs, and one-time resource development costs.

SUBJECTS

Unique patients eligible to complete a HEAL survey (N=24,018) and 74 clinic personnel.

MEASURES

The Stages of Implementation Completion guided documentation of preimplementation, implementation, and sustainment activities of HEAL pain clinic operations. These informed the calculations of the costs of implementation.

RESULTS

The total time for HEAL implementation is 7 months: preimplementation and implementation phases (4 mo) and sustainment (3 mo). One hour of HEAL implementation involving a future clinical site consisting of 2 attending physicians, 1 midlevel provider, 1 nurse manager, 1 nurse, 1 radiology technician, 2 medical assistants, and 1 front desk staff will cost $572. A 10-minute time increment for all clinic staff is $95. Total implementation costs based on hourly rates over 7 months, including non-site-based costs of consultations, materials, and technology development costs, is $28,287.

CONCLUSIONS

Documenting our implementation costs clarifies the resources needed for additional new sites to implement HEAL to measure pain care quality and to engage patients and clinic personnel.

Optimized segmented regression models for the transition period of intervention effects

BACKGROUND

The interrupted time series (ITS) design is a widely used approach to examine the effects of interventions. However, the classic segmented regression (CSR) method, the most popular statistical technique for analyzing ITS data, may not be adequate when there is a transitional period between the pre- and post-intervention phases.

METHODS

To address this issue and better capture the distribution patterns of intervention effects during the transition period, we propose using different cumulative distribution functions in the CSR model and developing corresponding optimized segmented regression (OSR) models. This study illustrates the application of OSR models to estimate the long-term impact of a national free delivery service policy intervention in Ethiopia.

RESULTS

Regardless of the choice of transition length ([Formula: see text]) and distribution patterns of intervention effects, the OSR models outperformed the CSR model in terms of mean square error (MSE), indicating the existence of a transition period and the validity of our model's assumptions. However, the estimates of long-term impacts using OSR models are sensitive to the selection of L, highlighting the importance of reasonable parameter specification. We propose a data-driven approach to select the transition period length to address this issue.

CONCLUSIONS

Overall, our OSR models provide a powerful tool for modeling intervention effects during the transition period, with a superior model fit and more accurate estimates of long-term impacts. Our study highlights the importance of appropriate statistical methods for analyzing ITS data and provides a useful framework for future research.

The relative value of Pre-Implementation stages for successful implementation of evidence-informed programs

BACKGROUND

Most implementations fail before the corresponding services are ever delivered. Measuring implementation process fidelity may reveal when and why these attempts fail. This knowledge is necessary to support the achievement of positive implementation milestones, such as delivering services to clients (program start-up) and competency in treatment delivery. The present study evaluates the extent to which implementation process fidelity at different implementation stages predicts achievement of those milestones.

METHODS

Implementation process fidelity data-as measured by the Stages of Implementation Completion (SIC)-from 1287 implementing sites across 27 evidence-informed programs were examined in mixed effects regression models with sites nested within programs. Implementation process fidelity, as measured by the proportion of implementation activities completed during the three stages of the SIC Pre-Implementation phase and overall Pre-Implementation (Phase 1) and Implementation (Phase 2) proportion scores, was assessed as a predictor of sites achieving program start-up (i.e., delivering services) and competency in program delivery.

RESULTS

The predicted probability of start-up across all sites was low at 35% (95% CI [33%, 38%]). When considering the evidence-informed program being implemented, that probability was nearly twice as high (64%; 95% CI [42%, 82%]), and 57% of the total variance in program start-up was attributable to the program. Implementation process fidelity was positively and significantly associated with achievement of program start-up and competency. The magnitude of this relationship varied significantly across programs for Pre-Implementation Stage 1 (i.e., Engagement) only. Compared to other stages, completing more Pre-Implementation Stage 3 (Readiness Planning) activities resulted in the most rapid gains in probability of achieving program start-up. The predicted probability of achieving competency was very low unless sites had high scores in both Pre-Implementation and Implementation phases.

CONCLUSIONS

Strong implementation process fidelity-as measured by SIC Pre-Implementation and Implementation phase proportion scores-was associated with sites' achievement of program start-up and competency in program delivery, with early implementation process fidelity being especially potent. These findings highlight the importance of a rigorous Pre-Implementation process.

Development of an Implementation Blueprint to Scale-Up Contraception Care for Adolescents with Psychiatric Conditions in a Pediatric Hospital

Implementation blueprints are comprehensive plans that describe implementation strategies, goals, timelines, and key personnel necessary for launching new interventions. Although blueprints are a foundational step in driving intervention rollout, little is known about how blueprints are developed, refined, and used in practice. The objective of this study was to describe a systematic, collaborative approach to developing, refining, and utilizing a formal implementation blueprint for scaling up the Contraception Care at Behavioral Health Pavilion (CC@BHP) intervention for adolescents hospitalized in psychiatric units within a pediatric hospital in the United States. In Stage 1 (Planning/Preparation), we assembled a Research Advisory Board (RAB) of 41 multidisciplinary members and conducted a formative evaluation to identify potential barriers to CC@BHP implementation. Barriers were mapped to implementation strategies using the Consolidated Framework for Implementation Research (CFIR) and Expert Recommendations for Implementing Change (ERIC) tool and used to create an initial blueprint. In Stage 2 (Development/Implementation), RAB members used activity logs to track implementation activities over the 18-month study period, which were then mapped to formal implementation strategies used to further develop the blueprint. About 30% of strategies were situated in the 'Train and Educate Stakeholders' ERIC category, 20% in 'Use Evaluative and Iterative Strategies,' and 16% in 'Develop Stakeholder Interrelationships' category. In Stage 3 (Synthesis/Refinement), the final blueprint was refined, consisting of 16 goals linked to 10 strategies for pre-implementation and 6 strategies for implementation. Feedback on the blueprint emphasized the role of the project champion in translating the blueprint into smaller, actionable steps for implementers.

Enhancing Mental and Physical Health of Women through Engagement and Retention (EMPOWER) 2.0 QUERI: study protocol for a cluster-randomized hybrid type 3 effectiveness-implementation trial

BACKGROUND

Women Veterans are the fastest-growing segment of Veterans Health Administration (VA) users. The VA has invested heavily in delivering care for women Veterans that is effective, comprehensive, and gender-tailored. However, gender disparities persist in cardiovascular (CV) and diabetes risk factor control, and the rate of perinatal depression among women Veterans is higher than that among civilian women. Challenges such as distance, rurality, negative perception of VA, discrimination (e.g., toward sexual and/or gender minority individuals), and harassment on VA grounds can further impede women's regular use of VA care. Enhancing Mental and Physical Health of Women through Engagement and Retention (EMPOWER) 2.0 builds on work to date by expanding access to evidence-based, telehealth preventive and mental health services for women Veterans with high-priority health conditions in rural and urban-isolation areas.

METHODS

EMPOWER 2.0 will evaluate two implementation strategies, Replicating Effective Practices (REP) and Evidence-Based Quality Improvement (EBQI), in supporting the implementation and sustainment of three evidence-based interventions (Virtual Diabetes Prevention Program; Telephone Lifestyle Coaching Program; and Reach Out, Stay Strong Essentials) focused on preventive and mental health care for women Veterans. We will conduct a mixed-methods implementation evaluation using a cluster-randomized hybrid type 3 effectiveness-implementation trial design to compare the effectiveness of REP and EBQI on improved access to and rates of engagement in telehealth preventive lifestyle and mental health services. Other outcomes of interest include (a) VA performance metrics for telehealth care delivery and related clinical outcomes; (b) progression along the Stages of Implementation Completion; (c) adaptation, sensemaking, and experiences of implementation among multilevel stakeholders; and (d) cost and return on investment. We will also generate implementation playbooks for program partners to support scale-up and spread of these and future evidence-based women's health programs and policies.

DISCUSSION

EMPOWER 2.0 provides a model for mixed-methods hybrid type 3 effectiveness-implementation trial design incorporating evaluation of performance metrics, implementation progress, stakeholder experience, and cost and return on investment, with the ultimate goal of improving access to evidence-based preventive and mental telehealth services for women Veterans with high-priority health conditions.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT05050266 . Registered on 20 September 2021.

Barriers and facilitators to implementing evidence-based interventions in the context of a randomized clinical trial in the United States: a qualitative study

BACKGROUND

Evidence-based interventions, which are typically supported by data from randomized controlled trials (RCTs), are highly valued by providers of human services like child welfare. However, implementing such interventions in the context of a randomized clinical trial is a complex process, as conducting an RCT adds extra tasks for providers and complicating factors for provider organizations. Utilizing the Exploration, Preparation, Implementation, and Sustainment Framework, this study examines factors that facilitate or impede success in the implementation of evidence-based interventions in the context of a largescale trial of SafeCare,® a child maltreatment intervention.

METHODS

Qualitative data were obtained as part of a larger mixed-methods study involving a cluster randomized trial comparing SafeCare to usual services for caregivers within nine child welfare agencies across four states. Between May and October 2017, individual interviews were conducted with a purposive sample of 21 child welfare administrators and 24 supervisors, and 19 focus groups were conducted with 84 providers. Data were coded iteratively and grouped into themes.

RESULTS

Several interconnected themes centered on facilitators and barriers to SafeCare implementation in the context of a randomized clinical trial. Facilitators included: (1) Benefits afforded through RCT participation; (2) Shared vision and sustained buy-in across system and organizational levels; and (3) Ongoing leadership support for SafeCare and the RCT. Barriers that hindered SafeCare were: (1) Insufficient preparation to incorporate SafeCare into services; (2) Perceived lack of fit, leading to mixed support for SafeCare and the RCT; and (3) Requirements of RCT participation at the provider level.

CONCLUSIONS

These data yield insight into an array of stakeholder perspectives on the experience of implementing a new intervention in the context of a largescale trial. This research also sheds light on how the dynamics of conducting an RCT may affect efforts to implement interventions in complex and high-pressure contexts. Findings highlight the importance of aligning knowledge and expectations among researchers, administrators of organizations, and supervisors and providers. Researchers should work to alleviate the burdens of study involvement and promote buy-in among frontline staff not only for the program but also for the research itself.

Sustainment of diverse evidence-informed practices disseminated in the Veterans Health Administration (VHA): initial development and piloting of a pragmatic survey tool

BACKGROUND

There are challenges associated with measuring sustainment of evidence-informed practices (EIPs). First, the terms sustainability and sustainment are often falsely conflated: sustainability assesses the likelihood of an EIP being in use in the future while sustainment assesses the extent to which an EIP is (or is not) in use. Second, grant funding often ends before sustainment can be assessed. The Veterans Health Administration (VHA) Diffusion of Excellence (DoE) program is one of few large-scale models of diffusion; it seeks to identify and disseminate practices across the VHA system. The DoE sponsors "Shark Tank" competitions, in which leaders bid on the opportunity to implement a practice with approximately 6 months of implementation support. As part of an ongoing evaluation of the DoE, we sought to develop and pilot a pragmatic survey tool to assess sustainment of DoE practices.

METHODS

In June 2020, surveys were sent to 64 facilities that were part of the DoE evaluation. We began analysis by comparing alignment of quantitative and qualitative responses; some facility representatives reported in the open-text box of the survey that their practice was on a temporary hold due to COVID-19 but answered the primary outcome question differently. As a result, the team reclassified the primary outcome of these facilities to Sustained: Temporary COVID-Hold. Following this reclassification, the number and percent of facilities in each category was calculated. We used directed content analysis, guided by the Consolidated Framework for Implementation Research (CFIR), to analyze open-text box responses.

RESULTS

A representative from forty-one facilities (64%) completed the survey. Among responding facilities, 29/41 sustained their practice, 1/41 partially sustained their practice, 8/41 had not sustained their practice, and 3/41 had never implemented their practice. Sustainment rates increased between Cohorts 1-4.

CONCLUSIONS

The initial development and piloting of our pragmatic survey allowed us to assess sustainment of DoE practices. Planned updates to the survey will enable flexibility in assessing sustainment and its determinants at any phase after adoption. This assessment approach can flex with the longitudinal and dynamic nature of sustainment, including capturing nuances in outcomes when practices are on a temporary hold. If additional piloting illustrates the survey is useful, we plan to assess the reliability and validity of this measure for broader use in the field.

Adapting the stages of implementation completion to an evidence-based implementation strategy: The development of the NIATx stages of implementation completion

Background

Dissemination and implementation frameworks provide the scaffolding to explore the effectiveness of evidence-based practices (EBPs) targeting process of care and organizational outcomes. Few instruments, like the stages of implementation completion (SIC) examine implementation fidelity to EBP adoption and how organizations differ in their approach to implementation. Instruments to measure organizational competency in the utilization of implementation strategies are lacking.

Method

An iterative process was utilized to adapt the SIC to the NIATx implementation strategies. The new instrument, NIATx-SIC, was applied in a randomized controlled trial involving 53 addiction treatment agencies in Washington state to improve agency co-occurring capacity. NIATx-SIC data were reported by state staff and external facilitators and through participating agency documentation. Proportion and duration scores for each stage and phase of the NIATx-SIC were calculated for each agency. Competency was assessed using the NIATx fidelity tool. Comparisons of proportion, duration, and NIATx activities completed were determined using independent sample t-tests by agency competency level.

Results

The NIATx-SIC distinguished between agencies achieving competency (n  =  23) and those not achieving competency (n  =  26). Agencies achieving competency completed a greater proportion of implementation phase activities and had a significantly longer Stage 7 duration. These agencies participated in significantly more individual and group coaching calls, attended more in-person meetings, implemented more change projects, and spent approximately 64 more days, on average, engaging in all NIATx activities.

Conclusions

Organizational participation in dissemination and implementation research requires a significant investment of staff resources. The inability of an organization to achieve competency when utilizing a set of implementation strategies waste an opportunity to institutionalize knowledge of how to apply implementation strategies to future change efforts. The NIATx-SIC provides evidence that competency is not an attribute of the organization but rather a result of the application of the NIATx implementation strategies to improve agency co-occurring capacity.

Trial Registration

ClinicalTrials.gov, NCT03007940. Registered January 2, 2017, https://clinicaltrials.gov/ct2/show/NCT03007940.

Understanding implementation completion of tailored motivational interviewing in multidisciplinary adolescent HIV clinics

Background

Understanding the barriers and facilitators of implementation completion is critical to determining why some implementation efforts fail and some succeed. Such studies provide the foundation for developing further strategies to support implementation completion when scaling up evidence-based practices (EBPs) such as Motivational Interviewing.

Method

This mixed-methods study utilized the Exploration, Preparation, Implementation, and Sustainment framework in an iterative analytic design to compare adolescent HIV clinics that demonstrated either high or low implementation completion in the context of a hybrid Type III trial of tailored motivational interviewing. Ten clinics were assigned to one of three completion categories (high, medium, and low) based on percentage of staff who adhered to three components of implementation strategies. Comparative analysis of staff qualitative interviews compared and contrasted the three high-completion clinics with the three low-completion clinics.

Results

Results suggested several factors that distinguished high-completion clinics compared to low-completion clinics including optimism, problem-solving barriers, leadership, and staff stress and turnover.

Conclusions

Implementation strategies targeting these factors can be added to EBP implementation packages to improve implementation success.

Plain Language Summary

While studies have begun to address adherence to intervention techniques, this is one of the first studies to address organizational adherence to implementation strategies. Youth HIV providers from different disciplines completed interviews about critical factors in both the inner and outer context that can support or hinder an organization's adherence to implementation strategies. Compared to less adherent clinics, more adherent clinics reported more optimism, problem-solving, and leadership strengths and less staff stress and turnover. Implementation strategies addressing these factors could be added to implementation packages to improve implementation success.

Evaluation of measures of sustainability and sustainability determinants for use in community, public health, and clinical settings: a systematic review

BACKGROUND

Sustainability is concerned with the long-term delivery and subsequent benefits of evidence-based interventions. To further this field, we require a strong understanding and thus measurement of sustainability and what impacts sustainability (i.e., sustainability determinants). This systematic review aimed to evaluate the quality and empirical application of measures of sustainability and sustainability determinants for use in clinical, public health, and community settings.

METHODS

Seven electronic databases, reference lists of relevant reviews, online repositories of implementation measures, and the grey literature were searched. Publications were included if they reported on the development, psychometric evaluation, or empirical use of a multi-item, quantitative measure of sustainability, or sustainability determinants. Eligibility was not restricted by language or date. Eligibility screening and data extraction were conducted independently by two members of the research team. Content coverage of each measure was assessed by mapping measure items to relevant constructs of sustainability and sustainability determinants. The pragmatic and psychometric properties of included measures was assessed using the Psychometric and Pragmatic Evidence Rating Scale (PAPERS). The empirical use of each measure was descriptively analyzed.

RESULTS

A total of 32,782 articles were screened from the database search, of which 37 were eligible. An additional 186 publications were identified from the grey literature search. The 223 included articles represented 28 individual measures, of which two assessed sustainability as an outcome, 25 covered sustainability determinants and one explicitly assessed both. The psychometric and pragmatic quality was variable, with PAPERS scores ranging from 14 to 35, out of a possible 56 points. The Provider Report of Sustainment Scale had the highest PAPERS score and measured sustainability as an outcome. The School-wide Universal Behaviour Sustainability Index-School Teams had the highest PAPERS score (score=29) of the measure of sustainability determinants.

CONCLUSIONS

This review can be used to guide selection of the most psychometrically robust, pragmatic, and relevant measure of sustainability and sustainability determinants. It also highlights that future research is needed to improve the psychometric and pragmatic quality of current measures in this field.

TRIAL REGISTRATION

This review was prospectively registered with Research Registry (reviewregistry1097), March 2021.

Economic evaluation of implementation science outcomes in low- and middle-income countries: a scoping review

BACKGROUND

Historically, the focus of cost-effectiveness analyses has been on the costs to operate and deliver interventions after their initial design and launch. The costs related to design and implementation of interventions have often been omitted. Ignoring these costs leads to an underestimation of the true price of interventions and biases economic analyses toward favoring new interventions. This is especially true in low- and middle-income countries (LMICs), where implementation may require substantial up-front investment. This scoping review was conducted to explore the topics, depth, and availability of scientific literature on integrating implementation science into economic evaluations of health interventions in LMICs.

METHODS

We searched Web of Science and PubMed for papers published between January 1, 2010, and December 31, 2021, that included components of both implementation science and economic evaluation. Studies from LMICs were prioritized for review, but papers from high-income countries were included if their methodology/findings were relevant to LMIC settings.

RESULTS

Six thousand nine hundred eighty-six studies were screened, of which 55 were included in full-text review and 23 selected for inclusion and data extraction. Most papers were theoretical, though some focused on a single disease or disease subset, including: mental health (n = 5), HIV (n = 3), tuberculosis (n = 3), and diabetes (n = 2). Manuscripts included a mix of methodology papers, empirical studies, and other (e.g., narrative) reviews. Authorship of the included literature was skewed toward high-income settings, with 22 of the 23 papers featuring first and senior authors from high-income countries. Of nine empirical studies included, no consistent implementation cost outcomes were measured, and only four could be mapped to an existing costing or implementation framework. There was also substantial heterogeneity across studies in how implementation costs were defined, and the methods used to collect them.

CONCLUSION

A sparse but growing literature explores the intersection of implementation science and economic evaluation. Key needs include more research in LMICs, greater consensus on the definition of implementation costs, standardized methods to collect such costs, and identifying outcomes of greatest relevance. Addressing these gaps will result in stronger links between implementation science and economic evaluation and will create more robust and accurate estimates of intervention costs.

TRIAL REGISTRATION

The protocol for this manuscript was published on the Open Science Framework. It is available at: https://osf.io/ms5fa/ (DOI: 10.17605/OSF.IO/32EPJ).

Stepped implementation-to-target: a study protocol of an adaptive trial to expand access to addiction medications

Background

In response to the US opioid epidemic, significant national campaigns have been launched to expand access to `opioid use disorder (MOUD). While adoption has increased in general medical care settings, specialty addiction programs have lagged in both reach and adoption. Elevating the quality of implementation strategy, research requires more precise methods in tailoring strategies rather than a one-size-fits-all-approach, documenting participant engagement and fidelity to the delivery of the strategy, and conducting an economic analysis to inform decision making and policy. Research has yet to incorporate all three of these recommendations to address the challenges of implementing and sustaining MOUD in specialty addiction programs.

Methods

This project seeks to recruit 72 specialty addiction programs in partnership with the Washington State Health Care Authority and employs a measurement-based stepped implementation-to-target approach within an adaptive trial design. Programs will be exposed to a sequence of implementation strategies of increasing intensity and cost: (1) enhanced monitoring and feedback (EMF), (2) 2-day workshop, and then, if outcome targets are not achieved, randomization to either internal facilitation or external facilitation. The study has three aims: (1) evaluate the sequential impact of implementation strategies on target outcomes, (2) examine contextual moderators and mediators of outcomes in response to the strategies, and (3) document and model costs per implementation strategy. Target outcomes are organized by the RE-AIM framework and the Addiction Care Cascade.

Discussion

This implementation project includes elements of a sequential multiple assignment randomized trial (SMART) design and a criterion-based design. An innovative and efficient approach, participating programs only receive the implementation strategies they need to achieve target outcomes. Findings have the potential to inform implementation research and provide key decision-makers with evidence on how to address the opioid epidemic at a systems level.

Trial registration

This trial was registered at ClinicalTrials.gov (NCT05343793) on April 25, 2022.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-022-01239-y.

An Initiative "that you do for one person": Identifying Barriers and Facilitators to Implementing an Immediate Postpartum LARC Initiative in Florida Hospitals

INTRODUCTION

In the United States (U.S.), perinatal quality improvement collaboratives have pursued implementing immediate postpartum long-acting reversible contraception (LARC) initiatives to increase people's access to contraception and support their fertility desires. This process evaluation aimed to identify barriers and facilitators to implementing an immediate postpartum LARC initiative in Florida.

METHODS

Data collection included in-depth qualitative assessments (i.e., interviews, small focus group discussions) with hospitals in pre- and early stages of the implementation process. Snowball sampling was used to recruit participants. Interviews were conducted in-person or via Zoom or phone and were audio-recorded and transcribed verbatim. Four of the five domains within the Consolidated Framework for Implementation Research (e.g., process, intervention characteristics, inner and outer settings) informed the study design and data collection/analysis.

RESULTS

Fourteen staff of diverse job roles from five hospitals participated. Factors that facilitated implementation were the strength of the evidence, relative advantage, internal and external networks, and engaging staff. Barriers to implementation included billing and reimbursement and needing significant support from external networks to progress through implementation phases.

DISCUSSION

Findings suggest that depending on the task or phase, multiple factors work in tandem to serve as implementation barriers and facilitators. Additionally, evaluating hospitals' progress at the pre- and early implementation phases was critical for quickly finding solutions and benefited other hospitals in different stages. As this initiative requires substantial support, health systems should create and sustain a culture of excellence and efficiency to facilitate implementing initiatives that improve care quality.

Michigan Model for HealthTM Learning to Enhance and Adapt for Prevention (Mi-LEAP): protocol of a pilot randomized trial comparing Enhanced Replicating Effective Programs versus standard implementation to deliver an evidence-based drug use prevention curriculum

Background

School-based drug use prevention programs have demonstrated notable potential to reduce the onset and escalation of drug use, including among youth at risk of poor outcomes such as those exposed to trauma. Researchers have found a robust relationship between intervention fidelity and participant (i.e., student) outcomes. Effective implementation of evidence-based interventions, such as the Michigan Model for Health^TM (MMH), is critical to achieving desired public health objectives. Yet, a persistent gap remains in what we know works and how to effectively translate these findings into routine practice. The objective of this study is to design and test a multi-component implementation strategy to tailor MMH to meet population needs (i.e., students exposed to trauma), and improve the population-context fit to enhance fidelity and effectiveness.

Methods

Using a 2-group, mixed-method randomized controlled trial design, this study will compare standard implementation versus Enhanced Replicating Effective Programs (REP) to deliver MMH. REP is a theoretically based implementation strategy that promotes evidence-based intervention (EBI) fidelity through a combination of EBI curriculum packaging, training, and as-needed technical assistance and is consistent with standard MMH implementation. Enhanced REP will tailor the intervention and training to integrate trauma-informed approaches and deploy customized implementation support (i.e., facilitation). The research will address the following specific aims: (1) design and test an implementation strategy (Enhanced REP) to deliver the MMH versus standard implementation and evaluate feasibility, acceptability, and appropriateness using mixed methods, (2) estimate the costs and cost-effectiveness of Enhanced REP to deliver MMH versus standard implementation.

Discussion

This research will design and test a multi-component implementation strategy focused on enhancing the fit between the intervention and population needs while maintaining fidelity to MMH core functions. We focus on the feasibility of deploying the implementation strategy bundle and costing methods and preliminary information on cost input distributions. The substantive focus on youth at heightened risk of drug use and its consequences due to trauma exposure is significant because of the public health impact of prevention. Pilot studies of implementation strategies are underutilized and can provide vital information on designing and testing effective strategies by addressing potential design and methods uncertainties and the effects of the implementation strategy on implementation and student outcomes.

Trial registration

NCT04752189—registered on 8 February 2021 on ClinicalTrials.gov PRS

Results of a multi-site pragmatic hybrid type 3 cluster randomized trial comparing level of facilitation while implementing an intervention in community-dwelling disabled and older adults in a Medicaid waiver

Background

Evidence-based interventions that optimize physical function for disabled and older adults living in the community who have difficulty with daily living tasks are available. However, uptake has been limited, particularly in resource-constrained (Medicaid) settings. Facilitation may be an effective implementation strategy. This study’s aim was to compare internal facilitation (IF) versus IF and external facilitation (EF) on adoption and sustainability of an intervention in a Medicaid home and community-based waiver.

Methods

In a hybrid type 3 trial, waiver sites (N = 18) were randomly assigned to implement the intervention using a bundle of strategies with either IF or IF and EF. Adoption and sustainability were assessed via Stages of Implementation Completion (SIC) for each site. Clinician attitudes toward evidence-based practice and self-efficacy were evaluated among 539 registered nurses, social workers, and occupational therapists. Medicaid beneficiary outcomes of activities of daily living, depression, pain, falls, emergency department visits, and hospitalizations were evaluated in a sample of N = 7030 as reflected by electronic health records data of the Medicaid waiver program. Linear mixed-effects models were used to compare outcomes between trial arms while accounting for cluster-randomized design.

Results

The mean SIC scores were 72.22 (standard deviation [SD] = 16.98) in the IF arm (9 sites) and 61.33 (SD = 19.29) in the IF + EF arm (9 sites). The difference was not statistically significant but corresponded to the medium clinically important effect size Cohen’s d = 0.60. Clinician implementation outcomes of attitudes and self-efficacy did not differ by trial arm. Beneficiary depression was reduced significantly in the IF + EF arm compared to the IF arm (p = .04, 95% confidence interval for the difference [0.01, 0.24]). No differences between trial arms were found for other beneficiary outcomes.

Conclusions

Level of facilitation did not enhance capacity for adoption and sustainability of an evidence-based intervention in a Medicaid setting that cares for disabled and older adults. Improved beneficiary depression favored use of IF and EF compared to IF alone, and no differences were found for other outcomes. These findings also suggest level of facilitation may not have impacted beneficiary outcomes.

Trial registration

ClinicalTrials.gov, NCT03634033; date registered August 16, 2018.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-022-01232-5.

How is leadership behavior associated with organization-related variables? Translation and psychometric evaluation of the implementation leadership scale in German primary healthcare

Background

The Implementation Leadership Scale (ILS) was developed to assess leadership behavior with regard to being proactive, knowledgeable, supportive, or perseverant in implementing evidence-based practices (EBPs). As part of a study on the implementation of a digitally supported polypharmacy management application in primary care, the original ILS was translated and validated for use in the German language.

Rationale

This study aimed to translate the original ILS into German and evaluate its psychometric properties.

Methods

The validation sample consisted of 198 primary care physicians in a cluster-randomized controlled trial in which the intervention group implemented a digitally supported clinical decision support system for polypharmacy management. The ILS was assessed using a 12-item scale. The study included a process evaluation with two evaluation waves between 2019 and 2021. The ILS was used within this process evaluation study to assess the leadership support with regard to the implementation of the polypharmacy management. The ILS was translated in a multi-step process, including pre-testing of the instrument and triple, back-and-forth translation of the instrument. We tested the reliability (Cronbach’s alpha) and validity (construct and criterion-related validity) of the scale.

Results

The four-dimensional structure of the instrument was confirmed (comparative fit index = .97; root mean square error of approximation = .06). Convergent validity was demonstrated by correlations with organizational innovation climate, social capital, and workload, which was consistent with the proposed hypothesis. Criterion-related validity of the ILS was demonstrated by predicting the organizational readiness for change scores using structural equation modeling. The reliability of the scale was good (α = .875).

Conclusion

The German version of the ILS created in this study is a reliable and valid measure. The original four-dimensional structure of the ILS was confirmed in a primary care setting. Further psychometric testing is needed to establish the validity and reliability of the ILS and to transfer it to other health care settings. It is a useful tool for identifying the areas for implementation leadership development. Further research is needed on how, why, and when distinct types of leadership behaviors have different effects on healthcare organizations in implementation processes.

Using Novel Implementation Tools for Evidence-based Intervention Delivery (UNITED) across public service systems for three evidence-based autism interventions in under-resourced communities: study protocol

BACKGROUND

There are a growing number of evidence-based interventions (EBIs) for autistic individuals, but few are successfully implemented with fidelity in under-resourced communities and with families from traditionally disenfranchised groups. Implementation science offers tools to increase EBI use in communities, but most implementation strategies are designed specific to a single EBI. It is not feasible to develop a new implementation strategy each time a new EBI is introduced in the community. Therefore, to test the effectiveness and generalizability of implementation strategies we are developing and testing a multifaceted implementation strategy with three EBIs concurrently. The goal of this protocol paper is to describe the randomized field trial of an implementation strategy for use across autism EBIs, diverse settings and participants, with the goal of increasing rapid uptake of effective practices to reach our most vulnerable children.

METHODS

We developed a multifaceted implementation strategy called Using Novel Implementation Tools for Evidence-based intervention Delivery (UNITED) to facilitate the implementation and sustainment of three EBIs in under-resourced settings. We will compare fidelity to, and effectiveness of, each intervention [Mind the Gap (MTG), Remaking Recess (RR), Self-Determined Learning Model of Instruction (SDLMI)] with and without UNITED in a randomized field trial. Randomization will be stratified using a minimization allocation method. We will train community practitioners using remote delivery of modules specific to the intervention, and active coaching via Zoom for at least 6 sessions and up to 12 as dictated by each EBI. Our primary outcome is fidelity to each EBI, and our secondary outcome is at the child or family level (family empowerment for MTG, child peer social engagement for RR, and adolescent self-determination for SDLMI, respectively). We will measure progress through the implementation phases using the Stages of Implementation Completion and cost-effectiveness of UNITED.

DISCUSSION

The results of this study will provide rigorous data on the effectiveness and generalizability of one relatively light-touch implementation strategy in increasing use of autism EBIs and associated outcomes in diverse under resourced public service settings for underrepresented autistic youth.

TRIAL REGISTRATION

Mind the Gap: Clinicaltrials.gov Identifier:  NCT04972825 (Date registered July 22, 2021); Remaking Recess: Clinicaltrials.gov Identifier:  NCT04972838 (Date registered July 22, 2021); Self-Determined Learning Model of Instruction: Clinicaltrials.gov Identifier:  NCT04972851 (Date registered July 22, 2021).

Barriers and facilitators to implementation of non-medical independent prescribing in primary care in the UK: a qualitative systematic review

OBJECTIVES

To support workforce deficits and rising demand for medicines, independent prescribing (IP) by nurses, pharmacists and allied health professionals is a key component of workforce transformation in UK healthcare. This systematic review of qualitative research studies used a thematic synthesis approach to explore stakeholders' views on IP in primary care and identify barriers and facilitators influencing implementation.

SETTING

UK primary/community care.

PARTICIPANTS

Inclusion criteria were UK qualitative studies of any design, published in the English language. Six electronic databases were searched between January 2010 and September 2021, supplemented by reference list searching. Papers were screened, selected and quality-appraised using the Quality Assessment Tool for Studies with Diverse Designs. Study data were extracted to a bespoke table and two reviewers used NVivo software to code study findings. An inductive thematic synthesis was undertaken to identify descriptive themes and interpret these into higher order analytical themes. The Diffusion of Innovations and Consolidated Framework for Implementation Research were guiding theoretical anchors.

PRIMARY AND SECONDARY OUTCOME MEASURES

N/A.

RESULTS

Twenty-three articles addressing nurse, pharmacist and physiotherapist IP were included. Synthesis identified barriers and facilitators in four key stages of implementation: (1) 'Preparation', (2) 'Training', (3) 'Transition' and 4) 'Sustainment'. Enhancement, substitution and role-specific implementation models reflected three main ways that the IP role was used in primary care.

CONCLUSIONS

In order to address global deficits, there is increasing need to optimise use of IP capability. Although the number of independent prescribers continues to grow, numerous barriers to implementation persist. A more coordinated and targeted approach is key to overcoming barriers identified in the four stages of implementation and would help ensure that IP is recognised as an effective approach to help alleviate workforce shortfalls in the UK, and around the world.

PROSPERO REGISTRATION NUMBER

CRD42019124400.

Using economic evaluations in implementation science to increase transparency in costs and outcomes for organizational decision-makers

Background

Economic evaluations frequently are utilized to compare the value of different interventions in medicine and health in concrete terms. Implementation science also would benefit from the incorporation of economic evaluations, but such studies are rare in the literature. The National Cancer Institute has supported a special collection of articles focusing on economic evaluations in implementation science. Even when interventions are supported by substantial evidence, they are implemented infrequently in the field. Implementation costs are important determinants for whether organizational decision-makers choose to adopt an intervention and whether the implementation process is successful. Economic evaluations, such as cost-effectiveness analyses, can help organizational decision-makers choose between implementation approaches for evidence-based interventions by accounting for costs and succinctly presenting cost/benefit tradeoffs.

Main text

This manuscript presents a discussion of important considerations for incorporating economic evaluations into implementation science. First, the distinction between intervention and implementation costs is presented, along with an explanation of why the comprehensive representation of implementation costs is elusive. Then, the manuscript describes how economic evaluations in implementation science may differ from those in medicine and health intervention studies, especially in terms of determining the perspectives and outcomes of interest. Finally, referencing a scale-up trial of an evidence-based behavioral health intervention, concrete case examples of how cost data can be collected and used in economic evaluations targeting implementation, rather than clinical outcomes, are described.

Conclusions

By gaining a greater understanding of the costs and economic impact associated with different implementation approaches, organizational decision-makers will have better transparency for future replication and scale-up. The use of economic evaluations can help to advance this understanding and provide researchers, purveyors or third-party intermediaries, and organizational decision-makers with essential information to facilitate implementation.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00295-1.

Sustainment of Integrated Care in Addiction Treatment Settings: Primary Outcomes From a Cluster-Randomized Controlled Trial

OBJECTIVE

Integrated treatment services are the gold standard for addressing co-occurring mental and substance use disorders, yet they are not readily available. The Network for the Improvement of Addiction Treatment (NIATx) was hypothesized to be an effective strategy to implement and sustain integrated mental health and substance use care in addiction treatment programs. This study examined sustainment of integrated services for up to 2 years after the active implementation phase.

METHODS

The effectiveness of NIATx strategies to implement and sustain integrated services was evaluated by using a cluster-randomized, waitlist control group design. Forty-nine addiction treatment organizations were randomly assigned to either NIATx1 (active implementation strategy) or NIATx2 (waitlist control). The Dual Diagnosis Capability in Addiction Treatment Index was used to evaluate organizations' capability to provide integrated care. The NIATx Stages of Implementation Completion scale was used to assess participation in and adherence to the NIATx implementation process. Linear mixed-effects modeling was used to evaluate changes from baseline to end of the sustainment period.

RESULTS

Both cohorts sustained their capability to provide integrated treatment services. Both groups achieved successful implementation and sustained integrated services to a similar degree, regardless of sustainment year. Sustainment did not vary as a function of NIATx adherence.

CONCLUSIONS

The delivery of integrated treatment services was sustained for 2 years after receipt of active implementation support. Future research should consider how contextual factors may predict, mediate, and moderate sustainment outcomes.

Six practical recommendations for improved implementation outcomes reporting

BACKGROUND

Implementation outcomes research spans an exciting mix of fields, disciplines, and geographical space. Although the number of studies that cite the 2011 taxonomy has expanded considerably, the problem of harmony in describing outcomes persists. This paper revisits that problem by focusing on the clarity of reporting outcomes in studies that examine them. Published recommendations for improved reporting and specification have proven to be an important step in enhancing the rigor of implementation research. We articulate reporting problems in the current implementation outcomes literature and describe six practical recommendations that address them.

RECOMMENDATIONS

Our first recommendation is to clearly state each implementation outcome and provide a definition that the study will consistently use. This includes providing an explanation if using the taxonomy in a new way or merging terms. Our second recommendation is to specify how each implementation outcome will be analyzed relative to other constructs. Our third recommendation is to specify "the thing" that each implementation outcome will be measured in relation to. This is especially important if you are concurrently studying interventions and strategies, or if you are studying interventions and strategies that have multiple components. Our fourth recommendation is to report who will provide data and the level at which data will be collected for each implementation outcome, and to report what kind of data will be collected and used to assess each implementation outcome. Our fifth recommendation is to state the number of time points and frequency at which each outcome will be measured. Our sixth recommendation is to state the unit of observation and the level of analysis for each implementation outcome.

CONCLUSION

This paper advances implementation outcomes research in two ways. First, we illustrate elements of the 2011 research agenda with concrete examples drawn from a wide swath of current literature. Second, we provide six pragmatic recommendations for improved reporting. These recommendations are accompanied by an audit worksheet and a list of exemplar articles that researchers can use when designing, conducting, and assessing implementation outcomes studies.

Rates of sustainment in the Universal Stages of Implementation Completion

BACKGROUND

Sustainment is a desirable outcome of implementation, but its precise definition remains unclear, contributing to the difficulty of identifying a generalized rate of sustainment. Several studies and reviews on the topic differ on both definition and levels of analysis. Furthermore, methodological limitations might have influenced the results, including the unknown quality with which some interventions were delivered. The Universal Stages of Implementation Completion (UniSIC) is a standardized measurement tool that tracks the implementation process and milestone completion across a wide range of real-world implementations-this provides a unique opportunity to identify a generalized rate of sustainment.

METHODS

UniSIC data was captured from the SIC website on 27 September 2020 and included data from all sites (n = 1778) that had been tracked to date. Data were restricted to sites that achieved competency in program delivery, and thus had a newly adopted program worthy of sustainment. Dates and indicator variables of implementation activities were combined to form two alternate definitions of sustainment: sustained (start-up) was achieved if sites continued to deliver services 2 years past their program start-up date; sustained (competent) was achieved if sites continued to deliver services 2 years past their competence and/or certification date. Of sites eligible for inclusion based on these definitions (N = 208), descriptive analyses were conducted to determine a rate of sustainment for all programs that successfully started a program. These definitions were also applied to a combined sample for a general rate of sustainment among all sites. Rates of competency among both a sample of sites that started up and a combined sample were also identified.

RESULTS

The rate of competence was 58.5% and the rate of sustained (start-up) was 37.1%, while the rate of sustained (competent) was 25.1%. The rates of competence and sustainment among the combined samples were far lower: 15.6% for competence, 6.8% for sustained (start-up), and 4.4% for sustained (competent).

CONCLUSIONS

These identified rates of sustainment are accurate initial estimates of sustainment of community-based practices, or in general. Future research on rates of sustainment should carefully define measures of sustainment and be transparent about the real-world conditions on which analyses are centered.

Fidelity and the impact of patient safety huddles on teamwork and safety culture: an evaluation of the Huddle Up for Safer Healthcare (HUSH) project

Background

The Patient Safety Huddle (PSH) is a brief multidisciplinary daily meeting held to discuss threats to patient safety and actions to mitigate risk. Despite growing interest and application of huddles as a mechanism for improving safety, evidence of their impact remains limited. There is also variation in how huddles are conceived and implemented with insufficient focus on their fidelity (the extent to which delivered as planned) and potential ways in which they might influence outcomes. The Huddle Up for Safer Healthcare (HUSH) project attempted to scale up the implementation of patient safety huddles (PSHs) in five hospitals – 92 wards - across three UK NHS Trusts. This paper aims to assess their fidelity, time to embed, and impact on teamwork and safety culture.

Methods

A multi-method Developmental Evaluation was conducted. The Stages of Implementation Checklist (SIC) was used to determine time taken to embed PSHs. Observations were used to check embedded status and fidelity of PSH. A Teamwork and Safety Climate survey (TSC) was administered at two time-points: pre- and post-embedding. Changes in TSC scores were calculated for Trusts, job role and clinical speciality.

Results

Observations confirmed PSHs were embedded in 64 wards. Mean fidelity score was 4.9/9. PSHs frequently demonstrated a ‘fear free’ space while Statistical Process Control charts and historical harms were routinely omitted. Analysis showed a positive change for the majority (26/27) of TSC questions and the overall safety grade of the ward.

Conclusions

PSHs are feasible and effective for improving teamwork and safety culture, especially for nurses. PSH fidelity criteria may need adjusting to include factors deemed most useful by frontline staff. Future work should examine inter-disciplinary and role-based differences in TSC outcomes.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-07080-1.

The multiphase optimization strategy (MOST) in child maltreatment prevention research

Each year hundreds of thousands of children and families receive behavioral interventions designed to prevent child maltreatment; yet rates of maltreatment have not declined in over a decade. To reduce the prevalence and prevent the life-long negative consequences of child maltreatment, behavioral interventions must not only be effective, but also affordable, scalable, and efficient to meet the demand for these services. An innovative approach to intervention science is needed. The purpose of this article is to introduce the multiphase optimization strategy (MOST) to the field of child maltreatment prevention. MOST is an engineering-inspired framework for developing, optimizing, and evaluating multicomponent behavioral interventions. MOST enables intervention scientists to empirically examine the performance of each intervention component, independently and in combination. Using a hypothetical example of a home visiting intervention and artificial data, this article demonstrates how MOST may be used to optimize the content of a parent-focused in-home intervention and the engagement strategies of an intervention to increase completion rate to identify an intervention that is effective, efficient, economical, and scalable. We suggest that MOST will ultimately improve prevention science and hasten the progress of translational science to prevent child maltreatment.

A Technology-Based Training Tool for a Health Promotion and Sex Education Program for Justice-Involved Youth: Development and Usability Study

Background

Justice-involved youth are especially vulnerable to mental health distress, substance misuse, and risky sexual activity, amplifying the need for evidence-based programs (EBPs). Yet, uptake of EBPs in the justice system is challenging because staff training is costly in time and effort. Hence, justice-involved youth experience increasing health disparities despite the availability of EBPs.

Objective

To counter these challenges, this study develops and pilot-tests a prototype of a technology-based training tool that teaches juvenile justice staff to deliver a uniquely tailored EBP for justice-involved youth—PHAT (Preventing HIV/AIDS Among Teens) Life. PHAT Life is a comprehensive sex education, mental health, and substance use EBP collaboratively designed and tested with guidance from key stakeholders and community members. The training tool addresses implementation barriers that impede uptake and sustainment of EBPs, including staff training and support and implementation costs.

Methods

Staff (n=11) from two juvenile justice settings pilot-tested the technology-based training tool, which included five modules. Participants completed measures of HIV and sexually transmitted infection (STI) knowledge, sex education confidence, and implementation outcomes such as training satisfaction, adoption, implementation, acceptability, appropriateness, and sustainability. PHAT Life trainers assessed fidelity through two activity role plays participants submitted upon completing the training modules.

Results

Participants demonstrated increases in HIV and STI knowledge (t₁₀=3.07; P=.01), and were very satisfied (mean 4.42, SD 0.36) with the training tool and the PHAT Life curriculum. They believed that the training tool and curriculum could be adopted, implemented, and sustained within their settings as an appropriate and acceptable intervention and training.

Conclusions

Overall, the results from this pilot test demonstrate feasibility and support continuing efforts toward completing the training tool and evaluating it within a fully powered randomized controlled trial. Ultimately, this study will provide a scalable option for disseminating an EBP and offers a more cost-effective and sustainable way to train staff in an EBP.

Alliances to disseminate addiction prevention and treatment (ADAPT): A statewide learning health system to reduce substance use among justice-involved youth in rural communities

BACKGROUND

Youth in the justice system (YJS) are more likely than youth who have never been arrested to have mental health and substance use problems. However, a low percentage of YJS receive SUD services during their justice system involvement. The SUD care cascade can identify potential missed opportunities for treatment for YJS. Steps along the continuum of the cascade include identification of treatment need, referral to services, and treatment engagement. To address gaps in care for YJS, we will (1) implement a learning health system (LHS) to develop, or improve upon, alliances between juvenile justice (JJ) agencies and community mental health centers (CMHC) and (2) present local cascade data during continuous quality improvement cycles within the LHS alliances.

METHODS/DESIGN

ADAPT is a hybrid Type II effectiveness implementation trial. We will collaborate with JJ and CMHCs in eight Indiana counties. Application of the EPIS (exploration, preparation, implementation, and sustainment) framework will guide the implementation of the LHS alliances. The study team will review local cascade data quarterly with the alliances to identify gaps along the continuum. The study will collect self-report survey measures longitudinally at each site regarding readiness for change, implementation climate, organizational leadership, and program sustainability. The study will use the Stages of Implementation Completion (SIC) tool to assess the process of implementation across interventions. Additionally, the study team will conduct focus groups and qualitative interviews with JJ and CMHC personnel across the intervention period to assess for impact.

DISCUSSION

Findings have the potential to increase SUD need identification, referral to services, and treatment for YJS.

Efficacy of active and passive evidence-based practice training for postgraduate medical residents: a non-randomized controlled trial

Objective

This study examined the effects of two evidence-based practice (EBP) educational programs for postgraduate medical residents on their attitude, behavior, knowledge, outcome, and competencies in EBP.

Results

Forty-five and thirty-five medical residents were recruited in the active and passive educational intervention groups, respectively. Among those, 39 and 30 participants were included in the final analysis. The participants of the active group received 12 h of EBP-structured presentation. The passive educational group received EBP education through their daily rounds, evidence-based journal clubs, and morning reports. Participants were evaluated with EBP-KABQ and ACE tools questionnaires. The active and passive intervention groups were not significantly different from each other at the baseline in the EBP-KABQ questionnaire and ACE tools score (p > 0.05). However, most questions in the EBP-KABQ questionnaire were significantly different from the pre-intervention measurement and the passive intervention group after the educational intervention. Educational intervention in both groups led to a significant difference in ACE tools score between groups (8.86 ± 2.62 vs. 7.31 ± 2.92, p = 0.029, in the active and passive groups, respectively). Paired t-test analysis revealed that our intervention led to a significant increase in ACE tool scores in both groups (p < 0.000, in both groups).

Supplementary Information

The online version contains supplementary material available at 10.1186/s13104-021-05732-3.

Mixed-methods economic evaluation of the implementation of tobacco treatment programs in National Cancer Institute-designated cancer centers

Background

The Cancer Center Cessation Initiative (C3I) was launched in 2017 as a part of the NCI Cancer Moonshot program to assist NCI-designated cancer centers in developing tobacco treatment programs for oncology patients. Participating centers have implemented varied evidence-based programs that fit their institutional resources and needs, offering a wide range of services including in-person and telephone-based counseling, point of care, interactive voice response systems, referral to the quitline, text- and web-based services, and medications.

Methods

We used a mixed methods comparative case study design to evaluate system-level implementation costs across 15 C3I-funded cancer centers that reported for at least one 6-month period between July 2018 and June 2020. We analyzed operating costs by resource category (e.g., personnel, medications) concurrently with transcripts from semi-structured key-informant interviews conducted during site visits. Personnel salary costs were estimated using Bureau of Labor Statistics wage data adjusted for area and occupation, and non-wage benefits. Qualitative findings provided additional information on intangible resources and contextual factors related to implementation costs.

Results

Median total monthly operating costs across funded centers were $11,045 (range: $5129–$20,751). The largest median operating cost category was personnel ($10,307; range: $4122–$19,794), with the highest personnel costs attributable to the provision of in-person program services. Monthly (non-zero) cost ranges for other categories were medications ($17–$573), materials ($6–$435), training ($96–$516), technology ($171–$2759), and equipment ($10–$620). Median cost-per-participant was $466 (range: $70–$2093) and cost-per-quit was $2688 (range: $330–$9628), with sites offering different combinations of program components, ranging from individually-delivered in-person counseling only to one program that offered all components. Site interviews provided context for understanding variations in program components and their cost implications.

Conclusions

Among most centers that have progressed in tobacco treatment program implementation, cost-per-quit was modest relative to other prevention interventions. Although select centers have achieved similar average costs by offering program components of various levels of intensity, they have varied widely in program reach and effectiveness. Evaluating implementation costs of such programs alongside reach and effectiveness is necessary to provide decision makers in oncology settings with the important additional information needed to optimize resource allocation when establishing tobacco treatment programs.

Cost analysis in implementation studies of evidence-based practices for mental health and substance use disorders: a systematic review

Background

This study is a systematic literature review of cost analyses conducted within implementation studies on behavioral health services. Cost analysis of implementing evidence-based practices (EBP) has become important within implementation science and is critical for bridging the research to practice gap to improve access to quality healthcare services. Costing studies in this area are rare but necessary since cost can be a barrier to implementation and sustainment of EBP.

Methods

We followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology and applied the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist. Key search terms included: (1) economics, (2) implementation, (3) EBP, and (4) behavioral health. Terms were searched within article title and abstracts in: EconLit, SocINDEX, Medline, and PsychINFO. A total of 464 abstracts were screened independently by two authors and reduced to 37 articles using inclusion and exclusion criteria. After a full-text review, 18 articles were included.

Results

Findings were used to classify costs into direct implementation, direct services, and indirect implementation. While all studies included phases of implementation as part of their design, only five studies examined resources across multiple phases of an implementation framework. Most studies reported direct service costs associated with adopting a new practice, usually summarized as total EBP cost, cost per client, cost per clinician, and/or cost per agency. For studies with detailed analysis, there were eleven direct cost categories represented. For five studies that reported costs per child served, direct implementation costs varied from $886 to $9470 per child, while indirect implementation costs ranged from $897 to $3805 per child.

Conclusions

This is the first systematic literature review to examine costs of implementing EBP in behavioral healthcare settings. Since 2000, 18 studies were identified that included a cost analysis. Given a wide variation in the study designs and economic methods, comparison across studies was challenging, which is a major limitation in the field, as it becomes difficult to replicate studies or to estimate future costs to inform policy decisions related to budgeting. We recommend future economic implementation studies to consider standard economic costing methods capturing costs across implementation framework phases to support comparisons and replicability.

Reducing Opioid Mortality in Illinois (ROMI): A case management/peer recovery coaching critical time intervention clinical trial protocol

Individuals with a history of opioid use are disproportionately represented in Illinois jails and prisons and face high risks of overdose and relapse at community reentry. Case management and peer recovery coaching are established interventions that may be leveraged to improve linkage to substance use treatment and supportive services during these critical periods of transition. We present the protocol for the Reducing Opioid Mortality in Illinois (ROMI), a type I hybrid effectiveness-implementation randomized trial of a case management, peer recovery coaching and overdose education and naloxone distribution (CM/PRC + OEND) critical time intervention (CTI) compared to OEND alone. The CM/PRC + OEND is a novel, 12-month intervention that involves linkage to substance use treatment and support for continuity of care, skills building, and navigation and engagement of social services that will be implemented using a hub-and-spoke model of training and supervision across the study sites. At least 1000 individuals released from jails and prisons spanning urban and rural settings will be enrolled. The primary outcome is engagement in medication for opioid use disorder. Secondary outcomes include health insurance enrollment, mental health service engagement, and re-arrest/recidivism, parole violation, and/or reincarceration. Mixed methods will be used to evaluate process and implementation outcomes including fidelity to, barriers to, facilitators of, and cost of the intervention. Videoconferencing and other remote processes will be leveraged to modify the protocol for safety during the COVID-19 pandemic. Results of the study may improve outcomes for vulnerable persons at the margin of behavioral health and the criminal legal system.

Bringing Evidence-Based Interventions into the Schools: An Examination of Organizational Factors and Implementation Outcomes

Although schools are one of the largest providers of behavioral health services for youth, many barriers exist to the implementation of evidence-based interventions in schools. This study used the Stages of Implementation Completion (SIC) to examine school-based implementation outcomes for a computer-assisted cognitive behavioral therapy intervention for anxious youth. Organizational factors and predictors of program startup also were examined. Results indicated that the SIC detected implementation variability in schools and suggested that spending more time completing pre-implementation activities may better prepare schools for active implementation of program delivery. Furthermore, proficiency emerged as a potentially important organizational factor to examine in future school-based implementation research.

Adapting the Comprehensive Unit Safety Program (CUSP) implementation strategy to increase delivery of evidence-based cardiovascular risk factor care in community mental health organizations: protocol for a pilot study

BACKGROUND

People with serious mental illnesses (SMI) such as schizophrenia and bipolar disorder experience excess mortality driven in large part by high rates of poorly controlled and under-treated cardiovascular risk factors. In the USA, integrated "behavioral health home" models in which specialty mental health organizations coordinate and manage physical health care for people with SMI are designed to improve guideline-concordant cardiovascular care for this group. Such models have been shown to improve cardiovascular care for clients with SMI in randomized clinical trials, but real-world implementation has fallen short. Key implementation barriers include lack of alignment of specialty mental health program culture and physical health care coordination and management for clients with SMI and lack of structured protocols for conducting effective physical health care coordination and management in the specialty mental health program context. This protocol describes a pilot study of an implementation intervention designed to overcome these barriers.

METHODS

This pilot study uses a single-group, pre/post-study design to examine the effects of an adapted Comprehensive Unit Safety Program (CUSP) implementation strategy designed to support behavioral health home programs in conducting effective cardiovascular care coordination and management for clients with SMI. The CUSP strategy, which was originally designed to improve inpatient safety, includes provider training, expert facilitation, and implementation of a five-step quality improvement process. We will examine the acceptability, appropriateness, and feasibility of the implementation strategy and how this strategy influences mental health organization culture; specialty mental health providers' self-efficacy to conduct evidence-based cardiovascular care coordination and management; and receipt of guideline-concordant care for hypertension, dyslipidemia, and diabetes mellitus among people with SMI.

DISCUSSION

While we apply CUSP to the implementation of evidence-based hypertension, dyslipidemia, and diabetes care, this implementation strategy could be used in the future to support the delivery of other types of evidence-based care, such as smoking cessation treatment, in behavioral health home programs. CUSP is designed to be fully integrated into organizations, sustained indefinitely, and used to continually improve evidence-based practice delivery.

TRIAL REGISTRATION

ClinicalTrials.gov, NCT04696653 . Registered on January 6, 2021.

Stepped-wedge randomized controlled trial of a novel opioid court to improve identification of need and linkage to medications for opioid use disorder treatment for court-involved adults

In response to the opioid crisis in New York State (NYS), the Unified Court System developed a new treatment court model-the opioid intervention court-designed around 10 Essential Elements of practice to address the flaws of existing drug courts in handling those with opioid addiction via broader inclusion criteria, rapid screening, and linkage to medications to treat opioid use disorder (MOUD). The new court model is now being rolled out statewide yet, given the innovation of the opioid court, the exact barriers to implementation in different counties with a range of resources are largely unknown. We describe a study protocol for the development and efficacy-test of a new implementation intervention (Opioid Court REACH; Research on Evidence-Based Approaches to Court Health) that will allow the opioid court, as framed by the 10 Essential Elements, to be scaled-up across 10 counties in NYS. Using a cluster-randomized stepped-wedge type-2 hybrid effectiveness-implementation design, we will test: (a) the implementation impact of Opioid Court REACH in improving implementation outcomes along the opioid cascade of care (screening, referral, treatment enrollment, MOUD initiation), and (b) the clinical and cost effectiveness of Opioid Court REACH in improving public health (treatment retention/court graduation) and public safety (recidivism) outcomes. Opioid Court REACH has the potential to improve management of individuals with opioid addiction in the court system via widespread scale-up of the opioid court model across the U.S., should this study find it to be effective.

Improving Medication Access within Integrated Treatment for Individuals with Co-Occurring Disorders in Substance Use Treatment Agencies

BACKGROUND

The best approach to provide comprehensive care for individuals with co-occurring disorders (CODs) related to substance use and mental health is to address both disorders through an integrated treatment approach. However, only 25% of behavioral health agencies offer integrated care and less than 7% of individuals who need integrated treatment receive it. A project used a cluster-randomized waitlist control group design to evaluate the effectiveness of Network for the Improvement of Addiction Treatment (NIATx) implementation strategies to improve access to addiction and psychotropic medications.

METHODS

This study represents a secondary analysis of data from the NIATx project. Forty-nine agencies were randomized to Cohort1 (active implementation group, receiving the NIATx strategy [n=25]) or Cohort2 (waitlist control group [n=24]). Data were collected at three time points (Baseline, Year1 and Year2). A two-level (patient within agency) multinomial logistic regression model investigated the effects of implementation strategy condition on one of four medication outcomes: both medication types, only psychotropic medication, only addiction medication, or neither medication type. A per-protocol analysis included time, NIATx fidelity, and agency focus as predictors.

RESULTS

The intent-to-treat analysis found a statistically significant change in access to addiction versus neither medication, but Cohort1 compared to Cohort2 at Year1 showed no differences. Changes were associated with the experimental intervention and occurred in the transition from Year 1 to Year 2, where greater increases were seen for agencies in Cohort2 versus Cohort1. The per-protocol analysis showed increased access to both medications and addiction medications from pre- to post-intervention for agencies in both cohorts; however, differences in change between high- and low-implementation agencies were not significant.

CONCLUSIONS

Access to integrated services for people with CODs is a long-standing problem. NIATx implementation strategies had limited effectiveness in improving medication access for individuals with CODs. Implementation strategy adherence is associated with increased medication access.

Cost-effectiveness of the Adaptive Implementation of Effective Programs Trial (ADEPT): approaches to adopting implementation strategies

BACKGROUND

Theory-based methods to support the uptake of evidence-based practices (EBPs) are critical to improving mental health outcomes. Implementation strategy costs can be substantial, and few have been rigorously evaluated. The purpose of this study is to conduct a cost-effectiveness analysis to identify the most cost-effective approach to deploying implementation strategies to enhance the uptake of Life Goals, a mental health EBP.

METHODS

We used data from a previously conducted randomized trial to compare the cost-effectiveness of Replicating Effective Programs (REP) combined with external and/or internal facilitation among sites non-responsive to REP. REP is a low-level strategy that includes EBP packaging, training, and technical assistance. External facilitation (EF) involves external expert support, and internal facilitation (IF) augments EF with protected time for internal staff to support EBP implementation. We developed a decision tree to assess 1-year costs and outcomes for four implementation strategies: (1) REP only, (2) REP+EF, (3) REP+EF add IF if needed, (4) REP+EF/IF. The analysis used a 1-year time horizon and assumed a health payer perspective. Our outcome was quality-adjusted life years (QALYs). The economic outcome was the incremental cost-effectiveness ratio (ICER). We conducted deterministic and probabilistic sensitivity analysis (PSA).

RESULTS

Our results indicate that REP+EF add IF is the most cost-effective option with an ICER of $593/QALY. The REP+EF/IF and REP+EF only conditions are dominated (i.e., more expensive and less effective than comparators). One-way sensitivity analyses indicate that results are sensitive to utilities for REP+EF and REP+EF add IF. The PSA results indicate that REP+EF, add IF is the optimal strategy in 30% of iterations at the threshold of $100,000/QALY.

CONCLUSIONS

Our results suggest that the most cost-effective implementation support begins with a less intensive, less costly strategy initially and increases as needed to enhance EBP uptake. Using this approach, implementation support resources can be judiciously allocated to those clinics that would most benefit. Our results were not robust to changes in the utility measure. Research is needed that incorporates robust and relevant utilities in implementation studies to determine the most cost-effective strategies. This study advances economic evaluation of implementation by assessing costs and utilities across multiple implementation strategy combinations.

TRIAL REGISTRATION

ClinicalTrials.gov Identifier: NCT02151331 , 05/30/2014.

A Longitudinal Mixed Method Approach for Assessing Implementation Context and Process Factors: Comparison of Three Sites from a Housing First Implementation Strategy Pilot

Background

Implementation science's focus on establishing implementation strategy effectiveness has overshadowed the need to understand differential performance of such strategies under various conditions. Methods allowing for assessment between implementation context and process can help address this gap. This paper provides a detailed description of a mixed method procedure for assessing factors related to the implementation context and process intersection, which was developed as part of the pilot study of the Housing First Technical Assistance and Training (HFTAT) Program, a multifaceted strategy designed to support Housing First model implementation.

Methods

The HFTAT was pilot tested among a sample of three organizations. Our mixed method approach combines two tools often used in implementation research-the Stages of Implementation Completion and the Consolidated Framework for Implementation Research-in a novel way. Several stages to analysis were completed, starting with a separate analysis of data pertaining to each measure and then two levels of mixed method analysis.

Results

The approach provided a better understanding of the issues that impacted the implementation guided by the HFTAT, suggesting: (1) individual determinants seemed to have a bigger impact based on the number of SIC phases they affected, (2) implementation context and process were connected through climate-related factors in the inner setting that made the sites more or less responsive to addressing identified barriers, and (3) there is a need to better assess context factors to identify areas where implementation drivers should be better targeted to facilitate change, and this is supported by prior research.

Conclusions

Understanding the underlying factors impacting a setting's performance related to a specific implementation strategy has potential to improve decision-making and optimize future implementation efforts. The approach likely be as successful combining the SIC with other determinant frameworks and should be utilized at the onset of an implementation project to maximize its usefulness.

Use of implementation science in tobacco control intervention studies in the USA between 2000 and 2020: a scoping review protocol

INTRODUCTION

Despite continuing efforts to reduce tobacco use in the USA, decline in smoking rates have stalled and smoking remains a major contributor to preventable death. Implementation science could potentially improve uptake and impact of evidence-based tobacco control interventions; however, no previous studies have systematically examined how implementation science has been used in this field. Our scoping review will describe the use of implementation science in tobacco control in the USA, identify relevant gaps in research and suggest future directions for implementation science application to tobacco control.

METHODS AND ANALYSIS

Our team, including a medical research librarian, will conduct a scoping review guided primarily by Arksey and O'Malley's methodology. We will search English language peer-reviewed literature published from 2000 to 31 December 2020 for terms synonymous with 'tobacco use', 'prevention', 'cessation' and 'implementation science'. The databases included in this search are MEDLINE (PubMed), Embase (Ovid), CINAHL (EBSCOhost), PsycINFO (ProQuest), ERIC (ProQuest) and the Cochrane Library (Wiley). We will include cohort and quasi-experimental studies, single-group experiments and randomised trials that report qualitative and/or quantitative data related to applying implementation science to the planning and/or delivery of interventions to prevent or decrease the use of tobacco products. Studies must target potential or active tobacco users, intervention providers such as educators or healthcare professionals, or US policy-makers. A minimum of two reviewers will independently examine each title and abstract for relevance, and each eligible full text for inclusion and analysis. Use of implementation science, demonstrated by explicit reference to implementation frameworks, strategies or outcomes, will be extracted from included studies and summarised.

ETHICS AND DISSEMINATION

This study is exempt from ethics board approval. We will document the equity-orientation of included studies with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses-Equity Extension checklist. Results will be submitted for conferences and peer-reviewed journals.

TRIAL REGISTRATION NUMBER

Open Science Framework Registry (6YRK8).

Aligning dissemination and implementation science with health policies to improve children's mental health

The prevalence of mental health problems among children (ages 0-21) in the United States remains unacceptably high and, post-COVID-19, is expected to increase dramatically. Decades of psychological knowledge about effective treatments should inform the delivery of better services. Dissemination and implementation (D&I) science has been heralded as a solution to the persistent problem of poor quality services and has, to some extent, improved our understanding of the contexts of delivery systems that implement effective practices. However, there are few studies demonstrating clear, population-level impacts of psychological interventions on children. Momentum is growing among communities, cities, states, and some federal agencies to build "health in all policies" to address broad familial, social, and economic factors known to affect children's healthy development and mental health. These health policy initiatives offer a rare opportunity to repurpose D&I science, shifting it from a primary focus on evidence-based practice implementation, to a focus on policy development and implementation to support child and family health and well-being. This shift is critical as states develop policy responses to address the health and mental health impacts of the COVID-19 pandemic on already-vulnerable families. We provide a typology for building research on D&I and children's mental health policy. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

A protocol study of participatory action research: integrated care pathway for pregnant women with heart disease in Indonesia

BACKGROUND

Heart diseases are increasingly identified as an important indirect cause of maternal mortality in several cities in Indonesia. The management of pregnancy with heart diseases requires a multidisciplinary approach, and interprofessional collaboration practice (IPCP) is critical to improving the quality of patient care. To enable the effective implementation of IPCP, integrated care pathways (ICPs) are needed to define the roles and responsibilities of the health professionals involved. This study aims to examine the obstacles and enabling factors of IPCP, to develop and use ICPs in the implementation of IPCP in health care services for pregnant women with heart diseases.

METHODS

A participatory action study consisting of four stages (diagnostic, planning, implementation, and evaluation) will take approximately 2 years after consensus of ICPs are made. The primary data collection process will employ consensus, observations, focus group discussions, and in-depth interviews throughout the four stages, while secondary data from referral documents and medical records will be collected mainly during the diagnostic and evaluation stages. The findings are being analysed and will then be used to develop an ICPs through consensus building at the planning stage to be applied in the implementation stage. Finally, the implementation outcome, including acceptability, adoption, appropriateness, and feasibility of IPCP, will be assessed in the evaluation stage. All qualitative data will be analysed thematically by two coders using NVIVO 12 software.

DISCUSSION

This research aims to assess the needs of IPCP, develop and use the ICPs in the implementation of IPCP in health care services for pregnant women with heart diseases. Findings from this study will be used for health service planning and policy making to strengthen practice of IPCP during the referral process. As a result, pregnant women with heart disease will have better access to high-quality services at every health care facility to reduce maternal mortality.

TRIAL REGISTRATION

Retrospectively registered in the ISRCTN registry with study ID ISRCTN82300061 on Feb 6, 2019.

Screening for Immigration-Related Health Concerns in a Federally Qualified Health Center Serving a Diverse Latinx Community: A Mixed Methods Study

Immigration-related concerns can impact health and are an important consideration while caring for a multinational Latinx immigrant community. Patients and caregivers waiting for a non-urgent clinic appointment were randomly screened with one of two social risk screening tools. One tool included a question about "any health or stability concerns related to immigration status." The other tool did not include an immigration health question. Immediately following, respondents were invited to participate in a semi-structured interview regarding their social risk screening experience. 201 screens were completed, and 20 patients agreed to an interview. There were no significant sociodemographic differences between groups. Of those screened for immigration, 11% reported a concern. In both arms, interviewees felt that social risk screening was acceptable in a clinic setting. Questions about immigration are timely, important, and relevant, and can be considered when implementing social assessments in communities where there are high levels of trust in providers.

Determinants of Implementing Evidence-Based Trauma-Focused Interventions for Children and Youth: A Systematic Review

A systematic review was conducted to identify determinants (barriers and facilitators) of implementing evidence-based psychosocial interventions for children and youth who experience emotional or behavioral difficulties due to trauma exposure. Determinants were coded, abstracted, and synthesized using the Exploration, Preparation, Implementation, and Sustainment framework. Twenty-three articles were included, all of which examined implementation of Trauma-Focused Cognitive Behavioral Therapy or Cognitive-Behavioral Intervention for Trauma in Schools. This review identified multilevel and multiphase determinants that can be addressed by implementation strategies to improve implementation and clinical outcomes, and suggests how future studies might address gaps in the evidence base.

The Use of Implementation Science Tools to Design, Implement, and Monitor a Community-Based mHealth Intervention for Child Health in the Amazon

It is essential to analyze the local context and implementation components to effectively deliver evidence-based solutions to public health problems. Tools provided by the field of implementation science can guide practitioners through a comprehensive implementation process, making innovations more adaptable, efficient, and sustainable. It is equally important to report on the design and implementation process so others can analyze, replicate, and improve on the progress made from an intervention. The current study reports on the design and implementation of an mHealth intervention to improve child health in the Amazon of Peru. The study aims to provide insight into how an implementation science tool can be used to improve implementation and reporting of an evidence-based intervention in a global health setting.

Methods: Implementation of a community-based mHealth intervention is analyzed and reported through the lens of the Active Implementation Frameworks (AIF). The AIF is used to analyze the design, implementation, adaptation, and monitoring of the intervention. The implementation process is categorized in the four stages of implementation. The results of the analysis and subsequent implementation activities are reported.

Results: The exploration stage was used to learn about the local context in the Amazonian communities and identify an evidence-based solution to address poor child health. Several potential solutions were combined to create an innovative mHealth tool. During the installation stage, the stakeholders worked together to improve the intervention and plan for implementation through human-centered design. The providers in the field were trained and data was gathered to monitor implementation. During initial implementation stage, electronic tablets were distributed to community health agents and continuous quality improvement activities allowed for rapid improvements to be implemented. The intervention moved on to full implementation stage as acceptance and fidelity approached 100%.

Conclusion: The AIF highlighted several potential barriers to implementation that may have been overlooked without the guidance of a science-based implementation tool. Reporting on the implementation process shows how implementation science tools can be used to foresee and address potential threats to successful implementation. The results of this study provide insight into the components of implementation in Amazonian communities, as well as the process of using implementation science tools in any global health setting.