Cost Analysis of a Randomized Trial of Getting to Outcomes Implementation Support of CHOICE in Boys and Girls Clubs in Southern California

Organizational Capacity for Sexual Assault Prevention Within a U.S. Army Installation

INTRODUCTION

Preventing military sexual assault (SA) is a DoD priority. Building prevention capacity could strengthen the impact of prevention programs and improve outcomes. Capacity was conceptualized as implementation knowledge and skills using the Getting To Outcomes (GTO) process and organizational-level capacities using the Prevention Evaluation Framework, a framework that applies best practices in prevention and implementation science to prevention at the program and organizational level. The present study assesses implementation knowledge and skills currently possessed by SA prevention personnel in one U.S. Army installation with the goal of identifying preexisting proficiencies and potential gaps. The study uses the Prevention Evaluation Framework organizational-level domains to identify organizational capacities that could be improved.

MATERIALS AND METHODS

Fifteen SA prevention program personnel were recruited for individual 1-hour interviews from one U.S. Army installation. A semi-structured interview guide was developed by the study team to assess knowledge and skills within the GTO framework. Applied thematic analysis guided data coding and analysis.

RESULTS

Participants reported a range of proficiencies and gaps in capacity for SA prevention. Within the prevention workforce capacity, proficiencies were skill in tailoring training, and an existing train-the-trainer model. Gaps were training to facilitate difficult conversations, and an inability to tailor trainings to audience needs. Two proficiencies related to the comprehensive approach were having a victim advocate present during trainings, and providing training across leadership levels. The final proficiency involved data. Existing data sources were viewed as useful for identifying the nature and extent of local problems. However, participants reported a lack of confidence and skill in selecting appropriate outcome measures and collecting and using primary data. Gaps in leadership capacity were cultural norms, lack of trust, leader investment, logistical difficulty working with leaders, and comfort engaging with leaders. Gaps in collaborative relationships were the lack of informal networking within the SA prevention area to include civilians.

CONCLUSIONS

The GTO-specific knowledge and skills of SA prevention personnel at one U.S. Army installation revealed proficiencies and gaps in organizational capacities influencing the prevention mission. Findings indicate that GTO support could be useful for improving the quality of program activities. However, results also indicate that GTO support would be more effective if they were paired with sufficient prevention infrastructure at the organizational level.

Adaptive Designs in Implementation Science and Practice: Their Promise and the Need for Greater Understanding and Improved Communication

The promise of adaptation and adaptive designs in implementation science has been hindered by the lack of clarity and precision in defining what it means to adapt, especially regarding the distinction between adaptive study designs and adaptive implementation strategies. To ensure a common language for science and practice, authors reviewed the implementation science literature and found that the term adaptive was used to describe interventions, implementation strategies, and trial designs. To provide clarity and offer recommendations for reporting and strengthening study design, we propose a taxonomy that describes fixed versus adaptive implementation strategies and implementation trial designs. To improve impact, (a) futureimplementation studies should prespecify implementation strategy core functions that in turn can be taught to and replicated by health system/community partners, (b) funders should support exploratory studies that refine and specify implementation strategies, and (c) investigators should systematically address design requirements and ethical considerations (e.g., randomization, blinding/masking) with health system/community partners.

The implementation & sustainment facilitation (ISF) strategy: Cost and cost-effectiveness results from a 39-site cluster randomized trial integrating substance use services in community-based HIV service organizations

Background: As part of the Substance Abuse Treatment to HIV Care Project, the Implementation & Sustainment Facilitation (ISF) strategy was found to be an effective adjunct to the Addiction Technology Transfer Center (ATTC) strategy for integrating a motivational interviewing-based brief intervention (MIBI) for substance use disorders. This study presents the cost and cost-effectiveness results. Methods: Thirty-nine HIV service organizations were randomized to receive the ATTC-only condition or the ATTC + ISF condition. Two staff from each organization received the ATTC-training. In ATTC + ISF organizations, the same two staff and additional support staff participated in facilitation sessions to support MIBI implementation. We estimated costs using primary data on the time spent in each strategy and the time spent delivering 409 MIBIs to clients. We estimated staff-level cost-effectiveness for the number of MIBIs delivered, average MIBI quality scores, and total client days abstinent per staff. We used sensitivity analyses to test how changes to key variables affect the results. Results: Adjusted per-staff costs were $2,915 for the ATTC strategy and $5,371 for ATTC + ISF, resulting in an incremental cost of $2,457. ATTC + ISF significantly increased the number of MIBIs delivered (3.73) and the average MIBI quality score (61.45), yielding incremental cost effectiveness ratios (ICERs) of $659 and $40. Client days abstinent increased by 59 days per staff with a quality-adjusted life-year ICER of $40,578 (95% confidence interval $29,795–$61,031). Conclusions: From the perspective of federal policymakers, ISF as an adjunct to the ATTC strategy may be cost-effective for improving the integration of MIBIs within HIV service organizations, especially if scaled up to reach more clients. Travel accounted for nearly half of costs, and virtual implementation may further increase value. We also highlight two considerations for cost-effectiveness analysis with hybrid trials: study protocols kept recruitment low and modeling choices affect how we interpret the effects on client-level outcomes.

Getting To Implementation (GTI)-Teach: A seven-step approach for teaching the fundamentals of implementation science

Introduction:

Implementation Science (IS) is a complex and rapidly evolving discipline, posing challenges for educators. We developed, implemented, and evaluated a novel, pragmatic approach to teach IS.

Methods:

Getting To Implementation (GTI)-Teach was developed as a seven-step educational model to guide students through the process of developing, conducting, and sustaining an IS research project. During the four-week online course, students applied the steps to self-selected implementation problems. Students were invited to complete two online post-course surveys to assess course satisfaction and self-reported changes in IS knowledge and relevance of GTI-Teach Steps to their work. Results were summarized using descriptive statistics; self-reported post-course changes in IS knowledge were compared using paired t-tests.

Results:

GTI-Teach was developed to include seven Steps: 1. Define the implementation problem; 2. Conceptualize the problem; 3. Prioritize implementation barriers and facilitators; 4. Select and tailor implementation strategies; 5. Design an implementation study; 6. Evaluate implementation; 7. Sustain implementation. Thirteen students, ranging in experience from medical students to full professors, enrolled in and completed the first GTI-Teach course. Of the seven students (54%) completing an end-of course survey, six (86%) were very satisfied with the course. Ten students (77%) responded to the tailored, 6-month post-course follow-up survey. They retrospectively reported a significant increase in their knowledge across all steps of GTI-Teach (1.3–1.8 points on a 5-point Likert scale) and rated each of the Steps as highly relevant to their work.

Conclusions:

GTI-Teach is a seven-step model for teaching IS fundamentals that students reported increased their knowledge and was relevant to their work.

Getting to implementation: a protocol for a Hybrid III stepped wedge cluster randomized evaluation of using data-driven implementation strategies to improve cirrhosis care for Veterans

BACKGROUND

Cirrhosis is a rapidly increasing cause of global mortality. To improve cirrhosis care, the Veterans Health Administration (VHA) developed the Hepatic Innovation Team (HIT) Collaborative to support VA Medical Centers (VAMCs) to deliver evidence-based cirrhosis care. This randomized HIT program evaluation aims to develop and assess a novel approach for choosing and applying implementation strategies to improve the quality of cirrhosis care.

METHODS

Evaluation aims are to (1) empirically determine which combinations of implementation strategies are associated with successful implementation of evidence-based practices (EBPs) for Veterans with cirrhosis, (2) manualize these "data-driven" implementation strategies, and (3) assess the effectiveness of data-driven implementation strategies in increasing cirrhosis EBP uptake. Aim 1 will include an online survey of all VAMCs' use of 73 implementations strategies to improve cirrhosis care, as defined by the Expert Recommendations for Implementing Change taxonomy. Traditional statistical as well as configurational comparative methods will both be employed to determine which combinations of implementation strategies are associated with site-level adherence to EBPs for cirrhosis. In aim 2, semi-structured interviews with high-performing VAMCs will be conducted to operationalize successful implementation strategies for cirrhosis care. These data will be used to inform the creation of a step-by-step guide to tailoring and applying the implementation strategies identified in aim 1. In aim 3, this manualized implementation intervention will be assessed using a hybrid type III stepped-wedge cluster randomized design. This evaluation will be conducted in 12 VAMCs, with four VAMCs crossing from control to intervention every 6 months, in order to assess the effectiveness of using data-driven implementation strategies to improve guideline-concordant cirrhosis care.

DISCUSSION

Successful completion of this innovative evaluation will establish the feasibility of using early evaluation data to inform a manualized, user-friendly implementation intervention for VAMCs with opportunities to improve care. This evaluation will provide implementation support tools that can be applied to enhance the implementation of other evidence-based practices.

TRIAL REGISTRATION

This project was registered at ClinicalTrials.Gov ( NCT04178096 ) on 4/29/20.