Measuring sustainment of an evidence based treatment for adolescent substance use

Multi-level alignment processes in the sustainment of a youth substance use treatment model following a federal implementation initiative: A mixed method study

INTRODUCTION

Government agencies have identified evidence-based practice (EBP) dissemination as a pathway to high-quality behavioral health care for youth. However, gaps remain about how to best sustain EBPs in treatment organizations in the U.S., especially in resource-constrained settings like publicly-funded youth substance use services. One important, but understudied, determinant of EBP sustainment is alignment: the extent to which multi-level factors that influence sustainment processes and outcomes are congruent, consistent, and/or coordinated. This study examined the role of alignment in U.S. states' efforts to sustain the Adolescent Community Reinforcement Approach (A-CRA), an EBP for youth substance use disorders, during the COVID-19 pandemic.

METHODS

In this mixed methods study, the qualitative investigation preceded and informed the quantitative investigation. We interviewed state administrators and providers (i.e., supervisors and clinicians) from 15 states that had completed a federal A-CRA implementation grant; providers also completed surveys. The sample included 50 providers from 35 treatment organizations that reported sustaining A-CRA when the COVID-19 pandemic began, and 20 state administrators. In qualitative thematic analyses, we applied the EPIS (Exploration, Preparation, Implementation, Sustainment) framework to characterize alignment processes that interviewees described as influential on sustainment. We then used survey items to quantitatively explore the associations described in qualitative themes, using bivariate linear regressions.

RESULTS

At the time of interview, staff from 80 % of the treatment organizations (n = 28), reported sustaining A-CRA. Providers from both sustainer and non-sustainer organizations, as well as state administrators, described major sources of misalignment when state agencies ceased technical assistance post-grant, and because limited staff capacity conflicted with A-CRA's training model, which was perceived as time-intensive. Participants described the pandemic as exacerbating preexisting challenges, including capacity issues. Sustainer organizations reported seeking new funding to help sustain A-CRA. Quantitative associations between self-rated extent of sustainment and other survey items mostly followed the pattern predicted from the qualitative findings.

CONCLUSIONS

The COVID-19 pandemic amplified longstanding A-CRA sustainment challenges, but treatment organizations already successfully sustaining A-CRA pre-pandemic largely continued. There are missed opportunities for state-level actors to coordinate with providers on the shared goal of EBP sustainment. A greater focus on alignment processes in research and practice could help states and providers strengthen sustainability planning.

Comparing two federal financing strategies on penetration and sustainment of the adolescent community reinforcement approach for substance use disorders: protocol for a mixed-method study

BACKGROUND

Sustained, widespread availability of evidence-based practices (EBPs) is essential to address the public health and societal impacts of adolescent substance use disorders (SUD). There remains a particularly significant need to identify effective financing strategies, which secure and direct financial resources to support the costs associated with EBP implementation and sustainment. This protocol describes a new project comparing two types of U.S. federal grant mechanisms (i.e., a type of financing strategy), which supported the implementation of the Adolescent Community Reinforcement Approach (A-CRA) EBP for SUD, through either organization-focused or state-focused granting of funds. The Exploration-Preparation-Implementation-Sustainment (EPIS) framework will guide our study aims, hypotheses, and selection of measures.

METHOD

We will employ a longitudinal, mixed-method (i.e., web surveys, semi-structured interviews, document review, focus groups, administrative data), quasi-experimental design to compare the grant types' outcomes and examine theoretically informed mediators and moderators. Aim 1 will examine the proportion of eligible clinicians certified in A-CRA with adequate fidelity levels (i.e., penetration outcomes) at the end of grant funding. Aim 2 will examine the sustainment of A-CRA up to 5 years post-funding, using a 10-element composite measure of treatment delivery and supervision activities. We will integrate the new data collected from state-focused grant recipients (~85 organizations in 19 states) with previously collected data from organization-focused grant recipients (Hunter et al., Implement Sci 9:104, 2014) (82 organizations in 26 states) for analysis. We will also use sensitivity analyses to characterize the effects of observed and unobserved secular trends in our quasi-experimental design. Finally, aim 3 will use comparative case study methods (integrating diverse quantitative and qualitative measures) to identify and disseminate policy implications about the roles of state- and organization-focused federal grants in efforts to promote adolescent SUD EBP implementation and sustainment.

DISCUSSION

The proposed research will have direct, practical implications for behavioral health administrators, policymakers, implementation experts, and the public. It will offer new knowledge that can directly inform financing strategies to support large-scale, sustained EBP delivery in behavioral health-while advancing implementation science through the use of novel methods to study financing strategies and sustainment.

Seeking to Understand: Qualitative Research on Sustainability of Evidence-Based Practice in Acute Care

OBJECTIVE

The aim of this study was to describe cultural characteristics, values, and beliefs that influence sustainability of an evidence-based practice (EBP) intervention in the acute care clinical setting.

BACKGROUND

There is an urgent need to identify best practices to sustain EBP to gain efficiencies in nursing care delivery and improve patient outcomes.

METHODS

A focused ethnographic qualitative study was conducted in a community hospital with nurses that used Screening, Brief Intervention, and Referral to Treatment (SBIRT).

RESULTS

Customizing the intervention to the unit culture evolved and was crucial for sustainability. Overlap in responsibilities, time, clinician confidence, and impact to workflow were noted as negative influences. The intervention was primarily viewed as a task to be checked off a list instead of a tool that informs the patient's plan of care.

CONCLUSIONS

Assessing clinician experiences, beliefs, and values of an EBP should be incorporated into a strategic sustainability plan. Clinician understanding of how an EBP can advance the patient plan of care could promote ownership of professional practice and sustainment.

What influences evidence-based treatment sustainment after implementation support ends? A mixed method study of the adolescent-community reinforcement approach

BACKGROUND

Little is known about clinician perspectives regarding the factors that support or hinder the long-term delivery (i.e., sustainment) of evidence-based treatments in community-based treatment settings.

METHODS

Clinical staff from 82 community-based treatment organizations that received federal grant funding to support the delivery of the Adolescent Community Reinforcement Approach (A-CRA), an evidence-based treatment for adolescent substance use, were asked to participate in interviews focused on understanding their perspectives about the sustainment of A-CRA. Qualitative themes were identified using inductive and deductive approaches. Then the themes were dichotomized (present/absent) so that quantitative comparisons could be made between staff from organizations that sustained and did not sustain delivery of A-CRA. Administrative data about each organization in relation to federal funding support and their primary focus was also examined to explore whether these characteristics were associated with A-CRA sustainment.

RESULTS

Staff (n = 134) representing 78 organizations participated in the interviews. Staff from organizations that had received multiple federal grants to support the delivery of A-CRA and whose primary focus was substance use rather than other conditions (mental health or primary care) were more likely to report sustaining A-CRA. Staff from sustaining organizations were more likely to report positive grant experiences and success with maintaining both organizational and external support in comparison to staff from non-sustaining organizations. Staff from non-sustaining organizations were more likely to report barriers to sustaining A-CRA, including more challenges with intervention delivery, and lack of internal support and external funding.

CONCLUSIONS

Our findings lend empirical support for implementation theories in that multiple factors appear to be associated with long-term delivery of an evidence-based treatment. Although A-CRA was generally perceived positively by staff from both organizations that sustained A-CRA and organizations that did not sustain A-CRA, inner setting factors (e.g., structural policies, leadership support and staff retention) along with outer setting factors (e.g., external funding support) were reported as key to A-CRA sustainment.

Assessing long-term sustainment of clinic participation in NIATx200: Results and a new methodological approach

BACKGROUND

Sustainability frameworks differentiate between sustainability capacity and sustainment of organizational change. Multiple studies have examined sustainability capacity. Methodological approaches to assess long-term sustainment have not been explored. This study addresses this gap by describing the development of a long-term sustainment methodology and evaluating its application utilizing data from substance abuse clinics participating in a quality improvement collaborative.

METHODS

The study involved clinics (n = 121) in three states (MI, NY and WA) participating in the 2007-2009 NIATx200 quality improvement (QI) intervention. It extended the primary analysis to focus on clinics' long-term sustainment of wait time, retention and admission improvements. Long-term sustainment was defined as two years post end of the active implementation period (Calendar Years 2010 and 2011). The analysis defined case exclusion criteria and spline "knot" time intervals; allowed for C_p statistic use to address clinic data volatility; established the structure of sustainment plots and explored differences between NIATx implementation strategies.

RESULTS

Example spline and sustain plots highlight the application of the long-term sustainment methodology for NIATx200 clinics. In clinics with available longitudinal outcome data, 40.8% (n = 31 of 76 clinics) sustained improvements in wait time, 26.7% (n = 20 of 75 clinics) in retention, and 28.1% (n = 32 of 114 clinics) for admissions. Clinic assignment to a NIATx200 implementation strategy did not significantly influence a clinics' long-term sustainment except for lower wait time changes in the interest circle interventions. Thirty clinics (24.8%) sustained improvements for two outcomes and six clinics (5.0%) did so for all three outcomes. The clinics that sustained multiple outcome improvements were assigned to the interest circle (n = 12), learning session (n = 10), combination (n = 8), and coaching (n = 6) implementation strategies. Guidance for applying the long-term sustainment methodology in other quality improvement settings is described.

CONCLUSIONS

Research about sustainability capacity and sustainment of change has become increasingly important in dissemination and implementation research. Assessment of long-term sustainment in a multi-organizational quality improvement collaborative (QIC) is needed to identify when program drift and intervention decay occurs. If "cut-points" indicate when effects diminish, specific sustainability modules could be developed and introduced within the structure of a QIC to improve organizational long-term sustainment. Coaches and change teams could be trained to focus on organizational change sustainment and strengthen the likelihood of institutionalization. ClinicalTrials.gov Identifier: NCT00934141 Registered July 6, 2009. Retrospectively registered.