A mixed-methods study of system-level sustainability of evidence-based practices in 12 large-scale implementation initiatives

The limits of learning engagement and academic leadership within the higher education digitalization process - analysis by using PLS SEM

There is convincing evidence that the learning environments digitalization of tools and equipment ultimately results in the speed and depth learning involvement of academia members, by raising attainment of each of the digital learning experiences. The majority of the research that was conducted on the topic of enhancing the digital skills of learners, which would ultimately lead to an increase in their active engagement, was conducted on students in primary and secondary education, leaving members of higher education outside of the scope of the study. Given the uninterrupted search for academic performance and innovation, the current research considers the technological changes that lead to the transformation of the traditional academic learning environments as previously known. The current paper considers the changes in the learners' engagement in the context of the dually digital transformation of the higher academic multi-institutional digitally-learning enhancements. An important factor to be considered regards the leadership evolution (in terms of teaching) that over time, led to a different speed contextual shift, according to its effectiveness, leading to higher or lower students learning (dis)engagement. The current manuscript aims to examine how the higher education digitalization levels could affect the student's learning engagement, under the close monitoring of the academia leadership styles practice. Data collection and analysis implied at first a qualitative approach by issuing an online-distributed survey that resulted in a number of 2272 valid responses. After performing structural equation modelling and proving a valid assessment tool, the analysis resulted into statistically proving the validity of two main hypotheses according to which students learning engagement has a positive effect on the practice of academic leadership. Additionally, results emphasized the fact that higher education digitalization has altogether a negative effect of students learning engagement. Consequently, the current study stresses on the importance of different peers' categories in the context of higher education institutions performance, with an emphasis on the different levels of students' engagement and the leadership styles evolution and practice, aspects uniformly developing within a continuously digitally transformation of the higher education environment.

Community-based Agency Delivery of Parent-Child Interaction Therapy: Comparing Outcomes for Children with and Without Autism Spectrum Disorder and/or Developmental Delays

While externalizing behaviors are common among children with autism spectrum disorder (ASD), there is a shortage of specialist community-based clinicians to provide treatment. Parent-Child Interaction Therapy (PCIT), an intervention designed to reduce child disruptive behaviors, may be effective for families of children with ASD but has rarely been studied outside of university-based research settings. We examined the effectiveness of PCIT delivered for children with (N = 109) and without (N = 2,324) ASD/developmental delays (DD) across community-based agencies in Oregon. Findings revealed significant reductions in disruptive behavior and positive changes in the parent-child relationship in both groups. These findings support PCIT as an efficacious intervention for children with ASD/DD and demonstrate PCIT's promise in community-based agencies with non-specialized clinicians.

Pilot evaluation of the Fiscal Mapping Process for sustainable financing of evidence-based youth mental health treatments: A comparative case study analysis

Background

Sustained delivery of evidence-based treatments (EBTs) is essential to addressing the public health impacts of youth mental health problems, but is complicated by the limited and fragmented funding available to youth mental health service agencies. Supports are needed that can guide service agencies in accessing sustainable funding for EBTs. We conducted a pilot evaluation of the Fiscal Mapping Process, an Excel-based strategic planning tool that helps service agency leaders identify and coordinate financing strategies for their EBT programs.

Method

Pilot testing of the Fiscal Mapping Process was completed with 10 youth mental health service agencies over a 12-month period, using trauma-focused cognitive-behavioral therapy or parent-child interaction therapy programs. Service agency representatives received initial training and monthly coaching in using the tool. We used case study methods to synthesize all available data (surveys, focus groups, coaching notes, document review) and contrast agency experiences to identify key findings through explanation building.

Results

Key evaluation findings related to the process and outcomes of using the Fiscal Mapping Process, as well as contextual influences. Process evaluation findings helped clarify the primary use case for the tool and identified the importance-and challenges-of engaging external collaborators. Outcome evaluation findings documented the impacts of the Fiscal Mapping Process on agency-reported sustainment capacities (strategic planning, funding stability), which fully explained reported improvements in outcomes (extent and likelihood)-although these impacts were incremental. Findings on contextual factors documented the influence of environmental and organizational capacities on engagement with the tool and concerns about equitable impacts, but also the view that the process could usefully generalize to other EBTs.

Conclusions

Our pilot evaluation of the Fiscal Mapping Process was promising. In future work, we plan to integrate the tool into EBT implementation initiatives and test its impact on long-term sustainment outcomes across various EBTs, while increasing attention to equity considerations.

Comparing organization-focused and state-focused financing strategies on provider-level reach of a youth substance use treatment model: a mixed-method study

BACKGROUND

Financial barriers in substance use disorder service systems have limited the widespread adoption-i.e., provider-level reach-of evidence-based practices (EBPs) for youth substance use disorders. Reach is essential to maximizing the population-level impact of EBPs. One promising, but rarely studied, type of implementation strategy for overcoming barriers to EBP reach is financing strategies, which direct financial resources in various ways to support implementation. We evaluated financing strategies for the Adolescent Community Reinforcement Approach (A-CRA) EBP by comparing two US federal grant mechanisms, organization-focused and state-focused grants, on organization-level A-CRA reach outcomes.

METHOD

A-CRA implementation took place through organization-focused and state-focused grantee cohorts from 2006 to 2021. We used a quasi-experimental, mixed-method design to compare reach between treatment organizations funded by organization-focused versus state-focused grants (164 organizations, 35 states). Using administrative training records, we calculated reach as the per-organization proportion of trained individuals who received certification in A-CRA clinical delivery and/or supervision by the end of grant funding. We tested differences in certification rate by grant type using multivariable linear regression models that controlled for key covariates (e.g., time), and tested threats to internal validity from our quasi-experimental design through a series of sensitivity analyses. We also drew on interviews and surveys collected from the treatment organizations and (when relevant) interviews with state administrators to identify factors that influenced reach.

RESULTS

The overall certification rates were 27 percentage points lower in state-focused versus organization-focused grants (p = .01). Sensitivity analyses suggested these findings were not explained by confounding temporal trends nor by organizational or state characteristics. We did not identify significant quantitative moderators of reach outcomes, but qualitative findings suggested certain facilitating factors were more influential for organization-focused grants (e.g., strategic planning) and certain barrier factors were more impactful for state-focused grants (e.g., states finding it difficult to execute grant activities).

DISCUSSION

As the first published comparison of EBP reach outcomes between financing strategies, our findings can help guide state and federal policy related to financing strategies for implementing EBPs that reduce youth substance use. Future work should explore contextual conditions under which different financing strategies can support the widespread implementation of EBPs for substance use disorder treatment.

Perspectives on Financing Strategies for Evidence-Based Treatment Implementation in Youth Mental Health Systems

BACKGROUND

Evidence-based treatments (EBTs) are critical to effectively address mental health problems among children and adolescents, but costly for mental health service agencies to implement and sustain. Financing strategies help agencies overcome cost-related barriers by obtaining financial resources to support EBT implementation and/or sustainment.

AIMS

We sought to (i) understand how youth mental health system decision-makers involved with EBT implementation and sustainment view key features (e.g., relevance, feasibility) that inform financing strategy selection and (ii) compare service agency, funding agency, and intermediary representative perspectives.

METHOD

Two surveys were disseminated to 48 representatives across U.S. youth mental health service agencies, funding agencies, and intermediaries who were participating in a larger study of financing strategies. Quantitative and qualitative data were gathered on 23 financing strategies through quantitative ratings and open-ended responses. Data were analyzed using descriptive statistics and rapid content analysis.

RESULTS

The financing strategies rated as most relevant include braided funding streams, contracts for EBTs, credentialing/rostering providers, fee-for-service reimbursement (regular and increased), and grant funding. All other strategies were unfamiliar to 1/3 to 1/2 of participants. The six strategies were rated between somewhat and quite available, feasible, and effective for EBT sustainment. For sustaining different EBT components (e.g., delivery, materials), the mix of financing strategies was rated as somewhat adequate. Qualitative analysis revealed challenges with strategies being non-recurring or unavailable in representatives' regions. Ratings were largely similar across participant roles, though funding agency representatives were the most familiar with financing strategies.

DISCUSSION

Despite the breadth of innovative financing strategies, expert representatives within the youth mental health services ecosystem had limited knowledge of most options. Experts relied on strategies that were familiar but often did not adequately support EBT implementation or sustainment. These findings underscore more fundamental issues with under-resourced mental health systems in the U.S.; financing strategies can help agencies navigate EBT use but must be accompanied by larger-scale system reforms. Limitations include difficulties generalizing results due to using a small sample familiar with EBTs, high agreement as a potential function of snowball recruiting, and limited responses to the open-ended survey questions.

IMPLICATIONS FOR HEALTH CARE PROVISION AND USE

Although EBTs have been found to effectively address mental health problems in children and adolescents, available strategies for financing their implementation and sustainment in mental health systems are insufficient. This constraint prevents many children and adolescents from receiving high-quality services.

IMPLICATIONS FOR HEALTH POLICIES

Financing strategies alone cannot solve systematic issues that prevent youth mental health service agencies from providing EBTs. Policy changes may be required, such as increased financial investment from the U.S. government into mental health services to support basic infrastructure (e.g., facility operations, measuring outcomes).

IMPLICATIONS FOR FURTHER RESEARCH

Future work should examine expert perspectives on EBT financing strategies in different contexts (e.g., substance use services), gathering targeted feedback on financing strategies that are less well known, and exploring topics such as strategic planning, funding stability, and collaborative decision-making as they relate to EBT implementation and sustainment.

The Role of Leadership and Digital Transformation in Higher Education Students' Work Engagement

Teaching and learning processes should be subject to continuous change due to the constant evolution of social, educational and technological environments, which ultimately results in higher levels of student engagement. The current paper describes the technological changes faced by higher education institutions as a result of digital transformation challenges. Further, transformational and transactional leadership styles' effectiveness is regarded within the context of higher education institutions' digital enhancements. Over time, these factors have led to contextual shifts that have disengaged students from learning and thus self-development. The current research aimed to examine how higher education institutions should apply different leadership styles within digitally transformed contexts so as to increase students' learning engagement and reduce the risk of failure in their future developments within (inter)national labor markets. Data gathering and analysis involved a qualitative approach: an online survey was distributed, resulting in 856 responses. Through structural equation modeling, the data revealed a valid higher education digital transformation assessment tool; the results also emphasize the increased role of transactional leadership, as opposed to the traditional transformational style, within a highly digitized higher education institutional framework. Consequently, the linear relationship of students' work engagement with leadership proved to also be enhanced by quadratic effects. The current study stresses the importance of internal and external peers in higher education performance through high levels of student learning (work) engagement through leadership and a uniformly developed digitally transformed higher education environment.

Assessment of a Workforce Sustainability Tool through Leadership and Digitalization

As organizational environment develops, the working environment increases in physical and mental demands. As a result, risk inadvertences could arise, along with organizational emotional and financial challenges. Within their efforts to diminish such risks, organizations strive for developing and training their workforce; a sustainable workforce can only be achieved through cultivating aptitudes and positive attitudes that will lead to organizational but also personal growth. Considered to be an important measuring instrument for social sustainability, workforce sustainability enhances organizational leadership projections and trajectories, along with digitalization initiatives. The aim of the current study is the development of an assessment tool for state and private organizational workforce sustainability, and to study it in relation to leadership and digitalization components. Through a quantitative approach, data was gathered by issuing an online survey that delivered 463 responses. By using structural equation modelling, the authors examined the aims and found that the designed workforce sustainability tool is reliable and valid; as predicted, all the leadership components contribute to organizational stability and a more favorable workforce sustainability development, along with enhancing digital learning. This study stresses the importance for state and private organizations to achieve workforce sustainability, while nurturing and providing the necessary tools for the development of leadership and digital learning.

Parent-Child Interaction Therapy for Children with Disruptive Behaviors and Autism: A Randomized Clinical Trial

A relatively large number of children with autism spectrum disorder (ASD) exhibit disruptive behavioral problems. While accumulating data have shown behavioral parent training programs to be efficacious in reducing disruptive behaviors for this population, there is a dearth of literature examining the impact of such programs across the range of ASD severity. To evaluate the effectiveness of Parent-Child Interaction Therapy (PCIT), an evidence-based treatment for children with problem behaviors and their families, in reducing disruptive behaviors among children (4-10 years) with ASD (without intellectual disabilities). Fifty-five children (85.5% male, 7.15 years; SD 1.72) were enrolled from pediatric offices and educational settings into a randomized clinical trial (PCIT: N = 30; Control: N = 25). PCIT families demonstrated a significant reduction in child disruptive behaviors, increase in positive parent-child communication, improvement in child compliance, and reduction in parental stress compared to the control group. Exploratory analyses revealed no differential treatment response based on ASD severity, receptive language, and age. Results are promising for the use of PCIT with children demonstrating disruptive behaviors across the autism spectrum.

Validation and endorsement of health system performance measures for opioid use disorder in British Columbia, Canada: A Delphi panel study

Background

Limited data exists on the performance of the healthcare system in opioid use disorder (OUD). We evaluated the face validity and potential risks of a set of health system performance measures for OUD collaboratively with clinicians, policymakers and people with lived experience of opioid use (PWLE) in the interest of establishing an endorsed set of measures for public reporting.

Methods

Through a two-stage Delphi-panel approach, a panel of clinical and policy experts validated and considered 102 previously constructed OUD performance measures for endorsement using information on measurement construction, sensitivity analyses, quality of evidence, predictive validity, and feedback from local PWLE. We collected quantitative and qualitative survey responses from 49 clinicians and policymakers, and 11 PWLE. We conducted inductive and deductive thematic analysis to present qualitative responses.

Results

A total of 37 measures of 102 were strongly endorsed (9/13 cascade of care, 2/27 clinical guideline compliance, 17/44 healthcare integration, and 9/18 healthcare utilization measures). Thematic analysis of responses revealed several themes regarding measurement validity, unintended consequences, and key contextual considerations. Overall, measures related to the cascade of care (excluding opioid agonist treatment dose tapering) received strong endorsements. PWLE highlighted barriers to accessing treatment, undignified aspects of treatment, and lack of a full continuum of care as their concerns.

Conclusion

We defined 37 endorsed health system performance measures for OUD and presented a range of perspectives on their validity and use. These measures provide critical considerations for health system improvement in the care of people with OUD.

Comparing two federal financing strategies on penetration and sustainment of the adolescent community reinforcement approach for substance use disorders: protocol for a mixed-method study

BACKGROUND

Sustained, widespread availability of evidence-based practices (EBPs) is essential to address the public health and societal impacts of adolescent substance use disorders (SUD). There remains a particularly significant need to identify effective financing strategies, which secure and direct financial resources to support the costs associated with EBP implementation and sustainment. This protocol describes a new project comparing two types of U.S. federal grant mechanisms (i.e., a type of financing strategy), which supported the implementation of the Adolescent Community Reinforcement Approach (A-CRA) EBP for SUD, through either organization-focused or state-focused granting of funds. The Exploration-Preparation-Implementation-Sustainment (EPIS) framework will guide our study aims, hypotheses, and selection of measures.

METHOD

We will employ a longitudinal, mixed-method (i.e., web surveys, semi-structured interviews, document review, focus groups, administrative data), quasi-experimental design to compare the grant types' outcomes and examine theoretically informed mediators and moderators. Aim 1 will examine the proportion of eligible clinicians certified in A-CRA with adequate fidelity levels (i.e., penetration outcomes) at the end of grant funding. Aim 2 will examine the sustainment of A-CRA up to 5 years post-funding, using a 10-element composite measure of treatment delivery and supervision activities. We will integrate the new data collected from state-focused grant recipients (~85 organizations in 19 states) with previously collected data from organization-focused grant recipients (Hunter et al., Implement Sci 9:104, 2014) (82 organizations in 26 states) for analysis. We will also use sensitivity analyses to characterize the effects of observed and unobserved secular trends in our quasi-experimental design. Finally, aim 3 will use comparative case study methods (integrating diverse quantitative and qualitative measures) to identify and disseminate policy implications about the roles of state- and organization-focused federal grants in efforts to promote adolescent SUD EBP implementation and sustainment.

DISCUSSION

The proposed research will have direct, practical implications for behavioral health administrators, policymakers, implementation experts, and the public. It will offer new knowledge that can directly inform financing strategies to support large-scale, sustained EBP delivery in behavioral health-while advancing implementation science through the use of novel methods to study financing strategies and sustainment.

Understanding the sustainment of population health programmes from a whole-of-system approach

BACKGROUND

Population health prevention programmes are needed to reduce the prevalence of chronic diseases. Nevertheless, sustaining programmes at a population level is challenging. Population health is highly influenced by social, economic and political environments and is vulnerable to these system-level changes. The aim of this research was to examine the factors and mechanisms contributing to the sustainment of population prevention programmes taking a systems thinking approach.

METHODS

We conducted a qualitative study through interviews with population health experts working within Australian government and non-government agencies experienced in sustaining public health programs at the local, state or national level (n = 13). We used a deductive thematic approach, grounded in systems thinking to analyse data.

RESULTS

We identified four key barriers affecting program sustainment: 1) short term political and funding cycles; 2) competing interests; 3) silo thinking within health service delivery; and 4) the fit of a program to population needs. To overcome these barriers various approaches have centred on the importance of long-range planning and resourcing, flexible program design and management, leadership and partnerships, evidence generation, and system support structures.

CONCLUSION

This study provides key insights for overcoming challenges to the sustainment of population health programmes amidst complex system-wide changes.

Cultural adaptation of psychological interventions for people with mental disorders delivered by lay health workers in Africa: scoping review and expert consultation

BACKGROUND

Lay Health Workers (LHW) are important providers of community mental health services and help mitigate access and treatment gaps in Africa. However, there is a paucity of knowledge about the role and performance of these workers, as well as about the extent to which the interventions delivered are culturally adapted to the African context.

AIMS

This scoping review aimed to explore the content and aspects concerning the cultural adaptation and sustainability of psychological interventions delivered by LHW to people with mental disorders in Africa.

METHODS

We conducted a scoping review of the peer-reviewed literature published from January 2000 to December 2018 to identify psychological interventions delivered by LHW for people with mental disorders in Africa. We systematically searched PubMed, Google scholar and Hinari to select relevant publications. The articles were evaluated for risk of bias according to study design with the National Heart, Lung, and Blood Institute's (NHLBI) Quality Assessment Tools. Expert consultation was performed according to Arksey & O'Malley framework and cultural adaptation analysis was performed according to Bernal framework.

RESULTS

Out of 14,549 retrieved records, we identified ten peer-reviewed articles conducted in Zimbabwe, Uganda, South Africa and Zambia describing four distinct interventions. Six were randomized controlled trials; none addressed implementation outcomes. Group-based interpersonal therapy (n = 5), trauma-focused cognitive behaviour therapy (n = 1), problem solving therapy (n = 3) and narrative exposure therapy (n = 1) emerged as psychological interventions delivered by LHW for people with depression, anxiety, trauma and suicidal behavior. Psychological interventions delivered by LHW in Africa were all culturally adapted to meet the competence of LHW. All the interventions were associated with symptom improvement, but the quality of this evidence varied widely with study design.

CONCLUSION

Task-shifting psychological interventions delivered by LHW after appropriate cultural adaptation show promise for addressing unmet mental health care needs in Africa. More effectiveness and implementation evidence is needed, especially with regard to psychological interventions delivered by LHW for adolescence, older people and those with severe mental disorders and suicidal behaviors.

Coordination of sustainable financing for evidence-based youth mental health treatments: protocol for development and evaluation of the fiscal mapping process

BACKGROUND

Sustained delivery of evidence-based treatments (EBTs) is essential to addressing the public health and economic impacts of youth mental health problems, but is complicated by the limited and fragmented funding available to youth mental health service agencies (hereafter, "service agencies"). Strategic planning tools are needed that can guide these service agencies in their coordination of sustainable funding for EBTs. This protocol describes a mixed-methods research project designed to (1) develop and (2) evaluate our novel fiscal mapping process that guides strategic planning efforts to finance the sustainment of EBTs in youth mental health services.

METHOD

Participants will be 48 expert stakeholder participants, including representatives from ten service agencies and their partners from funding agencies (various public and private sources) and intermediary organizations (which provide guidance and support on the delivery of specific EBTs). Aim 1 is to develop the fiscal mapping process: a multi-step, structured tool that guides service agencies in selecting the optimal combination of strategies for financing their EBT sustainment efforts. We will adapt the fiscal mapping process from an established intervention mapping process and will incorporate an existing compilation of 23 financing strategies. We will then engage participants in a modified Delphi exercise to achieve consensus on the fiscal mapping process steps and gather information that can inform the selection of strategies. Aim 2 is to evaluate preliminary impacts of the fiscal mapping process on service agencies' EBT sustainment capacities (i.e., structures and processes that support sustainment) and outcomes (e.g., intentions to sustain). The ten agencies will pilot test the fiscal mapping process. We will evaluate how the fiscal mapping process impacts EBT sustainment capacities and outcomes using a comparative case study approach, incorporating data from focus groups and document review. After pilot testing, the stakeholder participants will conceptualize the process and outcomes of fiscal mapping in a participatory modeling exercise to help inform future use and evaluation of the tool.

DISCUSSION

This project will generate the fiscal mapping process, which will facilitate the coordination of an array of financing strategies to sustain EBTs in community youth mental health services. This tool will promote the sustainment of youth-focused EBTs.

Time-Out with Young Children: A Parent-Child Interaction Therapy (PCIT) Practitioner Review

Time-out is a component of many evidence-based parent training programmes for the treatment of childhood conduct problems. Existing comprehensive reviews suggest that time-out is both safe and effective when used predictably, infrequently, calmly and as one component of a collection of parenting strategies-i.e., when utilised in the manner advocated by most parent training programmes. However, this research evidence has been largely oriented towards the academic community and is often in conflict with the widespread misinformation about time-out within communities of parents, and within groups of treatment practitioners. This dissonance has the potential to undermine the dissemination and implementation of an effective suite of treatments for common and disabling childhood conditions. The parent-practitioner relationship is integral to the success of Parent-Child Interaction Therapy (PCIT), an evidence-based treatment which involves live coaching of parent(s) with their young child(ren). Yet this relationship, and practitioner perspectives, attitudes and values as they relate to time-out, are often overlooked. This practitioner review explores the dynamics of the parent-practitioner relationship as they apply to the teaching and coaching of time-out to parents. It also acknowledges factors within the clinical setting that impact on time-out's use, such as the views of administrators and professional colleagues. The paper is oriented toward practitioners of PCIT but is of relevance to all providers of parent training interventions for young children.

How do Lay Health Worker Engage Caregivers? A Qualitative Study to Enhance Equity in Evidence-Based Parenting Programs

Engaging caregivers in their children's mental health treatment is critical for delivering high quality, evidence-based care, particularly for young children with externalizing behaviors. Lay health workers (LHWs), including peer providers and promotoras de salud, have been identified as important workforces in addressing structural and stigma-related barriers to engagement in mental health services. Importantly, research has suggested that LHWs may be integral in efforts to address engagement disparities in evidence-based behavioral parent training programs (BPTs) for Latinx caregivers. The purpose of the study was to understand how different LHW workforces engage caregivers within their usual services, in order to inform strategies that improve access to and engagement in BPTs. Qualitative interviews were conducted with two different LHW workforces: volunteer LHWs (i.e., promotoras de salud) (n = 14), who were part of a community-embedded network, and paid LHWs (i.e., parent support partners, home visitors) (n = 9) embedded within children's mental health agencies. Participants were predominately Latinx (79%) and female (96%). Qualitative analyses revealed three primary themes related to engagement strategies used by LHWs to address barriers to care: 1.) Building Trust, 2.) Empowerment, 3.) Increasing Access. Although the majority of themes and sub-themes were consistent across the two LHW workforces, agency-embedded LHWs often discussed having the means to provide resources through their organizations, whereas community-embedded LHWs discussed acting as a bridge to services by providing information and conducting outreach. Findings have implications for partnering with different workforces of LHWs to increase equity in access to BPTs.

Therapist Experiences and Attitudes About Implementing Internet-Delivered Parent-Child Interaction Therapy During COVID-19

It has been widely recognized that access to mental health treatment is imperative to address current and long-term stressors for children and parents during COVID-19. Internet-delivered Parent-Child Interaction Therapy (iPCIT, previously referred to as I-PCIT) is a strong model for remote service delivery during social distancing restrictions due to its empirical base. However, this treatment modality was not widely implemented before COVID-19, likely due to barriers to providing telehealth services. This mixed methods study conducted a follow-up survey to gather therapist experiences (N = 223) in delivering iPCIT during COVID-19, including qualitative data on the benefits and challenges to delivering iPCIT. The vast majority of therapists (82%) indicated that they transitioned to deliver PCIT via telehealth in response to COVID-19. PCIT caseloads decreased slightly from the first survey to the COVID-19 follow-up survey, but the racial and ethnic composition of caseloads were not significantly different between the two surveys. Of the 183 therapists who transitioned to deliver PCIT via telehealth, 82% expressed interest in continuing to provide iPCIT following the COVID-19 pandemic. Reported benefits of iPCIT included decreased barriers to access and the ability to practice skills within the naturalistic home environment. Challenges to iPCIT were primarily issues with technology as well as other logistical barriers, which could limit engagement for some families. Findings from this study may be beneficial in improving future implementation of iPCIT during and post-COVID-19.

An equivalence analysis of provider education in youth mental health care

OBJECTIVE

To test for equivalence between providers with and without advanced degrees in multiple domains related to delivery of evidence-based treatment.

DATA SOURCE

Provider and client data from an effectiveness trial of Alternatives for Families: A Cognitive Behavioral Therapy (AF-CBT) in a major metropolitan area in the United States.

STUDY DESIGN

We tested for equivalence between providers (N = 182) with and without advanced degrees in treatment-related knowledge, practices, and attitudes; job demands and stress; and training engagement and trainer-rated competence in AF-CBT. We also conducted exploratory analyses to test for equivalence in family clinical outcomes.

DATA COLLECTION

Providers completed measures prior to randomization and at 6-month follow-up, after completion of training and consultation in AF-CBT. Children and caregivers completed assessments at 0, 6, 12, and 18 months.

PRINCIPAL FINDINGS

Providers without advanced degrees were largely non-inferior to those with advanced degrees in treatment-related knowledge, practices, and attitudes, while findings for job demands and stress were mixed. Providers without advanced degrees were non-inferior to providers with advanced degrees in consultation attendance (B = -1.42; confidence interval (CI) = -3.01-0.16; margin of equivalence (Δ) = 2), number of case presentations (B = 0.64; CI = -0.49-1.76; Δ = 2), total training hours (B = -4.57; CI = -10.52-1.37; Δ = 3), and trainer-rated competence in AF-CBT (B = -0.04; CI = -3.04-2.96; Δ = 4), and they were significantly more likely to complete training (odds ratio = 0.66; CI = 0.10-0.96; Δ = 30%). Results for clinical outcomes were largely inconclusive.

CONCLUSIONS

Provider-level outcomes for those with and without advanced degrees were generally comparable. Additional research is needed to examine equivalence in clinical outcomes. Expanding evidence-based treatment training to individuals without advanced degrees may help to reduce workforce shortages and improve reach of evidence-based treatments.

Cost-Effectiveness Analysis in Implementation Science: a Research Agenda and Call for Wider Application

Purpose of Review

Cost-effectiveness analysis (CEA) can help identify the trade-offs decision makers face when confronted with alternative courses of action for the implementation of public health strategies. Application of CEA alongside implementation scientific studies remains limited. We aimed to identify areas for future development in order to enhance the uptake and impact of model-based CEA in implementation scientific research.

Recent Findings

Important questions remain about how to broadly implement evidence-based public health interventions in routine practice. Establishing population-level implementation strategy components and distinct implementation phases, including planning for implementation, the time required to scale-up programs, and sustainment efforts required to maintain them, can help determine the data needed to quantify each of these elements. Model-based CEA can use these data to determine the added value associated with each of these elements across systems, settings, population subgroups, and levels of implementation to provide tailored guidance for evidence-based public health action. There is a need to integrate implementation science explicitly into CEA to adequately capture diverse real-world delivery contexts and make detailed, informed recommendations on the aspects of the implementation process that provide good value.

Summary

We describe examples of how model-based CEA can integrate implementation scientific concepts and evidence to help tailor evaluations to local context. We also propose six distinct domains for methodological advancement in order to enhance the uptake and impact of model-based cost-effectiveness analysis in implementation scientific research.

Making Implementation Last: The Impact of Training Design on the Sustainability of an Evidence-Based Treatment in a Randomized Controlled Trial

Although advances have been made in facilitating the implementation of evidence-based treatments, little is known about the most effective way to sustain their use over time. The current study examined the sustainability of one evidence-based treatment, Parent-Child Interaction Therapy (PCIT), following a statewide implementation trial testing three training methods: Cascading Model, Learning Collaborative, and Distance Education. Participants included 100 clinicians and 50 administrators from 50 organizations across Pennsylvania. Clinicians and administrators reported on sustainability at 24-months, as measured by the number of clients receiving PCIT and the continued use of the PCIT protocol. Multi-level path analysis was utilized to examine the role of training on sustainability. Clinicians and administrators reported high levels of sustainability at 24-months. Clinicians in the Cascading Model reported greater average PCIT caseloads at 24-months, whereas clinicians in the Learning Collaborative reported greater full use of the PCIT protocol at 24-months. Attending consultation calls was associated with delivering PCIT to fewer families. Implications for the sustainable delivery of PCIT beyond the training year as well as for the broader field of implementation science are discussed.

Train-to-Sustain: Predictors of Sustainment in a Large-Scale Implementation of Parent-Child Interaction Therapy

Sustainment of evidence-based practices is necessary to ensure their public health impact. The current study examined predictors of sustainment of Parent-Child Interaction Therapy (PCIT) within a large-scale system-driven implementation effort in Los Angeles County. Data were drawn from PCIT training data and county administrative claims between January 2013 and March 2018. Participants included 241 therapists from 61 programs. Two sustainment outcomes were examined at the therapist- and program-levels: 1) PCIT claim volume and 2) PCIT claim discontinuation (discontinuation of claims during study period; survival time of claiming in months). Predictors included therapist- and program-level caseload, training, and workforce characteristics. On average, therapists and programs continued claiming to PCIT for 17.7 and 32.3 months, respectively. Across the sustainment outcomes, there were both shared and unshared significant predictors. For therapists, case-mix fit (higher proportions of young child clients with externalizing disorders) and participation in additional PCIT training activities significantly predicted claims volume. Furthermore, additional training activity participation was associated with lower likelihood of therapist PCIT claim discontinuation in the follow-up period. Programs with therapists eligible to be internal trainers were significantly less likely to discontinue PCIT claiming. Findings suggest that PCIT sustainment may be facilitated by implementation strategies including targeted outreach to ensure eligible families in therapist caseloads, facilitating therapist engagement in advanced trainings, and building internal infrastructure through train-the-trainer programs.

Making change last? Exploring the value of sustainability approaches in healthcare: a scoping review

Background

Numerous models, tools and frameworks have been produced to improve the sustainability of evidence-based interventions. Due to the vast number available, choosing the most appropriate one is increasingly difficult for researchers and practitioners. To understand the value of such approaches, evidence warranting their use is needed. However, there is limited understanding of how sustainability approaches have been used and how they have impacted research or practice. This review aims to consolidate evidence on the application and impact of sustainability approaches in healthcare settings.

Methods

A systematic scoping review was designed to search for peer-reviewed publications detailing the use of sustainability approaches in practice. A 5-stage framework for scoping reviews directed the search strategy, and quality assessment was performed using the Mixed Method Appraisal Tool. Searches were performed through electronic citation tracking and snowballing of references. Articles were obtained through Web of Science, PubMed and Google Scholar. Six outcome variables for sustainability were explored to ascertain impact of approaches.

Results

This review includes 68 articles demonstrating the application of sustainability approaches in practice. Results show an increase in the use of sustainability approaches in peer-reviewed studies. Approaches have been applied across a range of healthcare settings, including primary, secondary, tertiary and community healthcare. Approaches are used for five main purposes, namely analysis, evaluation, guidance, assessment and planning. Results outline benefits (e.g. improved conceptualisation of sustainability constructs and improved ability to interpret sustainability data) and challenges (e.g. issues with approach constructs and difficulty in application) associated with using a sustainability approach in practice. Few articles (14/68) reported the sustainability outcome variables explored; therefore, the impact of approaches on sustainability remains unclear. Additional sustainability outcome variables reported in retrieved articles are discussed.

Conclusions

This review provides practitioners and researchers with a consolidated evidence base on sustainability approaches. Findings highlight the remaining gaps in the literature and emphasise the need for improved rigour and reporting of sustainability approaches in research studies. To guide future assessment and study of sustainability in healthcare settings an updated list of sustainability outcome variables is proposed.

Trial Registration

This review was registered on the PROSPERO database CRD 42016040081 in June 2016.

Sustaining Treatment for Youth with Problematic Sexual Behavior: Administrator and Stakeholder Perspectives Following Implementation

This study examines administrator and stakeholder perspectives on factors influencing the sustainability of Problematic Sexual Behavior Cognitive Behavioral Therapy (PSB-CBT). After initial implementation of PSB-CBT, qualitative interviews (N = 42) on the sustainability of services were conducted with treatment program administrators (n = 10) and community stakeholders (n = 32) from six sites across the USA. Interviewees discussed key facilitators and barriers for sustainability and identified the following factors as impacting the sustainability of PSB-CBT: public relations, policy, inter-agency collaboration, funding, and workforce issues. Results were overall consistent with the EPIS (Exploration, Preparation, Implementation, Sustainment) model for implementation in public service sectors, but proposed modifications are also noted. Findings underscore the value of qualitative research in identifying best practices for sustaining valuable therapeutic interventions. The authors argue that future efforts to sustain evidence-based interventions, especially those dealing with sensitive topics around which misinformation and stigma are associated, should prioritize public relations (i.e., outreach, education) in addition to the intervention.

What influences evidence-based treatment sustainment after implementation support ends? A mixed method study of the adolescent-community reinforcement approach

BACKGROUND

Little is known about clinician perspectives regarding the factors that support or hinder the long-term delivery (i.e., sustainment) of evidence-based treatments in community-based treatment settings.

METHODS

Clinical staff from 82 community-based treatment organizations that received federal grant funding to support the delivery of the Adolescent Community Reinforcement Approach (A-CRA), an evidence-based treatment for adolescent substance use, were asked to participate in interviews focused on understanding their perspectives about the sustainment of A-CRA. Qualitative themes were identified using inductive and deductive approaches. Then the themes were dichotomized (present/absent) so that quantitative comparisons could be made between staff from organizations that sustained and did not sustain delivery of A-CRA. Administrative data about each organization in relation to federal funding support and their primary focus was also examined to explore whether these characteristics were associated with A-CRA sustainment.

RESULTS

Staff (n = 134) representing 78 organizations participated in the interviews. Staff from organizations that had received multiple federal grants to support the delivery of A-CRA and whose primary focus was substance use rather than other conditions (mental health or primary care) were more likely to report sustaining A-CRA. Staff from sustaining organizations were more likely to report positive grant experiences and success with maintaining both organizational and external support in comparison to staff from non-sustaining organizations. Staff from non-sustaining organizations were more likely to report barriers to sustaining A-CRA, including more challenges with intervention delivery, and lack of internal support and external funding.

CONCLUSIONS

Our findings lend empirical support for implementation theories in that multiple factors appear to be associated with long-term delivery of an evidence-based treatment. Although A-CRA was generally perceived positively by staff from both organizations that sustained A-CRA and organizations that did not sustain A-CRA, inner setting factors (e.g., structural policies, leadership support and staff retention) along with outer setting factors (e.g., external funding support) were reported as key to A-CRA sustainment.

Treatments for Early Childhood Trauma: Decision Considerations for Clinicians

The period from birth to age six represents a time of significant risk for exposure to trauma. Following trauma exposure, children may experience significant negative and lasting psychological, cognitive, and physical effects. Over the last two decades, the demand for and availability of evidence-based treatments (EBTs) for children under the age of six who have experienced trauma has dramatically increased. Three of the most well-supported and widely disseminated EBTs for early childhood trauma are Trauma-Focused Cognitive Behavioral Therapy, Parent-Child Interaction Therapy, and Child-Parent Psychotherapy. Increasingly, clinicians are receiving training in more than one EBT. This paper provides an overview of each intervention; presents clinicians with various child, caregiver, and environmental factors to consider when deciding amongst these three EBTs; and applies these considerations to three composite cases.

Dissemination and Implementation of Cognitive Behavioral Therapy for Depression in the Kaiser Permanente Health Care System: Evaluation of Initial Training and Clinical Outcomes

Notwithstanding its empirical status and strong recommendation in clinical practice guidelines, cognitive behavioral therapy (CBT) continues to be delivered infrequently and with low fidelity on the clinical front lines. Recently, organized efforts and policies within the public sector to disseminate and implement CBT and other evidence-based psychotherapies have yielded encouraging results and provided optimism for bridging the research-to-practice-gap. Following from these efforts, the current article examines the initial impact and experience of the implementation of an individualized approach to CBT training and treatment within the Kaiser Permanente health care system. Initial training outcomes, including changes in general and specific competencies, were assessed using divergent assessment methods within the initial cohort of therapists undergoing training. Initial patient outcomes, including changes in depression and anxiety, were assessed among patients receiving treatment from therapists in training. Results revealed training in and implementation of CBT-D was associated with overall large improvements in therapist competencies and in clinically significant improvements in both depression and anxiety among patients. Findings from the initial phase of dissemination and implementation within a large private system provide support for, and extend recent findings related to, the feasibility and effectiveness of training in and implementation of CBT-D in a real-world context.

Community pharmacy minor ailment services: Pharmacy stakeholder perspectives on the factors affecting sustainability

BACKGROUND

Self-care advice and management of minor ailments have long been provided in community pharmacies across England. However, formal pharmacy minor ailment service provision is geographically variable and has yet to gain recognition and political support as a valued sustainable service for nationwide adoption and commissioning.

OBJECTIVE

To investigate the sustainability potential of pharmacy minor ailment services from the perspective of community pharmacy stakeholders within the North East of England.

METHODS

A mixed methods approach was adopted to survey and interview stakeholders from the North East of England who commission; provide; and/or represent groups influencing the design, delivery and investment in community pharmacy clinical and public health services. The 40-item Programme Sustainability Assessment Tool, a validated instrument to assess a public health programme's capacity for sustainability across eight domains, was administered to fifty-three stakeholders, identified from a pharmacy minor ailments showcase event. The same stakeholders were invited for a semi-structured interview to explore issues further. Interviews were audio-recorded, transcribed verbatim, and underwent framework analysis.

RESULTS

Forty-two (79.2% response rate) stakeholders representing commissioning, provider and influencing (e.g. Local Professional Network) organisations completed the assessment tool. Pharmacy minor ailment services were rated as unsustainable across the majority of the domains. Elements within the domain 'Partnerships' demonstrated potential for sustainability. Stakeholder interviews provided detailed explanation for the low scoring sustainability domains, highlighting the multifaceted challenges threatening these services.

CONCLUSION

The Programme Sustainability Assessment Tool allowed stakeholders to evaluate the potential of pharmacy minor ailment services in England. Follow-up interviews highlighted that initial design and implementation of services was poorly conceived and lacked evidence, thereby impeding the services' sustainability. There are many challenges facing a widespread provision of pharmacy ailment services, but it is clear the profession needs to be clear on the service objectives to secure future interest and investment.