The Organizational Financial Context of Publicly-Funded Mental Health Clinics: Development and Preliminary Psychometric Evaluation of the Agency Financial Status Scales

Funding is a major barrier to implementation of evidence-based practices (EBPs) in publicly-funded community mental health clinics (CMHCs). Understanding how best to deploy implementation strategies that address this barrier requires greater clarity on the financial context within agencies. We developed the Agency Financial Status Scales (AFSS) to assess employee perceptions of the level of three hypothesized and theoretical funding related constructs in organizations: (a) perceptions of financial health, (b) financial attitudes toward EBPs, and (c) strategic financial climate. This investigation serves as a preliminary evaluation of this measure. Participants were 239 therapists and 40 supervisors from 25 publicly-funded CMHCs providing outpatient mental health services for young people. Confirmatory factor analysis was used to investigate the latent trait structure of the items. Internal consistency, interrater agreement, concordance between therapists and supervisors, and convergent validity were also examined. A two-factor model measuring perceptions of financial health and strategic financial climate best fit the data. For both of these scales, alpha reliability was acceptable and agreement statistics provided moderate support for aggregation at the organizational level. Analyses supported the convergent validity of the scales. The development and preliminary evaluation of the AFSS is an important first step in understanding the financial context of publicly-funded CMHCs. Though findings from this investigation are promising, additional development and testing are needed to develop a more thorough understanding of the constructs and to improve the validity and reliability of this measure.

Ethnography and user-centered design to inform context-driven implementation

Despite pervasive findings pointing to its inextricable role in intervention implementation, context remains poorly understood in implementation science. Existing approaches for describing context (e.g., surveys, interviews) may be narrow in scope or superficial in their elicitation of contextual data. Thus, in-depth and multilevel approaches are needed to meaningfully describe the contexts into which interventions will be implemented. Moreover, many studies assess context without subsequently using contextual information to enhance implementation. To be useful for improving implementation, though, methods are needed to apply contextual information during implementation. In the case example presented in this paper, we embedded an ethnographic assessment of context within a user-centered design approach to describe implementation context and apply that information to promote implementation. We developed a patient-reported outcome measure-based clinical intervention to assess and address the pervasive unmet needs of young adults with cancer: the Needs Assessment & Service Bridge (NA-SB). In this paper, we describe the user-centered design process that we used to anticipate context modifications needed to deliver NA-SB and implementation strategies needed to facilitate its implementation. Our ethnographic contextual inquiry yielded a rich understanding of local implementation context and contextual variation across potential scale-up contexts. Other methods from user-centered design (i.e., translation tables and a design team prototyping workshop) allowed us to translate that information into specifications for NA-SB delivery and a plan for implementation. Embedding ethnographic methods within a user-centered design approach can help us to tailor interventions and implementation strategies to their contexts of use to promote implementation.

Harmonizing evidence-based practice, implementation context, and implementation strategies with user-centered design: a case example in young adult cancer care

BACKGROUND

Attempting to implement evidence-based practices in contexts for which they are not well suited may compromise their fidelity and effectiveness or burden users (e.g., patients, providers, healthcare organizations) with elaborate strategies intended to force implementation. To improve the fit between evidence-based practices and contexts, implementation science experts have called for methods for adapting evidence-based practices and contexts and tailoring implementation strategies; yet, methods for considering the dynamic interplay among evidence-based practices, contexts, and implementation strategies remain lacking. We argue that harmonizing the three can be facilitated by user-centered design, an iterative and highly stakeholder-engaged set of principles and methods.

METHODS

This paper presents a case example in which we used a three-phase user-centered design process to design and plan to implement a care coordination intervention for young adults with cancer. Specifically, we used usability testing to redesign and augment an existing patient-reported outcome measure that served as the basis for our intervention to optimize its usability and usefulness, ethnographic contextual inquiry to prepare the context (i.e., a comprehensive cancer center) to promote receptivity to implementation, and iterative prototyping workshops with a multidisciplinary design team to design the care coordination intervention and anticipate implementation strategies needed to enhance contextual fit.

RESULTS

Our user-centered design process resulted in the Young Adult Needs Assessment and Service Bridge (NA-SB), including a patient-reported outcome measure and a collection of referral pathways that are triggered by the needs young adults report, as well as implementation guidance. By ensuring NA-SB directly responded to features of users and context, we designed NA-SB for implementation, potentially minimizing the strategies needed to address misalignment that may have otherwise existed. Furthermore, we designed NA-SB for scale-up; by engaging users from other cancer programs across the country to identify points of contextual variation which would require flexibility in delivery, we created a tool intended to accommodate diverse contexts.

CONCLUSIONS

User-centered design can help maximize usability and usefulness when designing evidence-based practices, preparing contexts, and informing implementation strategies-in effect, harmonizing evidence-based practices, contexts, and implementation strategies to promote implementation and effectiveness.

Evaluating the dissemination and scale-up of two evidence-based parenting interventions to reduce violence against children: study protocol

Background

Eliminating violence against children is a prominent policy goal, codified in the Sustainable Development Goals, and parenting programs are one approach to preventing and reducing violence. However, we know relatively little about dissemination and scale-up of parenting programs, particularly in low- and middle-income countries (LMICs). The scale-up of two parenting programs, Parenting for Lifelong Health (PLH) for Young Children and PLH for Parents and Teens, developed under Creative Commons licensing and tested in randomized trials, provides a unique opportunity to study their dissemination in 25 LMICs.

Methods

The Scale-Up of Parenting Evaluation Research (SUPER) study uses a range of methods to study the dissemination of these two programs. The study will examine (1) process and extent of dissemination and scale-up, (2) how the programs are implemented and factors associated with variation in implementation, (3) violence against children and family outcomes before and after program implementation, (4) barriers and facilitators to sustained program delivery, and (5) costs and resources needed for implementation.Primary data collection, focused on three case study projects, will include interviews and focus groups with program facilitators, coordinators, funders, and other stakeholders, and a summary of key organizational characteristics. Program reports and budgets will be reviewed as part of relevant contextual information. Secondary data analysis of routine data collected within ongoing implementation and existing research studies will explore family enrolment and attendance, as well as family reports of parenting practices, violence against children, child behavior, and child and caregiver wellbeing before and after program participation. We will also examine data on staff sociodemographic and professional background, and their competent adherence to the program, collected as part of staff training and certification.

Discussion

This project will be the first study of its kind to draw on multiple data sources and methods to examine the dissemination and scale-up of a parenting program across multiple LMIC contexts. While this study reports on the implementation of two specific parenting programs, we anticipate that our findings will be of relevance across the field of parenting, as well as other violence prevention and social programs.

NIATx-TI versus typical product training on e-health technology implementation: a clustered randomized controlled trial study protocol

BACKGROUND

Substance use disorders (SUDs) lead to tens-of-thousands of overdose deaths and other forms of preventable deaths in the USA each year. This results in over $500 billion per year in societal and economic costs as well as a considerable amount of grief for loved ones of affected individuals. Despite these health and societal consequences, only a small percentage of people seek treatment for SUDs, and the majority of those that seek help fail to achieve long-term sobriety. E-health applications in healthcare have proven to be effective at sustaining treatment and reaching patients traditional treatment pathways would have missed. However, e-health adoption and sustainment rates in healthcare are poor, especially in the SUD treatment sector. Implementation engineering can address this gap in the e-health field by augmenting existing implementation models, which explain organizational and individual e-health behaviors retrospectively, with prospective resources that can guide implementation.

METHODS

This cluster randomized control trial is designed to test two implementation strategies at adopting an evidence-based mobile e-health technology for SUD treatment. The proposed e-health implementation model is the Network for the Improvement of Addiction Treatment-Technology Implementation (NIATx-TI) Framework. This project, based in Iowa, will compare a control condition (using a typical software product training approach that includes in-person staff training followed by access to on-line support) to software implementation utilizing NIATx-TI, which includes change management training, followed by coaching on how to implement and use the mobile application. While e-health spans many modalities and health disciplines, this project will focus on implementing the Addiction Comprehensive Health Enhancement Support System (A-CHESS), an evidence-based SUD treatment recovery app framework. This trial will be conducted in Iowa at 46 organizational sites within 12 SUD treatment agencies. The control arm consists of 23 individual treatment sites based at five organizations, and the intervention arm consists of 23 individual SUD treatment sites based at seven organizations DISCUSSION: This study addresses an issue of substantial public health significance: enhancing the uptake of the growing inventory of patient-centered evidence-based addiction treatment e-health technologies.

TRIAL REGISTRATION

ClinicalTrials.gov , NCT03954184 . Posted 17 May 2019.

Occupational Therapists' Implementation of Evidence-Based Practice: A Cross Sectional Survey

Evidence-based practice (EBP) implementation has been associated with quality health care outcomes, yet only a small proportion of occupational therapists implement EBP. We conducted a cross-sectional survey measuring EBP implementation, skill and knowledge, self-reflection behavior, and practice demographics to clarify implementation of EBP and factors acting as supports. A random sample of occupational therapists practicing in the U.S. was invited to participate; 578 surveys were included in the final analysis. Participants implemented EBP less than 1 time over the previous 8 weeks; EBP use was highest in those with a doctorate (p = .002) and high self-reflection behavior (p = .001); EBP use was marginally higher in those with organizational support (p = .008; p = .009), time for EBP (p = .009), and access to full text articles (p = .006). Further investigation of how reflective practice and levels of doctoral education support EBP use are recommended.

Therapist-Observer Concordance in Ratings of EBP Strategy Delivery: Challenges and Targeted Directions in Pursuing Pragmatic Measurement in Children's Mental Health Services

Pragmatic measures of therapist delivery of evidence-based practice (EBP) are critical to assessing the impact of large-scale, multiple EBP implementation efforts. As an initial step in the development of pragmatic measurement, the current study examined the concordance between therapist and observer ratings of items assessing delivery of EBP strategies considered essential for common child EBP targets. Possible EBP-, session-, and therapist-levels factors associated with concordance were also explored. Therapists and independent observers rated the extensiveness of therapist (n = 103) EBP strategy delivery in 680 community psychotherapy sessions in which six EBPs were used. Concordance between therapist- and observer-report of the extensiveness of therapist EBP strategy use was at least fair (ICC ≥ .40) for approximately half of the items. Greater therapist-observer concordance was observed in sessions where a structured EBP was delivered and in sessions where therapists reported being able to carry out planned activities. Findings highlighted conditions that may improve or hinder therapists' ability to report on their own EBP strategy delivery in a way that is consistent with independent observers. These results can help inform the development of pragmatic therapist-report measures of EBP strategy delivery and implementation efforts more broadly.

Test of a workforce development intervention to expand opioid use disorder treatment pharmacotherapy prescribers: protocol for a cluster randomized trial

BACKGROUND

Overdoses due to non-medical use of prescription opioids and other opiates have become the leading cause of accidental deaths in the USA. Buprenorphine and extended-release naltrexone are key evidence-based pharmacotherapies available to addiction treatment providers to address opioid use disorder (OUD) and prevent overdose deaths. Treatment organizations' efforts to provide these pharmacotherapies have, however, been stymied by limited success in recruiting providers (physicians, nurse practitioners, and physician assistants) to prescribe these medications. Historically, the addiction treatment field has not attracted physicians, and many barriers to implementing OUD pharmacotherapy exist, ranging from lack of confidence in treating OUD patients to concerns regarding reimbursement. Throughout the USA, the prevalence of OUD far exceeds the capacity of the OUD pharmacotherapy treatment system. Poor access to OUD pharmacotherapy prescribers has become a workforce development need for the addiction treatment field and a significant health issue.

METHODS

This cluster randomized controlled trial (RCT) is designed to increase buprenorphine and extended-release naltrexone treatment capacity for OUD. The implementation intervention to be tested is a bundle of OUD pharmacotherapy capacity building practices called the Prescriber Recruitment Bundle (PRB), which was developed and piloted in a previous statewide buprenorphine implementation study. For this cluster RCT, organizational sites will be recruited and then randomized into one of two arms: (1) control, with treatment as usual and access to a website with PRB resources, or (2) intervention, with organizations implementing the PRB using the Network for the Improvement of Addiction Treatment organizational change model over a 24-month intervention period and a 10-month sustainability period. The primary treatment outcomes for each organizational site are self-reported monthly counts of buprenorphine slots, extended-release naltrexone capacity, number of buprenorphine patients, and number of extended-release naltrexone patients. This trial will be conducted in Florida, Ohio, and Wisconsin, resulting in 35 sites in each arm, for a total sample size of 70 organizations.

DISCUSSION

This study addresses three issues of substantial public health significance: (1) the pressing opioid misuse epidemic, (2) the low uptake of OUD treatment pharmacotherapies, and (3) the need to increase prescriber participation in the addiction treatment workforce.

TRIAL REGISTRATION

ClinicalTrials.gov NCT02926482.

Juvenile Justice-Translational Research on Interventions for Adolescents in the Legal System (JJ-TRIALS): a cluster randomized trial targeting system-wide improvement in substance use services

Background

The purpose of this paper is to describe the Juvenile Justice—Translational Research on Interventions for Adolescents in the Legal System (JJ-TRIALS) study, a cooperative implementation science initiative involving the National Institute on Drug Abuse, six research centers, a coordinating center, and Juvenile Justice Partners representing seven US states. While the pooling of resources across centers enables a robust implementation study design involving 36 juvenile justice agencies and their behavioral health partner agencies, co-producing a study protocol that has potential to advance implementation science, meets the needs of all constituencies (funding agency, researchers, partners, study sites), and can be implemented with fidelity across the cooperative can be challenging. This paper describes (a) the study background and rationale, including the juvenile justice context and best practices for substance use disorders, (b) the selection and use of an implementation science framework to guide study design and inform selection of implementation components, and (c) the specific study design elements, including research questions, implementation interventions, measurement, and analytic plan.

Methods/design

The JJ-TRIALS primary study uses a head-to-head cluster randomized trial with a phased rollout to evaluate the differential effectiveness of two conditions (Core and Enhanced) in 36 sites located in seven states. A Core strategy for promoting change is compared to an Enhanced strategy that incorporates all core strategies plus active facilitation. Target outcomes include improvements in evidence-based screening, assessment, and linkage to substance use treatment.

Discussion

Contributions to implementation science are discussed as well as challenges associated with designing and deploying a complex, collaborative project.

Trial registration

NCT02672150.