Assessing and changing organizational social contexts for effective mental health services

A Comparison of the Readiness of Youth Service Agencies to Implement a Technology-Based Toolkit to Support Treatment Delivery

Implementation initiatives and technology-based resources aim to address barriers to Evidence-Based Practice (EBP) use by creating generalizable techniques that can be used for a variety of youth-serving agencies. However, research has not carefully examined unique differences between agency types or individual programs in readiness to use such technologies and implementation strategies. The current study explored differences between community mental health clinics and child advocacy centers on organizational cultural factors (e.g., ability to change and commitment for change) to implement a novel technology-based toolkit to support delivery of Trauma Focused Cognitive Behavioral Therapy (TF-CBT). Results indicated that TF-CBT providers from child advocacy centers reported greater commitment to change and more support to use the technology-based system than those from community mental health centers. Findings suggest that implementation initiatives should address the needs of individual agencies and service settings and adaptations should be explored to best meet the needs of these settings.

Association of organizational culture and climate with variation in the clinical outcomes of collaborative care for maternal depression in community health centers

Background

Organizational factors may help explain variation in the effectiveness of evidence-based clinical innovations through implementation and sustainment. This study tested the relationship between organizational culture and climate and variation in clinical outcomes of the Collaborative Care Model (CoCM) for treatment of maternal depression implemented in community health centers.

Method

Organizational cultures and climates of 10 community health centers providing CoCM for depression among low-income women pregnant or parenting were assessed using the organizational social context (OSC) measure. Three-level hierarchical linear models tested whether variation in culture and climate predicted variation in improvement in depression symptoms from baseline to 6.5-month post-baseline for N = 468 women with care ±1 year of OSC assessment. Depression symptomology was measured using the Patient Health Questionnaire (PHQ-9).

Results

After controlling for patient characteristics, case mix, center size, and implementation support, patients served by centers with more proficient cultures improved significantly more from baseline to 6.5-month post-baseline than patients in centers with less proficient cultures (mean improvement = 5.08 vs. 0.14, respectively, p = .020), resulting in a large adjusted effect size of dadj = 0.78. A similar effect was observed for patients served by centers with more functional climates (mean improvement = 5.25 vs. 1.12, p < .044, dadj = 0.65). Growth models indicated that patients from all centers recovered on average after 4 months of care. However, those with more proficient cultures remained stabilized whereas patients served by centers with less proficient cultures deteriorated by 6.5-month post-baseline. A similar pattern was observed for functional climate.

Conclusions

Variation in clinical outcomes for women from historically underserved populations receiving Collaborative Care for maternal depression was associated with the organizational cultures and climates of community health centers. Implementation strategies targeting culture and climate may improve the implementation and effectiveness of integrated behavioral health care for depression.

Mental health services need action on organisational culture and justice

OBJECTIVE

A commentary on the usefulness of the concepts of organisational culture, organisational climate and justice on the quality and safety of mental health services and how conditions may be improved.

CONCLUSIONS

Organisational culture, organisational climate and justice impact upon the quality and safety of care, as well as well-being of staff, in mental health services. Psychiatrists and trainees, should consider, act and advocate for improved organisational culture, climate and justice. Improvement in these organisational domains can be achieved, through a coordinated framework that acts across all levels of administration, mid-level management and frontline clinical staff.

Promises and pitfalls in implementation science from the perspective of US-based researchers: learning from a pre-mortem

BACKGROUND

Implementation science is at a sufficiently advanced stage that it is appropriate for the field to reflect on progress thus far in achieving its vision, with a goal of charting a path forward. In this debate, we offer such reflections and report on potential threats that might stymie progress, as well as opportunities to enhance the success and impact of the field, from the perspective of a group of US-based researchers.

MAIN BODY

Ten mid-career extramurally funded US-based researchers completed a "pre-mortem" or a group brainstorming exercise that leverages prospective hindsight to imagine that an event has already occurred and to generate an explanation for it - to reduce the likelihood of a poor outcome. We came to consensus on six key themes related to threats and opportunities for the field: (1) insufficient impact, (2) too much emphasis on being a "legitimate science," (3) re-creation of the evidence-to-practice gap, (4) difficulty balancing accessibility and field coherence, (5) inability to align timelines and priorities with partners, and (6) overly complex implementation strategies and approaches.

CONCLUSION

We submit this debate piece to generate further discussion with other implementation partners as our field continues to develop and evolve. We hope the key opportunities identified will enhance the future of implementation research in the USA and spark discussion across international groups. We will continue to learn with humility about how best to implement with the goal of achieving equitable population health impact at scale.

The question of the question: impactful implementation science to address the HIV epidemic

INTRODUCTION

Questions about the implementation of evidence-based intervention to treat and prevent HIV have risen to the top of the field's scientific priorities. Despite the availability of highly efficacious treatment and prevention interventions, impact has fallen short of targets because these interventions are used with insufficient reach, consistency, sustainability and equity in diverse real-world settings. At present, substantial excitement for implementation science - defined as research methods and strategies to improve use of evidence-based interventions - has focused on developing and disseminating methods to conduct rigorous research. Yet, impactful answers depend on a sometimes less visible, but even more important, step: asking good questions about implementation.

DISCUSSION

In this commentary, we offer several considerations for researchers formulating implementation research questions based on several distinctive features of the field. First, as findings are used not only by other researchers but by implementers, scientific questions must incorporate a range of stakeholder and community perspectives to be most relevant. Second, real-world settings are contextually diverse, and the most relevant scientific questions must position answers to make sense within these contexts (whether geographical, organizational and sociological), rather than apart from them. Third, implementation is complex and dynamic; consequently, research questions must make use of emerging standards in describing implementation strategies and their effects whenever possible. Finally, the field of implementation science continues to evolve, so framing problems with a diverse disciplinary lens will enable researchers to pose insightful and impactful questions.

CONCLUSIONS

We are now at a juncture marked by both rich evidence-based interventions and a persistent global pandemic. To achieve continued scientific progress against the HIV epidemic, asking the right questions might be part of the answer itself.

Launching a Competency-Based Training Program in Evidence-Based Treatments for PTSD: Supporting Veteran-Serving Mental Health Providers in Texas

Community mental health providers play an essential role in delivering services to veterans who either have limited access to U.S. Department of Veterans Affairs (VA) facilities or who prefer to seek care outside of the VA. However, there are limited training opportunities in evidence-based treatments for posttraumatic stress disorder (PTSD) outside of the VA. In 2017, the STRONG STAR Training Initiative was established to develop competency-based training in two evidence-based therapies for PTSD and to provide that training for mental health providers serving veterans and their families in community settings in Texas. This article describes the program's development and implementation, baseline characteristics of participating clinicians, and lessons learned toward the scale-up and extension of this competency-based training effort to include other interventions and locations.

Rapid, Full-Scale Change to Virtual PCIT During the COVID-19 Pandemic: Implementation and Clinical Implications

Health agencies call for the immediate mobilization of existing interventions in response to numerous child and family mental health concerns that have arisen as result of the COVID-19 pandemic. Answering this call, this pilot study describes the rapid, full-scale change from a primarily clinic-based Parent–Child Interaction Therapy (PCIT) model to a virtual service model (i.e., I-PCIT) in an academic and community-based program in Miami, Florida. First, we describe the virtual service training model our program developed and its implementation with 17 therapists (M_Age = 32.35, 88.2% female, 47.1% Hispanic) to enable our clinic to shift from providing virtual services to a small portion of the families served (29.1%) to all of the families served. Second, we examine the effect of I-PCIT on child and caregiver outcomes during the 2-month stay-at-home period between March 16, 2020, and May 16, 2020, in 86 families (M_ChildAge = 4.75, 71% Hispanic). Due to the rapid nature of the current study, all active participants were transferred to virtual services, and therefore there was no comparison or control group, and outcomes represent the most recently available scores and not treatment completion. Results reveal that I-PCIT reduced child externalizing and internalizing problems and caregiver stress, and increased parenting skills and child compliance with medium to large effects even in the midst of the COVID-19 pandemic. Finally, the study examined components of our virtual service training model associated with the greatest improvements in child and caregiver outcomes. Preliminary findings revealed that locally and collaboratively developed strategies (e.g., online communities of practice, training videos and guides) had the strongest association with child and caregiver outcomes. Implications for virtual service delivery, implementation, and practice in the midst of the COVID-19 pandemic are discussed.

Incorporators, Early Investors, and Learners: a longitudinal study of organizational adaptation during EBP implementation and sustainment

BACKGROUND

The majority of literature on evidence-based practice (EBP) adaptation focuses on changes to clinical practices without explicitly addressing how organizations must adapt to accommodate a new EBP. This study explores the process of organizational-rather than EBP-adaptation during implementation and sustainment. To the authors' knowledge, there are no previous implementation studies that focus on organizational adaptation in this way.

METHODS

This analysis utilizes a case study approach to examine longitudinal qualitative data from 17 community-based organizations (CBOs) in one state and seven county-based child welfare systems. The CBOs had sustained a child-neglect intervention EBP (SafeCare®) for 2 to 10 years. The unit of analysis was the organization, and each CBO represented a case. Organizational-level profiles were created to describe the organizational adaptation process.

RESULTS

Three organizational-level adaptation profiles were identified as follows: incorporators (n = 7), early investors (n = 6), and learners (n = 4). Incorporators adapted by integrating SafeCare into existing operations to meet contractual or EBP fidelity requirements. Early Investors made substantial organizational adaptations during the early implementation period, then operated relatively consistently as the EBP became embedded in the organization and service system. Learners were characterized by steady and continuous attention to new ways that the organization could adapt to support the EBP.

CONCLUSION

The profiles demonstrated that there can be multiple effective paths to EBP sustainment. Organizational adaptation was calibrated to fit a CBO's operations (e.g., size of the program) and immediate environmental constraints (e.g., funding levels). Additionally, organizations fulfilled different functional roles in the network of entities involved in EBP implementation. Knowing organizational roles and adaptation profiles can guide implementation planning and help to structure contract designs that bridge the outer (system) and inner (organizational) contexts. Adaptation profiles can also inform the intensity of the implementation strategy tailoring process and the way that strategies are marketed to organizations.

Getting to the table: Agency characteristics and evidence-based intervention adoption in children's mental health care

Scaling evidence-based interventions (EBI) for children and families across healthcare systems can expand public health impact. Research has identified EBI adoption determinants. However, less understood are characteristics of agencies that opt in across the stages of adoption. This study examined the relationship between agency (N=69) characteristics (e.g., revenue) and four adoption stages during a large-scale trial of an EBI for children with significant behavioral difficulties and their families. 48 (70%) of agencies demonstrated interest, 28 (41%) scheduled an informational meeting, 20 (29%) received training, and 16 (22%) demonstrated EBI uptake. Analyses indicated no differences in characteristics and initial interest. However, agencies with small-sized revenue had significantly reduced odds at other adoption stages. Implications for strategies to bring EBI access to scale are discussed.

Impact of Stressful Climates on Provider Perceptions of Integrated Behavioral Health Services in Pediatric Primary Care: An Exploratory Study

OBJECTIVE

Pediatric primary care providers (PCPs) work in challenging environments and are increasingly called to implement complex interventions, such as behavioral health (BH) service integration. We explore how perceived stressful practice climates (1) change over time in and (2) influence provider perceptions of collaborative care versus usual care, 2 models of integrated BH care.

METHODS

Secondary exploratory analysis using hierarchical linear modeling was performed on an 18-month cluster-randomized trial of 8 pediatric primary care practices to Doctor-Office Collaborative Care (DOCC), where an on-site care manager delivered BH services in coordination with PCPs, or Enhanced Usual Care (EUC), where a care manager facilitated referrals to local BH providers. Various indicators of PCP perceptions of BH services, including satisfaction with practice, burdens and beliefs regarding psychosocial problems, and effectiveness in treating behavioral problems, were assessed as outcomes. Moderators were 2 domains of stressful climates, role conflict and role overload.

RESULTS

Role conflict and role overload stayed stable in both conditions. Role conflict strengthened the positive effect of DOCC on PCP perceived effectiveness in treating behavioral problems (β [SE], 0.04 [0.02]; p = 0.04) and improvement in managing oppositional/aggressive behavior (0.02 [0.01]; p = 0.02). Role overload strengthened the positive effect of DOCC on PCP-perceived improvement in managing attention-deficit hyperactivity disorder (0.03 [0.01]; p = 0.01). Stressful climates did not influence perceptions for EUC providers.

CONCLUSIONS

Providers experiencing more stressful practice climates developed more positive perceptions of collaborative care. This may encourage stressed providers to make effective practice changes and promote practice integration of BH services.

Effectiveness of 1-day trauma-informed care training programme on attitudes in psychiatric hospitals: A pre-post study

Many patients in mental health settings are likely to have histories of interpersonal traumatic experiences. Mental health providers are recommended to adopt trauma-informed care (TIC) to ensure sensitivity and responsiveness to the impact of trauma on patients. However, few studies have examined the effectiveness of a TIC training programme using standardized measures with follow-up assessments. The aim of the study was to evaluate the effects of a TIC training programme on attitudes towards TIC in mental health professionals. The study involved a pre-post design with 3-month follow-up assessments conducted between March and June 2018. In total, 65 mental health professionals from 29 psychiatric hospitals in Tokyo and its suburban prefectures participated in the study. Mental health professionals participated in a 1-day programme consisting of a 3.5-hour lecture and 1-hour group discussion. Development of favourable attitudes towards TIC was the primary outcome, as assessed by using the Attitude Related Trauma-Informed Care scale. The majority of participants were women (86%), and the mean age was 42.2 years. The mean score of the Attitude Related Trauma-Informed Care scale scores increased significantly from 5.1 during pre-training to 5.5 immediately after training (mean difference: 0.4; 95% confidence interval: 0.3-0.5) and 5.4 after 3 months (mean difference: 0.3; 95% confidence interval: 0.2-0.4). Furthermore, half of the participants claimed to have implemented TIC practice in daily clinical settings at the 3-month follow-up. These results suggested that this brief TIC training programme improved attitudes towards TIC practice significantly.

Annual Research Review: The state of implementation science in child psychology and psychiatry: a review and suggestions to advance the field

BACKGROUND

Scientists have developed evidence-based interventions that improve the symptoms and functioning of youth with psychiatric disorders; however, these interventions are rarely used in community settings. Eliminating this research-to-practice gap is the purview of implementation science, the discipline devoted to the study of methods to promote the use of evidence-based practices in routine care.

METHODS

We review studies that have tested factors associated with implementation in child psychology and psychiatry, explore applications of social science theories to implementation, and conclude with recommendations to advance implementation science through the development and testing of novel, multilevel, causal theories.

RESULTS

During its brief history, implementation science in child psychology and psychiatry has documented the implementation gap in routine care, tested training approaches and found them to be insufficient for behavior change, explored the relationships between variables and implementation outcomes, and initiated randomized controlled trials to test implementation strategies. This research has identified targets related to implementation (e.g., clinician motivation, organizational culture) and demonstrated the feasibility of activating these targets through implementation strategies. However, the dominant methodological approach has been atheoretical and predictive, relying heavily on a set of variables from heuristic frameworks.

CONCLUSIONS

Optimizing the implementation of effective treatments in community care for youth with psychiatric disorders is a defining challenge of our time. This review proposes a new direction focused on developing and testing integrated causal theories. We recommend implementation scientists: (a) move from observational studies of implementation barriers and facilitators to trials that include causal theory; (b) identify a core set of implementation determinants; (c) conduct trials of implementation strategies with clear targets, mechanisms, and outcomes; (d) ensure that behaviors that are core to EBPs are clearly defined; and (e) agree upon standard measures. This agenda will help fulfill the promise of evidence-based practice for improving youth behavioral health.

Cesarean overuse and the culture of care

OBJECTIVE

To assess hospital unit culture and clinician attitudes associated with varying rates of primary cesarean delivery.

DATA SOURCES/STUDY SETTING

Intrapartum nurses, midwives, and physicians recruited from 79 hospitals in California participating in efforts to reduce cesarean overuse.

STUDY DESIGN

Labor unit culture and clinician attitudes measured using a survey were linked to the California Maternal Data Center for birth outcomes and hospital covariates.

METHODS

Association with primary cesarean delivery rates was assessed using multivariate Poisson regression adjusted for hospital covariates.

PRINCIPAL FINDINGS

1718 respondents from 70 hospitals responded to the Labor Culture Survey. The "Unit Microculture" subscale was strongly associated with primary cesarean rate; the higher a unit scored on 8-items describing a culture supportive of vaginal birth (eg, nurses are encouraged to spend time in rooms with patients, and doulas are welcomed), the cesarean rate decreased by 41 percent (95% CI = -47 to -35 percent, P < 0.001). Discordant attitudes between nurses and physicians were associated with increased cesarean rates.

CONCLUSIONS

Hospital unit culture, clinician attitudes, and consistency between professions are strongly associated with primary cesarean rates. Improvement efforts to reduce cesarean overuse must address culture of care as a key part of the change process.

Training transfer: a systematic review of the impact of inner setting factors

Consistent with Baldwin and Ford's model (Pers Psychol 41(1):63-105, 1988), training transfer is defined as the generalization of learning from a training to everyday practice in the workplace. The purpose of this review was to examine the influence of work-environment factors, one component of the model hypothesized to influence training transfer within behavioral health. An electronic literature search guided by the Consolidated Framework for Implementation Research's inner setting domain was conducted was conducted on Medline OVID, Medline EMBASE, and PsycINFO databases. Of 9184 unique articles, 169 full-text versions of articles were screened for eligibility, yielding 26 articles meeting inclusion criteria. Results from the 26 studies revealed that overall, having more positive networks and communication, culture, implementation climate, and readiness for implementation can facilitate training transfer. Although few studies have examined the impact of inner setting factors on training transfer, these results suggest organizational context is important to consider with training efforts. These findings have important implications for individuals in the broader health professions educational field.

Organizational culture and climate profiles: relationships with fidelity to three evidence-based practices for autism in elementary schools

BACKGROUND

Implementation researchers have typically studied organizational culture and climate by testing whether individual dimensions are linked to the implementation of evidence-based practices (EBPs) rather than examining how the overarching social context influences implementation. This approach may limit implementation theory and strategy development to the extent that individual dimensions of culture and climate interact, mutually reinforce or counteract one another, or exhibit non-linear relationships. This study tests whether empirically identifiable culture and climate profiles emerge in a sample of organizations and examines how these profiles relate to EBP fidelity and work attitudes that support EBP sustainment, focusing on three EBPs for youth with autism delivered in schools as an example.

METHODS

The study included 65 elementary schools in the U.S. that implemented three EBPs-discrete trial training, pivotal response training, and visual schedules-for youth with autism. Organizational culture and climate and work attitudes were assessed using the Organizational Social Context measure at the beginning of the school year. Observations of EBP fidelity occurred mid school-year. We used bias-adjusted stepwise latent profile modeling to (1) identify subpopulations of schools that share similar culture and climate profiles, and (2) test for mean differences across profiles in observed EBP fidelity and teacher and staff work attitudes.

RESULTS

Controlling for region, four profiles best characterized the organizational cultures and climates of schools. Teachers and staff in schools with a comprehensive profile (high proficiency culture, positive climate) exhibited higher fidelity to two of three EBPs (d's = .95 to 1.64) and reported superior work attitudes (d's = .71 to 1.93) than teachers and staff in all other schools. Teachers and staff in supportive schools (low rigidity culture, positive climate) had better work attitudes, but not better fidelity, than those in schools with indifferent (low culture/climate, elevated stress) and constrained (high rigidity and resistance, high stress) profiles.

CONCLUSIONS

Organizational culture and climate profiles are a strong predictor of EBP fidelity and work attitudes that support EBP sustainment, highlighting the importance of an organization's overarching social context when developing implementation theory and strategies. Strategies that foster a comprehensive profile may improve EBP implementation.

The Cost of Quality: Organizational Financial Health and Program Quality

Purpose: Nonprofit health and human service organizations (NPHSOs) encounter pressure to improve program quality to continue to secure financial resources. Yet, organizations also struggle to innovate, evaluate, and monitor quality absent sufficient resources. This study explores the relationship between NPHSOs' financial health and quality indicators. Methods: Quality indicators were measured in an online survey with leaders from 32 nonprofit children's behavioral health organizations within a Midwestern urban region. Survey responses were matched with financial health metrics extracted from publicly available IRS 990 forms from 2010 to 2014. Results: Organizations struggled to maintain adequate reserve levels, although those with more quality capacities had greater assets and growth in those assets. Discussion: Although financial health appeared to be related to quality, it is unclear whether quality attracts financial resources, or if financial reserves support quality. Conclusion: Continuing research and dialogue are needed to understand the financial implications of an increasing emphasis on quality.

Connection between organizational culture and climate and empowerment: The perspective of nurse managers

Health-care organizations differ from other institutions due to their unique structure and management. Organizational culture and climate are the key constructs that compose the organizational social context, and might have an impact on employees, such as nurse managers, and the organization itself. The aim of this study was to analyze the connections between organizational culture and climate and work-related empowerment, and also to present culture and climate profiles at team and organization levels. The research was carried out in Lithuania among 193 nurse managers in seven hospitals using a questionnaire that measured organizational culture and climate (Organizational Social Context instrument), and work-related empowerment (Conditions for Work Effectiveness Questionnaire-II and Work Empowerment Questionnaire). The findings showed that nurse managers were both structurally and psychologically empowered when the organizational culture was proficient and resistant, and the climate was engaged and functional. These results suggest that nurse managers are more likely to be empowered when there is an appropriate organizational culture and climate in the workplace. Creating an empowering work environment might have a positive impact on individual and organizational outcomes.

The Role of Practice Research Networks (PRN) in the Development and Implementation of Evidence: The Northern Improving Access to Psychological Therapies PRN Case Study

Practice research networks (PRNs) can support the implementation of evidence based practice in routine services and generate practice based evidence. This paper describes the structure, processes and learning from a new PRN in the Improving Access to Psychological Therapies programme in England, in relation to an implementation framework and using one study as a case example. Challenges related to: ethics and governance processes; communications with multiple stakeholders; competing time pressures and linking outcome data. Enablers included: early tangible outputs and impact; a collaborative approach; engaging with local research leads; clarity of processes; effective dissemination; and committed leadership.

Effectiveness of a multi-level implementation strategy for ASD interventions: study protocol for two linked cluster randomized trials

BACKGROUND

The Centers for Disease Control (2018) estimates that 1 in 59 children has autism spectrum disorder, and the annual cost of ASD in the U.S. is estimated to be $236 billion. Evidence-based interventions have been developed and demonstrate effectiveness in improving child outcomes. However, research on generalizable methods to scale up these practices in the multiple service systems caring for these children has been limited and is critical to meet this growing public health need. This project includes two, coordinated studies testing the effectiveness of the Translating Evidence-based Interventions (EBI) for ASD: Multi-Level Implementation Strategy (TEAMS) model. TEAMS focuses on improving implementation leadership, organizational climate, and provider attitudes and motivation in order to improve two key implementation outcomes-provider training completion and intervention fidelity and subsequent child outcomes. The TEAMS Leadership Institute applies implementation leadership strategies and TEAMS Individualized Provider Strategies for training applies motivational interviewing strategies to facilitate provider and organizational behavior change.

METHODS

A cluster randomized implementation/effectiveness Hybrid, type 3, trial with a dismantling design will be used to understand the effectiveness of TEAMS and the mechanisms of change across settings and participants. Study #1 will test the TEAMS model with AIM HI (An Individualized Mental Health Intervention for ASD) in publicly funded mental health services. Study #2 will test TEAMS with CPRT (Classroom Pivotal Response Teaching) in education settings. Thirty-seven mental health programs and 37 school districts will be randomized, stratified by county and study, to one of four groups (Standard Provider Training Only, Standard Provider Training + Leader Training, Enhanced Provider Training, Enhanced Provider Training + Leader Training) to test the effectiveness of combining standard, EBI-specific training with the two TEAMS modules individually and together on multiple implementation outcomes. Implementation outcomes including provider training completion, fidelity (coded by observers blind to group assignment) and child behavior change will be examined for 295 mental health providers, 295 teachers, and 590 children.

DISCUSSION

This implementation intervention has the potential to increase quality of care for ASD in publicly funded settings by improving effectiveness of intervention implementation. The process and modules will be generalizable to multiple service systems, providers, and interventions, providing broad impact in community services.

TRIAL REGISTRATION

This study is registered with Clinicaltrials.gov ( NCT03380078 ). Registered 20 December 2017, retrospectively registered.

Applying the Policy Ecology Framework to Philadelphia's Behavioral Health Transformation Efforts

Raghavan et al. (Implement Sci 3(26):1-9, 2008) proposed that effective implementation of evidence-based practices requires implementation strategies deployed at multiple levels of the "policy ecology," including the organizational, regulatory or purchaser agency, political, and social levels. However, much of implementation research and practice targets providers without accounting for contextual factors that may influence provider behavior. This paper examines Philadelphia's efforts to work toward an evidence-based and recovery-oriented behavioral health system, and uses the policy ecology framework to illustrate how multifaceted, multilevel implementation strategies can facilitate the widespread implementation of evidence-based practices. Ongoing challenges and implications for research and practice are discussed.

Multilevel Mechanisms of Implementation Strategies in Mental Health: Integrating Theory, Research, and Practice

A step toward the development of optimally effective, efficient, and feasible implementation strategies that increase evidence-based treatment integration in mental health services involves identification of the multilevel mechanisms through which these strategies influence implementation outcomes. This article (a) provides an orientation to, and rationale for, consideration of multilevel mediating mechanisms in implementation trials, and (b) systematically reviews randomized controlled trials that examined mediators of implementation strategies in mental health. Nine trials were located. Mediation-related methodological deficiencies were prevalent and no trials supported a hypothesized mediator. The most common reason was failure to engage the mediation target. Discussion focuses on directions to accelerate implementation strategy development in mental health.

Exploring multi-level system factors facilitating educator training and implementation of evidence-based practices (EBP): a study protocol

BACKGROUND

This study examines how system-wide (i.e., region, district, and school) mechanisms such as leadership support, training requirements, structure, collaboration, and education affect the use of evidence-based practices (EBPs) in schools and how this affects the outcomes for students with autism spectrum disorder (ASD). Despite growing evidence for the positive effects of EBPs for ASD, these practices are not consistently or effectively used in schools. Although special education programs are mandated to use EBPs, there are very few evidence-based methods for selecting, implementing, and sustaining EBPs. Research focuses primarily on teacher training, without attention to contextual factors (e.g., implementation climate, attitudes toward EBPs, resource allocation, and social networks) that may impact outcomes. Using an implementation science framework, this project will prospectively examine relations between system-wide factors and teachers' use of EBPs and student education outcomes.

METHODS/DESIGN

Survey data will be collected from approximately 85 regional special education directors, 170 regional program specialists, 265 district special education directors, 265 behavior specialists, 925 school principals, 3538 special education teachers, and 2700 paraprofessionals. Administrative data for the students with ASD served by participating teachers will be examined. A total of 79 regional-, district-, and school-level personnel will also participate in social network interviews. Mixed methods, including surveys, administrative data, and observational checklists, will be used to gather in-depth information about system-wide malleable factors that relate to positive teacher implementation of EBPs and student outcomes. Multi-level modeling will be used to assess system-wide malleable factors related to EBP implementation which will be linked to the trainer, teacher, and student outcomes and examined based on moderators (e.g., district size, Special Education Local Plan Area structure, teachers' ASD experience). Finally, a dynamic social network approach will be used to map EBP-related connectivity across all levels of the system for selected regions. Dynamic network analysis will be used to gauge the degree to which and ways that EBP trainings, resources, and interventions are shared (or not shared) among school staff.

DISCUSSION

Results are expected to inform the development of system-wide interventions to improve the school-based implementation of EBPs for students with ASD.

Are general and strategic measures of organizational context and leadership associated with knowledge and attitudes toward evidence-based practices in public behavioral health settings? A cross-sectional observational study

BACKGROUND

Examining the role of modifiable barriers and facilitators is a necessary step toward developing effective implementation strategies. This study examines whether both general (organizational culture, organizational climate, and transformational leadership) and strategic (implementation climate and implementation leadership) organizational-level factors predict therapist-level determinants of implementation (knowledge of and attitudes toward evidence-based practices).

METHODS

Within the context of a system-wide effort to increase the use of evidence-based practices (EBPs) and recovery-oriented care, we conducted an observational, cross-sectional study of 19 child-serving agencies in the City of Philadelphia, including 23 sites, 130 therapists, 36 supervisors, and 22 executive administrators. Organizational variables included characteristics such as EBP initiative participation, program size, and proportion of independent contractor therapists; general factors such as organizational culture and climate (Organizational Social Context Measurement System) and transformational leadership (Multifactor Leadership Questionnaire); and strategic factors such as implementation climate (Implementation Climate Scale) and implementation leadership (Implementation Leadership Scale). Therapist-level variables included demographics, attitudes toward EBPs (Evidence-Based Practice Attitudes Scale), and knowledge of EBPs (Knowledge of Evidence-Based Services Questionnaire). We used linear mixed-effects regression models to estimate the associations between the predictor (organizational characteristics, general and strategic factors) and dependent (knowledge of and attitudes toward EBPs) variables.

RESULTS

Several variables were associated with therapists' knowledge of EBPs. Clinicians in organizations with more proficient cultures or higher levels of transformational leadership (idealized influence) had greater knowledge of EBPs; conversely, clinicians in organizations with more resistant cultures, more functional organizational climates, and implementation climates characterized by higher levels of financial reward for EBPs had less knowledge of EBPs. A number of organizational factors were associated with the therapists' attitudes toward EBPs. For example, more engaged organizational cultures, implementation climates characterized by higher levels of educational support, and more proactive implementation leadership were all associated with more positive attitudes toward EBPs.

CONCLUSIONS

This study provides evidence for the importance of both general and strategic organizational determinants as predictors of knowledge of and attitudes toward EBPs. The findings highlight the need for longitudinal and mixed-methods studies that examine the influence of organizational factors on implementation.

Bridging the Gap Between Research and Practice in Mental Health Service Settings: An Overview of Developments in Implementation Theory and Research

Twenty years after Sobell's (1996) influential call to integrate advances in behavior therapy into clinical settings, significant progress has been made in implementation science. In this narrative review, we provide an overview of implementation research findings and highlight recent findings that can inform efforts to bridge the gap between research and practice in mental health service settings. Key findings are summarized, organized according to levels of influence described in two implementation frameworks: The Exploration, Planning, Implementation, and Sustainment Framework (EPIS; Aarons et al., 2011) and the Consolidated Framework for Implementation Research (CFIR; Damschroder et al., 2009). Important levels of influence to consider when implementing new treatments include the outer context, inner context, characteristics of the individual, and characteristics of the innovation. Research on strategies to prepare clinicians to deliver evidence-based psychosocial treatments (EBPTs) and to address contextual barriers to implementation at each level is described, with discussion of implications for the implementation of EBPTs and next steps for research.

Assessing mental health clinicians' intentions to adopt evidence-based treatments: reliability and validity testing of the evidence-based treatment intentions scale

Background

Intentions play a central role in numerous empirically supported theories of behavior and behavior change and have been identified as a potentially important antecedent to successful evidence-based treatment (EBT) implementation. Despite this, few measures of mental health clinicians’ EBT intentions exist and available measures have not been subject to thorough psychometric evaluation or testing. This paper evaluates the psychometric properties of the evidence-based treatment intentions (EBTI) scale, a new measure of mental health clinicians’ intentions to adopt EBTs.

Methods

The study evaluates the reliability and validity of inferences made with the EBTI using multi-method, multi-informant criterion variables collected over 12 months from a sample of 197 mental health clinicians delivering services in 13 mental health agencies. Structural, predictive, and discriminant validity evidence is assessed.

Results

Findings support the EBTI’s factor structure (χ² = 3.96, df = 5, p = .556) and internal consistency reliability (α = .80). Predictive validity evidence was provided by robust and significant associations between EBTI scores and clinicians’ observer-reported attendance at a voluntary EBT workshop at a 1-month follow-up (OR = 1.92, p < .05), self-reported EBT adoption at a 12-month follow-up (R² = .17, p < .001), and self-reported use of EBTs with clients at a 12-month follow-up (R² = .25, p < .001). Discriminant validity evidence was provided by small associations with clinicians’ concurrently measured psychological work climate perceptions of functionality (R² = .06, p < .05), engagement (R² = .06, p < .05), and stress (R² = .00, ns).

Conclusions

The EBTI is a practical and theoretically grounded measure of mental health clinicians’ EBT intentions. Scores on the EBTI provide a basis for valid inferences regarding mental health clinicians’ intentions to adopt EBTs. Discussion focuses on research and practice applications.

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-016-0417-3) contains supplementary material, which is available to authorized users.

A model of integrated health care in a poverty-impacted community in New York City: Importance of early detection and addressing potential barriers to intervention implementation

Disruptive behavior disorders (DBDs) are chronic, impairing, and costly behavioral health conditions that are four times more prevalent among children of color living in impoverished communities as compared to the general population. This disparity is largely due to the increased exposure to stressors related to low socioeconomic status including community violence, unstable housing, under supported schools, substance abuse, and limited support systems. However, despite high rates and greater need, there is a considerably lower rate of mental health service utilization among these youth. Accordingly, the current study aims to describe a unique model of integrated health care for ethnically diverse youth living in a New York City borough. With an emphasis on addressing possible barriers to implementation, integrated models for children have the potential to prevent ongoing mental health problems through early detection and intervention.

Increasing clinicians' EBT exploration and preparation behavior in youth mental health services by changing organizational culture with ARC

OBJECTIVE

Clinician EBT exploration and preparation behavior is essential to the ongoing implementation of new EBTs. This study examined the effect of the ARC organizational intervention on clinician EBT exploration and preparation behavior and assessed the mediating role of organizational culture as a linking mechanism.

METHOD

Fourteen community mental health agencies that serve youth in a major Midwestern metropolis along with 475 clinicians who worked in those agencies, were randomly assigned to either the three-year ARC intervention or control condition. Organizational culture was assessed with the Organizational Social Context (OSC) measure at baseline and follow-up. EBT exploration and preparation behavior was measured as clinician participation in nine separate community EBT workshops held over a three-year period.

RESULTS

There was 69 percent greater odds (OR = 1.69, p < .003) of clinicians in the ARC condition (versus control) attending each subsequent workshop. Workshop attendance in the control group remained under two percent and declined over three years while attendance in the ARC condition grew from 3.6 percent in the first workshop to 12 percent in the ninth and final workshop. Improvement in proficient organizational culture mediated the positive effect of the ARC intervention on clinicians' workshop attendance (a × b = .21; 95% CI:LL = .05, UL = .40).

CONCLUSIONS

Organizational interventions that create proficient mental health agency cultures can increase clinician EBT exploration and preparation behavior that is essential to the ongoing implementation of new EBTs in community youth mental health settings.

Multilevel Mechanisms of Implementation Strategies in Mental Health: Integrating Theory, Research, and Practice

A step toward the development of optimally effective, efficient, and feasible implementation strategies that increase evidence-based treatment integration in mental health services involves identification of the multilevel mechanisms through which these strategies influence implementation outcomes. This article (a) provides an orientation to, and rationale for, consideration of multilevel mediating mechanisms in implementation trials, and (b) systematically reviews randomized controlled trials that examined mediators of implementation strategies in mental health. Nine trials were located. Mediation-related methodological deficiencies were prevalent and no trials supported a hypothesized mediator. The most common reason was failure to engage the mediation target. Discussion focuses on directions to accelerate implementation strategy development in mental health.