Quality indicators for family support services and their relationship to organizational social context

Managers' Micro-Communities Matter: The Impact of Clinical Supervision Team on Therapist Perception of the Organization

Positive organizational climate - employee perceptions of their work environment and the impact of this environment on well-being and functioning - is associated with desirable organizational and client-level outcomes in mental health organizations. Clinical supervisors are well-positioned to impact organizational climate, as they serve as intermediaries between higher-level administrators who drive the policies and procedures and the therapists impacted by such decisions. This cross-sectional study examined the role of clinical supervisors as drivers of therapist perceptions of organizational climate within supervisory teams. Specifically, the present study investigated: (1) shared perceptions of organizational climate among therapists on the same supervisory team; (2) predictors of therapist climate perceptions. Eighty-six therapists were supervised by 22 supervisors. Indices of interrater agreement and interrater reliability of therapists on the same supervisory team were examined to determine shared or distinct perceptions of organizational climate. Multi-level models were used to examine whether supervisor attitudes towards evidence-based practices and therapist perceptions of supervisor communication predicted perceived organizational climate. Results showed perceptions of organizational cohesion and autonomy were shared among therapists on the same supervisory team and distinct from therapists on different supervisory teams. Therapist perceptions of their supervisor's communication was positively associated with perceptions of organizational cohesion and autonomy. These findings align with emerging evidence that middle managers shape their employees' experience of their work environment through communication strategies. These findings also point to the potential for intervening at lower organizational levels to improve overall organizational climate.

Association of organizational culture and climate with variation in the clinical outcomes of collaborative care for maternal depression in community health centers

Background

Organizational factors may help explain variation in the effectiveness of evidence-based clinical innovations through implementation and sustainment. This study tested the relationship between organizational culture and climate and variation in clinical outcomes of the Collaborative Care Model (CoCM) for treatment of maternal depression implemented in community health centers.

Method

Organizational cultures and climates of 10 community health centers providing CoCM for depression among low-income women pregnant or parenting were assessed using the organizational social context (OSC) measure. Three-level hierarchical linear models tested whether variation in culture and climate predicted variation in improvement in depression symptoms from baseline to 6.5-month post-baseline for N = 468 women with care ±1 year of OSC assessment. Depression symptomology was measured using the Patient Health Questionnaire (PHQ-9).

Results

After controlling for patient characteristics, case mix, center size, and implementation support, patients served by centers with more proficient cultures improved significantly more from baseline to 6.5-month post-baseline than patients in centers with less proficient cultures (mean improvement = 5.08 vs. 0.14, respectively, p = .020), resulting in a large adjusted effect size of dadj = 0.78. A similar effect was observed for patients served by centers with more functional climates (mean improvement = 5.25 vs. 1.12, p < .044, dadj = 0.65). Growth models indicated that patients from all centers recovered on average after 4 months of care. However, those with more proficient cultures remained stabilized whereas patients served by centers with less proficient cultures deteriorated by 6.5-month post-baseline. A similar pattern was observed for functional climate.

Conclusions

Variation in clinical outcomes for women from historically underserved populations receiving Collaborative Care for maternal depression was associated with the organizational cultures and climates of community health centers. Implementation strategies targeting culture and climate may improve the implementation and effectiveness of integrated behavioral health care for depression.

The Organizational Financial Context of Publicly-Funded Mental Health Clinics: Development and Preliminary Psychometric Evaluation of the Agency Financial Status Scales

Funding is a major barrier to implementation of evidence-based practices (EBPs) in publicly-funded community mental health clinics (CMHCs). Understanding how best to deploy implementation strategies that address this barrier requires greater clarity on the financial context within agencies. We developed the Agency Financial Status Scales (AFSS) to assess employee perceptions of the level of three hypothesized and theoretical funding related constructs in organizations: (a) perceptions of financial health, (b) financial attitudes toward EBPs, and (c) strategic financial climate. This investigation serves as a preliminary evaluation of this measure. Participants were 239 therapists and 40 supervisors from 25 publicly-funded CMHCs providing outpatient mental health services for young people. Confirmatory factor analysis was used to investigate the latent trait structure of the items. Internal consistency, interrater agreement, concordance between therapists and supervisors, and convergent validity were also examined. A two-factor model measuring perceptions of financial health and strategic financial climate best fit the data. For both of these scales, alpha reliability was acceptable and agreement statistics provided moderate support for aggregation at the organizational level. Analyses supported the convergent validity of the scales. The development and preliminary evaluation of the AFSS is an important first step in understanding the financial context of publicly-funded CMHCs. Though findings from this investigation are promising, additional development and testing are needed to develop a more thorough understanding of the constructs and to improve the validity and reliability of this measure.

Embedding school cultures and climates that promote evidence-based practice implementation for youth with autism: A qualitative study

LAY ABSTRACT

Schools play a major role in providing services to youth with autism; however, not all schools use evidence-based practices, defined as interventions that are proven to improve youth well-being through rigorous research. School culture and climate are strong predictors of whether or not a school uses evidence-based practices; however, little is known about how principals can create school cultures and climates that support the use of these practices. This study interviewed 32 teachers in elementary schools that implemented three closely related evidence-based practices for youth with autism to better understand how principals create school cultures and climates that support effective services. Analysis of the teachers' responses identified seven strategies principals can use to create school cultures and climates that support the implementation of effective practices for youth with autism. The strategies include the following: (a) support teachers to obtain professional development focused on autism, (b) align performance expectations and evaluations with the needs of students with autism and evidence-based practice delivery, (c) allocate resources to ensure adequate staff, materials, and training are available to implement evidence-based practices, (d) be open and flexible to allow teachers to use the building and resources as needed to meet students' needs, (e) provide direct assistance, feedback, and coaching to troubleshoot challenges or involve outside experts to do so, (f) openly value the work of special education teachers and provide recognition to those who develop expertise in evidence-based practices, and (g) look for opportunities to integrate special and general education teachers and students to foster a truly inclusive climate.

A Qualitative Examination of a School-Based Implementation of Computer-Assisted Cognitive-Behavioral Therapy for Child Anxiety

Mental health treatment in schools has the potential to improve youth treatment access. However, school-specific barriers can make implementing evidence-based interventions difficult. Task-shifting (i.e., training lay staff to implement interventions) and computer-assisted interventions may mitigate these barriers. This paper reports on a qualitative examination of facilitators and barriers of a school-based implementation of a computer-assisted intervention for anxious youth (Camp Cope-A-Lot; CCAL). Participants (N = 45) included school staff in first through fourth grades. Providers attended a training in CCAL and received weekly, hour-long group consultation calls for three months. In the second year, the sustainability of CCAL use was assessed. Qualitative interviews were conducted after the first year (initial implementation) and second year (sustainability). Interviews were analyzed using the Consolidated Framework for Implementation Research domains to classify themes. Although participants reported that CCAL included useful skills, they expressed concerns about recommended session length (45 minutes) and frequency (weekly). Time burden of consultation calls was also a barrier. School staff facilitated implementation by enabling flexible scheduling for youth to be able to participate in the CCAL program. However, the sustainability of the program was limited due to competing school/time demands. Results suggest that even with computer assisted programs, there is a need to tailor interventions and implementation efforts to account for the time restrictions experienced by school-based service providers. Optimal fit between the intervention and specific school is important to maintain the potential benefits of computer-assisted treatments delivered by lay service providers in schools.

A Systematic Review of Organizational and Workforce Interventions to Improve the Culture and Climate of Youth-Service Settings

Both organizational culture and climate are associated with service quality and outcomes across youth-service settings. Increasing evidence indicates capacity of organizational interventions to promote a positive and effective culture and climate. Less is known about common intervention components across studies and service settings. The current systematic review reviewed 9223 citations and identified 31 studies, across six youth-service settings, measuring changes over time in organizational culture and climate following implementation of an organizational or workforce support intervention. Results highlight the promise of organizational interventions, a need for more comparison and randomized designs, and future directions for maximizing capacity of organizations to promote health for frontline providers and the children they serve.

Testing a theory of strategic implementation leadership, implementation climate, and clinicians' use of evidence-based practice: a 5-year panel analysis

BACKGROUND

Implementation theory suggests that first-level leaders, sometimes referred to as middle managers, can increase clinicians' use of evidence-based practice (EBP) in healthcare settings by enacting specific leadership behaviors (i.e., proactive, knowledgeable, supportive, perseverant with regard to implementation) that develop an EBP implementation climate within the organization; however, longitudinal and quasi-experimental studies are needed to test this hypothesis.

METHODS

Using data collected at three waves over a 5-year period from a panel of 30 outpatient children's mental health clinics employing 496 clinicians, we conducted a quasi-experimental difference-in-differences study to test whether within-organization change in implementation leadership predicted within-organization change in EBP implementation climate, and whether change in EBP implementation climate predicted within-organization change in clinicians' use of EBP. At each wave, clinicians reported on their first-level leaders' implementation leadership, their organization's EBP implementation climate, and their use of both EBP and non-EBP psychotherapy techniques for childhood psychiatric disorders. Hypotheses were tested using econometric two-way fixed effects regression models at the organization level which controlled for all stable organizational characteristics, population trends in the outcomes over time, and time-varying covariates.

RESULTS

Organizations that improved from low to high levels of implementation leadership experienced significantly greater increases in their level of EBP implementation climate (d = .92, p = .017) and within-organization increases in implementation leadership accounted for 11% of the variance in improvement in EBP implementation climate beyond all other covariates. In turn, organizations that improved from low to high levels of EBP implementation climate experienced significantly greater increases in their clinicians' average EBP use (d = .55, p = .007) and within-organization improvement in EBP implementation climate accounted for 14% of the variance in increased clinician EBP use. Mediation analyses indicated that improvement in implementation leadership had a significant indirect effect on clinicians' EBP use via improvement in EBP implementation climate (d = .26, 95% CI [.02 to .59]).

CONCLUSIONS

When first-level leaders increase their frequency of implementation leadership behaviors, organizational EBP implementation climate improves, which in turn contributes to increased EBP use by clinicians. Trials are needed to test strategies that target this implementation leadership-EBP implementation climate mechanism.

Lay counselors: Thoughts on the crossing of ecological frameworks and the use of lay counselors in the scale up of early infant mental health interventions

The recent call for the scale-up of evidence-based early childhood development interventions, in lower and middle-income countries and for minority groups in high-income countries, has seen numerous suggestions to train greater numbers of lay mental health workers to fulfill these functions. While studies have found that concepts from developed country settings, such as attachment, parental sensitivity, and containment, find purchase and relevance within developing settings, the management of contextual and cultural factors and the tensions of cultural interfacing in the rollout of these programs in developing country settings require consideration. Drawing on the experiences of two successful South African mother-infant home-visiting programs as examples, this article discusses some of the challenges in provision of attachment-based infant mental health programs and highlights the need for careful consideration of a number of factors pertaining to the recruitment, supervision, and management of lay mental health workers before large-scale rollout is conducted.

Developing and evaluating a lay health worker delivered implementation intervention to decrease engagement disparities in behavioural parent training: a mixed methods study protocol

INTRODUCTION

Behavioural parent training (BPT) programmes are effective in preventing and treating early-onset conduct problems and child maltreatment. Unfortunately, pervasive mental health service disparities continue to limit access to and engagement in these interventions. Furthermore, challenges with parental engagement can impede the successful implementation of evidence-based practices (EBPs) in community settings that serve low-income, ethnic minority families. Lay health workers (LHWs)-individuals without formal mental health training-represent an important workforce to increase engagement, as they are members of the communities they serve. However, the mobilisation of LHWs has not been well studied as an implementation strategy to extend the reach or effectiveness of EBPs in the USA. LHW-delivered implementation interventions that specifically support the engagement of Latinx parents in evidence-based BPT programmes have the potential to improve clinical and implementation outcomes.

METHODS AND ANALYSIS

A community-partnered approach will use the Quality Implementation Framework (QIF) to tailor and implement an LHW-delivered implementation intervention that aims to promote Latinx parent engagement in BPT programmes. Steps from the QIF will guide study activities to (1) conduct a mixed methods needs assessment to fit the implementation intervention to the local context, (2) adapt LHW-delivered implementation strategies to promote parent access to and engagement in Parent-Child Interaction Therapy and (3) conduct a hybrid effectiveness-implementation pilot trial to examine the feasibility, acceptability and preliminary effectiveness of the LHW implementation intervention at increasing engagement.

ETHICS AND DISSEMINATION

Study procedures have been approved by the Institutional Review Board at the University of California, Santa Barbara. Results will be shared with the community-advisory group, at community-based meetings for other stakeholders involved in the pilot project, and submitted for publication in peer-reviewed journals.

A repeated cross-sectional study of clinicians' use of psychotherapy techniques during 5 years of a system-wide effort to implement evidence-based practices in Philadelphia

Background

Little work investigates the effect of behavioral health system efforts to increase use of evidence-based practices or how organizational characteristics moderate the effect of these efforts. The objective of this study was to investigate clinician practice change in a system encouraging implementation of evidence-based practices over 5 years and how organizational characteristics moderate this effect. We hypothesized that evidence-based techniques would increase over time, whereas use of non-evidence-based techniques would remain static.

Method

Using a repeated cross-sectional design, data were collected three times from 2013 to 2017 in Philadelphia’s public behavioral health system. Clinicians from 20 behavioral health outpatient clinics serving youth were surveyed three times over 5 years (n = 340; overall response rate = 60%). All organizations and clinicians were exposed to system-level support provided by the Evidence-based Practice Innovation Center from 2013 to 2017. Additionally, approximately half of the clinicians participated in city-funded evidence-based practice training initiatives. The main outcome included clinician self-reported use of cognitive-behavioral and psychodynamic techniques measured by the Therapy Procedures Checklist-Family Revised.

Results

Clinicians were 80% female and averaged 37.52 years of age (SD = 11.40); there were no significant differences in clinician characteristics across waves (all ps > .05). Controlling for organizational and clinician covariates, average use of CBT techniques increased by 6% from wave 1 (M = 3.18) to wave 3 (M = 3.37, p = .021, d = .29), compared to no change in psychodynamic techniques (p = .570). Each evidence-based practice training initiative in which clinicians participated predicted a 3% increase in CBT use (p = .019) but no change in psychodynamic technique use (p = .709). In organizations with more proficient cultures at baseline, clinicians exhibited greater increases in CBT use compared to organizations with less proficient cultures (8% increase vs. 2% decrease, p = .048).

Conclusions

System implementation of evidence-based practices is associated with modest changes in clinician practice; these effects are moderated by organizational characteristics. Findings identify preliminary targets to improve implementation.

Comparing Agency Leader and Therapist Perspectives on Evidence-Based Practices: Associations with Individual and Organizational Factors in a Mental Health System-Driven Implementation Effort

Agency leaders and therapists are essential stakeholders in implementation of evidence-based practices (EBPs) within publicly-funded mental health services. Little is known about how these stakeholders differ in their perceptions of specific EBPs and which individual and organizational factors differentially influence these perceptions. Within the context of a system-driven implementation of multiple EBPs, survey data from 160 leaders and 720 therapists were examined to assess differences in perceptions of six EBPs. Findings indicated that leaders and therapists have unique perspectives and preferences regarding EBPs that are shaped by distinct sociodemographic and professional characteristics and aspects of organizational functioning.

Concordance Between Administrator and Clinician Ratings of Organizational Culture and Climate

Organizational culture and climate are important determinants of behavioral health service delivery for youth. The Organizational Social Context measure is a well validated assessment of organizational culture and climate that has been developed and extensively used in public sector behavioral health service settings. The degree of concordance between administrators and clinicians in their reports of organizational culture and climate may have implications for research design, inferences, and organizational intervention. However, the extent to which administrators' and clinicians' reports demonstrate concordance is just beginning to garner attention in public behavioral health settings in the United States. We investigated the concordance between 73 administrators (i.e., supervisors, clinical directors, and executive directors) and 247 clinicians in 28 child-serving programs in a public behavioral health system. Findings suggest that administrators, compared to clinicians, reported more positive cultures and climates. Organizational size moderated this relationship such that administrators in small programs (<466 youth clients served annually) provided more congruent reports of culture and climate in contrast to administrators in large programs (≥466 youth clients served annually) who reported more positive cultures and climates than clinicians. We propose a research agenda that examines the effect of concordance between administrators and clinicians on organizational outcomes in public behavioral health service settings.

Organizational culture and climate profiles: relationships with fidelity to three evidence-based practices for autism in elementary schools

BACKGROUND

Implementation researchers have typically studied organizational culture and climate by testing whether individual dimensions are linked to the implementation of evidence-based practices (EBPs) rather than examining how the overarching social context influences implementation. This approach may limit implementation theory and strategy development to the extent that individual dimensions of culture and climate interact, mutually reinforce or counteract one another, or exhibit non-linear relationships. This study tests whether empirically identifiable culture and climate profiles emerge in a sample of organizations and examines how these profiles relate to EBP fidelity and work attitudes that support EBP sustainment, focusing on three EBPs for youth with autism delivered in schools as an example.

METHODS

The study included 65 elementary schools in the U.S. that implemented three EBPs-discrete trial training, pivotal response training, and visual schedules-for youth with autism. Organizational culture and climate and work attitudes were assessed using the Organizational Social Context measure at the beginning of the school year. Observations of EBP fidelity occurred mid school-year. We used bias-adjusted stepwise latent profile modeling to (1) identify subpopulations of schools that share similar culture and climate profiles, and (2) test for mean differences across profiles in observed EBP fidelity and teacher and staff work attitudes.

RESULTS

Controlling for region, four profiles best characterized the organizational cultures and climates of schools. Teachers and staff in schools with a comprehensive profile (high proficiency culture, positive climate) exhibited higher fidelity to two of three EBPs (d's = .95 to 1.64) and reported superior work attitudes (d's = .71 to 1.93) than teachers and staff in all other schools. Teachers and staff in supportive schools (low rigidity culture, positive climate) had better work attitudes, but not better fidelity, than those in schools with indifferent (low culture/climate, elevated stress) and constrained (high rigidity and resistance, high stress) profiles.

CONCLUSIONS

Organizational culture and climate profiles are a strong predictor of EBP fidelity and work attitudes that support EBP sustainment, highlighting the importance of an organization's overarching social context when developing implementation theory and strategies. Strategies that foster a comprehensive profile may improve EBP implementation.

Navigating the storm: How proficient organizational culture promotes clinician retention in the shift to evidence-based practice

Objective

Clinician turnover is a major concern as mental health systems and organizations invest substantial resources in the implementation of evidence-based practice (EBP). In this study, we identify malleable factors associated with reduced clinician turnover during a system-wide EBP implementation initiative. Specifically, we examine how proficient organizational culture (i.e., norms and behavioral expectations that clinicians prioritize improvement in client well-being and exhibit competence in up-to-date treatment practices), EBP implementation climate (i.e., perceptions that the organization’s policies, procedures, and practices support EBP use), and change in these organizational characteristics relate to clinician turnover during a system-wide EBP transformation.

Method

Data were collected from 236 clinicians in 19 mental health clinics across 3 years of a system-wide EBP implementation initiative in the City of Philadelphia. Clinicians reported on proficient organizational culture and EBP implementation climate at baseline (T₁) and two-year follow-up (T₂). Administrators reported on clinician turnover at three-year follow-up (T₃). Hypotheses were tested via multilevel mediation analyses incorporating mixed effects logistic regression models.

Results

Controlling for organization size, clinician job satisfaction, attitudes towards EBP, job tenure, and age, higher levels of proficient organizational culture and improvement in proficient culture from baseline to two-year follow-up predicted reduced clinician turnover in the year following; these effects were mediated by EBP implementation climate and by improvement in EBP implementation climate, respectively.

Conclusions

Organizations with more proficient cultures have more supportive EBP implementation climates that predict reduced clinician turnover during system-wide EBP implementation initiatives. Strategies that target these antecedents in mental health service organizations may contribute to reduced clinician turnover.

Linking molar organizational climate and strategic implementation climate to clinicians' use of evidence-based psychotherapy techniques: cross-sectional and lagged analyses from a 2-year observational study

Background

Behavioral health organizations are characterized by multiple organizational climates, including molar climate, which encompasses clinicians’ shared perceptions of how the work environment impacts their personal well-being, and strategic implementation climate, which includes clinicians’ shared perceptions of the extent to which evidence-based practice implementation is expected, supported, and rewarded by the organization. Theory suggests these climates have joint, cross-level effects on clinicians’ implementation of evidence-based practice and that these effects may be long term (i.e., up to 2 years); however, no empirical studies have tested these relationships. We hypothesize that molar climate moderates implementation climate’s concurrent and long-term relationships with clinicians’ use of evidence-based practice such that strategic implementation climate will have its most positive effects when it is accompanied by a positive molar climate.

Methods

Hypotheses were tested using data collected from 235 clinicians in 20 behavioral health organizations. At baseline, clinicians reported on molar climate and implementation climate. At baseline and at a 2-year follow-up, all clinicians who were present in the organizations reported on their use of cognitive-behavioral psychotherapy techniques, an evidence-based practice for youth psychiatric disorders. Two-level mixed-effects regression models tested whether baseline molar climate and implementation climate interacted in predicting clinicians’ evidence-based practice use at baseline and at 2-year follow-up.

Results

In organizations with more positive molar climates at baseline, higher levels of implementation climate predicted increased evidence-based practice use among clinicians who were present at baseline and among clinicians who were present in the organizations at 2-year follow-up; however, in organizations with less positive molar climates, implementation climate was not related to clinicians’ use of evidence-based practice at either time point.

Conclusions

Optimizing clinicians’ implementation of evidence-based practice in behavioral health requires attention to both molar climate and strategic implementation climate. Strategies that focus exclusively on implementation climate may not be effective levers for behavior change if the organization does not also engender a positive molar climate. These findings have implications for the development of implementation theory and effective implementation strategies.

Lay Health Worker Involvement in Evidence-Based Treatment Delivery: A Conceptual Model to Address Disparities in Care

Mobilizing lay health workers (LHWs) to deliver evidence-based treatments (EBTs) is a workforce strategy to address mental health disparities in underserved communities. LHWs can be leveraged to support access to EBTs in a variety of ways, from conducting outreach for EBTs delivered by professional providers to serving as the primary treatment providers. This critical review provides an overview of how LHW-supported or -delivered EBTs have been leveraged in low-, middle-, and high-income countries (HICs). We propose a conceptual model for LHWs to address drivers of service disparities, which relate to the overall supply of the EBTs provided and the demand for these treatments. The review provides illustrative case examples that demonstrate how LHWs have been leveraged globally and domestically to increase access to mental health services. It also discusses challenges and recommendations regarding implementing LHW-supported or -delivered EBTs.

Mechanisms of Change in the ARC Organizational Strategy: Increasing Mental Health Clinicians' EBP Adoption Through Improved Organizational Culture and Capacity

The development of efficient and scalable implementation strategies in mental health is restricted by poor understanding of the change mechanisms that increase clinicians' evidence-based practice (EBP) adoption. This study tests the cross-level change mechanisms that link an empirically-supported organizational strategy for supporting implementation (labeled ARC for Availability, Responsiveness, and Continuity) to mental health clinicians' EBP adoption and use. Four hundred seventy-five mental health clinicians in 14 children's mental health agencies were randomly assigned to the ARC intervention or a control condition. Measures of organizational culture, clinicians' intentions to adopt EBPs, and job-related EBP barriers were collected before, during, and upon completion of the three-year ARC intervention. EBP adoption and use were assessed at 12-month follow-up. Multilevel mediation analyses tested changes in organizational culture, clinicians' intentions to adopt EBPs, and job-related EBP barriers as linking mechanisms explaining the effects of ARC on clinicians' EBP adoption and use. ARC increased clinicians' EBP adoption (OR = 3.19, p = .003) and use (81 vs. 56 %, d = .79, p = .003) at 12-month follow-up. These effects were mediated by improvement in organizational proficiency culture leading to increased clinician intentions to adopt EBPs and by reduced job-related EBP barriers. A combined mediation analysis indicated the organizational culture-EBP intentions mechanism was the primary carrier of ARC's effects on clinicians' EBP adoption and use. ARC increases clinicians' EBP adoption and use by creating proficient organizational cultures that increase clinicians' intentions to adopt EBPs.

Lay Health Worker Involvement in Evidence-Based Treatment Delivery: A Conceptual Model to Address Disparities in Care

Mobilizing lay health workers (LHWs) to deliver evidence-based treatments (EBTs) is a workforce strategy to address mental health disparities in underserved communities. LHWs can be leveraged to support access to EBTs in a variety of ways, from conducting outreach for EBTs delivered by professional providers to serving as the primary treatment providers. This critical review provides an overview of how LHW-supported or -delivered EBTs have been leveraged in low-, middle-, and high-income countries (HICs). We propose a conceptual model for LHWs to address drivers of service disparities, which relate to the overall supply of the EBTs provided and the demand for these treatments. The review provides illustrative case examples that demonstrate how LHWs have been leveraged globally and domestically to increase access to mental health services. It also discusses challenges and recommendations regarding implementing LHW-supported or -delivered EBTs.

Evaluation of a Train-The-Trainers Model for Family Peer Advocates in Children's Mental Health

Standardized training and credentialing is increasingly important to states and healthcare systems. Workforce shortages in children's mental health can be addressed through training and credentialing of professional peer parents (called family peer advocates or FPAs), who deliver a range of services to caregivers. A theory-based training program for FPAs targeting skills and knowledge about childhood mental health services (Parent Empowerment Program, or PEP) was developed through a partnership among a statewide family-run organization, state policy leaders, and academic researchers. Prior studies by this team using highly-experienced family peer advocates (who were also co-developers of the training program) as trainers found improvements in knowledge about mental health services and self-efficacy. In 2010, to meet demands and scale the model, a training of trainers (TOT) model was developed to build a cohort of locally-trained FPAs to deliver PEP training. A pre/post design was used to evaluate the impact of TOT model on knowledge and self-efficacy among 318 FPAs across the state. Participants showed significant pre-post (6 month) changes in knowledge about mental health services and self-efficacy. There were no significant associations between any FPA demographic characteristics and their knowledge or self-efficacy scores. A theory-based training model for professional peer parents working in the children's mental health system can be taught to local FPAs, and it improves knowledge about the mental health system and self-efficacy. Studies that evaluate the effectiveness of different training modalities are critical to ensure that high-quality trainings are maintained.

Implementing a Measurement Feedback System in Community Mental Health Clinics: A Case Study of Multilevel Barriers and Facilitators

Measurement feedback systems (MFSs) have been proposed as a means of improving practice. The present study examined the implementation of a MFS, the Contextualized Feedback System (CFS), in two community-based clinic sites. Significant implementation differences across sites provided a basis for examining factors that influenced clinician uptake of CFS. Following the theoretical implementation framework of Aarons et al. (Adm Policy Mental Health Mental Health Serv Res 38(1):4-23, 2011), we coded qualitative data collected from eighteen clinicians (13 from Clinic U and 5 from Clinic R) who participated in semi-structured interviews about their experience with CFS implementation. Results suggest that clinicians at both clinics perceived more barriers than facilitators to CFS implementation. Interestingly, clinicians at the higher implementing clinic reported a higher proportion of barriers to facilitators (3:1 vs. 2:1); however, these clinicians also reported a significantly higher level of organizational and leadership supports for CFS implementation. Implications of these findings are discussed.

Aligning organizational priorities with ARC to improve youth mental health service outcomes

OBJECTIVE

The Availability, Responsiveness, and Continuity (ARC) organizational intervention is designed to improve community-based youth mental health services by aligning organizational priorities with 5 principles of effective service organizations (i.e., mission driven, results oriented, improvement directed, relationship centered, participation based). This study assessed the effect of the ARC intervention on youth outcomes and the mediating role of organizational priorities as a mechanism linking the ARC intervention to outcomes.

METHOD

Fourteen community-based mental health agencies in a midwestern metropolis along with 475 clinicians and 605 youth (ages 5-18) served by those agencies were randomly assigned to the 3-year ARC intervention or control condition. The agencies' priorities were measured with the ARC Principles Questionnaire (APQ) completed by clinicians at the end of the intervention. Youth outcomes were measured as total problems in psychosocial functioning described by their caregivers using the Shortform Assessment for Children (SAC) at 6 monthly intervals.

RESULTS

The rate of improvement in youths' psychosocial functioning in agencies assigned to the ARC condition was 1.6 times the rate of improvement in agencies assigned to the control condition, creating a standardized difference in functioning of d = .23 between the 2 groups at the 6-month follow-up. The effect on youth outcomes was fully mediated by the alignment of organizational priorities described in the 5 ARC principles (d = .21).

CONCLUSION

The ARC organizational intervention improves youth outcomes by aligning organizational priorities with the 5 ARC principles. The findings suggest that organizational priorities explain why some community mental health agencies are more effective than others. (PsycINFO Database Record

Revised Conceptual Framework of Parent-to-Parent Support for Parents of Children Who Are Deaf or Hard of Hearing: A Modified Delphi Study

BACKGROUND

A scoping review of the literature was conducted, resulting in the development of a conceptual framework of parent-to-parent support for parents with children who are Deaf or hard of hearing. This is the 2nd stage of a dual-stage scoping review.

PURPOSE

This study sought stakeholder opinion and feedback with an aim to achieve consensus on the constructs, components, and design of the initial conceptual framework.

RESEARCH DESIGN

A modified electronic Delphi study was completed with 21 handpicked experts from 7 countries who have experience in provision, research, or experience in the area of parent-to-parent support. Participants completed an online questionnaire using an 11-point Likert scale (strongly disagree to strongly agree) and open-ended questions to answer various questions related to the descriptor terms, definitions, constructs, components, and overall design of the framework.

RESULTS

Participant responses led to the revision of the original conceptual framework.

CONCLUSION

The findings from this dual-stage scoping review and electronic Delphi study provide a conceptual framework that defines the vital contribution of parents in Early Hearing Detection and Intervention programs that will be a useful addition to these programs.

Assessing mental health clinicians' intentions to adopt evidence-based treatments: reliability and validity testing of the evidence-based treatment intentions scale

Background

Intentions play a central role in numerous empirically supported theories of behavior and behavior change and have been identified as a potentially important antecedent to successful evidence-based treatment (EBT) implementation. Despite this, few measures of mental health clinicians’ EBT intentions exist and available measures have not been subject to thorough psychometric evaluation or testing. This paper evaluates the psychometric properties of the evidence-based treatment intentions (EBTI) scale, a new measure of mental health clinicians’ intentions to adopt EBTs.

Methods

The study evaluates the reliability and validity of inferences made with the EBTI using multi-method, multi-informant criterion variables collected over 12 months from a sample of 197 mental health clinicians delivering services in 13 mental health agencies. Structural, predictive, and discriminant validity evidence is assessed.

Results

Findings support the EBTI’s factor structure (χ² = 3.96, df = 5, p = .556) and internal consistency reliability (α = .80). Predictive validity evidence was provided by robust and significant associations between EBTI scores and clinicians’ observer-reported attendance at a voluntary EBT workshop at a 1-month follow-up (OR = 1.92, p < .05), self-reported EBT adoption at a 12-month follow-up (R² = .17, p < .001), and self-reported use of EBTs with clients at a 12-month follow-up (R² = .25, p < .001). Discriminant validity evidence was provided by small associations with clinicians’ concurrently measured psychological work climate perceptions of functionality (R² = .06, p < .05), engagement (R² = .06, p < .05), and stress (R² = .00, ns).

Conclusions

The EBTI is a practical and theoretically grounded measure of mental health clinicians’ EBT intentions. Scores on the EBTI provide a basis for valid inferences regarding mental health clinicians’ intentions to adopt EBTs. Discussion focuses on research and practice applications.

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-016-0417-3) contains supplementary material, which is available to authorized users.

Assessing and changing organizational social contexts for effective mental health services

Culture and climate are critical dimensions of a mental health service organization's social context that affect the quality and outcomes of the services it provides and the implementation of innovations such as evidence-based treatments (EBTs). We describe a measure of culture and climate labeled Organizational Social Context (OSC), which has been associated with innovation, service quality, and outcomes in national samples and randomized controlled trials (RCTs) of mental health and social service organizations. The article also describes an empirically supported organizational intervention model labeled Availability, Responsiveness, and Continuity (ARC), which has improved organizational social context, innovation, and effectiveness in five RCTs. Finally, the article outlines a research agenda for developing more efficient and scalable organizational strategies to improve mental health services by identifying the mechanisms that link organizational interventions and social context to individual-level service provider intentions and behaviors associated with innovation and effectiveness.

Increasing clinicians' EBT exploration and preparation behavior in youth mental health services by changing organizational culture with ARC

OBJECTIVE

Clinician EBT exploration and preparation behavior is essential to the ongoing implementation of new EBTs. This study examined the effect of the ARC organizational intervention on clinician EBT exploration and preparation behavior and assessed the mediating role of organizational culture as a linking mechanism.

METHOD

Fourteen community mental health agencies that serve youth in a major Midwestern metropolis along with 475 clinicians who worked in those agencies, were randomly assigned to either the three-year ARC intervention or control condition. Organizational culture was assessed with the Organizational Social Context (OSC) measure at baseline and follow-up. EBT exploration and preparation behavior was measured as clinician participation in nine separate community EBT workshops held over a three-year period.

RESULTS

There was 69 percent greater odds (OR = 1.69, p < .003) of clinicians in the ARC condition (versus control) attending each subsequent workshop. Workshop attendance in the control group remained under two percent and declined over three years while attendance in the ARC condition grew from 3.6 percent in the first workshop to 12 percent in the ninth and final workshop. Improvement in proficient organizational culture mediated the positive effect of the ARC intervention on clinicians' workshop attendance (a × b = .21; 95% CI:LL = .05, UL = .40).

CONCLUSIONS

Organizational interventions that create proficient mental health agency cultures can increase clinician EBT exploration and preparation behavior that is essential to the ongoing implementation of new EBTs in community youth mental health settings.

An ARC-Informed Family Centered Care Intervention for Children's Community Based Mental Health Programs

The experience of parents in helping their children access and use mental health services is linked to service outcomes. Parent peer support service, based on the principles of family-centered care, is one model to improve parent experience and engagement in services. Yet, little is known about how best to integrate this service into the existing array of mental health services. Integration is challenged by philosophical differences between family-centered services and traditional children's treatment services, and is influenced by the organizational social contexts in which these services are embedded. We describe an organizational and frontline team intervention that draws on research in behavior change, technology transfer, and organizational social context for youth with serious emotional disturbance. The two-pronged intervention, called FAMILY (FCC and ARC Model to Improve the Lives of Youth) is guided by the evidence-based Availability, Responsiveness, and Continuity (ARC) organizational intervention, targeted primarily at program and upper management leadership and includes a family-centered care (FCC) intervention, targeted at frontline providers. The approach employs multilevel implementation strategies to promote the uptake, implementation and sustainability of new practices. We include examples of exercises and tools, and highlight implementation challenges and lessons learned in facilitating program and staff level changes in family-centered service delivery.

The Family Value of Information, Community Support, and Experience Study: Rationale, Design, and Methods of a "Family-Centered" Research Study

The Patient Protection and Affordable Care Act focuses on improving consumer engagement and patient-centered care. This article describes the design and rationale of a study targeting family engagement in pediatric mental health services. The study is a 90-day randomized trial of a telephone-delivered Family Navigator services versus usual care for parents of Medicaid-insured youth younger than 13 years with serious mental illness. Youth are identified through a pediatric antipsychotic medication preauthorization program. Family Navigators offer peer support to empower and engage parents in their child's recovery. Outcomes include parent report of empowerment, social support, satisfaction with child mental health services, and child functioning as well as claims-based measures of psychotherapy service utilization and antipsychotic medication dosage. The focus on "family-centered" care in this study is strongly supported by the active role of consumers in study design and implementation.

Future Directions for Dissemination and Implementation Science: Aligning Ecological Theory and Public Health to Close the Research to Practice Gap

Dissemination and implementation science (DI) has evolved as a major research model for children's mental health in response to a long-standing call to integrate science and practice and bridge the elusive research to practice gap. However, to address the complex and urgent needs of the most vulnerable children and families, future directions for DI require a new alignment of ecological theory and public health to provide effective, sustainable, and accessible mental health services. We present core principles of ecological theory to emphasize how contextual factors impact behavior and allow for the reciprocal impact individuals have on the settings they occupy, and an alignment of these principles with a public health model to ensure that services span the prevention to intervention continuum. We provide exemplars from our ongoing work in urban schools and a new direction for research to address the mental health needs of immigrant Latino families. Through these examples we illustrate how DI can expand its reach by embedding within natural settings to build on local capacity and indigenous resources, incorporating the local knowledge necessary to more substantively address long-standing mental health disparities. This paradigm shift for DI, away from an overemphasis on promoting program adoption, calls for fitting interventions within settings that matter most to children's healthy development and for utilizing and strengthening available community resources. In this way, we can meet the challenge of addressing our nation's mental health burden by supporting the needs and values of families and communities within their own unique social ecologies.

Parent-to-parent support for parents with children who are deaf or hard of hearing: a conceptual framework

BACKGROUND

Parent-to-parent support for parents with children who are deaf or hard of hearing (D/HH) is identified as an important component of Early Hearing Detection and Intervention (EHDI) programs for children with hearing loss.

PURPOSE

The specific aim of this review was to identify the constructs and components of parent-to-parent support for parents of children who are D/HH.

RESEARCH DESIGN

An extensive scoping literature review identified 39 peer-reviewed articles published from 2000 to 2014. Studies were selected and reviewed based on standardized procedures.

RESULTS

Data were identified, extracted, and organized into libraries of thematic and descriptive content. A conceptual framework of parent-to-parent support for parents of children who are D/HH was developed and presented in a comprehensive, bidirectional informational graphic. The constructs and components of the conceptual framework are (a) well-being: parent, family, and child; (b) knowledge: advocacy, system navigation, and education; and (c) empowerment: confidence and competence.

CONCLUSION

The findings from this scoping review led to the development of a structured conceptual framework of parent-to-parent support for parents of children who are D/HH. The conceptual framework provides an important opportunity to explore and clearly define the vital contribution of parents in EHDI programs.

Scaling up evidence-based practices for children and families in New York State: toward evidence-based policies on implementation for state mental health systems

Dissemination of innovations is widely considered the sine qua non for system improvement. At least two dozen states are rolling out evidence-based mental health practices targeted at children and families using trainings, consultations, webinars, and learning collaboratives to improve quality and outcomes. In New York State (NYS) a group of researchers, policymakers, providers, and family support specialists have worked in partnership since 2002 to redesign and evaluate the children's mental health system. Five system strategies driven by empirically based practices and organized within a state-supported infrastructure have been used in the child and family service system with more than 2,000 providers: (a) business practices, (b) use of health information technologies in quality improvement, (c) specific clinical interventions targeted at common childhood disorders, (d) parent activation, and (e) quality indicator development. The NYS system has provided a laboratory for naturalistic experiments. We describe these initiatives, key findings and challenges, lessons learned for scaling, and implications for creating evidence-based implementation policies in state systems.

Developing quality indicators for family support services in community team-based mental health care

Quality indicators for programs integrating parent-delivered family support services for children's mental health have not been systematically developed. Increasing emphasis on accountability under the Affordable Care Act highlights the importance of quality-benchmarking efforts. Using a modified Delphi approach, quality indicators were developed for both program level and family support specialist level practices. These indicators were pilot tested with 21 community-based mental health programs. Psychometric properties of these indicators are reported; variations in program and family support specialist performance suggest the utility of these indicators as tools to guide policies and practices in organizations that integrate parent-delivered family support service components.

Quality indicators for multidisciplinary team functioning in community-based children's mental health services

The current study examined the organizational social context in 21 community-based programs serving youth at-risk for out-of-home care due to emotional or behavioral disorders and their families and program performance on five quality indicators of team functioning in teams that included a family support specialist. Results indicate that programs with higher performance on structures to facilitate teamwork, informal communication mechanisms among team members, and the ability to integrate family support specialists as equal members of the team showed more positive organizational functioning. Implications for the role of quality indicators in health care reform efforts are discussed.

Reducing turnover is not enough: The need for proficient organizational cultures to support positive youth outcomes in child welfare

OBJECTIVE

High caseworker turnover has been identified as a factor in the poor outcomes of child welfare services. However, almost no empirical research has examined the relationship between caseworker turnover and youth outcomes in child welfare systems and there is an important knowledge gap regarding whether, and how, caseworker turnover relates to outcomes for youth. We hypothesized that the effects of caseworker turnover are moderated by organizational culture such that reduced caseworker turnover is only associated with improved youth outcomes in organizations with proficient cultures.

METHODS

The study applied hierarchical linear models (HLM) analysis to the second National Survey of Child and Adolescent Well-being (NSCAW II) with a U.S. nationwide sample of 2,346 youth aged 1.5- to 18-years-old and 1,544 caseworkers in 73 child welfare agencies. Proficient organizational culture was measured by caseworkers' responses to the Organizational Social Context (OSC) measure; staff turnover was reported by the agencies' directors; and youth outcomes were measured as total problems in psychosocial functioning with the Child Behavior Checklist (CBCL) completed by the youths' caregivers at intake and at 18 month follow-up.

RESULTS

The association between caseworker turnover and youth outcomes was moderated by organizational culture. Youth outcomes were improved with lower staff turnover in proficient organizational cultures and the best outcomes occurred in organizations with low turnover and high proficiency.

CONCLUSIONS

To be successful, efforts to improve child welfare services by lowering staff turnover must also create proficient cultures that expect caseworkers to be competent and responsive to the needs of the youth and families they serve.