Testing a theory of strategic implementation leadership, implementation climate, and clinicians' use of evidence-based practice: a 5-year panel analysis

Rapid jail-based implementation of overdose education and naloxone distribution in response to the COVID-19 pandemic

BACKGROUND

People incarcerated in jails are highly impacted by the opioid epidemic, and overdose education and naloxone distribution (OEND) is an effective strategy to reduce opioid overdose deaths. This study examines barriers and facilitators of fast-track OEND implementation within the jails in the Wave 1 Kentucky counties of the HEALing Communities Study during the COVID-19 pandemic.

METHODS

Meeting minutes with jail stakeholders were qualitatively coded using the Practical, Robust Implementation and Sustainability Model (PRISM) as the coding framework. The analysis highlighted the top barriers and facilitators to fast-track OEND implementation within the PRISM framework.

RESULTS

Space and staffing shortages related to the COVID-19 pandemic, disruptions in interorganizational programming from pandemic-related service suspensions, and a lack of technological solutions (e.g., reliable Internet access) for socially distanced delivery were the top barriers to fast-track OEND implementation. In addition, there were limitations on non-jail staff access to jails during COVID-19. Top facilitators included jail leadership support, the option to prioritize high-risk groups, and the incorporation of OEND processes into existing communications and management software. While the COVID-19 pandemic strained jail infrastructure, jail and partner agency collaboration led to creative implementation strategies for the successful integration of OEND into jail operations. Urban jails were more likely than rural jails to be early adopters of OEND during the public health emergency.

CONCLUSIONS

Understanding the barriers to and facilitators of OEND within jails will improve implementation efforts seeking to curb opioid overdose deaths. Jail leadership support and interorganizational efforts were key facilitators to implementation; therefore, it is recommended to increase buy-in with multiple agencies to promote success. Challenges brought on by COVID-19 have resulted in a need for innovative solutions for implementation.

CLINICAL TRIAL INFORMATION

ClinicalTrials.gov, NCT04111939, Submitted 30 September 2019, https://clinicaltrials.gov/study/NCT04111939?titles=HEALing%20Communities%20Study&rank=1 .

Randomized trial of the leadership and organizational change for implementation (LOCI) strategy in substance use treatment clinics

INTRODUCTION

Evidence-based practice (EBP) implementation represents a strategic change that requires alignment of leadership and support throughout organizations. Leadership and Organizational Change for Implementation (LOCI) is a multifaceted implementation strategy that aims to improve implementation leadership and climate within organizations through iterative cycles of leadership and climate assessment and feedback, leadership training and coaching, and strategic planning with upper-level leaders. This study tested the effects of LOCI on transformational and implementation leadership, implementation climate, implementation citizenship behavior, and EBP reach.

METHODS

A multiple cohort, cluster randomized trial tests the effect of LOCI in 60 clinics across nine behavioral health organizations in California and Arizona, USA. The study randomized clinics within organizations to either LOCI or a leadership training webinar control condition in three consecutive cohorts. Repeated web-based surveys of direct service providers (nLOCI = 201, nControl = 179) assessed leadership, implementation climate, and implementation citizenship over time. Multilevel autoregressive modeling was the primary statistical analysis such that providers (level-1) were nested within clinics (level-2). The study predicted between-condition differences at 4-, 8-, and 12-month follow-up assessments. Provider engagement in a fidelity monitoring process assessed reach of motivational interviewing (i.e., number of sessions recorded/submitted for fidelity coding). An independent sample t-test explored between condition differences in motivational interviewing reach.

RESULTS

Results indicated between condition differences at 4 months for implementation leadership, implementation climate, and implementation citizenship behavior such that greater improvements were evidenced in the LOCI condition compared to the control condition. Reach of MI was significantly greater in the LOCI vs control condition such that LOCI providers were significantly more likely to engage in the fidelity monitoring process (chi-square (1, n = 370) = 5.59, p = .018).

CONCLUSIONS

LOCI was developed based on organizational theories of strategic leadership and climate to affect organizational change processes that communicate that innovation implementation is expected, supported, and recognized as a value of the organization. The LOCI implementation strategy resulted in more positive hypothesized outcomes compared to the control condition. Organizational change strategies have utility for implementing health innovations in complex, multilevel contexts and for greater sustainment of facilitative leader behaviors, strategic implementation climate, and improved implementation outcomes.

TRIAL REGISTRATION

This study is registered with Clinicaltrials.gov gov (NCT03042832, 2 February 2017; retrospectively registered).

Statistical analysis plan for the NU IMPACT stepped-wedge cluster randomized trial

BACKGROUND

As part of the IMPACT Consortium of three effectiveness-implementation trials, the NU IMPACT trial was designed to evaluate implementation and effectiveness outcomes for an electronic health record (EHR)-embedded symptom monitoring and management program for outpatient cancer care. NU IMPACT uses a unique stepped-wedge cluster randomized design, involving six clusters of 26 clinics, for evaluation of implementation outcomes with an embedded patient-level randomized trial to evaluate effectiveness outcomes. Collaborative, consortium-wide efforts to ensure use of the most robust and recent analytic methodologies for stepped-wedge trials motivated updates to the statistical analysis plan for implementation outcomes in the NU IMPACT trial.

METHODS

In the updated statistical analysis plan for NU IMPACT, the primary implementation outcome patient adoption, as measured by clinic-level monthly proportions of patient engagement with the EHR-based cancer symptom monitoring system, will be analyzed using generalized least squares linear regression with auto-regressive errors and adjustment for cluster and time effects (underlying secular trends). A similar strategy will be used for secondary patient and provider implementation outcomes.

DISCUSSION

The analytic updates described here resulted from highly iterative, collaborative efforts among statisticians, implementation scientists, and trial leads in the IMPACT Consortium. This updated statistical analysis plan will serve as the a priori specified approach for analyzing implementation outcomes for the NU IMPACT trial.

Inter-organizational alignment and implementation outcomes in integrated mental healthcare for children and adolescents: a cross-sectional observational study

BACKGROUND

Integrated care involves care provided by a team of professionals, often in non-traditional settings. A common example worldwide is integrated school-based mental health (SBMH), which involves externally employed clinicians providing care at schools. Integrated mental healthcare can improve the accessibility and efficiency of evidence-based practices (EBPs) for vulnerable populations suffering from fragmented traditional care. However, integration can complicate EBP implementation due to overlapping organizational contexts, diminishing the public health impact. Emerging literature suggests that EBP implementation may benefit from the similarities in the implementation context factors between the different organizations in integrated care, which we termed inter-organizational alignment (IOA). This study quantitatively explored whether and how IOA in general and implementation context factors are associated with implementation outcomes in integrated SBMH.

METHODS

SBMH clinicians from community-based organizations (CBOs; nclinician = 27) and their proximal student-support school staff (nschool = 99) rated their schools and CBOs (clinician only) regarding general (organizational culture and molar climate) and implementation context factors (Implementation Climate and Leadership), and nine common implementation outcomes (e.g., treatment integrity, service access, acceptability). The levels of IOA were estimated by intra-class correlations (ICCs). We fitted multilevel models to estimate the standalone effects of context factors from CBOs and schools on implementation outcomes. We also estimated the 2-way interaction effects between CBO and school context factors (i.e., between-setting interdependence) on implementation outcomes.

RESULTS

The IOA in general context factors exceeded those of implementation context factors. The standalone effects of implementation context factors on most implementation outcomes were larger than those of general context factors. Similarly, implementation context factors between CBOs and schools showed larger 2-way interaction effects on implementation outcomes than general context factors.

CONCLUSIONS

This study preliminarily supported the importance of IOA in context factors for integrated SBMH. The findings shed light on how IOA in implementation and general context factors may be differentially associated with implementation outcomes across a broad array of integrated mental healthcare settings.

"Hypertension is such a difficult disease to manage": federally qualified health center staff- and leadership-perceived readiness to implement a technology-facilitated team-based hypertension model

BACKGROUND

Despite decades of evidence demonstrating the efficacy of hypertension care delivery in reducing morbidity and mortality, a majority of hypertension cases remain uncontrolled. There is an urgent need to elucidate and address multilevel facilitators and barriers clinical staff face in delivering evidence-based hypertension care, patients face in accessing it, and clinical systems face in sustaining it. Through a rigorous pre-implementation evaluation, we aimed to identify facilitators and barriers bearing the potential to affect the planned implementation of a multilevel technology-facilitated hypertension management trial across six primary care sites in a large federally qualified health center (FQHC) in New York City.

METHODS

During a dedicated pre-implementation period (3-9 months/site, 2021-2022), a capacity assessment was conducted by trained practice facilitators, including (1) online anonymous surveys (n = 124; 70.5% of eligible), (2) hypertension training analytics (n = 69; 94.5% of assigned), and (3) audio-recorded semi-structured interviews (n = 67; 48.6% of eligible) with FQHC leadership and staff. Surveys measured staff sociodemographic characteristics, adaptive reserve, evidence-based practice attitudes, and implementation leadership scores via validated scales. Training analytics, derived from end-of-course quizzes, included mean score and number attempts needed to pass. Interviews assessed staff-reported facilitators and barriers to current hypertension care delivery and uptake; following audio transcription, trained qualitative researchers employed a deductive coding approach, informed by the Consolidated Framework for Implementation Research (CFIR).

RESULTS

Most survey respondents reported moderate adaptive reserve (mean = 0.7, range = 0-1), evidence-based practice attitudes (mean = 2.7, range = 0-4), and implementation leadership (mean = 2.5, range = 0-4). Most staff passed training courses on first attempt and demonstrated high scores (means > 80%). Findings from interviews identified potential facilitators and barriers to implementation; specifically, staff reported that complex barriers to hypertension care, control, and clinical communication exist; there is a recognized need to improve hypertension care; in-clinic challenges with digital tool access imposes workflow delays; and despite high patient loads, staff are motivated to provide high-quality cares.

CONCLUSIONS

This study serves as one of the first to apply the CFIR to a rigorous pre-implementation evaluation within the understudied context of a FQHC and can serve as a model for similar trials seeking to identify and address contextual factors known to impact implementation success.

TRIAL REGISTRATION

ClinicalTrials.gov NCT03713515 , date of registration: October 19, 2018.

Strategies to implement evidence-informed decision making at the organizational level: a rapid systematic review

BACKGROUND

Achievement of evidence-informed decision making (EIDM) requires the integration of evidence into all practice decisions by identifying and synthesizing evidence, then developing and executing plans to implement and evaluate changes to practice. This rapid systematic review synthesizes evidence for strategies for the implementation of EIDM across organizations, mapping facilitators and barriers to the COM-B (capability, opportunity, motivation, behaviour) model for behaviour change. The review was conducted to support leadership at organizations delivering public health services (health promotion, communicable disease prevention) to drive change toward evidence-informed public health.

METHODS

A systematic search was conducted in multiple databases and by reviewing publications of key authors. Articles that describe interventions to drive EIDM within teams, departments, or organizations were eligible for inclusion. For each included article, quality was assessed, and details of the intervention, setting, outcomes, facilitators and barriers were extracted. A convergent integrated approach was undertaken to analyze both quantitative and qualitative findings.

RESULTS

Thirty-seven articles are included. Studies were conducted in primary care, public health, social services, and occupational health settings. Strategies to implement EIDM included the establishment of Knowledge Broker-type roles, building the EIDM capacity of staff, and research or academic partnerships. Facilitators and barriers align with the COM-B model for behaviour change. Facilitators for capability include the development of staff knowledge and skill, establishing specialized roles, and knowledge sharing across the organization, though staff turnover and subsequent knowledge loss was a barrier to capability. For opportunity, facilitators include the development of processes or mechanisms to support new practices, forums for learning and skill development, and protected time, and barriers include competing priorities. Facilitators identified for motivation include supportive organizational culture, expectations for new practices to occur, recognition and positive reinforcement, and strong leadership support. Barriers include negative attitudes toward new practices, and lack of understanding and support from management.

CONCLUSION

This review provides a comprehensive analysis of facilitators and barriers for the implementation of EIDM in organizations for public health, mapped to the COM-B model for behaviour change. The existing literature for strategies to support EIDM in public health illustrates several facilitators and barriers linked to realizing EIDM. Knowledge of these factors will help senior leadership develop and implement EIDM strategies tailored to their organization, leading to increased likelihood of implementation success.

REVIEW REGISTRATION

PROSPERO CRD42022318994.

Improving measurement-based care implementation in youth mental health through organizational leadership and climate: a mechanistic analysis within a randomized trial

BACKGROUND

Theory and correlational research indicate organizational leadership and climate are important for successful implementation of evidence-based practices (EBPs) in healthcare settings; however, experimental evidence is lacking. We addressed this gap using data from the WISDOM (Working to Implement and Sustain Digital Outcome Measures) hybrid type III effectiveness-implementation trial. Primary outcomes from WISDOM indicated the Leadership and Organizational Change for Implementation (LOCI) strategy improved fidelity to measurement-based care (MBC) in youth mental health services. In this study, we tested LOCI's hypothesized mechanisms of change, namely: (1) LOCI will improve implementation and transformational leadership, which in turn will (2) mediate LOCI's effect on implementation climate, which in turn will (3) mediate LOCI's effect on MBC fidelity.

METHODS

Twenty-one outpatient mental health clinics serving youth were randomly assigned to LOCI plus MBC training and technical assistance or MBC training and technical assistance only. Clinicians rated their leaders' implementation leadership, transformational leadership, and clinic implementation climate for MBC at five time points (baseline, 4-, 8-, 12-, and 18-months post-baseline). MBC fidelity was assessed using electronic metadata for youth outpatients who initiated treatment in the 12 months following MBC training. Hypotheses were tested using longitudinal mixed-effects models and multilevel mediation analyses.

RESULTS

LOCI significantly improved implementation leadership and implementation climate from baseline to follow-up at 4-, 8-, 12-, and 18-month post-baseline (all ps < .01), producing large effects (range of ds = 0.76 to 1.34). LOCI's effects on transformational leadership were small at 4 months (d = 0.31, p = .019) and nonsignificant thereafter (ps > .05). LOCI's improvement of clinic implementation climate from baseline to 12 months was mediated by improvement in implementation leadership from baseline to 4 months (proportion mediated [pm] = 0.82, p = .004). Transformational leadership did not mediate LOCI's effect on implementation climate (p = 0.136). Improvement in clinic implementation climate from baseline to 12 months mediated LOCI's effect on MBC fidelity during the same period (pm = 0.71, p = .045).

CONCLUSIONS

LOCI improved MBC fidelity in youth mental health services by improving clinic implementation climate, which was itself improved by increased implementation leadership. Fidelity to EBPs in healthcare settings can be improved by developing organizational leaders and strong implementation climates.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT04096274. Registered September 18, 2019.

A mixed-methods evaluation of organization and individual factors influencing provider intentions to use caregiver coaching in community-based early intervention

BACKGROUND

Most psycho-social interventions contain multiple components. Practitioners often vary in their implementation of different intervention components. Caregiver coaching is a multicomponent intervention for young autistic children that is highly effective but poorly implemented in community-based early intervention (EI). Previous research has shown that EI providers' intentions, and the determinants of their intentions, to implement caregiver coaching vary across components. Organizational culture and climate likely influence these psychological determinants of intention by affecting beliefs that underlie attitudes, norms, and self-efficacy to implement an intervention. Research in this area is limited, which limits the development of theoretically driven, multilevel implementation strategies to support multi-component interventions. This mixed methods study evaluated the relationships among organizational leadership, culture and climate, attitudes, norms, self-efficacy, and EI providers' intentions to implement the components of caregiver coaching.

METHODS

We surveyed 264 EI providers from 37 agencies regarding their intentions and determinants of intentions to use caregiver coaching. We also asked questions about the organizational culture, climate, and leadership in their agencies related to caregiver coaching. We used multilevel structural equation models to estimate associations among intentions, psychological determinants of intentions (attitudes, descriptive norms, injunctive norms, and self-efficacy), and organizational factors (implementation climate and leadership). We conducted qualitative interviews with 36 providers, stratified by strength of intentions to use coaching. We used mixed-methods analysis to gain an in-depth understanding of the organization and individual-level factors.

RESULTS

The associations among intentions, psychological determinants of intentions, and organizational factors varied across core components of caregiver coaching. Qualitative interviews elucidated how providers describe the importance of each component. For example, providers' attitudes toward coaching caregivers and their perceptions of caregivers' expectations for service were particularly salient themes related to their use of caregiver coaching.

CONCLUSION

Results highlight the importance of multi-level strategies that strategically target individual intervention components as well as organization-level and individual-level constructs. This approach holds promise for improving the implementation of complex, multicomponent, psychosocial interventions in community-based service systems.

Exploring Shared Implementation Leadership of Point of Care Nursing Leadership Teams on Inpatient Hospital Units: Protocol for a Collective Case Study

BACKGROUND

Nursing leadership teams at the point of care (POC), consisting of both formal and informal leaders, are regularly called upon to support the implementation of evidence-based practices (EBPs) in hospital units. However, current conceptualizations of effective leadership for successful implementation typically focus on the behaviors of individual leaders in managerial roles. Little is known about how multiple nursing leaders in formal and informal roles share implementation leadership (IL), representing an important knowledge gap.

OBJECTIVE

This study aims to explore shared IL among formal and informal nursing leaders in inpatient hospital units. The central research question is as follows: How is IL shared among members of POC nursing leadership teams on inpatient hospital units? The subquestions are as follows: (1) What IL behaviors are enacted and shared by formal and informal leaders? (2) What social processes enable shared IL by formal and informal leaders? and (3) What factors influence shared IL in nursing leadership teams?

METHODS

We will use a collective case study approach to describe and generate an in-depth understanding of shared IL in nursing. We will select nursing leadership teams on 2 inpatient hospital units that have successfully implemented an EBP as instrumental cases. We will construct data through focus groups and individual interviews with key informants (leaders, unit staff, and senior nurse leaders), review of organizational documents, and researcher-generated field notes. We have developed a conceptual framework of shared IL to guide data analysis, which describes effective IL behaviors, formal and informal nursing leaders' roles at the POC, and social processes generating shared leadership and influencing contextual factors. We will use the Framework Method to systematically generate data matrices from deductive and inductive thematic analysis of each case. We will then generate assertions about shared IL following a cross-case analysis.

RESULTS

The study protocol received research ethics approval (2022-8408) on February 24, 2022. Data collection began in June 2022, and we have recruited 2 inpatient hospital units and 25 participants. Data collection was completed in December 2023, and data analysis is ongoing. We anticipate findings to be published in a peer-reviewed journal by late 2024.

CONCLUSIONS

The anticipated results will shed light on how multiple and diverse members of the POC nursing leadership team enact and share IL. This study addresses calls to advance knowledge in promoting effective implementation of EBPs to ensure high-quality health care delivery by further developing the concept of shared IL in a nursing context. We will identify strategies to strengthen shared IL in nursing leadership teams at the POC, informing future intervention studies.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

DERR1-10.2196/54681.

Climates and associated factors for evidence-based practice implementation among nurses: a cross-sectional study

BACKGROUND

The organizational climate that fosters and supports the implementation of evidence is a key factor influencing the effective implementation of evidence-based practice (EBP). Nurses, being the largest group of medical staff, play a crucial role in EBP implementation. The perception of the climate for EBP implementation among nurses in their organizations is unknown, especially among Chinese nurses.

AIMS

To clarify the implementation climate of EBP among Chinese nurses and identify the factors associated with the implementation and development of evidence-based nursing practices.

METHODS

This study employed a descriptive cross-sectional study design. From March 2023 to April 2023, a sample of nurses (n = 1260) from two Tertiary care hospitals in central China were selected and surveyed by self-designed social-demographic questionnaire and Implementation Climate Scale. Multiple linear stepwise regression analysis was conducted to determine the predictors of implementation climate.

RESULTS

The nurses achieved a mean ICS score of 59.10 ± 11.22, with mean scores exceeding 3 points for each sub-dimension and item. In the results of multiple linear regression, income satisfaction, implementation of evidence-based nursing practice project(s) within the unit, experience of evidence-based nursing practice during working life, and specific training or courses in evidence-based nursing practice were predictors of ICS. Of these, income satisfaction was the most significant predictor. These factors could explain 17.5% of the total variance in implementation climate.

CONCLUSION

The climate for implementing EBP in Chinese nursing organizations was relatively strong. Nursing managers can enhance the climate for implementing EBP in their organizations by actively improving salary and enhancing EBP-related trainings and practices.

RELEVANCE TO CLINICAL PRACTICE

Understanding nurses' perceptions of the EBP implementation climate in their organizations can help to identify specific barriers and facilitators to the development of EBP and facilitate its successful implementation.

PATIENT OR PUBLIC CONTRIBUTION

Clinical nurses were involved in data collection and completed the questionnaires on EBP implementation climate.

Inter-Organizational Alignment and Implementation Outcomes in Integrated Mental Healthcare for Children and Adolescents: A Cross-Sectional Observational Study

Background

Integrated care involves care provided by a team of professionals, often in non-traditional settings. A common example worldwide is integrated school-based mental health (SBMH), which involves externally employed clinicians providing care at schools. Integrated mental healthcare can improve the accessibility and efficiency of evidence-based practices (EBPs) for vulnerable populations suffering from fragmented traditional care. However, integration can complicate EBP implementation due to overlapping organizational contexts, diminishing the public health impact. Emerging literature suggests that EBP implementation may benefit from the similarities in the implementation context factors between the different organizations in integrated care, which we termed inter-organizational alignment (IOA). This study quantitatively explored whether and how IOAs in general and implementation context factors are associated with implementation outcomes in integrated SBMH.

Methods

SBMH clinicians from community-based organizations (CBOs; nclinician=27) and their proximal student-support school staff (nschool=99) rated their schools and CBOs (clinician only) regarding general (organizational culture and molar climate) and implementation context factors (Implementation Climate and Leadership), and nine common implementation outcomes (e.g., treatment integrity, service access, acceptability). The levels of IOA were estimated by intra-class correlations (ICCs). We fitted multilevel models to estimate the standalone effects of context factors from CBOs and schools on implementation outcomes. We also estimated the 2-way interaction effects between CBO and school context factors (i.e., between-setting interdependence) on implementation outcomes.

Results

The IOA in general context factors exceeded those of implementation context factors. The standalone effects of implementation context factors on most implementation outcomes were larger than those of general context factors. Similarly, implementation context factors between CBOs and schools showed larger 2-way interaction effects on implementation outcomes than general context factors.

Conclusions

This study preliminarily supported the importance of IOA in context factors for integrated SBMH. The findings shed light on how IOA in implementation and general context factors may be differentially associated with implementation outcomes across a broad array of integrated mental healthcare settings.

Uncovering determinants of perceived feasibility of TF-CBT through coincidence analysis

Introduction

A mental health provider's perception of how well an intervention can be carried out in their context (i.e., feasibility) is an important implementation outcome. This article aims to identify determinants of feasibility of trauma-focused cognitive behavioral therapy (TF-CBT) through a case-based causal approach.

Method

Data come from an implementation-effectiveness study in which lay counselors (teachers and community health volunteers) implemented a culturally adapted manualized mental health intervention, TF-CBT, delivered to teens who were previously orphaned and were experiencing posttraumatic stress symptoms and prolonged grief in Western Kenya. The intervention team identified combinations of determinants that led to feasibility among teacher- and community health volunteer-counselors through coincidence analysis.

Results

Among teacher-counselors, organizational-level factors (implementation climate, implementation leadership) determined moderate and high levels of feasibility. Among community health volunteer-counselors, a strong relationship between a clinical supervisor and the supervisee was the most influential determinant of feasibility.

Conclusion

Methodology and findings from this article can guide the assessment of determinants of feasibility and the development of implementation strategies for manualized mental health interventions in contexts like Western Kenya.

Plain Language Summary

A mental health provider's perception of how easy a therapy is to use in their work setting (i.e., feasibility) can impact whether the provider uses the therapy in their setting. Implementation researchers have recommended finding practices and constructs that lead to important indicators that a therapy will be used. However, limited research to our knowledge has searched and found practices and constructs that might determine feasibility of a therapy. This article uses existing data from a large trial looking at the continued use of a trauma-focused therapy to find practices and constructs that lead to moderate and high levels of feasibility. We found that in settings with a strong organizational structure that organization and leadership support for the therapy led to teachers in Kenya to perceive the therapy as easy to use. On the other hand, in settings with a weaker organizational structure, outside support from a clinical supervisor led to community health volunteers in Kenya perceiving the therapy as easy to use. The findings from this article can guide context-specific recommendations for increasing perceived therapy feasibility at the provider-, organization-, and policy levels.

Implementation leadership and implementation climate in context: A single organization intrinsic case study for implementation of digital measurement-based care

Background

Although studies have demonstrated that implementation leadership and climate are important constructs in predicting evidence-based practice (EBP) implementation, concrete descriptions of how they operate during organizational implementation efforts are lacking. This case study fills that gap through an in-depth description of an organization with effective implementation leadership that successfully built a strong implementation climate. This case study provides an illustration of implementation leadership and climate in tangible, replicable terms to assist managers, practitioners, and researchers in addressing the organizational context in their own implementation projects.

Method

A single organization, intrinsic case study was employed to paint a multifaceted picture of how one organization leveraged implementation leadership to strengthen a climate for the successful implementation of digital measurement-based care. The case was drawn from a cluster-randomized trial designed to test the effects of a leadership-focused implementation strategy on youth-level fidelity and clinical outcomes of digital measurement-based care. Following the completion of the trial, case study activities commenced. Descriptive summaries of multiple data sources (including quantitative data on implementation leadership and climate, coaching call and organizational alignment meeting recordings and notes, and development plans) were produced and revised iteratively until consensus was reached. Leadership actions were analyzed for corresponding dimensions of implementation leadership and climate.

Results

Specific actions organizational leaders took, as well as the timing specific strategies were enacted, to create a climate for implementation are presented, along with lessons learned from this experience.

Conclusion

This case study offers concrete steps organizational leaders took to create a consistent and aligned message that the implementation of a specific EBP was a top priority in the agency. The general approach taken to create an implementation climate provides several lessons for leaders, especially for EBPs that have broad implications across an organization.

Helping Educational Leaders Mobilize Evidence (HELM): The iterative redesign of the Leadership and Organizational Change for Implementation (LOCI) intervention for use in schools

Background

Few "intervention agnostic" strategies have been developed that can be applied to the broad array of evidence-based practices (EBPs) in schools. This paper describes two studies that reflect the initial iterative redesign phases of an effective leadership-focused implementation strategy-Leadership and Organizational Change for Implementation (LOCI)-to ensure its acceptability, feasibility, contextual appropriateness, and usability when used in elementary schools. Our redesigned strategy-Helping Educational Leaders Mobilize Evidence (HELM)-is designed to improve principals' use of strategic implementation leadership to support the adoption and high-fidelity delivery of a universal EBP to improve student outcomes.

Method

In Study 1, focus groups were conducted (n = 6) with 54 district administrators, principals, and teachers. Stakeholders provided input on the appropriateness of original LOCI components to maximize relevance and utility in schools. Transcripts were coded using conventional content analysis. Key themes referencing low appropriateness were summarized to inform LOCI adaptations. We then held a National Expert Summit (Study 2) with 15 research and practice experts. Participants provided feedback via a nominal group process (NGP; n = 6 groups) and hackathon (n = 4 groups). The research team rated each NGP suggestion for how actionable, impactful/effective, and feasible it was. We also coded hackathon notes for novel ideas or alignment with LOCI components.

Results

Study 1 suggestions included modifications to LOCI content and delivery. Study 2's NGP results revealed most recommendations to be actionable, impactful/effective, and feasible. Hackathon results surfaced two novel ideas (distributed leadership teams and leaders' knowledge to support educators EBP use) and several areas of alignment with LOCI components.

Conclusion

Use of these iterative methods informed the redesign of LOCI and the development of HELM. Because it was collaboratively constructed, HELM has the potential to be an effective implementation strategy to support the use of universal EBP in schools.

An evaluation of implementation climate in inpatient maternity care: a cross-sectional survey study

OBJECTIVE

Researchers in obstetrics and gynecology are continuously generating new evidence to inform clinical care delivery. Yet, much of this newly emerging evidence fails to be rapidly and effectively integrated into routine clinical practice. Implementation climate refers to clinicians' perceptions of to what degree organizations support and reward use of an evidence-based practice (EBP) and is an important construct in the science of implementation in healthcare. Little is known about implementation climate for EBPs in maternity care. Thus, we aimed to (a) determine the reliability of the Implementation Climate Scale (ICS) for use in inpatient maternity care, (b) describe implementation climate in inpatient maternity care overall, and (c) compare individual perceptions of implementation climate between physician and nursing clinicians on these units.

STUDY DESIGN

We performed a cross-sectional survey of clinicians in inpatient maternity units across 2 urban, academic hospitals in the northeastern United States in 2020. Clinicians completed the 18-question validated ICS [scored 0-4]. Scale reliability by role was evaluated using Cronbach's α. Subscale and total scores were described overall and compared by physician versus nursing role using independent t-tests, as well as linear regression to control for confounders.

RESULTS

111 clinicians completed the survey (physicians = 65; nursing = 46). Physicians were less likely to identify as female (75.4% vs. 100.0%, p < .001), but were of similar age and years of experience as nursing clinicians. Reliability of the ICS was excellent, with Cronbach's α of 0.91 and 0.86 among physicians and nursing clinicians, respectively. Scores were notably low for implementation climate in maternity care overall and for all subscales. ICS total scores were also higher among physicians as compared to nurses (2.18(±0.56) vs. 1.92(±0.50), p = .02), which remained significant in multivariable modeling (p = .02). Unadjusted subscale scores were higher among physicians in Recognition for EBP (2.68(±0.89) vs. 2.30(±0.86), p = .03) and Selection for EBP (2.24(±0.93) vs. 1.62(±1.04), p = .002). After adjustment for potential confounders, subscale scores for Focus on EBP (p = .04) and Selection for EBP (p = .002) were all higher among physicians.

CONCLUSIONS

This study supports the ICS as a reliable scale for measuring implementation climate in the inpatient maternity care setting. Notably lower implementation climate scores across subscales and roles compared to other settings may underlie the vast evidence to practice gap in obstetrics. In order to successfully implement practices that reduce maternal morbidity, we may need to focus on building educational support and rewarding EBP utilization on labor and delivery units, with an emphasis on nursing clinicians.

Eliminating Monitor Overuse (EMO) type III effectiveness-deimplementation cluster-randomized trial: Statistical analysis plan

Background

Deimplementing overused health interventions is essential to maximizing quality and value while minimizing harm, waste, and inefficiencies. Three national guidelines discourage continuous pulse oximetry (SpO2) monitoring in children who are not receiving supplemental oxygen, but the guideline-discordant practice remains prevalent, making it a prime target for deimplementation. This paper details the statistical analysis plan for the Eliminating Monitor Overuse (EMO) SpO2 trial, which compares the effect of two competing deimplementation strategies (unlearning only vs. unlearning plus substitution) on the sustainment of deimplementation of SpO2 monitoring in children with bronchiolitis who are in room air.

Methods

The EMO Trial is a hybrid type 3 effectiveness-deimplementation trial with a longitudinal cluster-randomized design, conducted in Pediatric Research in Inpatient Settings Network hospitals. The primary outcome is deimplementation sustainment, analyzed as a longitudinal difference-in-differences comparison between study arms. This analysis will use generalized hierarchical mixed-effects models for longitudinal clustering outcomes. Secondary outcomes include the length of hospital stay and oxygen supplementation duration, modeled using linear mixed-effects regressions. Using the well-established counterfactual approach, we will also perform a mediation analysis of hospital-level mechanistic measures on the association between the deimplementation strategy and the sustainment outcome.

Discussion

We anticipate that the EMO Trial will advance the science of deimplementation by providing new insights into the processes, mechanisms, and likelihood of sustained practice change using rigorously designed deimplementation strategies. This pre-specified statistical analysis plan will mitigate reporting bias and support data-driven approaches.

Trial registration

ClinicalTrials.gov NCT05132322. Registered on 24 November 2021.

Top-level leaders and implementation strategies to support organizational diversity, equity, inclusion, and belonging (DEIB) interventions: a qualitative study of top-level DEIB leaders in healthcare organizations

BACKGROUND

The Black Lives Matter movement and COVID-19 pandemic motivated the wide-scale adoption of diversity, equity, inclusion, and belonging (DEIB) initiatives within healthcare organizations and the creation of DEIB top-level leader positions. The next step is to understand how these leaders contribute to the implementation of DEIB interventions, a task with notable salience due to not only the historical difficulties associated with DEIB strategy execution, but also the substantial evidence that leadership plays a significant role in implementation processes. Therefore, the objective of this qualitative study is to understand the role of top-level DEIB leaders in the implementation of healthcare organizational DEIB interventions.

METHODS

A qualitative research approach which used an in-depth semi-structured interview approach was employed. We conducted thirty-one 60-90-min semi-structured interviews with DEIB top-level leaders between February 2022 and October 2022 over Zoom. An iterative coding process was used to identify the key implementation strategies and activities of DEIB top-level leaders.

RESULTS

Interviewees were mostly Black, majority female, and mostly heterosexual and had a variety of educational backgrounds. We identified the DEIB top-level leader as the DEIB strategy implementation champion. These leaders drive five DEIB implementation strategies: (1) People, (2) Health Equity, (3) Monitoring and Feedback, (4) Operational Planning and Communication, and (5) External Partners. Within these, we identified 19 significant activities that describe the unique implementation strategies supported by the DEIB top-level leaders.

CONCLUSIONS

To move toward sustained commitment to DEIB, the organization must focus on not only establishing DEIB interventions, but on their successful implementation. Our findings help explicate the implementation activities that drive the DEIB initiatives of healthcare organizations and the role of DEIB leaders. Our work can help healthcare organizations systematically identify how to support the success of DEIB organizational interventions.

The role of therapy delivery and clinic organizational factors in explaining therapist effects for trauma-focused psychotherapies in the Veterans Health Administration

OBJECTIVE

This study estimated the size of therapist effects (TEs) for dropout and clinical effectiveness of two trauma-focused psychotherapies (TFPs) and evaluated whether therapy delivery and clinic organizational factors explained observed TEs.

METHOD

Participants were 180 therapists (54.4% psychologists, 42.2% social workers) from 137 Veterans Health Administration facilities and 1,735 patients (24.7% women; 27.2% people of color) who completed at least two TFP sessions. Outcomes were dropout (< 8 TFP sessions) and for a subsample (n = 1,273), clinically meaningful improvement and recovery based on posttraumatic stress disorder checklist for DSM-5 (PCL-5) scores. Therapist-level predictors were ascertained through survey, manual chart review, and administrative data. Multilevel models estimated TEs.

RESULTS

Over half (51.2%) of patients dropped out and those who dropped out were less likely to meet criteria for clinically meaningful improvement or recovery (ps < .001). Adjusting for case-mix and TFP type, therapists accounted for 5.812% (p < .001) of the unexplained variance in dropout. The average dropout rate for the 45 therapists in the top performing quartile was 27.0%, while the average dropout rate for the 45 therapists in the bottom performing quartile was 78.8%. Variation between therapists was reduced to 2.031% (p = .140) when therapists' mean of days between sessions, adherence, implementation climate, and caseload were added to multilevel models. TEs were nonsignificant for clinically meaningful improvement and recovery.

CONCLUSIONS

Interventions targeting therapy delivery and clinic organization have the potential to reduce variation between therapists in TFP dropout, so that more patients stay engaged long enough to experience clinical benefit. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A Structural Model of Organization and Clinician Factors Associated with Standardized Measure Use in a National Survey of Youth Mental Health Providers

Standardized assessment measures are important for accurate diagnosis of mental health problems and for treatment planning and evaluation. However, little is known about youth mental health providers' typical use of standardized measures across disciplines and outside the context of evidence-based practice initiatives. A multidisciplinary national survey examined the frequency with which 674 youth mental health providers administer standardized and unstandardized measures, and the extent to which organizational (i.e., implementation climate, rigid hierarchical organizational structure) and provider (i.e., attitudes toward standardized assessment measures, highest degree, practice setting) characteristics are associated with standardized measure use. Providers used unstandardized measures far more frequently than standardized measures. Providers' perceptions (a) that standardized measures are practical or feasible, (b) that their organization supports and values evidence-based practices, and (c) that their organization has a rigid hierarchical structure predicted greater use of standardized measures. Working in schools predicted less frequent SMU, while working in higher education and other professional settings predicted more frequent SMU. Standardized measures were not routinely used in this community-based sample. A rigid hierarchical organizational structure may be conducive to more frequent administration of standardized measures, but it is unclear whether such providers actually utilize these measures for clinical decision-making. Alternative strategies to promote standardized measure use may include promoting organizational cultures that value empirical data and encouraging use of standardized measures and training providers to use pragmatic standardized measures for clinical decision making.

The Price of Practice Change: Assessing the Cost of Integrating Research Findings Into Clinical Practice

BACKGROUND

Clinicians, health care administrators, and implementation scientists know that it takes intentional effort, resources, and implementation strategies to integrate research findings into routine clinical practice. An oft-cited concern for those considering whether and how to implement an evidence-based program is how much it will cost to implement the change. Yet information about the cost of implementation is not often available to health care decision-makers. Teams that received Implementation Award funding from PCORI are conducting implementation projects to promote the uptake of evidence-based practices in health care settings. As part of their implementation efforts, a number of teams have examined the costs of implementation. In this Topical Collection, 5 teams will report their findings on implementation costs and discuss their methods for data collection and analysis.

DISCUSSION

The teams' costing efforts provide specific information about the costs sites can expect to incur in promoting the uptake of specific evidence-based programs. In addition, the papers illuminate 3 key features of the teams' approaches to measuring the cost of implementation: (1) the use of specific micro-costing methods with time-driven activity-based costing serving as the most popular method; (2) different ways to categorize and organize costs, including a site-based and non-site-based framework; and (3) cost collection challenges experienced by the teams.

CONCLUSION

The cost of implementation is a critical consideration for organizations seeking to improve practice in accordance with research findings. This Topical Collection describes detailed approaches to providing this type of cost information and highlights insights to be gained from a rigorous focus on implementation cost.

Costs of Implementing Electronic Context Factor Assessments and Patient-reported Outcomes in Pain Clinic Settings

BACKGROUND

The Healing Encounters and Attitudes Lists (HEALs) patient-reported measures, consisting of 6 separate context factor questionnaires, predict patients' pain improvements. Our Patient-centered Outcomes Research Initiative-funded implementation project demonstrated success in using HEAL data during clinic consultations to enhance patient engagement, improve patient outcomes, and reduce opioid prescribing.

OBJECTIVE

We aimed to determine the resources needed for additional sites to implement HEAL to improve pain care treatment.

RESEARCH DESIGN

An observational study from March 1 to November 30, 2021, assessing implementation cost data from invoices, time and salary requirements for clinic personnel training, estimates of non-site-based costs, and one-time resource development costs.

SUBJECTS

Unique patients eligible to complete a HEAL survey (N=24,018) and 74 clinic personnel.

MEASURES

The Stages of Implementation Completion guided documentation of preimplementation, implementation, and sustainment activities of HEAL pain clinic operations. These informed the calculations of the costs of implementation.

RESULTS

The total time for HEAL implementation is 7 months: preimplementation and implementation phases (4 mo) and sustainment (3 mo). One hour of HEAL implementation involving a future clinical site consisting of 2 attending physicians, 1 midlevel provider, 1 nurse manager, 1 nurse, 1 radiology technician, 2 medical assistants, and 1 front desk staff will cost $572. A 10-minute time increment for all clinic staff is $95. Total implementation costs based on hourly rates over 7 months, including non-site-based costs of consultations, materials, and technology development costs, is $28,287.

CONCLUSIONS

Documenting our implementation costs clarifies the resources needed for additional new sites to implement HEAL to measure pain care quality and to engage patients and clinic personnel.

Individual-level associations between implementation leadership, climate, and anticipated outcomes: a time-lagged mediation analysis

BACKGROUND

Leaders can improve implementation outcomes by developing an organizational climate conducive to the implementation of evidence-based practices (EBP). This study tested the lagged associations between individual-level perceptions of implementation leadership, implementation climate, and three anticipated implementation outcomes, that is EBP acceptability, appropriateness, and feasibility.

METHODS

Screening tools and treatment methods for posttraumatic stress disorder were implemented in 43 Norwegian mental health services. A sample of 494 child and adult mental health care professionals (M = 43 years, 78% female) completed surveys addressing perceptions of first-level leaders' (n = 47) implementation leadership and their clinics' implementation climate. Single-level structural equation models estimating both direct, indirect, and total effects were used to investigate whether perceived implementation climate mediated the association between perceived implementation leadership and perceived acceptability, appropriateness, and feasibility of screening tools and treatment methods.

RESULTS

Regarding the treatment methods, implementation leadership was associated with therapists' perceptions of acceptability, appropriateness, and feasibility. Implementation climate also mediated between implementation leadership and the outcomes. Regarding the screening tools, implementation leadership was not associated with the outcomes. However, implementation climate mediated between implementation leadership and therapists' perceptions of acceptability and feasibility, but not appropriateness. Analyses with the implementation climate subscales showed stronger associations for therapists' perceptions of the treatment methods than of screening tools.

CONCLUSIONS

Leaders may promote positive implementation outcomes, both directly and through implementation climate. With regard to the effect sizes and explained variance, results indicated that both implementation leadership and implementation climate were more strongly associated with the therapists' perceptions of the treatment methods, implemented by one group of therapists, than the screening tools, implemented by all therapists. This may imply that implementation leadership and climate may have stronger effects for smaller implementation teams within a larger system than for system-wide implementations or when the clinical interventions being implemented are more complex rather than simple ones.

TRIAL REGISTRATION

ClinicalTrials NCT03719651, 25 October 2018.

Dynamic interplay between available resources and implementation climate across phases of implementation: a qualitative study of a VA national population health tool

BACKGROUND

Available resources within an organization can determine the implementation success of an intervention. However, few studies have investigated how the required resources change over the phases of implementation. Using stakeholder interviews, we examined the changes in and interactions between available resources and implementation climate in the implementation and sustainment phases of a national implementation effort for a population health tool.

METHODS

We conducted a secondary analysis of the interviews with 20 anticoagulation professionals at 17 clinical sites in the Veterans Health Administration health system about their experiences with a population health dashboard for anticoagulant management. Interview transcripts were coded using constructs from the Consolidated Framework for Implementation Research (CFIR) and according to the phase of implementation (pre-implementation, implementation, and sustainment) as defined by the VA Quality Enhancement Research Initiative (QUERI) Roadmap. We analyzed the factors that may determine successful implementation by examining the co-occurrence patterns between available resources and implementation climate across different implementation phases. To illustrate the variations in these determinants across phases, we aggregated and scored coded statements using a previously published CFIR scoring system (- 2 to + 2). Key relationships between available resources and implementation climate were identified and summarized using thematic analysis.

RESULTS

The resources necessary to support the successful implementation of an intervention are not static; both the quantity and types of resources shift based on the phases of the intervention. Furthermore, increased resource availability does not guarantee the sustainment of intervention success. Users need different types of support beyond the technical aspects of an intervention, and this support varies over time. Specifically, available resources in the form of technological support and social/emotional support help users establish trust in a new technological-based intervention during the implementation phase. Resources that foster and maintain collaboration between users and other stakeholders help them stay motivated during sustainment.

CONCLUSIONS

Our findings highlight the dynamic nature of available resources and their impacts on the implementation climate across different phases of implementation. A better understanding of the dynamics of available resources over time from the users' perspectives will allow the adaptation of resources to better meet the needs of the intervention stakeholders.

Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy

BACKGROUND

Valid and reliable measurement of implementation strategies is essential to advancing implementation science; however, this area lags behind the measurement of implementation outcomes and determinants. Clinical supervision is a promising and highly feasible implementation strategy in behavioral healthcare for which pragmatic measures are lacking. This research aimed to develop and psychometrically evaluate a pragmatic measure of clinical supervision conceptualized in terms of two broadly applicable, discrete clinical supervision techniques shown to improve providers' implementation of evidence-based psychosocial interventions-(1) audit and feedback and (2) active learning.

METHODS

Items were generated based on a systematic review of the literature and administered to a sample of 154 outpatient mental health clinicians serving youth and 181 community-based mental health providers serving adults. Scores were evaluated for evidence of reliability, structural validity, construct-related validity, and measurement invariance across the two samples.

RESULTS

In sample 1, confirmatory factor analysis (CFA) supported the hypothesized two-factor structure of scores on the Evidence-Based Clinical Supervision Strategies (EBCSS) scale (χ2=5.89, df=4, p=0.208; RMSEA=0.055, CFI=0.988, SRMR=0.033). In sample 2, CFA replicated the EBCSS factor structure and provided discriminant validity evidence relative to an established supervisory alliance measure (χ2=36.12, df=30, p=0.204; RMSEA=0.034; CFI=0.990; SRMR=0.031). Construct-related validity evidence was provided by theoretically concordant associations between EBCSS subscale scores and agency climate for evidence-based practice implementation in sample 1 (d= .47 and .55) as well as measures of the supervision process in sample 2. Multiple group CFA supported the configural, metric, and partial scalar invariance of scores on the EBCSS across the two samples.

CONCLUSIONS

Scores on the EBCSS provide a valid basis for inferences regarding the extent to which behavioral health providers experience audit and feedback and active learning as part of their clinical supervision in both clinic- and community-based behavioral health settings.

TRIAL REGISTRATION

ClinicalTrials.gov NCT04096274 . Registered on 19 September 2019.

Climate for evidence-based mental health care implementation in Germany: psychometric investigation of the Implementation Climate Scale (ICS)

Organizational implementation climate is an important construct in implementation research to describe to what extent implementation is expected, supported, and rewarded. Efforts in bridging the research-practice gap by implementing evidence-based practice (EBP) can benefit from consideration of implementation climate. The Implementation Climate Scale (ICS) is a psychometrically strong measure assessing employees' perceptions of the implementation climate. The present cross-sectional study aimed at providing a German translation and investigating its psychometric properties. The translation followed standard procedures for adapting psychometric instruments. German psychotherapists (N = 425) recruited online completed the ICS, the Evidence Based Practice Attitudes Scale (EBPAS-36D) and the Intention Scale for Providers (ISP). We conducted standard item and reliability analyses. Factorial validity was assessed by comparing an independent cluster model of Confirmatory Factorial Analysis (ICM-CFA), a Bifactor CFA, a Second-order CFA and an (Bifactor) Exploratory Structural Equation Model (ESEM). Measurement invariance was tested using multiple-group CFA and ESEM, convergent validity with correlation analysis between the ICS and the ISP subjective norms subscale (ISP-D-SN). The mean item difficulty was pi = .47, mean inter-item correlation r = .34, and mean item-total correlation ritc = .55. The total scale (ω = 0.91) and the subscales (ω = .79-.92) showed acceptable to high internal consistencies. The model fit indices were comparable and acceptable (Second-order CFA: RMSEA [90% CI] = .077 [.069; .085], SRMR = .078, CFI = .93). Multiple-group CFA and ESEM indicated scalar measurement invariance across gender and presence of a psychotherapy license. Psychotherapists in training reported higher educational support for EBP than licensed psychotherapists (T = 2.09, p = .037, d = 0.25). The expected high correlation between the ICS and the ISP-D-SN was found (r = .59, p < .001). Results for the German ICS confirm good psychometric properties including validity.

Expanding school wellness policies to encompass the Whole School, Whole Community, Whole Child model

Schools influence children’s developmental outcomes across multiple domains, including academic, social, emotional, behavioral, and physical. School district wellness policies are powerful mechanisms that set clear expectations for health-related practices in school buildings and the surrounding community. A current challenge is that many health-related school policies are narrow, siloed, and reactive instead of proactive. In this paper, we: (a) describe how written food, nutrition, and physical activity district and state policies were strengthened in the United States in response to specific concerns about childhood obesity; (b) present how schools have historically addressed policies concerning children’s social, emotional, and behavioral health; and (c) propose using the Whole School, Whole Community, Whole Child (WSCC) model to strengthen the coordination and integration of school wellness policies. We conclude by describing recently developed tools to assist school districts in implementing the WSCC model. The Wellness School Assessment Tool (WellSAT) WSCC is a quantitative measure that districts can use to code their current written policies for alignment with the WSCC model. The WSCC Policy and Practice Blueprints are action planning tools that lead school and district leaders through a series of activities to strengthen the implementation of coordinated and integrated policies and practices. By using the WSCC model and accompanying implementation tools, schools can support the development of the whole child.

Creating an Implementation Enhancement Plan for a Digital Patient Fall Prevention Platform Using the CFIR-ERIC Approach: A Qualitative Study

(1) Background: Inpatient falls are a major cause of hospital-acquired complications (HAC) and inpatient harm. Interventions to prevent falls exist, but it is unclear which are most effective and what implementation strategies best support their use. This study uses existing implementation theory to develop an implementation enhancement plan to improve the uptake of a digital fall prevention workflow. (2) Methods: A qualitative approach using focus groups/interview included 12 participants across four inpatient wards, from a newly built, 300-bed rural referral hospital. Interviews were coded to the Consolidated Framework for Implementation Research (CFIR) and then converted to barrier and enabler statements using consensus agreement. Barriers and enablers were mapped to the Expert Recommendations for Implementing Change (ERIC) tool to develop an implementation enhancement plan. (3) Results: The most prevalent CFIR enablers included: relative advantage (n = 12), access to knowledge and information (n = 11), leadership engagement (n = 9), patient needs and resources (n = 8), cosmopolitanism (n = 5), knowledge and beliefs about the intervention (n = 5), self-efficacy (n = 5) and formally appointed internal implementation leaders (n = 5). Commonly mentioned CFIR barriers included: access to knowledge and information (n = 11), available resources (n = 8), compatibility (n = 8), patient needs and resources (n = 8), design quality and packaging (n = 10), adaptability (n = 7) and executing (n = 7). After mapping the CFIR enablers and barriers to the ERIC tool, six clusters of interventions were revealed: train and educate stakeholders, utilize financial strategies, adapt and tailor to context, engage consumers, use evaluative and iterative strategies and develop stakeholder interrelations. (4) Conclusions: The enablers and barriers identified are similar to those described in the literature. Given there is close agreement between the ERIC consensus framework recommendations and the evidence, this approach will likely assist in enhancing the implementation of Rauland's Concentric Care fall prevention platform and other similar workflow technologies that have the potential to disrupt team and organisational routines. The results of this study will provide a blueprint to enhance implementation that will be tested for effectiveness at a later stage.

Implementation of physical activity interventions in a community-based youth mental healthcare service: A case study of context, strategies, and outcomes

AIMS

Physical activity interventions are recommended for community-based youth mental health services to prevent physical health disparities. Implementation is challenging, and studies focusing on the methods to achieve change are needed. This study aims to identify the context, implementation strategies, and implementation outcomes that illustrate how physical activity interventions were implemented within an early intervention service in Australia.

METHODS

A theoretically informed case study was undertaken. Data from a community-based youth mental health service that delivers an early psychosis programme were collected between July and November 2020. Three data sources were accessed (1) interviews with service managers, mental health clinicians and exercise physiologists; (2) document review of organizational policies and procedures; and (3) survey using the Program Sustainability Assessment Tool. The implementation outcomes investigated were acceptability, fidelity, penetration, and sustainability. Framework analysis was used, and a logic model developed guided by an established template, to interpret findings.

RESULTS

Forty-three contextual factors and 43 implementation strategies were identified. The data suggests that creating a new clinical team and auditing and feedback are critical for implementation. High levels of acceptability and sustainability were described, while fidelity of implementation was difficult to establish, and penetration was low.

CONCLUSIONS

The relationship between constructs suggests several mechanisms underpinned implementation. These include changing professional beliefs, establishing new organizational norms, augmenting existing work processes, and aligning physical activity with priorities of the mental healthcare system and existing work tasks. This case study provides direction for future health service planning of physical activity interventions in community-based youth mental health service.

Perceptual differences in nursing implementation leadership and climate: a cross-sectional study

BACKGROUND

The literature on perceptual differences between managers and staff regarding social dynamic factors (e.g., leadership, climate) in nursing settings is sparse. Addressing this gap in knowledge is critical for informing implementation efforts and improving patient and organizational outcomes. The purpose of this study was to test the perceptual differences regarding implementation leadership and implementation climate between nursing staff and their managers.

METHODS

This study was a secondary analysis of cross-sectional survey data collected in 2016-2017. The setting included 22 adult medical-surgical units nested in 7 acute care hospitals in the Eastern and Midwestern United States. Participants were registered nurses (N = 261) and nurse managers (N = 22) who completed an electronic survey consisting of the Implementation Leadership Scale (ILS), the Implementation Climate Scale (ICS), and demographic items. Differences in perception were analyzed at the unit level using structural equation modeling to develop latent difference score models (LDS). We assessed associations of the LDSs with manager ILS and ICS scores, years of nursing experience, and years of experience working on the current unit. The association of ILS LDS with the observed nursing staff ICS scores was also analyzed.

RESULTS

Higher manager scores on the ILS and ICS were associated with greater perceptual differences in implementation leadership and implementation climate. Greater years of experience as a nurse were associated with greater perceptual differences in ILS and ICS scores. Greater tenure on the unit was associated with smaller differences on the ILS knowledge domain. Greater perceptual differences regarding implementation leadership were associated with worse staff ratings of implementation climate.

CONCLUSIONS

Although this study observed significant relationships among manager ILS and ICS scores, staff-manager perceptual differences, and staff ratings of implementation climate in nursing settings, it is still unclear why perceptual differences in implementation leadership and climate exist and how to address them. Future studies are warranted to test the effect of perceptual differences on implementation and patient outcomes.

Implementation Leadership in the Point of Care Nursing Context: A Systematic Review Comparing Two Measurement Tools

Introduction

Implementation leadership (IL) are effective point of care (POC) nursing leadership behaviors that facilitate contexts conducive to the successful implementation of evidence-based practices (EBPs). However, no systematic evaluation of IL tools validated for the nursing context existed.

Aims

The purpose of this systematic review was to compare iterations of two IL measurement tools, the Implementation Leadership Scale (ILS) and the iLEAD, for application in a nursing context; and to critically appraise and summarize the methodological quality of studies assessing their psychometric properties.

Methods

A comprehensive search was conducted in four databases. Two reviewers independently screened titles and abstracts, reviewed full-text articles, and performed extraction into data tables. Statisticians appraised the quality control aspects. Findings were narratively summarized.

Results

A total of 247 records were included, where 10 for the ILS (including different versions) and one for the iLEAD met the inclusion criteria. Three studies evaluated the psychometric properties of the ILS in nursing, and its translations into Chinese and Greek. Content validity was deemed to be doubtful for both tools, but the ILS had adequate rating for comprehensiveness; methodological quality was very good for structural validity, internal consistency, hypothesis testing, and responsiveness where applicable for both scales, with the exception of cross-cultural validity which had ratings of adequate and inadequate for versions of the scales. Several study findings met the criteria for good measurement properties. No studies for either tool formally assessed feasibility.

Conclusion

Applying validated and contextually relevant tools to evaluate the capacity of nursing leadership to engage in IL in real-world contexts are needed. The ILS shows promise but requires further validation for contexts with diverse and multiple nursing leaders at the POC. Feasibility needs to be further studied.

Individual and organizational factors as predictors of early evidence-based practice adoption in Michigan high schools: Baseline data from an implementation trial

Background

Adolescents increasingly access mental health services in schools. School mental health professionals (SPs; school counselors, social workers, etc.) can offer evidence-based mental health practices (EBPs) in schools, which may address access gaps and improve clinical outcomes. Although some studies have assessed factors associated with EBP adoption in schools, additional research focusing on SP- and school-level factors is warranted to support EBP implementation as SPs’ mental health delivery grows.

Methods

Baseline data were collected from SPs at Michigan high schools participating in a statewide trial to implement SP-delivered cognitive behavioral therapy (CBT) to students. Models examined factors associated with attitudes about EBPs, implementation climate, and implementation leadership, and their associations with CBT knowledge, training attendance, and pre-training CBT delivery.

Results

One hundred ninety-eight SPs at 107 schools (87%) completed a baseline survey. The mean Evidence-Based Practice Attitude Scale (EBPAS) total score was 2.9, and school-aggregated mean scores of the Implementation Climate Scale (ICS) and Implementation Leadership Scale (ILS) were 1.83 and 1.77, respectively, all on a scale ranging from 0 ( low) to 4 ( high). ICS and ILS scores were lower than typically reported in clinical settings, while EBPAS scores were higher. School characteristics were not significantly associated with EBPAS, ICS, or ILS scores, but scores did differ by SP role. Higher EBPAS scores were associated with more CBT knowledge (average marginal effect for 1 SD change [AME] = 0.15 points) and a higher probability of training completion (AME = 8 percentage points). Higher ICS scores were associated with a higher probability of pre-training CBT delivery (AME = 6 percentage points), and higher ILS scores were associated with higher probability of training completion (AME = 10 percentage points).

Conclusions

Our findings suggest that SPs’ attitudes toward EBPs and organizational support were positively associated with early signs of implementation success. As schools increasingly fill the adolescent mental healthcare access gap, efforts to strengthen both provider attitudes toward EBP and strategic organizational factors supporting EBP delivery will be key to encouraging EBP uptake in schools.

Plain Language Summary

Schools are an important setting in which adolescents receive mental healthcare. We need to better understand how to implement evidence-based practices (EBPs) in this setting to improve student mental health. This study examined the attitudes and perceptions of school professionals (SPs) as key contributors to the implementation of a particular EBP, the delivery of cognitive behavioral therapy (CBT) in schools. The study found that implementation climate and leadership scores in participating schools were lower than scores typically reported in clinical settings, while scores for SP attitudes about EBP adoption were higher than typical scores in clinical settings. Results further suggest that SPs with more positive attitudes toward EBPs are more knowledgeable of CBT and more likely to complete a 1-day CBT training. We also found that higher implementation climate scores were associated with SPs reporting pre-training CBT delivery (although this association was not statistically significant), and more implementation leadership was associated with SPs completing the CBT training. These findings suggest that SP attitudes toward EBPs and organizational support in schools are positively associated with early signs of implementation success. Early, low-intensity efforts to (1) improve SP attitudes about mental health EBPs, and (2) increase schools’ support for implementation may scaffold more intensive implementation efforts in schools down the road.

Implementation context and burnout among Department of Veterans Affairs psychotherapists prior to and during the COVID-19 pandemic

BACKGROUND

The first goal of this study was to assess longitudinal changes in burnout among psychotherapists prior to (T1) and during the COVID-19 pandemic (T2). The second objective was to assess the effects of job demands, job resources (including organizational support for evidence-based psychotherapies, or EBPs) and pandemic-related stress (T2 only) on burnout.

METHOD

Psychotherapists providing EBPs for posttraumatic stress disorder in U.S. Department of Veterans Affairs (VA) facilities completed surveys assessing burnout, job resources, and job demands prior to (T1; n = 346) and during (T2; n = 193) the COVID-19 pandemic.

RESULTS

Burnout prevalence increased from 40 % at T1 to 56 % at T2 (p < .001). At T1, stronger implementation climate and implementation leadership (p < .001) and provision of only cognitive processing therapy (rather than use of prolonged exposure therapy or both treatments; p < .05) reduced burnout risk. Risk factors for burnout at T2 included T1 burnout, pandemic-related stress, less control over when and how to deliver EBPs, being female, and being a psychologist rather than social worker (p < .02). Implementation leadership did not reduce risk of burnout at T2.

LIMITATIONS

This study involved staff not directly involved in treating COVID-19, in a healthcare system poised to transition to telehealth delivery.

CONCLUSION

Organizational support for using EBPs reduced burnout risk prior to but not during the pandemic. Pandemic related stress rather than increased work demands contributed to elevated burnout during the pandemic. A comprehensive approach to reducing burnout must address the effects of both work demands and personal stressors.

Organization-level variation in therapists' attitudes toward and use of measurement-based care

PURPOSE

Despite significant interest in improving behavioral health therapists' implementation of measurement-based care (MBC)-and widespread acknowledgment of the potential importance of organization-level determinants-little is known about the extent to which therapists' use of, and attitudes toward, MBC vary across and within provider organizations or the multilevel factors that predict this variation.

METHODS

Data were collected from 177 therapists delivering psychotherapy to youth in 21 specialty outpatient clinics in the USA. Primary outcomes were use of MBC for progress monitoring and treatment modification, measured by the nationally-normed Current Assessment of Practice Evaluation-Revised. Secondary outcomes were therapist attitudes towards MBC. Linear multilevel regression models tested the association of theory-informed clinic and therapist characteristics with these outcomes.

RESULTS

Use of MBC varied significantly across clinics, with means on progress monitoring ranging from values at the 25th to 93rd percentiles and means on treatment modification ranging from the 18th to 71st percentiles. At the clinic level, the most robust predictor of both outcomes was clinic climate for evidence-based practice implementation; at the therapist level, the most robust predictors were: attitudes regarding practicality, exposure to MBC in graduate training, and prior experience with MBC. Attitudes were most consistently related to clinic climate for evidence-based practice implementation, exposure to MBC in graduate training, and prior experience with MBC.

CONCLUSIONS

There is important variation in therapists' attitudes toward and use of MBC across clinics. Implementation strategies that target clinic climate for evidence-based practice implementation, graduate training, and practicality may enhance MBC implementation in behavioral health.

Testing a Theory of Implementation Leadership and Climate Across Autism Evidence-Based Interventions of Varying Complexity

Prominent theories within the field of implementation science contend that organizational leaders can improve providers' fidelity to evidence-based practices (EBPs) by using focused implementation leadership behaviors that create an organizational climate for EBP implementation. However, this work has been criticized for overreliance on nonspecific, self-report fidelity measures and poor articulation of the boundary conditions that may attenuate leadership and climate's influence. This study tests the predictions of EBP implementation leadership and climate theory on observed fidelity to three school-based EBPs for autism that vary in complexity: pivotal response training (PRT), discrete trial training (DTT), and visual schedules (VS). Educators in kindergarten to third-grade autism support classrooms in 65 schools assessed their principals' EBP implementation leadership and school EBP implementation climate prior to the school year. Mid-school year, trained observers rated educator fidelity to all three interventions. Expert raters confirmed PRT was significantly more complex than DTT or VS using the Intervention Complexity Assessment Tool for Systematic Reviews. Linear regression analyses at the school level indicated principals' increased frequency of EBP implementation leadership predicted a higher school EBP implementation climate, which in turn predicted higher educator fidelity to PRT-however, there was no evidence of a relationship between implementation climate and fidelity to DTT or VS. Comparing principals whose EBP implementation leadership was ±1 SD from the mean, there was a significant indirect association of EBP implementation leadership with PRT fidelity through EBP implementation climate (d = 0.49, 95% CI [0.04, 0.93]). Strategies that target EBP implementation leadership and climate may support fidelity to complex behavioral interventions.

Teacher Perceptions of Implementation Climate Related to Feasibility of Implementing Schoolwide Positive Behavior Supports and Interventions

For Schoolwide Positive Behavior Interventions and Supports (SWPBIS) to be effective, educators must use the practices as intended. Whether a teacher uses a practice as intended can depend, in part, on how feasible the practice is perceived to be. Identifying malleable factors associated with teachers’ perceptions of SWPBIS feasibility can help schools identify targeted supports to improve feasibility. Implementation climate, or the shared perception among implementers that their school supports implementation efforts, is known to promote high quality implementation. However, little is known about how individuals’ perceptions of their school’s implementation climate may influence their perceptions of feasibility. The lack of empirical evidence points to a need to explore whether educators’ shared and individual perceptions of implementation climate influence feasibility of implementing SWPBIS. Therefore, this study examines the link between teachers’ individual and shared perceptions of implementation climate related to the feasibility of implementing SWPBIS in a sample of 348 K-5 general education teachers across 39 elementary schools in the pacific northwestern United States. Results indicate that teachers who experience their schools’ implementation climate as positive are more likely to report SWPBIS as feasible, controlling for teachers’ shared perceptions of implementation climate. Implications for schools aiming to improve their implementation of SWPBIS, including the development of individualized implementation supports (e.g., tailoring implementation strategies to support each and every teacher), are discussed.

Generation and Use of Evidence by Local Health Departments: The Role of Leader Attributes

OBJECTIVE

The purpose of this study was to examine whether certain types of leaders were associated with the degree to which local health departments (LHDs) generate and use evidence to inform their service offering.

DESIGN

Pooled, cross-sectional analysis using 4 waves (2010, 2013, 2016, and 2019) of the National Profiles of Local Health Departments sponsored by the National Association of County and City Health Officials (NACCHO). Univariate analysis was used to assess the extent to which LHDs were generating and using evidence to improve the health of their local communities and whether this changed over time. Multinomial logistic regression models were used to examine the relationships between LHD leader attributes and the extent to which LHDs were generating and using evidence.

PARTICIPANTS

Between 1496 and 2087 (varied by survey round) LHDs from throughout the United States.

MAIN OUTCOME MEASURES

Two outcome variables pertaining to the generation of evidence: (1) how recently an LHD completed a community health assessment and (2) how recently an LHD completed a community health improvement plan. A third outcome variable reflected how extensively an LHD used the Community Guide, a compendium of evidence-based findings.

RESULTS

In 2010, 25.1% and 41.4% of all LHDs had not completed a community health assessment or a community health improvement plan, respectively; by 2019, those figures declined significantly to 14.6% and 24.7%. Similarly, in 2010, 61.7% of all LHDs were not using the Community Guide; by 2019, that percentage declined significantly to 42.5%. Multivariable analysis revealed that leader experience was a more robust correlate of evidence generation and use by LHDs than leader education.

CONCLUSIONS

While LHDs' generation and use of evidence have grown over the past decade, there is room for improvement. Local health department leader attributes-education and experience-highlight targeted opportunities to fill gaps in the use of evidence-based public health practices.

Linking organizational climate for evidence-based practice implementation to observed clinician behavior in patient encounters: a lagged analysis

Background

Theory and empirical research suggest organizational climate for evidence-based practice (EBP) implementation may be an important and malleable target to improve clinician use of EBPs in healthcare; however, this work has been criticized for overreliance on self-report measures of implementation outcomes and cross-sectional designs. This study combines data from two studies spanning 7 years to test the hypothesis that higher levels of organizational EBP implementation climate prospectively predicts improved clinician adherence to an EBP, cognitive behavioral therapy (CBT), as rated by expert observers.

Methods

Biennial assessments of EBP implementation climate collected in 10 community mental health agencies in Philadelphia as part of a systemwide evaluation (time 1) were linked to subsequent observer ratings of clinician adherence to CBT in clinical encounters with 108 youth (time 2). Experts rated clinician adherence to CBT using the Therapy Process Observation Coding System which generated two primary outcomes (a) maximum CBT adherence per session (i.e., highest rated CBT intervention per session; depth of delivery) and (b) average CBT adherence per session (i.e., mean rating across all CBT interventions used; depth and breadth of delivery).

Results

On average, time 2 clinician adherence observations occurred 19.8 months (SD = 10.15) after time 1 organizational climate assessments. Adjusting for organization, clinician, and client covariates, a one standard deviation increase in organizational EBP implementation climate at time 1 predicted a 0.63-point increase in clinicians’ maximum CBT adherence per session at time 2 (p = 0.000), representing a large effect size (d = 0.93; 95% CI = 0.63–1.24) when comparing organizations in the upper (k = 3) versus lower tertiles (k = 3) of EBP implementation climate. Higher levels of time 1 organizational EBP implementation climate also predicted higher time 2 average CBT adherence per session (b = 0.23, p < 0.001, d = 0.72). Length of time between assessments of climate and adherence did not moderate these relationships.

Conclusions

Organizational EBP implementation climate is a promising predictor of clinicians’ subsequent observed adherence to CBT. Implementation strategies that target this antecedent may improve the delivery of EBPs in healthcare settings.

Frontline leadership for implementing clinical guidelines in Norwegian mental health services: a qualitative study

Purpose

The purpose of this study was to explore and interpret how frontline leaders define, experience and rationalise their approaches to the successful implementation of clinical guidelines in mental health care.

Design/methodology/approach

Employing an interpretative phenomenological design, the authors conducted and analysed individual interviews of frontline leaders at 14 psychiatric clinics involved in a national study of implementing evidence-based clinical guidelines in mental health.

Findings

The authors found a broad spectrum of attitudes and attributes, as well as a wide repertoire of strategies for frontline implementation leadership. Three main approaches were revealed, comprising “Curious and welcoming”, “Integrity and setting standards” and “Caring and collegial”.

Research limitations/implications

The study present what experienced frontline leaders emphasise to enable implementation of guidelines, not empirical pieces of evidences for what they in fact do or if these actions lead to implementation. The generalisability to other settings is unknown. Another sample profile, context or organisational level may have impacted the result. The concreteness of the frontline leaders’ considerations, approaches and actions gives important knowledge about frontline leaders leadership across traditional leadership theories.

Originality/value

Existing leadership theories describe different leadership styles, while this study reveals the need for a wide range of approaches to balance the many needs and demands. The complexity of leadership approaches this study found is in line with implementation theories; thus, the present study incorporates implementation science into the leadership literature.

A stepped-wedge randomized trial investigating the effect of the Leadership and Organizational Change for Implementation (LOCI) intervention on implementation and transformational leadership, and implementation climate

BACKGROUND

This study evaluates the Leadership and Organizational Change for Implementation (LOCI) strategy and its effect on implementation leadership, transformational leadership, and implementation climate.

METHODS

A stepped wedge cluster randomized study design enrolling 47 first-level leaders from child- and adult-specialized mental health clinics within Norwegian health trusts across three cohorts. All therapists (n = 790) received training in screening of trauma exposure and posttraumatic stress, and a subgroup of therapists (n = 248) received training in evidence-based treatment methods for posttraumatic stress disorder (PTSD). First-level leaders and therapists completed surveys at baseline, 4, 8-, 12-, 16-, and 20-months assessing leadership and implementation climate. General linear mixed-effects models were used to investigate whether the LOCI strategy would lead to greater therapist-rated scores on implementation leadership, transformational leadership, and implementation climate.

RESULTS

After introducing the LOCI strategy, there was a significant increase in therapist-rated implementation and transformational leadership and implementation climate. The increase was sustained at all measurement time points compared to non-LOCI conditions, which demonstrated a steady decrease in scores before LOCI.

CONCLUSIONS

The LOCI strategy can develop better transformational and implementation leadership skills and contribute to a more positive implementation climate, which may enhance successful EBP implementation. Thus, LOCI can help leaders create an organizational context conducive for effective EBP implementation.

TRIAL REGISTRATION

Retrospectively registered: ClinicalTrials NCT03719651 , 25th of October 2018. The trial protocol can be accessed from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6417075/ .

Moving toward equity: fostering transdisciplinary research between the social and behavioral sciences and implementation science to end the HIV epidemic

PURPOSE OF REVIEW

Since the beginning of the HIV epidemic, social and behavioral scientists have developed interventions to stem the spread of the virus. The dissemination of these interventions has traditionally been a lengthy process; however, implementation science (IS) offers a route toward hastening delivery of effective interventions. A transdisciplinary approach, wherein IS informs and is informed by social and behavioral sciences (SBS) as well as community participation, offers a strategy for more efficiently moving toward health equity and ending the HIV epidemic.

RECENT FINDINGS

There has been considerable growth in HIV research utilizing IS theories, methods and frameworks. Many of these studies have been multi or interdisciplinary in nature, demonstrating the ways that IS and SBS can strengthen one another. We also find areas for continued progress toward transdisciplinarity.

SUMMARY

We review literature from 2020 to 2021, exploring the ways IS and SBS have been used in tandem to develop, evaluate and disseminate HIV interventions. We highlight the interplay between disciplines and make a case for moving toward transdisciplinarity, which would yield new, integrated frameworks that can improve prevention and treatment efforts, moving us closer to achieving health equity.

In Search of the Common Elements of Clinical Supervision: A Systematic Review

The importance of clinical supervision for supporting effective implementation of evidence-based treatments (EBTs) is widely accepted; however, very little is known about which supervision practice elements contribute to implementation effectiveness. This systematic review aimed to generate a taxonomy of empirically-supported supervision practice elements that have been used in treatment trials and shown to independently predict improved EBT implementation. Supervision practice elements were identified using a two-phase, empirically-validated distillation process. In Phase I, a systematic review identified supervision protocols that had evidence of effectiveness based on (a) inclusion in one or more EBT trials, and (b) independent association with improved EBT implementation in one or more secondary studies. In Phase II, a hybrid deductive-inductive coding process was applied to the supervision protocols to characterize the nature and frequency of supervision practice elements across EBTs. Twenty-one of the 876 identified articles assessed the associations of supervision protocols with implementation or clinical outcomes, representing 13 separate studies. Coding and distillation of the supervision protocols resulted in a taxonomy of 21 supervision practice elements. The most frequently used elements were: reviewing supervisees' practice (92%; n = 12), clinical suggestions (85%; n = 11), behavioral rehearsal (77%; n = 10), elicitation (77%; n = 10), and fidelity assessment (77%; n = 10). This review identified supervision practice elements that could be targets for future research testing which elements are necessary and sufficient to support effective EBT implementation. Discrepancies between supervision practice elements observed in trials as compared to routine practice highlights the importance of research addressing supervision-focused implementation strategies.

Measuring implementation climate: psychometric properties of the Implementation Climate Scale (ICS) in Norwegian mental health care services

BackgroundEmployees' perceptions of organizational climate for implementation of new methods are important in assessing and planning for implementation efforts. More specifically, feedback from employees' points to which implementation strategies to select, adopt, and tailor in building positive climate for implementation of new evidence-based practices within the organization. Implementation climate can be measured with the Implementation Climate Scale (ICS). The purpose of this study was to investigate the psychometric properties of the Norwegian version of the ICS in outpatient mental health clinics.MethodsThe ICS was administered to 383 clinicians within 47 different child and adult mental health clinics across the country. We conducted confirmatory factor analysis to assess the psychometric functioning of the ICS. Cronbach's alpha was examined to assess internal consistency. We also examined criterion related validity of the scale by comparing it with an alternative measure of implementation climate (concurrent validity) and by examining correlations with clinicians' intentions to use evidence-based practices.ResultsResults supported the 6-factor structure and the internal consistency reliability of the ICS. One exception was poor functioning of the Reward scale. Concurrent validity was stronger at the group than at the individual level, and assessment of associations with clinicians' intentions to use evidence- based practices showed positive correlations.ConclusionsThe Norwegian version of the ICS is a promising tool for assessing implementation climate which can provide organizations with specific feedback concerning which aspects of the implementation climate to attend to. Due to poor functioning of the Reward scale, adaptations and further testing of this is recommended.

Practice change in community health centers: A qualitative study of leadership attributes

Introduction

This paper explores leadership attributes important for practice change in community health centers (CHCs) and assesses attributes' fit with the Full-Range Leadership Theory (FRLT).

Methods

We conducted four focus groups and 15 in-depth interviews with 48 CHC leaders from several U.S. states using a modified appreciative inquiry approach. Thematic analysis was used to review transcripts for leadership concepts and code with a priori FRLT-derived and inductive codes.

Results

CHC leaders most often noted attributes associated with transformational leadership as essential for practice change. Important attributes included emphasizing a collective sense of mission and a compelling, achievable vision; expressing enthusiasm about what needs to be done; and appealing to employees' analytical reasoning and challenging others to think creatively to problem solve. Few expressions of leadership fit with the transactional typology, though some did mention active vigilance to ensure standards are met, clarifying role and task requirements, and rewarding followers. Passive-avoidant attributes were rarely mentioned.

Conclusions

Our results enhance understanding of leadership attributes supportive of successful practice change in CHCs.

Organizational factors associated with community therapists’ self-efficacy in EBP delivery: The interplay between sustainment leadership, sustainment climate, and psychological safety

Background

Inner context organizational factors proximally shape therapist experiences with evidence-based practice (EBP) implementation and may influence therapist self-efficacy, which has been linked to sustained use of EBPs in community mental health settings. Research has primarily focused on constructs such as implementation leadership and climate. However, the effects of such factors may depend upon other inner context dimensions, such as psychological safety. Psychologically safe environments are conducive to taking risks, speaking up about problems, and requesting feedback and may promote therapist self-efficacy during implementation. This study examines whether organizational sustainment leadership and sustainment climate relate to therapist EBP self-efficacy only under conditions of psychological safety.

Methods

Data were collected from 410 clinicians in 85 programs during the sustainment phase of a system-driven implementation of multiple EBPs in children's mental health services. Therapists reported on their organization's sustainment leadership, sustainment climate, psychological safety, and their own self-efficacy in delivering specific EBPs. Multilevel regression analyses were conducted to account for nested data structure.

Results

Among program-level variables, sustainment leadership and psychological safety both significantly predicted therapist self-efficacy. However, there were no significant interactions between program-level sustainment climate and psychological safety. Exploratory post-hoc analyses revealed a significant interaction between program-level sustainment leadership and therapist-level perceptions of psychological safety such that that the conditional effect of psychological safety on EBP self-efficacy was significant at high levels of sustainment leadership, but not at low or average levels.

Conclusion

We noted independent links between sustainment leadership, organizational psychological safety and therapists feelings of confidence and mastery with EBPs. Therapists’ individual perceptions of psychological safety were linked to self-efficacy only in programs with high sustainment leadership. Thus, sustainment leadership and psychological safety may both represent implementation intervention targets, but it may not be critical to assess for perceptions of psychological safety before deploying organizational leadership strategies. Plain language abstract Therapist self-efficacy is a therapist's belief that they are capable, knowledgeable, and skilled enough to deliver evidence-based practices (EBPs), and is thought to promote improved clinical and implementation outcomes, such as therapists’ sustained use of EBPs. Conditions within community mental health organizations may influence therapists’ sense of EBP self-efficacy. Leaders’ support and expectations for EBP implementation, and collective staff perceptions about the organization's climate to support EBPs are linked to positive therapist attitudes and EBP adoption. However, less is known about how these implementation-specific organizational factors associated with therapist EBP self-efficacy in the long-term, and how this may depend on general workplace conditions. Specifically, psychologically safe environments – where therapists feel safe taking risks such as asking questions, admitting mistakes, and trying new skills – may be needed to promote self-efficacy when therapists are tasked with learning and using complex multi-component EBP innovations. The current study tested the prediction that leader-driven and program-wide focus on EBP sustainment may promote therapist EBP self-efficacy only in organizations where conditions for learning are psychologically safe. Our findings confirmed that fostering strong sustainment focused leadership and psychologically safe environments may each be important for increasing therapists’ EBP self-efficacy. The model results suggested that individual therapist perceptions of psychological safety were more strongly related to EBP self-efficacy in programs with greater implementation leadership. Findings suggest the importance of increasing EBP leadership behavior to fully potentiate other facilitating conditions for therapist learning in the sustainment phase of EBP implementation initiatives.

Organizational Context in General and Special Education: An Exploratory Investigation to Describe the Perspective of School Leaders

Organizational context (e.g., climate, culture, resources) can impede or enhance implementation of evidence-based practices in general education settings or special education settings serving students with autism spectrum disorder. We examined the relations between organizational context and individual (i.e., implementation leadership, administrator- or service provider-role) or school (i.e., enrollment size, public/nonpublic school type) characteristics. Participants were administrative or service providing leaders (n = 34) from 11 schools in one state on the East Coast of the United States. School leaders' average ratings of the organizational context were generally more positive for special education than general education; however, greater culture stress was reported for special education. Correlation analyses indicated being an administrator and implementation leadership were positively associated with implementation climate in both education settings. Being an administrator was also positively associated with cultural effort (i.e., how hard people work towards achieving goals) in special education, but negatively associated with culture stress in general education. In special education, nonpublic schools had better climates (both learning and implementation), but more culture stress. Additionally, school enrollment size was negatively related to available resources and implementation climate in special education. Investigating the similarities and differences in organizational context across educational settings is needed in future research.

Project MIMIC (Maximizing Implementation of Motivational Incentives in Clinics): A cluster-randomized type 3 hybrid effectiveness-implementation trial

BACKGROUND

Opioid-related overdoses and harms have been declared a public health emergency in the United States, highlighting an urgent need to implement evidence-based treatments. Contingency management (CM) is one of the most effective behavioral interventions when delivered in combination with medication for opioid use disorder, but its implementation in opioid treatment programs is woefully limited. Project MIMIC (Maximizing Implementation of Motivational Incentives in Clinics) was funded by the National Institute on Drug Abuse to identify effective strategies for helping opioid treatment programs improve CM implementation as an adjunct to medication. Specific aims will test the impact of two different strategies on implementation outcomes (primary aim) and patient outcomes (secondary aims), as well as test putative mediators of implementation effectiveness (exploratory aim).

METHODS

A 3-cohort, cluster-randomized, type 3 hybrid design is used with the opioid treatment programs as the unit of randomization. Thirty programs are randomized to one of two conditions. The control condition is the Addiction Technology Transfer Center (ATTC) Network implementation strategy, which consists of three core approaches: didactic training, performance feedback, and on-going consultation. The experimental condition is an enhanced ATTC strategy, with the same core ATTC elements plus two additional theory-driven elements. The two additional elements are Pay-for-Performance, which aims to increase implementing staff's extrinsic motivations, and Implementation & Sustainment Facilitation, which targets staff's intrinsic motivations. Data will be collected using a novel, CM Tracker tool to document CM session delivery, session audio recordings, provider surveys, and patient surveys. Implementation outcomes include CM Exposure (number of CM sessions delivered per patient), CM Skill (ratings of CM fidelity), and CM Sustainment (number of patients receiving CM after removal of support). Patient outcomes include self-reported opioid abstinence and opioid-related problems (both assessed at 3- and 6-months post-baseline).

DISCUSSION

There is urgent public health need to improve the implementation of CM as an adjunct to medication for opioid use disorder. Consistent with its hybrid type 3 design, Project MIMIC is advancing implementation science by comparing impacts of these two multifaceted strategies on both implementation and patient outcomes, and by examining the extent to which the impacts of those strategies can be explained by putative mediators.

TRIAL REGISTRATION

This clinical trial has been registered with clinicaltrials.gov (NCT03931174). Registered April 30, 2019. https://clinicaltrials.gov/ct2/show/NCT03931174?term=project+mimic&draw=2&rank=1.

The Organizational Financial Context of Publicly-Funded Mental Health Clinics: Development and Preliminary Psychometric Evaluation of the Agency Financial Status Scales

Funding is a major barrier to implementation of evidence-based practices (EBPs) in publicly-funded community mental health clinics (CMHCs). Understanding how best to deploy implementation strategies that address this barrier requires greater clarity on the financial context within agencies. We developed the Agency Financial Status Scales (AFSS) to assess employee perceptions of the level of three hypothesized and theoretical funding related constructs in organizations: (a) perceptions of financial health, (b) financial attitudes toward EBPs, and (c) strategic financial climate. This investigation serves as a preliminary evaluation of this measure. Participants were 239 therapists and 40 supervisors from 25 publicly-funded CMHCs providing outpatient mental health services for young people. Confirmatory factor analysis was used to investigate the latent trait structure of the items. Internal consistency, interrater agreement, concordance between therapists and supervisors, and convergent validity were also examined. A two-factor model measuring perceptions of financial health and strategic financial climate best fit the data. For both of these scales, alpha reliability was acceptable and agreement statistics provided moderate support for aggregation at the organizational level. Analyses supported the convergent validity of the scales. The development and preliminary evaluation of the AFSS is an important first step in understanding the financial context of publicly-funded CMHCs. Though findings from this investigation are promising, additional development and testing are needed to develop a more thorough understanding of the constructs and to improve the validity and reliability of this measure.

The relationship between first-level leadership and inner-context and implementation outcomes in behavioral health: a scoping review

BACKGROUND

First-level leadership is uniquely positioned to support evidence-based practice (EBP) implementation for behavioral health due to first-level leaders' access to and relationship with service providers. First-level leaders are individuals who directly supervise and manage frontline employees who do not manage others. However, first-level leadership is underrepresented in existing reviews of the impact of leadership on EBP implementation. This review describes the relationship between first-level leadership and implementation determinants and outcomes.

METHODS

A scoping review was performed to synthesize the literature on the relationship between first-level leadership and inner-context and implementation outcomes. A literature search was conducted in PubMed, Eric, PsycINFO, CINAHL, Scopus, and Web of Science. To be eligible, studies had to examine first-level leadership, be conducted in settings providing behavioral health services, and examine the relationship between first-level leadership and an implementation or inner-context outcome. Data extraction and synthesis were performed to describe study characteristics, leader-outcome relationships, and overlap in leadership frameworks.

RESULTS

Twenty-one records met our inclusion criteria. Studies primarily relied on observational designs and were often cross-sectional. Studies more often examined general leadership rather than leadership strategically focused on EBP implementation (i.e., strategic implementation leadership). Our findings suggest that several forms of first-level leadership are inconsistently related to a broad set of implementation determinants, with infrequent examination of specific implementation outcomes. The broad set of implementation determinants studied, limited number of replications, and inconsistent findings have resulted in sparse evidence for any specific leadership-outcome relationship. The greatest accumulation of evidence exists for general leadership's positive relationship with providers' EBP attitudes, most notably in the form of transformational leadership. This was followed by evidence for strategic implementation leadership facilitating general implementation. Our synthesis revealed moderate conceptual overlap of strategic implementation leadership behaviors described in the theory of implementation leadership and theory of middle managers' role in implementation.

CONCLUSIONS

Our findings suggest that first-level leadership may play an important role in shaping implementation determinants and outcomes, but consistent empirical support is sparse and confidence dampened by methodological issues. To advance the field, we need studies that adopt stronger methodological rigor, address the conceptual overlap in leadership frameworks, examine a broader set of implementation outcomes, and examine conditions under which leadership impacts implementation.

TRIAL REGISTRATION

This review was not registered.

Conceptualizing the clinical decision-making process in managing temporomandibular disorders: A qualitative study

Management of patients with temporomandibular disorders (TMD) appears to be more challenging than for other dental conditions. This study aimed to explore the decision-making process in TMD management, and thereby to conceptualize the decision-making process in dentistry. Individual semi-structured interviews were conducted during 2018 and 2019 with a purposive sample of 22 general dental practitioners from the Public Dental Healthcare Services and private practices in the Region of Västerbotten, Northern Sweden. The interviews were analysed using the Grounded Theory approach of Charmaz. Data analysis resulted in the core category 'Combining own competence and others' expectations in the desire to do the right thing'. The dentists showed interest in and a desire to apply professional knowledge, but also reflected on challenges and complexity in the decision-making process for TMD. The challenges were primarily related to organisational factors and lack of self-confidence. This identifies a need for re-organisation of daily clinical management in dentistry, and a need for more postgraduate training to improve self-confidence. The complexity of the decision-making process for TMD makes the study findings applicable in other dental situations.

Recommendations for strengthening the role of embedded researchers to accelerate implementation in health systems: Findings from a state-of-the-art (SOTA) conference workgroup

BACKGROUND

Traditional research approaches do not promote timely implementation of evidence-based innovations (EBIs) to benefit patients. Embedding research within health systems can accelerate EBI implementation by blending rigorous methods with practical considerations in real-world settings. A state-of-the-art (SOTA) conference was convened in February 2019 with five workgroups that addressed five facets of embedded research and its potential to impact healthcare. This article reports on results from the workgroup focused on how embedded research programs can be implemented into heath systems for greatest impact.

METHODS

Based on a pre-conference survey, participants indicating interest in accelerating implementation were invited to participate in the SOTA workgroup. Workgroup participants (N = 26) developed recommendations using consensus-building methods. Ideas were grouped by thematic clusters and voted on to identify top recommendations. A summary was presented to the full SOTA membership. Following the conference, the workgroup facilitators (LJD, CDH, NR) summarized workgroup findings, member-checked with workgroup members, and were used to develop recommendations.

RESULTS

The workgroup developed 12 recommendations to optimize impact of embedded researchers within health systems. The group highlighted the tension between "ROI vs. R01" goals-where health systems focus on achieving return on their investments (ROI) while embedded researchers focus on obtaining research funding (R01). Recommendations are targeted to three key stakeholder groups: researchers, funders, and health systems. Consensus for an ideal foundation to support optimal embedded research is one that (1) maximizes learning; (2) aligns goals across all 3 stakeholders; and (3) implements EBIs in a consistent and timely fashion.

CONCLUSIONS

Four cases illustrate a variety of ways that embedded research can be structured and conducted within systems, by demonstrating key embedded research values to enable collaborations with academic affiliates to generate actionable knowledge and meaningfully accelerate implementation of EBIs to benefit patients.

IMPLICATIONS

Embedded research approaches have potential for transforming health systems and impacting patient health. Accelerating embedded research should be a focused priority for funding agencies to maximize a collective return on investment.