Validation of the Implementation Leadership Scale (ILS) in Substance use Disorder Treatment Organizations

Randomized trial of the leadership and organizational change for implementation (LOCI) strategy in substance use treatment clinics

INTRODUCTION

Evidence-based practice (EBP) implementation represents a strategic change that requires alignment of leadership and support throughout organizations. Leadership and Organizational Change for Implementation (LOCI) is a multifaceted implementation strategy that aims to improve implementation leadership and climate within organizations through iterative cycles of leadership and climate assessment and feedback, leadership training and coaching, and strategic planning with upper-level leaders. This study tested the effects of LOCI on transformational and implementation leadership, implementation climate, implementation citizenship behavior, and EBP reach.

METHODS

A multiple cohort, cluster randomized trial tests the effect of LOCI in 60 clinics across nine behavioral health organizations in California and Arizona, USA. The study randomized clinics within organizations to either LOCI or a leadership training webinar control condition in three consecutive cohorts. Repeated web-based surveys of direct service providers (nLOCI = 201, nControl = 179) assessed leadership, implementation climate, and implementation citizenship over time. Multilevel autoregressive modeling was the primary statistical analysis such that providers (level-1) were nested within clinics (level-2). The study predicted between-condition differences at 4-, 8-, and 12-month follow-up assessments. Provider engagement in a fidelity monitoring process assessed reach of motivational interviewing (i.e., number of sessions recorded/submitted for fidelity coding). An independent sample t-test explored between condition differences in motivational interviewing reach.

RESULTS

Results indicated between condition differences at 4 months for implementation leadership, implementation climate, and implementation citizenship behavior such that greater improvements were evidenced in the LOCI condition compared to the control condition. Reach of MI was significantly greater in the LOCI vs control condition such that LOCI providers were significantly more likely to engage in the fidelity monitoring process (chi-square (1, n = 370) = 5.59, p = .018).

CONCLUSIONS

LOCI was developed based on organizational theories of strategic leadership and climate to affect organizational change processes that communicate that innovation implementation is expected, supported, and recognized as a value of the organization. The LOCI implementation strategy resulted in more positive hypothesized outcomes compared to the control condition. Organizational change strategies have utility for implementing health innovations in complex, multilevel contexts and for greater sustainment of facilitative leader behaviors, strategic implementation climate, and improved implementation outcomes.

TRIAL REGISTRATION

This study is registered with Clinicaltrials.gov gov (NCT03042832, 2 February 2017; retrospectively registered).

Improving measurement-based care implementation in youth mental health through organizational leadership and climate: a mechanistic analysis within a randomized trial

BACKGROUND

Theory and correlational research indicate organizational leadership and climate are important for successful implementation of evidence-based practices (EBPs) in healthcare settings; however, experimental evidence is lacking. We addressed this gap using data from the WISDOM (Working to Implement and Sustain Digital Outcome Measures) hybrid type III effectiveness-implementation trial. Primary outcomes from WISDOM indicated the Leadership and Organizational Change for Implementation (LOCI) strategy improved fidelity to measurement-based care (MBC) in youth mental health services. In this study, we tested LOCI's hypothesized mechanisms of change, namely: (1) LOCI will improve implementation and transformational leadership, which in turn will (2) mediate LOCI's effect on implementation climate, which in turn will (3) mediate LOCI's effect on MBC fidelity.

METHODS

Twenty-one outpatient mental health clinics serving youth were randomly assigned to LOCI plus MBC training and technical assistance or MBC training and technical assistance only. Clinicians rated their leaders' implementation leadership, transformational leadership, and clinic implementation climate for MBC at five time points (baseline, 4-, 8-, 12-, and 18-months post-baseline). MBC fidelity was assessed using electronic metadata for youth outpatients who initiated treatment in the 12 months following MBC training. Hypotheses were tested using longitudinal mixed-effects models and multilevel mediation analyses.

RESULTS

LOCI significantly improved implementation leadership and implementation climate from baseline to follow-up at 4-, 8-, 12-, and 18-month post-baseline (all ps < .01), producing large effects (range of ds = 0.76 to 1.34). LOCI's effects on transformational leadership were small at 4 months (d = 0.31, p = .019) and nonsignificant thereafter (ps > .05). LOCI's improvement of clinic implementation climate from baseline to 12 months was mediated by improvement in implementation leadership from baseline to 4 months (proportion mediated [pm] = 0.82, p = .004). Transformational leadership did not mediate LOCI's effect on implementation climate (p = 0.136). Improvement in clinic implementation climate from baseline to 12 months mediated LOCI's effect on MBC fidelity during the same period (pm = 0.71, p = .045).

CONCLUSIONS

LOCI improved MBC fidelity in youth mental health services by improving clinic implementation climate, which was itself improved by increased implementation leadership. Fidelity to EBPs in healthcare settings can be improved by developing organizational leaders and strong implementation climates.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT04096274. Registered September 18, 2019.

A new scale to assess health-facility level management: the development and validation of the facility management scale in Ghana, Uganda, and Malawi

BACKGROUND

The increased recognition of governance, leadership, and management as determinants of health system performance has prompted calls for research focusing on the nature, quality, and measurement of this key health system building block. In low- or middle-income contexts (LMIC), where facility-level management and performance remain a challenge, valid tools to measure management have the potential to boost performance and accelerate improvements. We, therefore, sought to develop a Facility-level Management Scale (FMS) and test its reliability in the psychometric properties in three African contexts.

METHODS

The FMS was administered to 881 health workers in; Ghana (n = 287; 32.6%), Malawi (n = 66; 7.5%) and Uganda (n = 528; 59.9%). Half of the sample data was randomly subjected to exploratory factor analysis (EFA) and Monte Carlo Parallel Component Analysis to explore the FMS' latent structure. The construct validity of this structure was then tested on the remaining half of the sample using confirmatory factor analysis (CFA). The FMS' convergent and divergent validity, as well as internal consistency, were also tested.

RESULTS

Findings from the EFA and Monte Carlo PCA suggested the retention of three factors (labelled 'Supportive Management', 'Resource Management' and 'Time management'). The 3-factor solution explained 51% of the variance in perceived facility management. These results were supported by the results of the CFA (N = 381; χ2 = 256.8, df = 61, p < 0.001; CFI = 0.94; TLI = 0.92; RMSEA [95% CI] = 0.065 [0.057-0.074]; SRMR = 0.047).

CONCLUSION

The FMS is an open-access, short, easy-to-administer scale that can be used to assess how health workers perceive facility-level management in LMICs. When used as a regular monitoring tool, the FMS can identify key strengths or challenges pertaining to time, resources, and supportive management functions at the health facility level.

Implementation leadership and implementation climate in context: A single organization intrinsic case study for implementation of digital measurement-based care

Background

Although studies have demonstrated that implementation leadership and climate are important constructs in predicting evidence-based practice (EBP) implementation, concrete descriptions of how they operate during organizational implementation efforts are lacking. This case study fills that gap through an in-depth description of an organization with effective implementation leadership that successfully built a strong implementation climate. This case study provides an illustration of implementation leadership and climate in tangible, replicable terms to assist managers, practitioners, and researchers in addressing the organizational context in their own implementation projects.

Method

A single organization, intrinsic case study was employed to paint a multifaceted picture of how one organization leveraged implementation leadership to strengthen a climate for the successful implementation of digital measurement-based care. The case was drawn from a cluster-randomized trial designed to test the effects of a leadership-focused implementation strategy on youth-level fidelity and clinical outcomes of digital measurement-based care. Following the completion of the trial, case study activities commenced. Descriptive summaries of multiple data sources (including quantitative data on implementation leadership and climate, coaching call and organizational alignment meeting recordings and notes, and development plans) were produced and revised iteratively until consensus was reached. Leadership actions were analyzed for corresponding dimensions of implementation leadership and climate.

Results

Specific actions organizational leaders took, as well as the timing specific strategies were enacted, to create a climate for implementation are presented, along with lessons learned from this experience.

Conclusion

This case study offers concrete steps organizational leaders took to create a consistent and aligned message that the implementation of a specific EBP was a top priority in the agency. The general approach taken to create an implementation climate provides several lessons for leaders, especially for EBPs that have broad implications across an organization.

Validity and reliability of the transculturally adapted Spanish version of the Implementation Leadership Scale (ILS)

BACKGROUND

There is a need for pragmatic and reliable measures of sound factors that affect evidence-based practice (EBP) adoption and implementation in different languages and cultural environments. The Implementation Leadership Scale (ILS) is a brief and efficient measurement tool of strategic leadership for EBP implementation. The objective of this study was to assess the psychometric properties of the Spanish version of the ILS.

METHODS

The process of translation of the original ILS into Spanish consisted of forward translation, panel meeting, and back-translation. Scale face and content validity compared to that of the original version were assessed and ensured before agreement on the final version. Psychometric properties were examined in 144 healthcare professionals (family physicians, pediatricians, practice and pediatric nurses) involved in implementation or improvement research projects. ILS factor structure was tested by confirmatory factor analysis (CFA). Reliability was assessed by internal consistency analysis. The Pearson correlation between the ILS and the Organizational Support dimension of the Organizational Readiness for Knowledge Translation (OR4KT) questionnaire in the subsample of pediatricians and pediatric nurses (n = 52) was estimated for convergent validity analysis.

RESULTS

The CFA results indicated that the original four theorized first-order factors with a second-order Implementation Leadership factor fit the data well (χ2 = 107.70; df = 45; p < 0.001). All standardized first- and second-order factor loadings were statistically significant. Fit indexes showed acceptable figures (GFI = 0.90; CFI = 0.97; RMSEA = 0.10; SRMR = 0.053). Cronbach's alpha coefficient for the four dimensions of ILS ranged from 0.90 to 0.97, while the reliability estimated for the total scale was 0.95. Results of convergent validity revealed high correlation (r = 0.56) between the ILS and the OR4KT's Organizational Support dimension.

CONCLUSION

The CFA results demonstrated that the tested first- and second-order factor structure of the 12-item Spanish version of the ILS is consistent with the factor structure of the original tool. The availability of the ILS will allow Spanish-speaking researchers to assess and advance understanding of the implementation leadership construct as a predictor of organizational implementation context.

Effectiveness and implementation of an electronic health record-integrated digital health intervention for managing depressive symptoms in ambulatory oncology: The My Well-Being Guide study rationale and protocol

BACKGROUND

Rates of clinically elevated depressive symptoms among ambulatory oncology patients are higher than in the general population and are associated with poorer health-related quality of life. Furthermore, a reduction in depressive symptoms may be associated with improved cancer survival. Several interventions have demonstrated efficacy in reducing oncologic depressive symptoms, including cognitive-behavioral stress management (CBSM). However, more work is needed to understand how to best implement CBSM into practice, such as through stepped-care approaches and digital health interventions linked to electronic health records (EHR). This manuscript presents the protocol of the My Well-Being Guide study, a pragmatic type 1 effectiveness-implementation hybrid study. This trial will test the effectiveness of My Well-Being Guide, a seven-week structured, CBSM-based digital health intervention designed to reduce depressive symptoms. This trial will also evaluate My Well-Being Guide's implementation across two health systems.

METHODS

The final sample (N = 4561) will be oncology patients at Northwestern Medicine or University of Miami Health System who are ≥18 years of age; have a cancer diagnosis; elevated depressive symptoms on the Patient-Reported Outcomes Measurement Information System Depression; and primary language is English or Spanish. Data collection will occur at baseline, and 2-, 6-, and 12-months post baseline. Outcome domains include depressive symptoms and implementation evaluation.

DISCUSSION

This study may provide valuable data on the effectiveness of our depressive symptom management digital health intervention linked to the EHR and the scalability of digital health interventions in general.

Using the Consolidated Framework for Implementation Research to evaluate a nationwide depression prevention project (ImplementIT) from the perspective of health care workers and implementers: Results on the implementation of digital interventions for farmers

Introduction

Depression has a significant impact on individuals and society, which is why preventive measures are important. Farmers represent an occupational group exposed to many risk factors for depression. The potential of guided, tailored internet-based interventions and a personalized telephone coaching is evaluated in a German project of the Social Insurance for Agriculture, Forestry and Horticulture (SVLFG). While user outcomes are promising, not much is known about actual routine care use and implementation of the two digital health interventions. This study evaluates the implementation from the perspective of social insurance employees to understand determinants influencing the uptake and implementation of digital interventions to prevent depression in farmers.

Methods

The data collection and analysis are based on the Consolidated Framework for Implementation Research (CFIR). Health care workers (n = 86) and implementers (n = 7) completed online surveys and/or participated in focus groups. The surveys consisted of validated questionnaires used in implementation research, adapted items from the CFIR guide or from other CFIR studies. In addition, we used reporting data to map implementation based on selected CFIR constructs.

Results

Within the five CFIR dimensions, many facilitating factors emerged in relation to intervention characteristics (e.g., relative advantage compared to existing services, evidence and quality) and the inner setting of the SVLFG (e.g., tension for change, compatibility with values and existing working processes). In addition, barriers to implementation were identified in relation to the outer setting (patient needs and resources), inner setting (e.g., available resources, access to knowledge and information) and characteristics of individuals (e.g., self-efficacy). With regard to the implementation process, facilitating factors (formal implementation leaders) as well as hindering factors (reflecting and evaluating) were identified.

Discussion

The findings shed light on the implementation of two digital prevention services in an agricultural setting. While both offerings seem to be widely accepted by health care workers, the results also point to revealed barriers and contribute to recommendations for further service implementation. For instance, special attention should be given to "patient needs and resources" by raising awareness of mental health issues among the target population as well as barriers regarding the inner setting.

Clinical Trial Registration

German Clinical Trial Registration: [DRKS00017078]. Registered on 18.04.2019.

Implementation Strategy Fidelity and Organizational Outcomes in a Randomized Trial: Implications for Reentry from Corrections to Community Treatment

Background: This study examined fidelity of implementation strategies used in an organizational process improvement intervention (OPII) designed to improve evidence-based practices related to assessments for drug-involved clients exiting incarceration. Leadership was studied as a moderating factor between fidelity and outcomes. Methods: A mixed-method cluster randomized design was used to randomize 21 sites to early- or delayed-start within 9 research centers. Parent study protocol was reviewed to develop fidelity constructs (i.e., responsiveness, dose, quality, adherence). Outcomes were site-level success in achieving goals and objectives completed during the OPII (e.g., percent goals achieved). Correlations, analyses of covariance, regressions and moderation analyses were performed. Qualitative interviews assessed facilitators/barriers to implementation. Results: Fidelity constructs related to outcomes. No differences were found in fidelity by early or delayed condition. At low levels of leadership, high staff responsiveness (i.e., engagement in the OPII) related to poorer outcome. Conclusions: It is important to consider contextual factors (e.g., leadership) that may influence implementation strategy fidelity when deploying evidence-based practices. Findings are relevant to researchers, clinicians, administrators and policy makers, and suggest that goal completion during implementation of evidence-based practices requires monitoring of leadership competence, fidelity to implementation strategies (i.e., staff responsiveness to strategies) and attendance to goal importance.

Implementation Leadership in the Point of Care Nursing Context: A Systematic Review Comparing Two Measurement Tools

Introduction

Implementation leadership (IL) are effective point of care (POC) nursing leadership behaviors that facilitate contexts conducive to the successful implementation of evidence-based practices (EBPs). However, no systematic evaluation of IL tools validated for the nursing context existed.

Aims

The purpose of this systematic review was to compare iterations of two IL measurement tools, the Implementation Leadership Scale (ILS) and the iLEAD, for application in a nursing context; and to critically appraise and summarize the methodological quality of studies assessing their psychometric properties.

Methods

A comprehensive search was conducted in four databases. Two reviewers independently screened titles and abstracts, reviewed full-text articles, and performed extraction into data tables. Statisticians appraised the quality control aspects. Findings were narratively summarized.

Results

A total of 247 records were included, where 10 for the ILS (including different versions) and one for the iLEAD met the inclusion criteria. Three studies evaluated the psychometric properties of the ILS in nursing, and its translations into Chinese and Greek. Content validity was deemed to be doubtful for both tools, but the ILS had adequate rating for comprehensiveness; methodological quality was very good for structural validity, internal consistency, hypothesis testing, and responsiveness where applicable for both scales, with the exception of cross-cultural validity which had ratings of adequate and inadequate for versions of the scales. Several study findings met the criteria for good measurement properties. No studies for either tool formally assessed feasibility.

Conclusion

Applying validated and contextually relevant tools to evaluate the capacity of nursing leadership to engage in IL in real-world contexts are needed. The ILS shows promise but requires further validation for contexts with diverse and multiple nursing leaders at the POC. Feasibility needs to be further studied.

Defining, conceptualizing and evaluating pragmatic qualities of quantitative instruments measuring implementation determinants and outcomes: a scoping and critical review of the literature and recommendations for future research

The pragmatic (i.e., practical) quality of quantitative implementation measures has received increased attention in the implementation science literature in recent years. Implementation measures that are judged to be pragmatic by implementation stakeholders are thought to be more likely to be applied in research and practice. Despite the need for pragmatic implementation measures, ambiguity and uncertainty regarding what constitutes a pragmatic measure remains. This study sought to identify and critically appraise the published literature to understand (i) how pragmatism is defined as a measurement construct/quality of implementation determinants and outcome instruments; (ii) how pragmatic qualities of instruments are evaluated; (iii) identify key gaps and limitations of the current evidence-base and (iv) identify recommendations for future research. We conducted a scoping review of the literature also employing methods of critical review. PubMed and PsycINFO databases, using the OVID interface, were searched for relevant articles published between January 2010 and September 2020. Articles that contained a definition and/or described characteristics of "pragmatism" as a measurement construct of quantitative implementation outcomes (as defined by Proctor's Implementation Outcomes taxonomy) and/or implementation determinants were eligible for inclusion. Nine articles met inclusion criteria. A degree of overlap in definitions and terms used to describe the pragmatic qualities of quantitative implementation determinant and outcome instruments were found. The most frequently cited descriptors of pragmatism were "not burdensome", "brief", "reliable", "valid" and "sensitive to change". 3 of the 9 included articles involved international implementation stakeholders in defining and conceptualizing pragmatism and employed specific methods to do so, including a systematic literature review, stakeholder interviews, concept mapping, and a Delphi process. All other articles defined pragmatism, with or without citing relevant literature. One article objectively assessed the pragmatic qualities, above and beyond the psychometric qualities, of implementation measures, using the Psychometric and Pragmatic Evidence Rating Scale (PAPERS). The evidence base within the implementation instrumentation literature on what pragmatism is and how it might be assessed is limited. Some of the research identified in the review provides a strong foundation to build upon, by testing its applicability in other settings (including healthcare areas and countries) and among a more diverse group of stakeholders. We discuss directions for further development of the concept of pragmatism relating to the measurement of implementation determinants and outcomes.

Stepped implementation-to-target: a study protocol of an adaptive trial to expand access to addiction medications

Background

In response to the US opioid epidemic, significant national campaigns have been launched to expand access to `opioid use disorder (MOUD). While adoption has increased in general medical care settings, specialty addiction programs have lagged in both reach and adoption. Elevating the quality of implementation strategy, research requires more precise methods in tailoring strategies rather than a one-size-fits-all-approach, documenting participant engagement and fidelity to the delivery of the strategy, and conducting an economic analysis to inform decision making and policy. Research has yet to incorporate all three of these recommendations to address the challenges of implementing and sustaining MOUD in specialty addiction programs.

Methods

This project seeks to recruit 72 specialty addiction programs in partnership with the Washington State Health Care Authority and employs a measurement-based stepped implementation-to-target approach within an adaptive trial design. Programs will be exposed to a sequence of implementation strategies of increasing intensity and cost: (1) enhanced monitoring and feedback (EMF), (2) 2-day workshop, and then, if outcome targets are not achieved, randomization to either internal facilitation or external facilitation. The study has three aims: (1) evaluate the sequential impact of implementation strategies on target outcomes, (2) examine contextual moderators and mediators of outcomes in response to the strategies, and (3) document and model costs per implementation strategy. Target outcomes are organized by the RE-AIM framework and the Addiction Care Cascade.

Discussion

This implementation project includes elements of a sequential multiple assignment randomized trial (SMART) design and a criterion-based design. An innovative and efficient approach, participating programs only receive the implementation strategies they need to achieve target outcomes. Findings have the potential to inform implementation research and provide key decision-makers with evidence on how to address the opioid epidemic at a systems level.

Trial registration

This trial was registered at ClinicalTrials.gov (NCT05343793) on April 25, 2022.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-022-01239-y.

Testing a Theory of Implementation Leadership and Climate Across Autism Evidence-Based Interventions of Varying Complexity

Prominent theories within the field of implementation science contend that organizational leaders can improve providers' fidelity to evidence-based practices (EBPs) by using focused implementation leadership behaviors that create an organizational climate for EBP implementation. However, this work has been criticized for overreliance on nonspecific, self-report fidelity measures and poor articulation of the boundary conditions that may attenuate leadership and climate's influence. This study tests the predictions of EBP implementation leadership and climate theory on observed fidelity to three school-based EBPs for autism that vary in complexity: pivotal response training (PRT), discrete trial training (DTT), and visual schedules (VS). Educators in kindergarten to third-grade autism support classrooms in 65 schools assessed their principals' EBP implementation leadership and school EBP implementation climate prior to the school year. Mid-school year, trained observers rated educator fidelity to all three interventions. Expert raters confirmed PRT was significantly more complex than DTT or VS using the Intervention Complexity Assessment Tool for Systematic Reviews. Linear regression analyses at the school level indicated principals' increased frequency of EBP implementation leadership predicted a higher school EBP implementation climate, which in turn predicted higher educator fidelity to PRT-however, there was no evidence of a relationship between implementation climate and fidelity to DTT or VS. Comparing principals whose EBP implementation leadership was ±1 SD from the mean, there was a significant indirect association of EBP implementation leadership with PRT fidelity through EBP implementation climate (d = 0.49, 95% CI [0.04, 0.93]). Strategies that target EBP implementation leadership and climate may support fidelity to complex behavioral interventions.

Leading the charge in the education sector: development and validation of the School Implementation Leadership Scale (SILS)

Background

Strategic implementation leadership is a critical determinant of successful implementation, hypothesized to create a more supportive implementation climate conducive to the adoption and use of evidence-based practices. Implementation leadership behaviors may vary significantly across contexts, necessitating studies that examine the validity of established measurement tools in novel health service delivery sectors. The education sector is the most common site for delivering mental health services to children and adolescents in the USA, but research focused on implementation leadership in schools is in the early phases, and there is a need for adaptation and expansion of instruments in order to tailor to the school context. The current study adapted and validated the School Implementation Leadership Scale (SILS) (based on the Implementation Leadership Scale) in a sample of elementary school personnel from six school districts who were implementing one of two well-established prevention programs for supporting children’s mental health.

Methods

Participants were 441 public school teachers from 52 elementary schools in the Midwest and West Coast of the USA. Participants completed a survey that contained: (1) an adapted and expanded version of the SILS with additional items generated for four existing subscales as well as three new subscales (communication, vision/mission, and availability), and (2) additional tools to evaluate convergent and divergent validity (i.e., measures of general/molar leadership and teaching attitudes). Data underwent (1) examination of item characteristic curves to reduce items and ensure a pragmatic instrument, (2) confirmatory factor analyses to establish structural validity, and (3) evaluation of convergent and divergent validity.

Results

Item reduction analyses resulted in seven subscales of three items each. Results indicated acceptable fit for a seven-factor structural model (CFI = .995, TLI = .99, RMSEA = .07, SRMR = 0.02). Second-order factor loadings were high (λ = .89 to .96), suggesting that the SILS subscales comprise a higher-order implementation leadership factor. All subscales demonstrated good inter-item reliability (α = .91–.96). Convergent and divergent validity results were generally as hypothesized, with moderate to high correlations between SILS subscales and general leadership, moderate correlations with teaching attitudes, and low correlations with school demographics.

Conclusions

Overall, results provided strong structural, convergent, and divergent validity evidence for the 21-item, 7-factor SILS instrument. Implications for the measurement of implementation leadership in schools are discussed, as well as strategies to support leaders to enhance their strategic behaviors related to the implementation of mental health prevention programs (e.g., adaptation of existing leadership-focused implementation strategies).

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-022-01222-7.

Alliances to disseminate addiction prevention and treatment (ADAPT): A statewide learning health system to reduce substance use among justice-involved youth in rural communities

BACKGROUND

Youth in the justice system (YJS) are more likely than youth who have never been arrested to have mental health and substance use problems. However, a low percentage of YJS receive SUD services during their justice system involvement. The SUD care cascade can identify potential missed opportunities for treatment for YJS. Steps along the continuum of the cascade include identification of treatment need, referral to services, and treatment engagement. To address gaps in care for YJS, we will (1) implement a learning health system (LHS) to develop, or improve upon, alliances between juvenile justice (JJ) agencies and community mental health centers (CMHC) and (2) present local cascade data during continuous quality improvement cycles within the LHS alliances.

METHODS/DESIGN

ADAPT is a hybrid Type II effectiveness implementation trial. We will collaborate with JJ and CMHCs in eight Indiana counties. Application of the EPIS (exploration, preparation, implementation, and sustainment) framework will guide the implementation of the LHS alliances. The study team will review local cascade data quarterly with the alliances to identify gaps along the continuum. The study will collect self-report survey measures longitudinally at each site regarding readiness for change, implementation climate, organizational leadership, and program sustainability. The study will use the Stages of Implementation Completion (SIC) tool to assess the process of implementation across interventions. Additionally, the study team will conduct focus groups and qualitative interviews with JJ and CMHC personnel across the intervention period to assess for impact.

DISCUSSION

Findings have the potential to increase SUD need identification, referral to services, and treatment for YJS.

Psychometric properties of leadership scales for health professionals: a systematic review

Background

The important role of leaders in the translation of health research is acknowledged in the implementation science literature. However, the accurate measurement of leadership traits and behaviours in health professionals has not been directly addressed. This review aimed to identify whether scales which measure leadership traits and behaviours have been found to be reliable and valid for use with health professionals.

Methods

A systematic review was conducted. MEDLINE, EMBASE, PsycINFO, Cochrane, CINAHL, Scopus, ABI/INFORMIT and Business Source Ultimate were searched to identify publications which reported original research testing the reliability, validity or acceptability of a leadership-related scale with health professionals.

Results

Of 2814 records, a total of 39 studies met the inclusion criteria, from which 33 scales were identified as having undergone some form of psychometric testing with health professionals. The most commonly used was the Implementation Leadership Scale (n = 5) and the Multifactor Leadership Questionnaire (n = 3). Of the 33 scales, the majority of scales were validated in English speaking countries including the USA (n = 15) and Canada (n = 4), but also with some translations and use in Europe and Asia, predominantly with samples of nurses (n = 27) or allied health professionals (n = 10). Only two validation studies included physicians. Content validity and internal consistency were evident for most scales (n = 30 and 29, respectively). Only 20 of the 33 scales were found to satisfy the acceptable thresholds for good construct validity. Very limited testing occurred in relation to test-re-test reliability, responsiveness, acceptability, cross-cultural revalidation, convergent validity, discriminant validity and criterion validity.

Conclusions

Seven scales may be sufficiently sound to be used with professionals, primarily with nurses. There is an absence of validation of leadership scales with regard to physicians. Given that physicians, along with nurses and allied health professionals have a leadership role in driving the implementation of evidence-based healthcare, this constitutes a clear gap in the psychometric testing of leadership scales for use in healthcare implementation research and practice.

Trial registration

This review follows the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) (see Additional File 1) (PLoS Medicine. 6:e1000097, 2009) and the associated protocol has been registered with the PROSPERO International Prospective Register of Systematic Reviews (Registration Number CRD42019121544).

Supplementary Information

The online version contains supplementary material available at 10.1186/s13012-021-01141-z.

Advancing theory on the multilevel role of leadership in the implementation of evidence-based health care practices

Background:

Top managers’ transformational leadership is associated with significant influence on subordinates. Yet little is known about the extent to which top managers’ transformational leadership influences middle managers’ implementation leadership and, ultimately, frontline staff delivery of evidence-based health care practices.

Purpose:

To test a multilevel leadership model examining the extent to which top managers’ transformational leadership, as mediated by implementation leadership of middle managers (i.e., those who supervise direct clinical services), affects staff attitudes toward evidence-based practices (EBPs) and their implementation.

Methodology/Approach:

We used data collected in 2013 from 427 employees in 112 addiction health services programs in Los Angeles County, California. We relied on hierarchical linear models with robust standard errors to analyze multilevel data, individuals nested in programs. We conducted two path models to estimate multilevel relationships with two EBPs: contingency management and medication-assisted treatment.

Results:

Findings partially supported our theory-driven multilevel leadership model. Specifically, results demonstrated that middle managers’ implementation leadership mediated the relationship between top managers’ transformational leadership and attitudes toward EBPs. At the same time, they showed the mediated relationship for delivery of contingency management treatment was only marginally significant (standardized indirect effect = .006, bootstrap p = .091). We did not find a mediation effect for medication-assisted treatment.

Discussion:

Findings advance leadership theory in health care, highlighting the importance of middle managers’ implementation leadership in transmitting the influence of top managers’ transformational leadership on staff attitudes toward EBPs. The full path model shows the extent to which transformational leadership may influence staff implementation of innovative practices as mediated through staff attitudes toward EBPs and middle managers’ implementation leadership.

Practice Implications:

Our findings have implications for developing a multilevel leadership approach to implementation in health care. Leadership development should build on different competencies based on managers' level but align managers' priorities on the same implementation goals.

Validating the Implementation Leadership Scale in Chinese nursing context: A cross-sectional study

AIM

This study aimed to evaluate the validity, reliability and acceptability of the Implementation Leadership Scale in the Chinese nursing context.

DESIGN

This study utilized a cross-sectional design.

METHODS

This study was conducted in one general tertiary hospital with 234 nurses (85.3% response rate) from 35 clinical units in China. Content validity, structural validity, convergent validity, reliability (internal consistency), agreement indices and acceptability were evaluated. The data collection was from December 1st, 2017 to June 30th, 2018.

RESULTS

Confirmatory factor analysis demonstrated a good model fit to the four-factor implementation leadership model. The psychometric testing also indicated good convergent validity, high internal consistency and acceptable aggregation. Most participants completed the scale in two minutes or less and agreed or strongly agreed that the questions were relevant to implementation leadership, clear and easy to answer.

CONCLUSIONS

This study demonstrated that the Chinese Implementation Leadership Scale is a valid, reliable and pragmatic tool for measuring strategic leadership for implementing evidence-based practices.

Using the Practical, Robust Implementation and Sustainability Model (PRISM) to qualitatively assess multilevel contextual factors to help plan, implement, evaluate, and disseminate health services programs

There is consensus in dissemination and implementation (D&I) science that addressing contextual factors is critically important for understanding translation of health care delivery interventions but little agreement on which contextual factors are key determinants of implementation outcomes. We describe the application of the Practical Robust Implementation and Sustainability Model (PRISM), which expands the Reach, Effectiveness, Adoption, Implementation, Maintenance (RE-AIM) framework to identify contextual factors across four diverse programs. Multiple qualitative methods were used to collect multilevel, multistakeholder perspectives from the adopting organizations and staff. We identified measures for evaluating context through the various domains of PRISM to guide health services research across the phases of program implementation. The PRISM domains of Recipients, Implementation and Sustainability Infrastructure, and External Environment identified important multilevel contextual factors, including variability in operational processes and available resources. These domains helped to facilitate planning and implementation phases of the four interventions and guide purposeful adaptations. We found assessments of PRISM domains useful to systematically assess multilevel contextual factors across various content areas as well as phases of program implementation. Additionally, these contextual factors were found to be relevant to RE-AIM outcomes. Lessons learned can be applied to future research as there is a need to investigate the measurement properties of PRISM and continue to test which contextual factors are most important to successful implementation and for which outcomes.

Implementing internet- and tele-based interventions to prevent mental health disorders in farmers, foresters and gardeners (ImplementIT): study protocol for the multi-level evaluation of a nationwide project

BACKGROUND

Farmers are a vulnerable population for developing depression or other mental health disorders due to a variety of risk factors in their work context. Beyond face-to-face resources, preventive internet- and tele-based interventions could extend available treatment options to overcome barriers to care. The German Social Insurance Company for Agriculture, Forestry and Horticulture (SVLFG) implements several guided internet- and mobile-based interventions and personalised tele-based coaching for this specific target group provided by external companies within a nation-wide prevention project for their insured members. The current study aims to evaluate the implementation process and to identify determinants of successful implementation on various individual and organisational levels.

METHODS

The current study includes two groups of participants: 1) insured persons with an observable need for prevention services, and 2) staff-participants who are involved in the implementation process. The Reach, Effectiveness, Adoption, Implementation and Maintenance (RE-AIM) framework and the Consolidated Framework for Implementation Research (CFIR) will be used to track and evaluate the implementation process. A mixed-method approach will provide insights on individual and organizational level (e.g. degree of normalization, readiness for change) and helps to identify determinants of successful implementation. In-depth insights on experiences of the participants (e.g. acceptance, satisfaction, barriers and facilitating factors of intervention use) will be yielded through qualitative interviews. Focus groups with field workers provide insights into barriers and facilitators perceived during their consultations. Furthermore, intervention as well as implementation costs will be evaluated. According to the stepwise, national rollout, data collection will occur at baseline and continuously across 24 months.

DISCUSSION

The results will show to what extent the implementation of the internet- and tele-based services as a preventive offer will be accepted by the participants and involved employees and which critical implementation aspects will occur within the process. If the implementation of the internet- and tele-based services succeeds, these services may be feasible in the long-term.

TRIAL REGISTRATION

German Clinical Trial Registration: DRKS00017078 . Registered on 18.04.2019.

Protocol: examining the effectiveness of an adaptive implementation intervention to improve uptake of the VA suicide risk identification strategy: a sequential multiple assignment randomized trial

Background

In 2018, the Veterans Health Administration (VHA) mandated implementation of a national suicide risk identification strategy (Risk ID). The goal of Risk ID is to improve the detection and management of suicide risk by standardizing suicide risk screening and evaluation enterprise-wide. In order to ensure continuous quality improvement (QI), ongoing evaluation and targeted interventions to improve implementation of Risk ID are needed. Moreover, given that facilities will vary with respect to implementation needs and barriers, the dose and type of intervention needed may vary across facilities. Thus, the objective of this study is to examine the effectiveness of an adaptive implementation strategy to improve the uptake of suicide risk screening and evaluation in VHA ambulatory care settings. In addition, this study will examine specific factors that may impact the uptake of suicide risk screening and evaluation and the adoption of different implementation strategies. This protocol describes the stepped implementation approach and proposed evaluation plan.

Methods

Using a sequential multiple assignment randomized trial (SMART) design, two evidence-based implementation strategies will be evaluated: (1) audit and feedback (A&F); (2) A&F plus external facilitation (A&F + EF). Implementation outcomes of interest include uptake of secondary suicide risk screening and uptake of comprehensive suicide risk evaluation (stages 2 and 3 of Risk ID). Secondary outcomes include rates of other clinical outcomes (i.e., safety planning) and organizational factors that may impact Risk ID implementation (i.e., leadership climate and leadership support).

Discussion

This national QI study will use a SMART design to evaluate whether an adaptive implementation strategy is effective in improving uptake of a mandated VHA-wide suicide risk screening and evaluation initiative. If this study finds that the proposed stepped implementation strategy is effective at increasing uptake and maintaining performance improvements, this approach may be used as an overarching QI strategy for other national suicide prevention programs.

Trial registration

ClinicalTrials.gov NCT04243330. Registered 28 January 2020

Establishing cross-systems collaborations for implementation: protocol for a longitudinal mixed methods study

Background

Cross-system interventions can help integrate services across different service delivery systems but require organizations to establish strong collaborative relationships for implementation. Contingency theory suggests that the effectiveness of different collaborative strategies (i.e. specific ways organizations align operations and services) varies by context. This paper describes a study of different strategies for fostering collaboration between child welfare and substance abuse treatment agencies and the conditions under which they are effective for implementation. We also describe the development and piloting of the Collaborating Across Systems for Program Implementation (CASPI) tool—a decision-making guide intended to help researchers and organizational leaders identify and use appropriate collaborative strategies for their context.

Methods/design

This multisite longitudinal, mixed methods study, leverages a naturally occurring implementation initiative -- in up to 17 Ohio counties -- to implement Ohio START (Sobriety Treatment and Reducing Trauma). START is a child welfare model that requires strong collaboration with local substance use treatment organizations to promote integrated services. During the first two years, we will identify collaborative strategies associated with improved START implementation (penetration and fidelity) and service delivery outcomes (timeliness), given system, and organizational features. We will conduct a convergent mixed methods study drawing on worker surveys, agency documents, administrative data, formal partner agreements, and group interviews. Data will be integrated and analyzed using Qualitative Comparative Analysis (QCA). To develop the CASPI, an expert panel comprised of implementation experts, and community stakeholders will convene to synthesize our findings and develop contents (including a decision tree). During the final year of the study, we will assess the acceptability, appropriateness, and feasibility of the CASPI in a randomized vignette experiment, and a pilot-test with 3 child welfare agencies that have not yet implemented START.

Discussion

Our results will lay the groundwork for a larger controlled trial that will test the CASPI’s effectiveness for supporting effective and efficient implementation of cross-system interventions like START. The CASPI is expected to help leaders and researchers select and use collaboration strategies tailored to their context and be applicable in a wide range of settings including rural communities. Our work also advances system-level implementation strategies.

Trial registration

NCT03931005, Registered April 29, 2019.

Validity and reliability of the Greek version of Implementation Leadership Scale (ILS)

Background

The need for developing pragmatic and reliable measures that affect evidence-based practice has been highlighted in organizational studies. The aim of the current study is to evaluate the psychometric properties of the Greek version of Implementation Leadership Scale (ILS). ILS is a brief and effective tool for measuring leadership when implementing evidence based practices.

Methods

The translation process followed World Health Organization guidelines. Face and content validity were examined. Then, the psychometric properties of ILS were tested with a sample of 143 nurses and midwifes working in a private Greek hospital. Confirmatory Factor Analyses for structural validity testing, Pearson coefficient for convergent and discriminant validity testing as well as internal consistency analysis for reliability testing were conducted. Quality of leadership scale from COPSOQ II and Organizational Climate Measure were used for assessing convergent and discriminant validity, respectively.

Results

Greek version of ILS show good face and content validity. CFA results (x² = 100. 69 (50); CFI = 0.93; GFI = 0.83; RMSEA = 0.06) confirmed the four-factor structure of the scale (Proactive, Knowledgeable, Supportive and Perseverant leadership). The internal consistency was excellent (a = 0.94 for total scale and between 0.85 and 0.91 for subscales). Analyses also revealed good convergent and discriminant validity.

Conclusions

The findings suggest that the Greek Version of ILS is a valid and reliable tool for measuring leadership of evidence based practices implementation. However, further research for assessing its psychometric properties in various samples and more professional groups is suggested.

Pragmatic adaptation of implementation research measures for a novel context and multiple professional roles: a factor analysis study

BACKGROUND

Although some advances have been made in recent years, the lack of measures remains a major challenge in the field of implementation research. This results in frequent adaptation of implementation measures for different contexts-including different types of respondents or professional roles-than those for which they were originally developed and validated. The psychometric properties of these adapted measures are often not rigorously evaluated or reported. In this study, we examined the internal consistency, factor structure, and structural invariance of four well-validated measures of inner setting factors across four groups of respondents. The items in these measures were adapted as part of an evaluation of a large-scale organizational change in a rehabilitation hospital, which involved transitioning to a new building and a new model of patient care, facilitated by a significant redesign of patient care and research spaces.

METHODS

Items were tailored for the context and perspective of different respondent groups and shortened for pragmatism. Confirmatory factor analysis was then used to test study hypotheses related to fit, internal consistency, and invariance across groups.

RESULTS

The survey was administered to approximately 1208 employees; 785 responded (65% response rate) across the roles of clinician, researcher, leader, support staff, or dual clinician and researcher. For each of the four scales, confirmatory factor analysis demonstrated adequate fit that largely replicated the original measure. However, a few items loaded poorly and were removed from the final models. Internal consistencies of the final scales were acceptable. For scales that were administered to multiple professional roles, factor structures were not statistically different across groups, indicating structural invariance.

CONCLUSIONS

The four inner setting measures were robust for use in this new context and across the multiple stakeholder groups surveyed. Shortening these measures did not significantly impair their measurement properties; however, as this study was cross sectional, future studies are required to evaluate the predictive validity and test-retest reliability of these measures. The successful use of adapted measures across contexts, across and between respondent groups, and with fewer items is encouraging, given the current emphasis on designing pragmatic implementation measures.

EBP Implementation Leadership of Frontline Nurse Managers: Validation of the Implementation Leadership Scale in Acute Care

BACKGROUND

Frontline nurse managers influence the implementation of evidence-based practices (EBP); however, there is a need for valid and reliable instruments to measure their leadership behaviors for EBP implementation in acute care settings.

AIM

The aim of this study was to evaluate the validity and reliability of the Implementation Leadership Scale (ILS) in acute care settings using two unique nurse samples.

METHODS

This study is a secondary analysis of ILS data obtained through two distinct multisite cross-sectional studies. Sample 1 included 200 registered nurses from one large Californian health system. Sample 2 was 284 registered nurses from seven Midwest and Northeast U.S. hospitals. Two separate studies by different research teams collected responses using written and electronic questionnaires. We analyzed each sample independently. Descriptive statistics described individual item, total, and subscale scores. We analyzed validity using confirmatory factor analysis and within-unit agreement (awg). We evaluated factorial invariance using multigroup confirmatory factor analyses and evaluating change in chi-square and comparative fit index values. We evaluated reliability using Cronbach's alpha.

RESULTS

Confirmatory factor analyses in both samples provided strong support for first- and second-order factor structure of the ILS. The factor structure did not differ between the two samples. Across both samples, internal consistency reliability was strong (Cronbach's alpha: 0.91-0.98), as was within-unit agreement (awg: 0.70-0.80).

LINKING EVIDENCE TO ACTION

Frontline manager implementation leadership is a critical contextual factor influencing EBP implementation. This study provides strong evidence supporting the validity and reliability of the ILS to measure implementation leadership behaviors of nursing frontline managers in acute care. The ILS can help clinicians, researchers, and leaders in nursing contexts assess frontline manager implementation leadership, deliver interventions to target areas needing improvement, and improve implementation of EBP.

Attitudes Towards Evidence-Based Practice in Substance Use Treatment Programs Serving American Indian Native Communities

The objective of this paper was to compare attitudes towards evidence-based practice (EBP) of substance use disorder treatment (SUDT) center employees' serving American Indian and Alaskan Native (AIAN) populations to those serving non-AIAN populations. Survey data on the openness and divergence subscales of the Evidence-Based Practice Attitude Scale (EBPAS) were collected and analyzed. Independent samples T tests were performed to compare the two samples. For all comparisons, except the divergence subscale between counselors, the SUDT employees serving AIANs had significantly lower mean openness scores and higher mean divergence scores than those serving non-AIANs. This study suggests that employees of SUDT centers serving AIAN population hold less positive attitudes towards the adoption and use of EBP than non-AIAN.

When east meets west: a qualitative study of barriers and facilitators to evidence-based practice in Hunan China

Background

Research into evidence-based practice has been extensively explored in nursing and there is strong recognition that the organizational context influences implementation. A range of barriers has been identified; however, the research has predominantly taken place in Western cultures, and there is little information about factors that influence evidence-based practice in China. The purpose of this study was to explore barriers and facilitators to evidence-based practice in Hunan province, a less developed region in China.

Methods

A descriptive qualitative methodology was employed. Semi-structured interviews were conducted with staff nurses, head nurses and directors (n = 13). Interviews were translated into English and verified for accuracy by two bilingual researchers. Both Chinese and English data were simultaneously analyzed for themes related to factors related to the evidence to be implemented (Innovation), nurses’ attitudes and beliefs (Potential Adopters), and the organizational setting (Practice Environment).

Results

Barriers included lack of available evidence in Chinese, nurses’ lack of understanding of what evidence-based practice means, and fear that patients will be angry about receiving care that is perceived as non-traditional. Nurses believed evidence-based practice was to be used when clinical problems arose, and not as a routine way to practice. Facilitators included leadership support and the pervasiveness of web based social network services such as Baidu (百度) for easy access to information.

Conclusion

While several parallels to previous research were found, our study adds to the knowledge base about factors related to evidence-based practice in different contextual settings. Findings are important for international comparisons to develop strategies for nurses to provide evidence-based care.

Electronic supplementary material

The online version of this article (10.1186/s12912-018-0295-x) contains supplementary material, which is available to authorized users.

Associations among unit leadership and unit climates for implementation in acute care: a cross-sectional study

Background

Nurse managers have a pivotal role in fostering unit climates supportive of implementing evidence-based practices (EBPs) in care delivery. EBP leadership behaviors and competencies of nurse managers and their impact on practice climates are widely overlooked in implementation science. The purpose of this study was to examine the contributions of nurse manager EBP leadership behaviors and nurse manager EBP competencies in explaining unit climates for EBP implementation in adult medical-surgical units.

Methods

A multi-site, multi-unit cross-sectional research design was used to recruit the sample of 24 nurse managers and 553 randomly selected staff nurses from 24 adult medical-surgical units from 7 acute care hospitals in the Northeast and Midwestern USA. Staff nurse perceptions of nurse manager EBP leadership behaviors and unit climates for EBP implementation were measured using the Implementation Leadership Scale and Implementation Climate Scale, respectively. EBP competencies of nurse managers were measured using the Nurse Manager EBP Competency Scale. Participants were emailed a link to an electronic questionnaire and asked to respond within 1 month. The contributions of nurse manager EBP leadership behaviors and competencies in explaining unit climates for EBP implementation were estimated using mixed-effects models controlling for nurse education and years of experience on current unit and accounting for the variability across hospitals and units. Significance level was set at α < .05.

Results

Two hundred sixty-four staff nurses and 22 nurse managers were included in the final sample, representing 22 units in 7 hospitals. Nurse manager EBP leadership behaviors (p < .001) and EBP competency (p = .008) explained 52.4% of marginal variance in unit climate for EBP implementation. Leadership behaviors uniquely explained 45.2% variance. The variance accounted for by the random intercepts for hospitals and units (p < .001) and years of nursing experience in current unit (p < .05) were significant but level of nursing education was not.

Conclusion

Nurse managers are significantly related to unit climates for EBP implementation primarily through their leadership behaviors. Future implementation studies should consider the leadership of nurse managers in creating climates supportive of EBP implementation.

Electronic supplementary material

The online version of this article (10.1186/s13012-018-0753-6) contains supplementary material, which is available to authorized users.

Leadership rounds to reduce health care-associated infections

BACKGROUND

Evidence-based guidelines exist to reduce health care-associated infections (HAIs). Leadership rounds are one tool leaders can use to ensure compliance with guidelines, but have not been studied specifically for the reduction of HAIs. This study examines HAI leadership rounds at one facility.

METHODS

We explored unit-based HAI leadership rounds led by 2 hospital leaders at a large academic hospital. Leadership rounds were observed on 19 units, recorded, and coded to identify themes. Themes were linked to the Consolidated Framework for Implementation Research and used to guide interviews with frontline staff members.

RESULTS

Staff members disclosed unit-specific problems and readily engaged in problem-solving with top hospital leaders. These themes appeared over 350 times within 22 rounds. Findings revealed that leaders used words that demonstrated fallibility and modeled curiosity, 2 factors associated with learning climate and psychologic safety. These 2 themes appeared 115 and 142 times, respectively. The flexible nature of the rounds appeared to be conducive for reflection and evaluation, which was coded 161 times.

CONCLUSIONS

Each interaction between leaders and frontline staff can foster psychologic safety, which can lead to open problem-solving to reduce barriers to implementation. Discovering specific communication and structural factors that contribute to psychologic safety may be powerful in reducing HAIs.

Ottawa Model of Implementation Leadership and Implementation Leadership Scale: mapping concepts for developing and evaluating theory-based leadership interventions

Purpose

Leadership in health care is instrumental to creating a supportive organizational environment and positive staff attitudes for implementing evidence-based practices to improve patient care and outcomes. The purpose of this study is to demonstrate the alignment of the Ottawa Model of Implementation Leadership (O-MILe), a theoretical model for developing implementation leadership, with the Implementation Leadership Scale (ILS), an empirically validated tool for measuring implementation leadership. A secondary objective is to describe the methodological process for aligning concepts of a theoretical model with an independently established measurement tool for evaluating theory-based interventions.

Methods

Modified template analysis was conducted to deductively map items of the ILS onto concepts of the O-MILe. An iterative process was used in which the model and scale developers (n=5) appraised the relevance, conceptual clarity, and fit of each ILS items with the O-MILe concepts through individual feedback and group discussions until consensus was reached.

Results

All 12 items of the ILS correspond to at least one O-MILe concept, demonstrating compatibility of the ILS as a measurement tool for the O-MILe theoretical constructs.

Conclusion

The O-MILe provides a theoretical basis for developing implementation leadership, and the ILS is a compatible tool for measuring leadership based on the O-MILe. Used together, the O-MILe and ILS provide an evidence- and theory-based approach for developing and measuring leadership for implementing evidence-based practices in health care. Template analysis offers a convenient approach for determining the compatibility of independently developed evaluation tools to test theoretical models.

Testing the leadership and organizational change for implementation (LOCI) intervention in substance abuse treatment: a cluster randomized trial study protocol

BACKGROUND

Evidence-based practice (EBP) implementation represents a strategic change in organizations that requires effective leadership and alignment of leadership and organizational support across organizational levels. As such, there is a need for combining leadership development with organizational strategies to support organizational climate conducive to EBP implementation. The leadership and organizational change for implementation (LOCI) intervention includes leadership training for workgroup leaders, ongoing implementation leadership coaching, 360° assessment, and strategic planning with top and middle management regarding how they can support workgroup leaders in developing a positive EBP implementation climate.

METHODS

This test of the LOCI intervention will take place in conjunction with the implementation of motivational interviewing (MI) in 60 substance use disorder treatment programs in California, USA. Participants will include agency executives, 60 program leaders, and approximately 360 treatment staff. LOCI will be tested using a multiple cohort, cluster randomized trial that randomizes workgroups (i.e., programs) within agency to either LOCI or a webinar leadership training control condition in three consecutive cohorts. The LOCI intervention is 12 months, and the webinar control intervention takes place in months 1, 5, and 8, for each cohort. Web-based surveys of staff and supervisors will be used to collect data on leadership, implementation climate, provider attitudes, and citizenship. Audio recordings of counseling sessions will be coded for MI fidelity. The unit of analysis will be the workgroup, randomized by site within agency and with care taken that co-located workgroups are assigned to the same condition to avoid contamination. Hierarchical linear modeling (HLM) will be used to analyze the data to account for the nested data structure.

DISCUSSION

LOCI has been developed to be a feasible and effective approach for organizations to create a positive climate and fertile context for EBP implementation. The approach seeks to cultivate and sustain both effective general and implementation leadership as well as organizational strategies and support that will remain after the study has ended. Development of a positive implementation climate for MI should result in more positive service provider attitudes and behaviors related to the use of MI and, ultimately, higher fidelity in the use of MI.

TRIAL REGISTRATION

This study is registered with Clinicaltrials.gov ( NCT03042832 ), 2 February 2017, retrospectively registered.