The meaning and measurement of implementation climate

Adaptability, Scalability and Sustainability (ASaS) of complex health interventions: a systematic review of theories, models and frameworks

BACKGROUND

Complex health interventions (CHIs) are increasingly used in public health, clinical research and education to reduce the burden of disease worldwide. Numerous theories, models and frameworks (TMFs) have been developed to support implementation of CHIs. This systematic review aims to identify and critique theoretical frameworks concerned with three features of implementation; adaptability, scalability and sustainability (ASaS). By dismantling the constituent theories, analysing their component concepts and then exploring factors that influence each theory the review team hopes to offer an enhanced understanding of considerations when implementing CHIs.

METHODS

This review searched PubMed MEDLINE, CINAHL, Web of Science, and Google Scholar for research investigating the TMFs of complex health interventions. Narrative synthesis was employed to examine factors that may influence the adaptability, scalability and sustainability of complex health interventions.

RESULTS

A total of 9763 studies were retrieved from the five databases (PubMed, MEDLINE, CINAHL, Web of Science, and Google Scholar). Following removal of duplicates and application of the eligibility criteria, 35 papers were eligible for inclusion. Influencing factors can be grouped within outer context (socio-political context; leadership funding, inter-organisational networks), inner context; (client advocacy; organisational characteristics), intervention characteristics (supervision, monitoring and evaluation), and bridging factors (individual adopter or provider characteristics).

CONCLUSION

This review confirms that identified TMFS do not typically include the three components of adaptability, scalability, and sustainability. Current approaches focus on high income countries or generic "whole world" approaches with few frameworks specific to low- and middle-income countries. The review offers a starting point for further exploration of adaptability, scalability and sustainability, within a low- and middle-income context.

TRIAL REGISTRATION

Not registered.

The Role of Implementation Climate in Moderating Educator Use of Evidence-Based Practices and Outcomes for Autistic Students

Ensuring effective use of evidence-based practice (EBP) for autism in schools is imperative due to the significantly increasing number of autistic students receiving school services each year. High-quality EBP use has proven challenging in schools. Research indicates implementation climate, or how EBP are supported, rewarded, and valued, and EBP resources are related to successful implementation. However, limited understanding of system-level contextual factors that impact EBP implementation for school-based providers makes development of appropriate implementation supports challenging. Understanding these factors is crucial for selecting and tailoring implementation strategies to support EBP scale up. In this observational study, California school-based providers (n = 1084) completed surveys related to implementation climate, leadership, autism experience and EBP implementation (use, competence, knowledge). Student outcomes included state level academic and behavioral indicators. Using an implementation science framework (Aarons et al., in Administration and Policy in Mental Health and Mental Health Services Research 38:4-23, 2011) and multilevel modeling, we examined the relationship between EBP Implementation and student outcomes and the moderation effects of provider and district level factors. Higher implementation climate predicted better EBP implementation outcomes, and proved more impactful when provider hands-on autism experience was low. Greater EBP resources predicted a higher percentage of students who met math standards only when district poverty level was high. Our findings suggested moderating effects on EBP implementation from both provider and system level factors. Implementation climate and resources may be especially key in addressing equity issues related to high poverty schools in which teachers often have less autism experience.

Randomized trial of the leadership and organizational change for implementation (LOCI) strategy in substance use treatment clinics

INTRODUCTION

Evidence-based practice (EBP) implementation represents a strategic change that requires alignment of leadership and support throughout organizations. Leadership and Organizational Change for Implementation (LOCI) is a multifaceted implementation strategy that aims to improve implementation leadership and climate within organizations through iterative cycles of leadership and climate assessment and feedback, leadership training and coaching, and strategic planning with upper-level leaders. This study tested the effects of LOCI on transformational and implementation leadership, implementation climate, implementation citizenship behavior, and EBP reach.

METHODS

A multiple cohort, cluster randomized trial tests the effect of LOCI in 60 clinics across nine behavioral health organizations in California and Arizona, USA. The study randomized clinics within organizations to either LOCI or a leadership training webinar control condition in three consecutive cohorts. Repeated web-based surveys of direct service providers (nLOCI = 201, nControl = 179) assessed leadership, implementation climate, and implementation citizenship over time. Multilevel autoregressive modeling was the primary statistical analysis such that providers (level-1) were nested within clinics (level-2). The study predicted between-condition differences at 4-, 8-, and 12-month follow-up assessments. Provider engagement in a fidelity monitoring process assessed reach of motivational interviewing (i.e., number of sessions recorded/submitted for fidelity coding). An independent sample t-test explored between condition differences in motivational interviewing reach.

RESULTS

Results indicated between condition differences at 4 months for implementation leadership, implementation climate, and implementation citizenship behavior such that greater improvements were evidenced in the LOCI condition compared to the control condition. Reach of MI was significantly greater in the LOCI vs control condition such that LOCI providers were significantly more likely to engage in the fidelity monitoring process (chi-square (1, n = 370) = 5.59, p = .018).

CONCLUSIONS

LOCI was developed based on organizational theories of strategic leadership and climate to affect organizational change processes that communicate that innovation implementation is expected, supported, and recognized as a value of the organization. The LOCI implementation strategy resulted in more positive hypothesized outcomes compared to the control condition. Organizational change strategies have utility for implementing health innovations in complex, multilevel contexts and for greater sustainment of facilitative leader behaviors, strategic implementation climate, and improved implementation outcomes.

TRIAL REGISTRATION

This study is registered with Clinicaltrials.gov gov (NCT03042832, 2 February 2017; retrospectively registered).

Inter-organizational alignment and implementation outcomes in integrated mental healthcare for children and adolescents: a cross-sectional observational study

BACKGROUND

Integrated care involves care provided by a team of professionals, often in non-traditional settings. A common example worldwide is integrated school-based mental health (SBMH), which involves externally employed clinicians providing care at schools. Integrated mental healthcare can improve the accessibility and efficiency of evidence-based practices (EBPs) for vulnerable populations suffering from fragmented traditional care. However, integration can complicate EBP implementation due to overlapping organizational contexts, diminishing the public health impact. Emerging literature suggests that EBP implementation may benefit from the similarities in the implementation context factors between the different organizations in integrated care, which we termed inter-organizational alignment (IOA). This study quantitatively explored whether and how IOA in general and implementation context factors are associated with implementation outcomes in integrated SBMH.

METHODS

SBMH clinicians from community-based organizations (CBOs; nclinician = 27) and their proximal student-support school staff (nschool = 99) rated their schools and CBOs (clinician only) regarding general (organizational culture and molar climate) and implementation context factors (Implementation Climate and Leadership), and nine common implementation outcomes (e.g., treatment integrity, service access, acceptability). The levels of IOA were estimated by intra-class correlations (ICCs). We fitted multilevel models to estimate the standalone effects of context factors from CBOs and schools on implementation outcomes. We also estimated the 2-way interaction effects between CBO and school context factors (i.e., between-setting interdependence) on implementation outcomes.

RESULTS

The IOA in general context factors exceeded those of implementation context factors. The standalone effects of implementation context factors on most implementation outcomes were larger than those of general context factors. Similarly, implementation context factors between CBOs and schools showed larger 2-way interaction effects on implementation outcomes than general context factors.

CONCLUSIONS

This study preliminarily supported the importance of IOA in context factors for integrated SBMH. The findings shed light on how IOA in implementation and general context factors may be differentially associated with implementation outcomes across a broad array of integrated mental healthcare settings.

Improving measurement-based care implementation in youth mental health through organizational leadership and climate: a mechanistic analysis within a randomized trial

BACKGROUND

Theory and correlational research indicate organizational leadership and climate are important for successful implementation of evidence-based practices (EBPs) in healthcare settings; however, experimental evidence is lacking. We addressed this gap using data from the WISDOM (Working to Implement and Sustain Digital Outcome Measures) hybrid type III effectiveness-implementation trial. Primary outcomes from WISDOM indicated the Leadership and Organizational Change for Implementation (LOCI) strategy improved fidelity to measurement-based care (MBC) in youth mental health services. In this study, we tested LOCI's hypothesized mechanisms of change, namely: (1) LOCI will improve implementation and transformational leadership, which in turn will (2) mediate LOCI's effect on implementation climate, which in turn will (3) mediate LOCI's effect on MBC fidelity.

METHODS

Twenty-one outpatient mental health clinics serving youth were randomly assigned to LOCI plus MBC training and technical assistance or MBC training and technical assistance only. Clinicians rated their leaders' implementation leadership, transformational leadership, and clinic implementation climate for MBC at five time points (baseline, 4-, 8-, 12-, and 18-months post-baseline). MBC fidelity was assessed using electronic metadata for youth outpatients who initiated treatment in the 12 months following MBC training. Hypotheses were tested using longitudinal mixed-effects models and multilevel mediation analyses.

RESULTS

LOCI significantly improved implementation leadership and implementation climate from baseline to follow-up at 4-, 8-, 12-, and 18-month post-baseline (all ps < .01), producing large effects (range of ds = 0.76 to 1.34). LOCI's effects on transformational leadership were small at 4 months (d = 0.31, p = .019) and nonsignificant thereafter (ps > .05). LOCI's improvement of clinic implementation climate from baseline to 12 months was mediated by improvement in implementation leadership from baseline to 4 months (proportion mediated [pm] = 0.82, p = .004). Transformational leadership did not mediate LOCI's effect on implementation climate (p = 0.136). Improvement in clinic implementation climate from baseline to 12 months mediated LOCI's effect on MBC fidelity during the same period (pm = 0.71, p = .045).

CONCLUSIONS

LOCI improved MBC fidelity in youth mental health services by improving clinic implementation climate, which was itself improved by increased implementation leadership. Fidelity to EBPs in healthcare settings can be improved by developing organizational leaders and strong implementation climates.

TRIAL REGISTRATION

ClinicalTrials.gov identifier: NCT04096274. Registered September 18, 2019.

Developing and testing a produce prescription implementation blueprint to improve food security in a clinical setting: a pilot study protocol

BACKGROUND

Food insecurity is common in the United States, especially in Rhode Island, where it affects up to 33% of residents. Food insecurity is associated with adverse health outcomes and disproportionally affects people from minoritized backgrounds. Produce prescription programs, in which healthcare providers write "prescriptions" for free or reduced cost vegetables, have been used to address food insecurity and diet-related chronic disease. Although there is growing evidence for the effectiveness of produce prescription programs in improving food security and diet quality, there have been few efforts to use implementation science methods to improve the adoption of these programs.

METHODS

This two-phase pilot study will examine determinants and preliminary implementation and effectiveness outcomes for an existing produce prescription program. The existing program is funded by an Accountable Care Organization in Rhode Island and delivered in primary care practices. For the first phase, we conducted a formative evaluation, guided by the Consolidated Framework for Implementation Research 2.0, to assess barriers, facilitators, and existing implementation strategies for the produce prescription program. Responses from the formative evaluation were analyzed using a rapid qualitative analytic approach to yield a summary of existing barriers and facilitators. In the second phase, we presented our formative evaluation findings to a community advisory board consisting of primary care staff, Accountable Care Organization staff, and staff who source and deliver the vegetables. The community advisory board used this information to identify and refine a set of implementation strategies to support the adoption of the program via an implementation blueprint. Guided by the implementation blueprint, we will conduct a single-arm pilot study to assess implementation antecedents (i.e., feasibility, acceptability, appropriateness, implementation climate, implementation readiness), implementation outcomes (i.e., adoption), and preliminary program effectiveness (i.e., food and nutrition security). The first phase is complete, and the second phase is ongoing.

DISCUSSION

This study will advance the existing literature on produce prescription programs by formally assessing implementation determinants and developing a tailored set of implementation strategies to address identified barriers. Results from this study will inform a future fully powered hybrid type 3 study that will use the tailored implementation strategies and assess implementation and effectiveness outcomes for a produce prescription program.

TRIAL REGISTRATION

Clinical trials: NCT05941403 , Registered June 9, 2023.

Climates and associated factors for evidence-based practice implementation among nurses: a cross-sectional study

BACKGROUND

The organizational climate that fosters and supports the implementation of evidence is a key factor influencing the effective implementation of evidence-based practice (EBP). Nurses, being the largest group of medical staff, play a crucial role in EBP implementation. The perception of the climate for EBP implementation among nurses in their organizations is unknown, especially among Chinese nurses.

AIMS

To clarify the implementation climate of EBP among Chinese nurses and identify the factors associated with the implementation and development of evidence-based nursing practices.

METHODS

This study employed a descriptive cross-sectional study design. From March 2023 to April 2023, a sample of nurses (n = 1260) from two Tertiary care hospitals in central China were selected and surveyed by self-designed social-demographic questionnaire and Implementation Climate Scale. Multiple linear stepwise regression analysis was conducted to determine the predictors of implementation climate.

RESULTS

The nurses achieved a mean ICS score of 59.10 ± 11.22, with mean scores exceeding 3 points for each sub-dimension and item. In the results of multiple linear regression, income satisfaction, implementation of evidence-based nursing practice project(s) within the unit, experience of evidence-based nursing practice during working life, and specific training or courses in evidence-based nursing practice were predictors of ICS. Of these, income satisfaction was the most significant predictor. These factors could explain 17.5% of the total variance in implementation climate.

CONCLUSION

The climate for implementing EBP in Chinese nursing organizations was relatively strong. Nursing managers can enhance the climate for implementing EBP in their organizations by actively improving salary and enhancing EBP-related trainings and practices.

RELEVANCE TO CLINICAL PRACTICE

Understanding nurses' perceptions of the EBP implementation climate in their organizations can help to identify specific barriers and facilitators to the development of EBP and facilitate its successful implementation.

PATIENT OR PUBLIC CONTRIBUTION

Clinical nurses were involved in data collection and completed the questionnaires on EBP implementation climate.

Determinants of factors affecting readiness of academic institutions to conduct knowledge translation in low- and middle-income countries

Introduction

Capacity building strategies have been used to improve uptake of knowledge translation (KT) activities among academic institutions, but little is known about their effectiveness, contextual responsiveness, and adaptability. Many of these strategies target individuals while few address institutional gaps. This research describes the determinants for conducting KT (or readiness to conduct such activities) at the institutional level across diverse LMIC contexts to inform the development of capacity building strategies.

Methods

We conducted a survey to assess organizational readiness to conduct KT to public health researchers and practitioners from six academic institutions in Bangladesh, Ethiopia, DRC, India, Indonesia and Nigeria and members of a global knowledge-to-action working group. We assessed the frequency of barriers and facilitators to KT and their relationship to age, gender, country, and KT experience. We then performed logistic regression to identify determinants of five underlying factors demonstrated to influence KT readiness in LMICs (Institutional Climate, Organization Change Efficacy, Prioritization and Cosmopolitanism, Self-Efficacy and Financial Resource) along with their composite score, which represented an overall readiness score to conduct KT.

Results

A total of 111 responses were included in the final analysis. Participants represented 10 LMICs; a majority were 30-49 years old (57%) and most were male (53%). Most participants had professional foci in research (84%), teaching (62%), and project coordination (36%) and 59% indicated they had experience with KT. Common facilitators included motivated faculty (57%) and dedicated personnel (40%). Funding (60%), training (37%), and time (37%) were the most frequently reported barriers. In the adjusted model, age, gender, country, and professional focus were significantly associated with at least one factor. Prior experience with KT was significantly and positively (OR = 9.07; CI: 1.60-51.58; p < 0.05) associated with the overall KT readiness to conduct KT.

Discussion

Different KT readiness factors are relevant for younger (institutional climate) vs. older (self-efficacy) academic professionals, suggesting value in cross-generational collaborations. Leadership and gender were both relevant for organizational change efficacy indicating a need to engage leaders and promote women to influence organizational change. Institutions in different countries may be at different stages of change; readiness assessments can be used to systematically identify needs and develop targeted strategies.

Inter-Organizational Alignment and Implementation Outcomes in Integrated Mental Healthcare for Children and Adolescents: A Cross-Sectional Observational Study

Background

Integrated care involves care provided by a team of professionals, often in non-traditional settings. A common example worldwide is integrated school-based mental health (SBMH), which involves externally employed clinicians providing care at schools. Integrated mental healthcare can improve the accessibility and efficiency of evidence-based practices (EBPs) for vulnerable populations suffering from fragmented traditional care. However, integration can complicate EBP implementation due to overlapping organizational contexts, diminishing the public health impact. Emerging literature suggests that EBP implementation may benefit from the similarities in the implementation context factors between the different organizations in integrated care, which we termed inter-organizational alignment (IOA). This study quantitatively explored whether and how IOAs in general and implementation context factors are associated with implementation outcomes in integrated SBMH.

Methods

SBMH clinicians from community-based organizations (CBOs; nclinician=27) and their proximal student-support school staff (nschool=99) rated their schools and CBOs (clinician only) regarding general (organizational culture and molar climate) and implementation context factors (Implementation Climate and Leadership), and nine common implementation outcomes (e.g., treatment integrity, service access, acceptability). The levels of IOA were estimated by intra-class correlations (ICCs). We fitted multilevel models to estimate the standalone effects of context factors from CBOs and schools on implementation outcomes. We also estimated the 2-way interaction effects between CBO and school context factors (i.e., between-setting interdependence) on implementation outcomes.

Results

The IOA in general context factors exceeded those of implementation context factors. The standalone effects of implementation context factors on most implementation outcomes were larger than those of general context factors. Similarly, implementation context factors between CBOs and schools showed larger 2-way interaction effects on implementation outcomes than general context factors.

Conclusions

This study preliminarily supported the importance of IOA in context factors for integrated SBMH. The findings shed light on how IOA in implementation and general context factors may be differentially associated with implementation outcomes across a broad array of integrated mental healthcare settings.

Improving integrated depression and non-communicable disease care in Malawi through engaged leadership and supportive implementation climate

BACKGROUND

Low- and middle-income countries often lack access to mental health services, leading to calls for integration within other primary care systems. In sub-Saharan Africa, integration of depression treatment in non-communicable disease (NCD) settings is feasible, acceptable, and effective. However, leadership and implementation climate challenges often hinder effective integration and quality of services. The aim of this study was to identify discrete leadership strategies that facilitate overcoming barriers to the integration of depression care in NCD clinics in Malawi and to understand how clinic leadership shapes the implementation climate.

METHODS

We conducted 39 in-depth interviews with the District Medical Officer, the NCD coordinator, one NCD provider, and the research assistant from each of the ten Malawian NCD clinics (note one District Medical Officer served two clinics). Based on semi-structured interview guides, participants were asked their perspectives on the impact of leadership and implementation climate on overcoming barriers to integrating depression care into existing NCD services. Thematic analysis used both inductive and deductive approaches to identify emerging themes and compare among participant type.

RESULTS

The results revealed how engaged leadership can fuel a positive implementation climate where clinics had heightened capacity to overcome implementation barriers. Effective leaders were approachable and engaged in daily operations of the clinic and problem-solving. They held direct involvement with and mentorship during the intervention, providing assistance in patient screening and consultation with treatment plans. Different levels of leadership utilized their respective standings and power dynamics to influence provider attitudes and perceptions surrounding the intervention. Leaders acted by informing providers about the intervention source and educating them on the importance of mental healthcare, as it was often undervalued. Lastly, they prioritized teamwork and collective ownership for the intervention, increasing provider responsibility.

CONCLUSION

Training that prioritizes leadership visibility and open communication will facilitate ongoing Malawi Ministry of Health efforts to scale up evidence-based depression treatment within NCD clinics. This proves useful where extensive and external monitoring may be limited. Ultimately, these results can inform successful strategies to close implementation gaps to achieve integration of mental health services in low-resource settings through improved leadership and implementation climate.

TRIAL REGISTRATION

These findings are reported from ClinicalTrials.gov, NCT03711786. Registered on 18/10/2018. https://clinicaltrials.gov/ct2/show/NCT03711786 .

An evaluation of implementation climate in inpatient maternity care: a cross-sectional survey study

OBJECTIVE

Researchers in obstetrics and gynecology are continuously generating new evidence to inform clinical care delivery. Yet, much of this newly emerging evidence fails to be rapidly and effectively integrated into routine clinical practice. Implementation climate refers to clinicians' perceptions of to what degree organizations support and reward use of an evidence-based practice (EBP) and is an important construct in the science of implementation in healthcare. Little is known about implementation climate for EBPs in maternity care. Thus, we aimed to (a) determine the reliability of the Implementation Climate Scale (ICS) for use in inpatient maternity care, (b) describe implementation climate in inpatient maternity care overall, and (c) compare individual perceptions of implementation climate between physician and nursing clinicians on these units.

STUDY DESIGN

We performed a cross-sectional survey of clinicians in inpatient maternity units across 2 urban, academic hospitals in the northeastern United States in 2020. Clinicians completed the 18-question validated ICS [scored 0-4]. Scale reliability by role was evaluated using Cronbach's α. Subscale and total scores were described overall and compared by physician versus nursing role using independent t-tests, as well as linear regression to control for confounders.

RESULTS

111 clinicians completed the survey (physicians = 65; nursing = 46). Physicians were less likely to identify as female (75.4% vs. 100.0%, p < .001), but were of similar age and years of experience as nursing clinicians. Reliability of the ICS was excellent, with Cronbach's α of 0.91 and 0.86 among physicians and nursing clinicians, respectively. Scores were notably low for implementation climate in maternity care overall and for all subscales. ICS total scores were also higher among physicians as compared to nurses (2.18(±0.56) vs. 1.92(±0.50), p = .02), which remained significant in multivariable modeling (p = .02). Unadjusted subscale scores were higher among physicians in Recognition for EBP (2.68(±0.89) vs. 2.30(±0.86), p = .03) and Selection for EBP (2.24(±0.93) vs. 1.62(±1.04), p = .002). After adjustment for potential confounders, subscale scores for Focus on EBP (p = .04) and Selection for EBP (p = .002) were all higher among physicians.

CONCLUSIONS

This study supports the ICS as a reliable scale for measuring implementation climate in the inpatient maternity care setting. Notably lower implementation climate scores across subscales and roles compared to other settings may underlie the vast evidence to practice gap in obstetrics. In order to successfully implement practices that reduce maternal morbidity, we may need to focus on building educational support and rewarding EBP utilization on labor and delivery units, with an emphasis on nursing clinicians.

Champion and audit and feedback strategy fidelity and their relationship to depression intervention fidelity: A mixed method study

Background

Globally, mental health disorders rank as the greatest cause of disability. Low and middle-income countries (LMICs) hold a disproportionate share of the mental health burden, especially as it pertains to depression. Depression is highly prevalent among those with non-communicable diseases (NCDs), creating a barrier to successful treatment. While some treatments have proven efficacy in LMIC settings, wide dissemination is challenged by multiple factors, leading researchers to call for implementation strategies to overcome barriers to care provision. However, implementation strategies are often not well defined or documented, challenging the interpretation of study results and the uptake and replication of strategies in practice settings. Assessing implementation strategy fidelity (ISF), or the extent to which a strategy was implemented as designed, overcomes these challenges. This study assessed fidelity of two implementation strategies (a 'basic' champion strategy and an 'enhanced' champion + audit and feedback strategy) to improve the integration of a depression intervention, measurement based care (MBC), at 10 NCD clinics in Malawi. The primary goal of this study was to assess the relationship between the implementation strategies and MBC fidelity using a mixed methods approach.

Methods

We developed a theory-informed mixed methods fidelity assessment that first combined an implementation strategy specification technique with a fidelity framework. We then created corresponding fidelity indicators to strategy components. Clinical process data and one-on-one in-depth interviews with 45 staff members at 6 clinics were utilized as data sources. Our final analysis used descriptive statistics, reflexive-thematic analysis (RTA), data merging, and triangulation to examine the relationship between ISF and MBC intervention fidelity.

Results

Our mixed methods analysis revealed how ISF may moderate the relationship between the strategies and MBC fidelity. Leadership engagement and implementation climate were critical for clinics to overcome implementation barriers and preserve implementation strategy and MBC fidelity. Descriptive statistics determined champion strategy fidelity to range from 61 to 93% across the 10 clinics. Fidelity to the audit and feedback strategy ranged from 82 to 91% across the 5 clinics assigned to that condition. MBC fidelity ranged from 54 to 95% across all clinics. Although correlations between ISF and MBC fidelity were not statistically significant due to the sample of 10 clinics, associations were in the expected direction and of moderate effect size. A coefficient for shared depression screening among clinicians had greater face validity compared to depression screening coverage and functioned as a proximal indicator of implementation strategy success.

Conclusion

Fidelity to the basic and enhanced strategies varied by site and were influenced by leadership engagement and implementation climate. Champion strategies may benefit from the addition of leadership strategies to help address implementation barriers outside the purview of champions. ISF may moderate the relationship between strategies and implementation outcomes.

Top-level leaders and implementation strategies to support organizational diversity, equity, inclusion, and belonging (DEIB) interventions: a qualitative study of top-level DEIB leaders in healthcare organizations

BACKGROUND

The Black Lives Matter movement and COVID-19 pandemic motivated the wide-scale adoption of diversity, equity, inclusion, and belonging (DEIB) initiatives within healthcare organizations and the creation of DEIB top-level leader positions. The next step is to understand how these leaders contribute to the implementation of DEIB interventions, a task with notable salience due to not only the historical difficulties associated with DEIB strategy execution, but also the substantial evidence that leadership plays a significant role in implementation processes. Therefore, the objective of this qualitative study is to understand the role of top-level DEIB leaders in the implementation of healthcare organizational DEIB interventions.

METHODS

A qualitative research approach which used an in-depth semi-structured interview approach was employed. We conducted thirty-one 60-90-min semi-structured interviews with DEIB top-level leaders between February 2022 and October 2022 over Zoom. An iterative coding process was used to identify the key implementation strategies and activities of DEIB top-level leaders.

RESULTS

Interviewees were mostly Black, majority female, and mostly heterosexual and had a variety of educational backgrounds. We identified the DEIB top-level leader as the DEIB strategy implementation champion. These leaders drive five DEIB implementation strategies: (1) People, (2) Health Equity, (3) Monitoring and Feedback, (4) Operational Planning and Communication, and (5) External Partners. Within these, we identified 19 significant activities that describe the unique implementation strategies supported by the DEIB top-level leaders.

CONCLUSIONS

To move toward sustained commitment to DEIB, the organization must focus on not only establishing DEIB interventions, but on their successful implementation. Our findings help explicate the implementation activities that drive the DEIB initiatives of healthcare organizations and the role of DEIB leaders. Our work can help healthcare organizations systematically identify how to support the success of DEIB organizational interventions.

A Structural Model of Organization and Clinician Factors Associated with Standardized Measure Use in a National Survey of Youth Mental Health Providers

Standardized assessment measures are important for accurate diagnosis of mental health problems and for treatment planning and evaluation. However, little is known about youth mental health providers' typical use of standardized measures across disciplines and outside the context of evidence-based practice initiatives. A multidisciplinary national survey examined the frequency with which 674 youth mental health providers administer standardized and unstandardized measures, and the extent to which organizational (i.e., implementation climate, rigid hierarchical organizational structure) and provider (i.e., attitudes toward standardized assessment measures, highest degree, practice setting) characteristics are associated with standardized measure use. Providers used unstandardized measures far more frequently than standardized measures. Providers' perceptions (a) that standardized measures are practical or feasible, (b) that their organization supports and values evidence-based practices, and (c) that their organization has a rigid hierarchical structure predicted greater use of standardized measures. Working in schools predicted less frequent SMU, while working in higher education and other professional settings predicted more frequent SMU. Standardized measures were not routinely used in this community-based sample. A rigid hierarchical organizational structure may be conducive to more frequent administration of standardized measures, but it is unclear whether such providers actually utilize these measures for clinical decision-making. Alternative strategies to promote standardized measure use may include promoting organizational cultures that value empirical data and encouraging use of standardized measures and training providers to use pragmatic standardized measures for clinical decision making.

Predicting Therapists' Intentions to Use Innovations: Comparing the Role of Individual, Organizational, and Innovation Characteristics

Theories emphasize the role of individual and organizational characteristics in implementation outcomes, yet research indicates that these characteristics account for only a small amount of variance in those outcomes. Innovation characteristics might be important proximal determinants of implementation outcomes but are infrequently examined in mental health services research. This study examined the relative variance explained by individual, organizational, and innovation characteristics on behavioral intentions, a central implementation outcome in implementation theories. Data were collected from 95 therapists and 28 supervisors who participated in a cluster randomized trial that tested the effectiveness of two clinical decision-making innovations. Multilevel models compared individual, organizational, and innovation characteristics as predictors of therapists' intentions to use the innovations. Subsequent mediational path analyses tested whether innovation characteristics mediated the effect of innovation type on intentions. Individual and organizational characteristics explained 29% of the variability in therapists' intentions. Approximately 75% of the variability in therapists' intentions was accounted for by innovation characteristics. Individual and organizational characteristics were not statistically significant predictors of intentions after controlling for innovation characteristics. The indirect effect of innovation type on intentions through therapists' beliefs was statistically significant (B = 0.410, 95% Bootstrapped CI = [0.071, 0.780]), but the direct effect of innovation type was not (B = 0.174, p = .365). Innovation characteristics are related to therapist intentions and might explain why some innovations are received more favorably than others. Future studies should explore the complex interrelationships between these beliefs alongside other individual or organizational characteristics.

Assessment of implementation methods in sepsis: study protocol for a cluster-randomized hybrid type 2 trial

BACKGROUND

Sepsis is the leading cause of intensive care unit (ICU) admission and ICU death. In recognition of the burden of sepsis, the Surviving Sepsis Campaign (SSC) and the Institute for Healthcare Improvement developed sepsis "bundles" (goals to accomplish over a specific time period) to facilitate SSC guideline implementation in clinical practice. Using the SSC 3-h bundle as a base, the Centers for Medicare and Medicaid Services developed a 3-h sepsis bundle that has become the national standard for early management of sepsis. Emerging observational data, from an analysis conducted for the AIMS grant application, suggest there may be additional mortality benefit from even earlier implementation of the 3-h bundle, i.e., the 1-h bundle.

METHOD

The primary aims of this randomized controlled trial are to: (1) examine the effect on clinical outcomes of Emergency Department initiation of the elements of the 3-h bundle within the traditional 3 h versus initiating within 1 h of sepsis recognition and (2) examine the extent to which a rigorous implementation strategy will improve implementation and compliance with both the 1-h bundle and the 3-h bundle. This study will be entirely conducted in the Emergency Department at 18 sites. A secondary aim is to identify clinical sepsis phenotypes and their impact on treatment outcomes.

DISCUSSION

This cluster-randomized trial, employing implementation science methodology, is timely and important to the field. The hybrid effectiveness-implementation design is likely to have an impact on clinical practice in sepsis management by providing a rigorous evaluation of the 1- and 3-h bundles.

FUNDING

NHLBI R01HL162954.

TRIAL REGISTRATION

ClinicalTrials.gov NCT05491941. Registered on August 8, 2022.

Individual-level associations between implementation leadership, climate, and anticipated outcomes: a time-lagged mediation analysis

BACKGROUND

Leaders can improve implementation outcomes by developing an organizational climate conducive to the implementation of evidence-based practices (EBP). This study tested the lagged associations between individual-level perceptions of implementation leadership, implementation climate, and three anticipated implementation outcomes, that is EBP acceptability, appropriateness, and feasibility.

METHODS

Screening tools and treatment methods for posttraumatic stress disorder were implemented in 43 Norwegian mental health services. A sample of 494 child and adult mental health care professionals (M = 43 years, 78% female) completed surveys addressing perceptions of first-level leaders' (n = 47) implementation leadership and their clinics' implementation climate. Single-level structural equation models estimating both direct, indirect, and total effects were used to investigate whether perceived implementation climate mediated the association between perceived implementation leadership and perceived acceptability, appropriateness, and feasibility of screening tools and treatment methods.

RESULTS

Regarding the treatment methods, implementation leadership was associated with therapists' perceptions of acceptability, appropriateness, and feasibility. Implementation climate also mediated between implementation leadership and the outcomes. Regarding the screening tools, implementation leadership was not associated with the outcomes. However, implementation climate mediated between implementation leadership and therapists' perceptions of acceptability and feasibility, but not appropriateness. Analyses with the implementation climate subscales showed stronger associations for therapists' perceptions of the treatment methods than of screening tools.

CONCLUSIONS

Leaders may promote positive implementation outcomes, both directly and through implementation climate. With regard to the effect sizes and explained variance, results indicated that both implementation leadership and implementation climate were more strongly associated with the therapists' perceptions of the treatment methods, implemented by one group of therapists, than the screening tools, implemented by all therapists. This may imply that implementation leadership and climate may have stronger effects for smaller implementation teams within a larger system than for system-wide implementations or when the clinical interventions being implemented are more complex rather than simple ones.

TRIAL REGISTRATION

ClinicalTrials NCT03719651, 25 October 2018.

Dynamic interplay between available resources and implementation climate across phases of implementation: a qualitative study of a VA national population health tool

BACKGROUND

Available resources within an organization can determine the implementation success of an intervention. However, few studies have investigated how the required resources change over the phases of implementation. Using stakeholder interviews, we examined the changes in and interactions between available resources and implementation climate in the implementation and sustainment phases of a national implementation effort for a population health tool.

METHODS

We conducted a secondary analysis of the interviews with 20 anticoagulation professionals at 17 clinical sites in the Veterans Health Administration health system about their experiences with a population health dashboard for anticoagulant management. Interview transcripts were coded using constructs from the Consolidated Framework for Implementation Research (CFIR) and according to the phase of implementation (pre-implementation, implementation, and sustainment) as defined by the VA Quality Enhancement Research Initiative (QUERI) Roadmap. We analyzed the factors that may determine successful implementation by examining the co-occurrence patterns between available resources and implementation climate across different implementation phases. To illustrate the variations in these determinants across phases, we aggregated and scored coded statements using a previously published CFIR scoring system (- 2 to + 2). Key relationships between available resources and implementation climate were identified and summarized using thematic analysis.

RESULTS

The resources necessary to support the successful implementation of an intervention are not static; both the quantity and types of resources shift based on the phases of the intervention. Furthermore, increased resource availability does not guarantee the sustainment of intervention success. Users need different types of support beyond the technical aspects of an intervention, and this support varies over time. Specifically, available resources in the form of technological support and social/emotional support help users establish trust in a new technological-based intervention during the implementation phase. Resources that foster and maintain collaboration between users and other stakeholders help them stay motivated during sustainment.

CONCLUSIONS

Our findings highlight the dynamic nature of available resources and their impacts on the implementation climate across different phases of implementation. A better understanding of the dynamics of available resources over time from the users' perspectives will allow the adaptation of resources to better meet the needs of the intervention stakeholders.

Healthcare provider experiences of deploying a continuous remote patient monitoring pilot program during the COVID-19 pandemic: a structured qualitative analysis

Objective

To describe the healthcare provider (HCP) experience of launching a COVID-19 remote patient monitoring (CRPM) program during the global COVID-19 pandemic.

Methods

We conducted qualitative, semi-structured interviews with eight HCPs involved in deploying the CRPM pilot program in the Military Health System (MHS) from June to December 2020. Interviews were audio recorded, transcribed, and analyzed thematically using an inductive approach. We then deductively mapped themes from interviews to the updated Consolidated Framework for Implementation Research (CFIR).

Results

We identified the following main themes mapped to CFIR domains listed in parentheses: external and internal environments (outer and inner settings), processes around implementation (implementation process domain), the right people (individuals domain), and program characteristics (innovation domain). Participants believed that buy-in from leadership and HCPs was critical for successful program implementation. HCP participants showed qualities of clinical champions and believed in the CRPM program.

Conclusion

The MHS deployed a successful remote patient monitoring pilot program during the global COVID-19 pandemic. HCPs found the CRPM program and the technology enabling the program to be acceptable, feasible, and usable. HCP participants exhibited characteristics of clinical champions. Leadership engagement was the most often-cited key factor for successful program implementation.

Artificial Intelligence in Health Care-Understanding Patient Information Needs and Designing Comprehensible Transparency: Qualitative Study

Background

Artificial intelligence (AI) is as a branch of computer science that uses advanced computational methods such as machine learning (ML), to calculate and/or predict health outcomes and address patient and provider health needs. While these technologies show great promise for improving healthcare, especially in diabetes management, there are usability and safety concerns for both patients and providers about the use of AI/ML in healthcare management.

Objectives

To support and ensure safe use of AI/ML technologies in healthcare, the team worked to better understand: 1) patient information and training needs, 2) the factors that influence patients' perceived value and trust in AI/ML healthcare applications; and 3) on how best to support safe and appropriate use of AI/ML enabled devices and applications among people living with diabetes.

Methods

To understand general patient perspectives and information needs related to the use of AI/ML in healthcare, we conducted a series of focus groups (n=9) and interviews (n=3) with patients (n=40) and interviews with providers (n=6) in Alaska, Idaho, and Virginia. Grounded Theory guided data gathering, synthesis, and analysis. Thematic content and constant comparison analysis were used to identify relevant themes and sub-themes. Inductive approaches were used to link data to key concepts including preferred patient-provider-interactions, patient perceptions of trust, accuracy, value, assurances, and information transparency.

Results

Key summary themes and recommendations focused on: 1) patient preferences for AI/ML enabled device and/or application information; 2) patient and provider AI/ML-related device and/or application training needs; 3) factors contributing to patient and provider trust in AI/ML enabled devices and/or application; and 4) AI/ML-related device and/or application functionality and safety considerations. A number of participant (patients and providers) recommendations to improve device functionality to guide information and labeling mandates (e.g., links to online video resources, and access to 24/7 live in-person or virtual emergency support). Other patient recommendations include: 1) access to practice devices; 2) connection to local supports and reputable community resources; 3) simplified display and alert limits.

Conclusion

Recommendations from both patients and providers could be used by Federal Oversight Agencies to improve utilization of AI/ML monitoring of technology use in diabetes, improving device safety and efficacy.

Facilitation of team-based care to improve HTN management and outcomes: a protocol for a randomized stepped wedge trial

BACKGROUND

There are well-established guidelines for treating hypertension (HTN), yet only half of patients with HTN meet the defined target of < 140/90. Team-based care (TBC) is an evidence-based strategy for improving blood pressure (BP) management and control. TBC is defined as the provision of health services by at least two health professionals "who work collaboratively with patients and their caregivers to accomplish shared goals to achieve coordinated, high-quality care". However, primary care practices experience challenges to implementing TBC principles and care processes; these are more pronounced in small independent practice settings (SIPs). Practice facilitation (PF) is an implementation strategy that may overcome barriers to adopting evidence-based TBC to improve HTN management in SIPs.

METHODS

Using a stepped wedge randomized controlled trial design, we will test the effect of PF on the adoption of TBC to improve HTN management in small practices (< 5 FTE clinicians) in New York City, and the impact on BP control compared with usual care. We will enroll 90 SIPs and randomize them into one of three 12-month intervention waves. Practice facilitators will support SIPs to adopt TBC principles to improve implementation of five HTN management strategies (i.e., panel management, population health, measuring BP, supporting medication adherence, self-management). The primary outcome is the adoption of TBC for HTN management measured at baseline and 12 months. Secondary outcomes include the rate of BP control and sustainability of TBC and BP outcomes at 18 months. Aggregated data on BP measures are collected every 6 months in all clusters so that each cluster provides data points in both the control and intervention conditions. Using a mixed methods approach, we will also explore factors that influence the effectiveness of PF at the organization and team level.

DISCUSSION

This study will provide much-needed guidance on how to optimize adoption and sustainability of TBC in independent primary care settings to reduce the burden of disease related to suboptimal BP control and advance understanding of how facilitation works to improve implementation of evidence-based interventions.

TRIAL REGISTRATION

ClinicalTrials.gov; NCT05413252 .

Misalignment in Community Mental Health Leader and Therapist Ratings of Psychological Safety Climate Predicts Therapist Self-Efficacy with Evidence-Based Practices (EBPs)

Therapist self-efficacy in delivering evidence-based practices (EBPs) is associated with implementation outcomes, including adoption and sustainment in community mental health settings. Inner context organizational climate, including psychological safety, can proximally shape therapist learning experiences within EBP implementation. Psychologically safe environments are conducive to learning behaviors including taking risks, admitting mistakes, and seeking feedback. Organization leaders are instrumental in facilitating psychological safety, but may have differing perspectives of organizational climate than front-line therapists. Discrepant leader and therapist views of psychological safety may have independent associations with therapist EBP learning and implementation outcomes over and above average therapist perceptions of climate. This study examined survey data from 337 therapists and 123 leaders from 49 programs contracted to deliver multiple EBPs within a study examining determinants of sustainment within a large system-driven implementation. Both leaders and therapists completed measures of psychological safety climate and therapists reported on their self-efficacy in delivering multiple EBPs in children's mental health services. Polynomial regression and response surface analysis models were conducted to examine the associations of therapist and leader reports of psychological safety and therapist EBP self-efficacy. Greater discrepancies between leader and therapist reports of psychological safety, in either direction, were associated with lower therapist EBP self-efficacy. Alignment in leader and therapist views of psychological safety climate may impact EBP implementation outcomes. Strategies for improving alignment in perceptions and priorities among organizational members can be included in organizational implementation interventions and may represent unexamined implementation mechanisms of action.

The Interaction Between General and Strategic Leadership and Climate on Their Multilevel Associations with Implementer Attitudes Toward Universal Prevention Programs for Youth Mental Health: A Cross-Sectional Study

Emerging literature has highlighted the importance of discerning general and strategic organizational context (OC) factors (e.g., leadership and climate) and their interaction effect on individual implementation behaviors (e.g., attitudes toward evidence-based practices; EBPs) in youth mental healthcare. This study aimed to examine how leadership and climate (general and strategic) are associated with implementer attitudes toward EBPs across the individual and organizational levels and their interaction effect in schools. A series of multilevel models (MLMs) were fitted on a diverse sample of schools actively implementing universal prevention programs for youth mental health (441 implementers from 52 schools). The organization-level aggregates and individual educators' perceptions of general and strategic leadership and climate, and their interaction terms, were entered as level-2 and level-1 predictors of four attitudinal dimensions (Requirement, Openness, Appeal, and Divergence) based on their level of measurement. At the organizational level, higher levels of strategic leadership and climate, but not their general counterparts, were consistently associated with more favorable attitudes in all four dimensions. At the individual/within-school level, higher levels of perceived general and strategic leadership and climate were associated with more favorable attitudes of Requirement and Openness. At the organizational/between-school level, general climate moderated the positive effect of strategic climate on implementers' perception of appeal and divergence of EBPs. Our findings indicate that leaders should make data-based decisions to allocate resources on strategic and/or general leadership and climate to foster favorable staff attitudes toward EBPs based on the level of measurement, implementation-specificity, and attitudinal dimensions.

Value of implementation science and hybrid implementation research designs for nursing education research: A discussion paper

INTRODUCTION

Nursing education research is complex and contextualized. The evaluation and impact of educational innovations on students, educators, and educational outcomes are influenced by the complexity of environments. Most interventional research in nursing is designed and implemented without considering the behavioral and contextual issues affecting educational innovations, uptake and change processes, and outcomes. Implementation science has emerged as a valuable methodology for designing and conducting interventional research that has the potential to translate evidence and innovations quickly into practice.

PURPOSE

This paper aims to explore the value of implementation science theories, models, and frameworks and hybrid designs for interventional nursing education research and illustrate how these can be used in nursing education research.

METHODS

A brief overview of implementation science, the various types of theories, models and frameworks and Hybrid designs are provided. Illustrative examples demonstrating the incorporation of these methodologies in interventional nursing education research are provided.

RESULTS

A brief overview of implementation and its key concepts namely context, implementation strategies, fidelity, outcomes, adaptation, and sustainability is provided. Three types of hybrid designs are discussed with examples in nursing education research.

DISCUSSION

The implications of implementation science for nursing education research are: a) Accelerating uptake of innovations to improve educational outcomes, b) targeting systematic change in individual and organizational behaviour and c) ensuring the sustainability of teaching and learning innovations.

CONCLUSION

Incorporating implementation science in nursing education research can optimize the uptake of educational innovations in practice in a sustainable manner. Nurse educators should equip themselves with implementation science skills and develop competencies to enhance the delivery of effective and quality nursing education.

Climate for evidence-based mental health care implementation in Germany: psychometric investigation of the Implementation Climate Scale (ICS)

Organizational implementation climate is an important construct in implementation research to describe to what extent implementation is expected, supported, and rewarded. Efforts in bridging the research-practice gap by implementing evidence-based practice (EBP) can benefit from consideration of implementation climate. The Implementation Climate Scale (ICS) is a psychometrically strong measure assessing employees' perceptions of the implementation climate. The present cross-sectional study aimed at providing a German translation and investigating its psychometric properties. The translation followed standard procedures for adapting psychometric instruments. German psychotherapists (N = 425) recruited online completed the ICS, the Evidence Based Practice Attitudes Scale (EBPAS-36D) and the Intention Scale for Providers (ISP). We conducted standard item and reliability analyses. Factorial validity was assessed by comparing an independent cluster model of Confirmatory Factorial Analysis (ICM-CFA), a Bifactor CFA, a Second-order CFA and an (Bifactor) Exploratory Structural Equation Model (ESEM). Measurement invariance was tested using multiple-group CFA and ESEM, convergent validity with correlation analysis between the ICS and the ISP subjective norms subscale (ISP-D-SN). The mean item difficulty was pi = .47, mean inter-item correlation r = .34, and mean item-total correlation ritc = .55. The total scale (ω = 0.91) and the subscales (ω = .79-.92) showed acceptable to high internal consistencies. The model fit indices were comparable and acceptable (Second-order CFA: RMSEA [90% CI] = .077 [.069; .085], SRMR = .078, CFI = .93). Multiple-group CFA and ESEM indicated scalar measurement invariance across gender and presence of a psychotherapy license. Psychotherapists in training reported higher educational support for EBP than licensed psychotherapists (T = 2.09, p = .037, d = 0.25). The expected high correlation between the ICS and the ISP-D-SN was found (r = .59, p < .001). Results for the German ICS confirm good psychometric properties including validity.

Individual and organizational factors as predictors of early evidence-based practice adoption in Michigan high schools: Baseline data from an implementation trial

Background

Adolescents increasingly access mental health services in schools. School mental health professionals (SPs; school counselors, social workers, etc.) can offer evidence-based mental health practices (EBPs) in schools, which may address access gaps and improve clinical outcomes. Although some studies have assessed factors associated with EBP adoption in schools, additional research focusing on SP- and school-level factors is warranted to support EBP implementation as SPs’ mental health delivery grows.

Methods

Baseline data were collected from SPs at Michigan high schools participating in a statewide trial to implement SP-delivered cognitive behavioral therapy (CBT) to students. Models examined factors associated with attitudes about EBPs, implementation climate, and implementation leadership, and their associations with CBT knowledge, training attendance, and pre-training CBT delivery.

Results

One hundred ninety-eight SPs at 107 schools (87%) completed a baseline survey. The mean Evidence-Based Practice Attitude Scale (EBPAS) total score was 2.9, and school-aggregated mean scores of the Implementation Climate Scale (ICS) and Implementation Leadership Scale (ILS) were 1.83 and 1.77, respectively, all on a scale ranging from 0 ( low) to 4 ( high). ICS and ILS scores were lower than typically reported in clinical settings, while EBPAS scores were higher. School characteristics were not significantly associated with EBPAS, ICS, or ILS scores, but scores did differ by SP role. Higher EBPAS scores were associated with more CBT knowledge (average marginal effect for 1 SD change [AME] = 0.15 points) and a higher probability of training completion (AME = 8 percentage points). Higher ICS scores were associated with a higher probability of pre-training CBT delivery (AME = 6 percentage points), and higher ILS scores were associated with higher probability of training completion (AME = 10 percentage points).

Conclusions

Our findings suggest that SPs’ attitudes toward EBPs and organizational support were positively associated with early signs of implementation success. As schools increasingly fill the adolescent mental healthcare access gap, efforts to strengthen both provider attitudes toward EBP and strategic organizational factors supporting EBP delivery will be key to encouraging EBP uptake in schools.

Plain Language Summary

Schools are an important setting in which adolescents receive mental healthcare. We need to better understand how to implement evidence-based practices (EBPs) in this setting to improve student mental health. This study examined the attitudes and perceptions of school professionals (SPs) as key contributors to the implementation of a particular EBP, the delivery of cognitive behavioral therapy (CBT) in schools. The study found that implementation climate and leadership scores in participating schools were lower than scores typically reported in clinical settings, while scores for SP attitudes about EBP adoption were higher than typical scores in clinical settings. Results further suggest that SPs with more positive attitudes toward EBPs are more knowledgeable of CBT and more likely to complete a 1-day CBT training. We also found that higher implementation climate scores were associated with SPs reporting pre-training CBT delivery (although this association was not statistically significant), and more implementation leadership was associated with SPs completing the CBT training. These findings suggest that SP attitudes toward EBPs and organizational support in schools are positively associated with early signs of implementation success. Early, low-intensity efforts to (1) improve SP attitudes about mental health EBPs, and (2) increase schools’ support for implementation may scaffold more intensive implementation efforts in schools down the road.

Implementation context and burnout among Department of Veterans Affairs psychotherapists prior to and during the COVID-19 pandemic

BACKGROUND

The first goal of this study was to assess longitudinal changes in burnout among psychotherapists prior to (T1) and during the COVID-19 pandemic (T2). The second objective was to assess the effects of job demands, job resources (including organizational support for evidence-based psychotherapies, or EBPs) and pandemic-related stress (T2 only) on burnout.

METHOD

Psychotherapists providing EBPs for posttraumatic stress disorder in U.S. Department of Veterans Affairs (VA) facilities completed surveys assessing burnout, job resources, and job demands prior to (T1; n = 346) and during (T2; n = 193) the COVID-19 pandemic.

RESULTS

Burnout prevalence increased from 40 % at T1 to 56 % at T2 (p < .001). At T1, stronger implementation climate and implementation leadership (p < .001) and provision of only cognitive processing therapy (rather than use of prolonged exposure therapy or both treatments; p < .05) reduced burnout risk. Risk factors for burnout at T2 included T1 burnout, pandemic-related stress, less control over when and how to deliver EBPs, being female, and being a psychologist rather than social worker (p < .02). Implementation leadership did not reduce risk of burnout at T2.

LIMITATIONS

This study involved staff not directly involved in treating COVID-19, in a healthcare system poised to transition to telehealth delivery.

CONCLUSION

Organizational support for using EBPs reduced burnout risk prior to but not during the pandemic. Pandemic related stress rather than increased work demands contributed to elevated burnout during the pandemic. A comprehensive approach to reducing burnout must address the effects of both work demands and personal stressors.

Implementation readiness for evidence-based autism practices in school systems

Background

The increase in the number of autistic children being identified has led to increased demand on public schools to provide high-quality services. Effectively scaling up evidence-based practice (EBP) use for autistic students is challenging, given the complicated organization of special education. Teachers have significant challenges implementing autism EBP with fidelity. Factors such as implementation leadership and climate and attitudes toward EBP are linked to successful EBP use and may vary at different levels of the education system. Examining mechanisms of successful implementation is a critical step to support scale-up.

Method

In this observational study, conducted from September 2018 to March 2020, California school personnel (n = 2273) at multiple levels of the system completed surveys related to implementation climate, leadership, and attitudes toward EBP. Data were collected throughout California at the Special Education Local Plan Areas, County Office of Education, and district and school levels from educators and administrators working in public schools supporting autistic students. Multi-level modeling was conducted to characterize implementation readiness.

Results

Overall, implementation climate and leadership scores are low across levels with regional levels rated more positively than districts or schools. Attitudes toward EBP were moderate, with those working in schools having the poorest ratings and specialists/trainers and related service providers (e.g., speech-language pathologists) having the highest ratings.

Conclusions

Outcomes provide a unique opportunity to compare implementation factors across organizational levels with a large, statewide sample. These data provide guidance for developing implementation interventions at multiple levels of the education system to increase readiness for effective scale-up of autism EBP in schools. Personnel and leaders at different organizational levels may need differentiated training targeting improved implementation climate and leadership. Personnel within districts and schools may experience a particular benefit from leadership support for EBP implementation.

Use of the consolidated framework for implementation research in a mixed methods evaluation of the EQUIPPED medication safety program in four academic health system emergency departments

Background

Enhancing Quality of Prescribing Practices for Older Adults Discharged from the Emergency Department (EQUIPPED) is an effective quality improvement program initially designed in the Veterans Administration (VA) health care system to reduce potentially inappropriate medication prescribing for adults aged 65 years and older. This study examined factors that influence implementation of EQUIPPED in EDs from four distinct, non-VA academic health systems using a convergent mixed methods design that operationalized the Consolidated Framework for Implementation Research (CFIR). Fidelity of delivery served as the primary implementation outcome.

Materials and methods

Four EDs implemented EQUIPPED sequentially from 2017 to 2021. Using program records, we scored each ED on a 12-point fidelity index calculated by adding the scores (1-3) for each of four components of the EQUIPPED program: provider receipt of didactic education, one-on-one academic detailing, monthly provider feedback reports, and use of order sets. We comparatively analyzed qualitative data from focus groups with each of the four implementation teams (n = 22) and data from CFIR-based surveys of ED providers (108/234, response rate of 46.2%) to identify CFIR constructs that distinguished EDs with higher vs. lower levels of implementation.

Results

Overall, three sites demonstrated higher levels of implementation (scoring 8-9 of 12) and one ED exhibited a lower level (scoring 5 of 12). Two constructs distinguished between levels of implementation as measured through both quantitative and qualitative approaches: patient needs and resources, and organizational culture. Implementation climate distinguished level of implementation in the qualitative analysis only. Networks and communication, and leadership engagement distinguished level of implementation in the quantitative analysis only.

Discussion

Using CFIR, we demonstrate how a range of factors influence a critical implementation outcome and build an evidence-based approach on how to prime an organizational setting, such as an academic health system ED, for successful implementation.

Conclusion

This study provides insights into implementation of evidence-informed programs targeting medication safety in ED settings and serves as a potential model for how to integrate theory-based qualitative and quantitative methods in implementation studies.

Implementing community pharmacy-based influenza point-of-care test-and-treat under collaborative practice agreement

Background

Early and accessible testing for influenza with point-of-care testing (POCT) can be a critical factor for deciding to begin antiviral treatment. More than 10,000 pharmacies across the USA offer Clinical Laboratory Improvement Amendments-waived POCT for infectious diseases, such as influenza A/B. Knowledge of barriers and facilitators to large-scale POCT implementation may be useful in scaling POCT for influenza test-and-treat services (Flu POCT). The objective of this study was to explore the experiences of pharmacists who were early adopters of Flu POCT and treatment under collaborative practice agreement in community pharmacy settings.

Methods

Qualitative research design with in-depth, semi-structured virtual video interviews of licensed US community pharmacists. Interview questions were derived from the Consolidated Framework for Implementation Research (CFIR). Interviewees were selected via a purposeful sampling of pharmacists who were enrolled in a nationwide clinical trial involving pharmacy-based influenza test-and-treat under a collaborative agreement. Interviews were recorded and transcribed. A deductive analytic approach was used via constructs from the CFIR.

Results

Six pharmacists were interviewed. Interviews ranged from 28 to 70 min, with an average length of 46 min. Four broad themes emerged from the data, and each had corresponding subthemes and supporting quotes: influence of the Flu POCT service characteristics on pharmacy implementation, influence of factors outside of the pharmacy setting in Flu POCT implementation, factors within the pharmacy setting influencing implementation, and process of implementing Flu POCT. A novel pharmacy-based Flu POCT implementation framework is presented.

Conclusions

Implementation of community pharmacy-based Flu POCT services is feasible; but, a thorough understanding of both barriers and facilitators to their implementation is needed to increase the spread and scale of these programs. Specifically, pharmacy stakeholders should focus efforts on increasing patient and provider awareness, pharmacist acceptance, leadership support, and support of health providers external to the pharmacy to improve implementation success.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00324-z.

Implementation and outcomes of a system-wide women’s health ‘team goal’ to reduce maternal morbidity for black women: a prospective quality improvement study

Objective

In response to the unacceptable racial disparities in US obstetric outcomes, our health system established a formal goal to reduce maternal morbidity for black women. Here, we describe our process for meeting this equity-focused goal in the context of diverse implementation climates at 5 inpatient sites.

Study design

To meet the system goal, we established a collaborative of multidisciplinary, site-based teams. The validated 18-question Implementation Climate Scale (ICS) was distributed to site clinicians at baseline. Sites focused on haemorrhage, performing case reviews of black women meeting morbidity criteria. Comparing cases by site, site-specific areas for improvement in haemorrhage risk assessment, prevention and management emerged. Evidence-based practices (EBPs) were then selected, tailored and implemented by site. Monthly system-wide team meetings included (1) metric tracking and (2) site presentations with discussions around barriers/facilitators to EBP implementation. Maternal morbidity rates among black women were compared the year before goal development (1 July 2019–30 June 2020) to the year after (1 July 2020–30 June 2021).

Results

Mean ICS scores for inpatient obstetric units differed by site (p=0.005), with climates more supportive of implementation at urban/academic hospitals. In response to case reviews, sites reported implementing 2 to 8 EBPs to meet the goal. Despite different ICS scores, this process was associated with significant reductions in maternal morbidity for black women from pregoal to postgoal development overall and at sites 1, 2 and 3, with non-statistically significant reductions at sites 4 and 5 (overall: −29.4% reduction, p<0.001).

Conclusions

A health system goal of reducing maternal morbidity for black women led to a data-driven, collaborative model for implementing site-tailored interventions. If health systems prioritise equity-focused goals, sites can be supported in implementing EBPs that improve care.

Lessons learnt from the implementation of new models of care delivery through alliance governance in the Southern health region of New Zealand: a qualitative study

OBJECTIVES

To explore the process of implementation of the primary and community care strategy (new models of care delivery) through alliance governance in the Southern health region of New Zealand (NZ).

DESIGN

Qualitative semistructured interviews were undertaken. A framework-guided rapid analysis was conducted, informed by implementation science theory-the Consolidated Framework for Implementation Research.

SETTING

Southern health region of NZ (Otago and Southland).

PARTICIPANTS

Eleven key informants (Alliance Leadership Team members and senior health professionals) who were involved in the development and/or implementation of the strategy.

RESULTS

The large number of strategy action plans and interdependencies of activities made implementation of the strategy complex. In the inner setting, communication and relationships between individuals and organisations were identified as an important factor for joint and integrated working. Key elements of a positive implementation climate were not adequately addressed to better align the interests of health providers, and there were multiple competing priorities for the project leaders. A perceived low level of commitment from the leadership of both organisations to joint working and resourcing indicated poor organisational readiness. Gaps in the implementation process included no detailed implementation plan (reflected in poor execution), ambitious targets, the lack of a clear performance measurement framework and an inadequate feedback mechanism.

CONCLUSIONS

This study identified factors for the successful implementation of the PCSS using an alliancing approach in Southern NZ. A key enabler is the presence of a stable and committed senior leadership team working through high trust relationships and open communication across all partner organisations. With alliances, partnerships and networks increasingly held up as models for integration, this evaluation identifies important lessons for policy makers, managers and services providers both in NZ and internationally.

Quantitative Measurements for Factors Influencing Implementation in School Settings: Protocol for A Systematic Review and A Psychometric and Pragmatic Analysis

INTRODUCTION

In order to address the effectiveness and sustainability of school-based interventions, there is a need to consider the factors affecting implementation success. The rapidly growing field of implementation-focused research is struggling to determine how to assess and measure implementation-relevant constructs. Earlier research has identified the need for strong psychometric and pragmatic measures. The aims of this review are therefore to (i) systematically review the literature to identify measurements of the factors influencing implementations which have been developed or adapted in school settings, (ii) describe each measurement's psychometric and pragmatic properties, (iii) describe the alignment between each measurement and the corresponding domain and/or construct of the Consolidated Framework for Implementation Research (CFIR).

METHODS

Six databases (Medline, ERIC, PsycInfo, Cinahl, Embase, and Web of Science) will be searched for peer-reviewed articles reporting on school settings, published from the year 2000. The identified measurements will be mapped against the CFIR, and analyzed for their psychometric and pragmatic properties.

DISCUSSION

By identifying measurements that are psychometrically and pragmatically impactful in the field, this review will contribute to the identification of feasible, effective, and sustainable implementation strategies for future research in school settings.

Barriers and Facilitators to Data Use for Decision Making: The Experience of the African Health Initiative Partnerships in Ethiopia, Ghana, and Mozambique

BACKGROUND

Three African Health Initiative (AHI) partnership projects in Ethiopia, Ghana, and Mozambique implemented strategies to improve the quality and evaluation of routinely collected data at the primary health care level and stimulate its use in evidence-based decision making. We compare how these programs designed and carried out data for decision-making (DDM) strategies, elaborate on barriers and facilitators to implementation success, and offer recommendations for future DDM programming.

METHODS

Researchers from each project collaboratively wrote a cross-country protocol based on these objectives. By adapting the Consolidated Framework for Implementation Research (CFIR) through a qualitative theme reduction process, they harmonized lines of inquiry on the design of the respective DDM strategies and the barriers and facilitators of effective implementation. We conducted in-depth interviews and focus group discussions with stakeholders from the primary health care systems in each country, and we carried out multistage, thematic analyses using a deductive lens.

RESULTS

Effective implementation of DDM depended on whether implementers felt that DDM was adaptable to context, feasible to trial, and easy to introduce and maintain. The prevailing policy and political environment in the wider health system, learning climate and absorptive capacity for evidence-based change in DDM settings, engagement of external change agents and internal change leaders, and promotion of opportunities and means for team-based reflection and evaluations of what works influenced the success or failure of DDM strategies.

CONCLUSION

Opportunities for team-based capacity building and individual mentorship led to effective DDM programming. External policies and associated incentives bolstered this but occasionally led to unintended consequences. Leadership engagement and availability of resources to act on recommendations; respond to capacity-building needs; and facilitate collaborations between peers, within hierarchies, and across the local health system proved crucial to DDM, as was encouraging adaptation and opportunities for iterative on-the-job learning.

A Qualitative Force Field Analysis of Facilitators and Barriers to Evidence-Based Practice in Healthcare Using an Implementation Framework

Research has identified facilitators and barriers to implementation of evidence-based practices (EBPs). Few studies have evaluated which factors persist among healthcare clinicians with extensive education and training on EBP implementation. Therefore, the purpose of this study was to examine facilitators and barriers to EBP implementation across a national sample of specialty-prepared EBP mentors in healthcare settings. Healthcare clinicians participating in an immersive 5-day EBP knowledge and skill building program were invited to complete a follow-up survey 12 months later to report on implementation experiences. The Consolidated Framework for Implementation Research (CFIR) guided content analysis of responses. A force field analysis using Lewin’s change theory was used to assign numerical ‘weights’ to factors. Eighty-four individuals reported facilitators and barriers to implementation. The majority occurred within the inner setting of the CFIR model. Facilitators were strong leadership engagement (n = 15), positive EBP culture (n = 9), and resources (n = 4). Barriers included lack of resources (n = 21), poor leadership engagement (n = 19), implementation climate (n = 17), lack of relative priority (n = 12), and organizational characteristics (n = 9). Respondents also identified simultaneous facilitators and barriers within the process domain of the CFIR model. The construct of stakeholder engagement was a barrier when absent from the implementation process (n = 23), yet was a strong facilitator when present (n = 23). Implementation in healthcare settings appears most effective when conducted by an interprofessional team with strong leadership, resources, stakeholder engagement, and positive EBP culture. When these same factors are absent, they remain persistent barriers to implementation, even among specialty-trained healthcare clinicians.

Teacher Perceptions of Implementation Climate Related to Feasibility of Implementing Schoolwide Positive Behavior Supports and Interventions

For Schoolwide Positive Behavior Interventions and Supports (SWPBIS) to be effective, educators must use the practices as intended. Whether a teacher uses a practice as intended can depend, in part, on how feasible the practice is perceived to be. Identifying malleable factors associated with teachers’ perceptions of SWPBIS feasibility can help schools identify targeted supports to improve feasibility. Implementation climate, or the shared perception among implementers that their school supports implementation efforts, is known to promote high quality implementation. However, little is known about how individuals’ perceptions of their school’s implementation climate may influence their perceptions of feasibility. The lack of empirical evidence points to a need to explore whether educators’ shared and individual perceptions of implementation climate influence feasibility of implementing SWPBIS. Therefore, this study examines the link between teachers’ individual and shared perceptions of implementation climate related to the feasibility of implementing SWPBIS in a sample of 348 K-5 general education teachers across 39 elementary schools in the pacific northwestern United States. Results indicate that teachers who experience their schools’ implementation climate as positive are more likely to report SWPBIS as feasible, controlling for teachers’ shared perceptions of implementation climate. Implications for schools aiming to improve their implementation of SWPBIS, including the development of individualized implementation supports (e.g., tailoring implementation strategies to support each and every teacher), are discussed.

The implementation and validation of the NoMAD during a complex primary care intervention

Background

Normalization process theory (NPT) has been widely used to better understand how new interventions are implemented and embedded. The NoMAD (Normalization Measurement Development questionnaire) is a 23-item NPT instrument based on NPT. As the NoMAD is a relatively new instrument, the objectives of this paper are: to describe the experience of implementing the NoMAD, to describe it being used as a feedback mechanism to gain insight into the normalization process of a complex health intervention, and to further explore the psychometric properties of the instrument.

Methods

Health TAPESTRY was implemented in six Family Health Teams (total of seven sites) across Ontario. Healthcare team members at each site were invited to complete the NoMAD, and three general questions about normalization, six times over a 12-month period. Each site was then provided a visual traffic light summary (TLS) reflecting the implementation of the Health TAPESTRY. The internal consistency of each sub-scale and validity of the NoMAD were assessed. Learnings from the implementation of the NoMAD and subsequent feedback mechanism (TLS) are reported descriptively.

Results

In total, 56 diverse health care team members from six implementation sites completed the NoMAD. Each used it at least once during the 12-month study period. The implementation of the NoMAD and TLS was time consuming to do with multiple collection (and feedback) points. Most (60%) internal consistency values of the four subscales (pooled across site) across each collection point were satisfactory. All correlations were positive, and most (86%) were statistically significant among NoMAD subscales. All but one correlation between the NoMAD subscales and the general questions were positive, and most (72%) were significant. Generally, scores on the subscales were higher at 12-month than baseline, albeit did not follow a linear pattern of change across implementation. Generally, scores were higher for experienced sites compared to first-time implementors.

Conclusion

Our experience would suggest fewer collection points; three timepoints spaced out by several months are adequate, if repeated administration of the NoMAD is used for feedback loops. We provide additional evidence of the psychometric properties of the NoMAD.

Trial Registration

Registered at ClinicalTrials.gov: NCT03397836.

Linking organizational climate for evidence-based practice implementation to observed clinician behavior in patient encounters: a lagged analysis

Background

Theory and empirical research suggest organizational climate for evidence-based practice (EBP) implementation may be an important and malleable target to improve clinician use of EBPs in healthcare; however, this work has been criticized for overreliance on self-report measures of implementation outcomes and cross-sectional designs. This study combines data from two studies spanning 7 years to test the hypothesis that higher levels of organizational EBP implementation climate prospectively predicts improved clinician adherence to an EBP, cognitive behavioral therapy (CBT), as rated by expert observers.

Methods

Biennial assessments of EBP implementation climate collected in 10 community mental health agencies in Philadelphia as part of a systemwide evaluation (time 1) were linked to subsequent observer ratings of clinician adherence to CBT in clinical encounters with 108 youth (time 2). Experts rated clinician adherence to CBT using the Therapy Process Observation Coding System which generated two primary outcomes (a) maximum CBT adherence per session (i.e., highest rated CBT intervention per session; depth of delivery) and (b) average CBT adherence per session (i.e., mean rating across all CBT interventions used; depth and breadth of delivery).

Results

On average, time 2 clinician adherence observations occurred 19.8 months (SD = 10.15) after time 1 organizational climate assessments. Adjusting for organization, clinician, and client covariates, a one standard deviation increase in organizational EBP implementation climate at time 1 predicted a 0.63-point increase in clinicians’ maximum CBT adherence per session at time 2 (p = 0.000), representing a large effect size (d = 0.93; 95% CI = 0.63–1.24) when comparing organizations in the upper (k = 3) versus lower tertiles (k = 3) of EBP implementation climate. Higher levels of time 1 organizational EBP implementation climate also predicted higher time 2 average CBT adherence per session (b = 0.23, p < 0.001, d = 0.72). Length of time between assessments of climate and adherence did not moderate these relationships.

Conclusions

Organizational EBP implementation climate is a promising predictor of clinicians’ subsequent observed adherence to CBT. Implementation strategies that target this antecedent may improve the delivery of EBPs in healthcare settings.

Results of the Kigali Imbereheza Project: A 2-Arm Individually Randomized Trial of TI-CBT Enhanced to Address ART Adherence and Mental Health for Rwandan Youth Living With HIV

BACKGROUND

Adolescents living with HIV have elevated mental distress and suboptimal antiretroviral therapy (ART) adherence.

SETTING

Two urban clinics in Kigali, Rwanda.

METHODS

A 2-arm individual randomized controlled trial compared Trauma-Informed Cognitive Behavioral Therapy enhanced to address HIV (TI-CBTe) with usual care (time-matched, long-standing, unstructured support groups) with 356 12- to 21-year-old (M = 16.78) Rwandans living with HIV. TI-CBTe included 6 group-based 2-hour sessions led by trained and supervised 21- to 25-year-old Rwandans living with HIV. Participants reported their ART adherence, depression/anxiety, and Post-Traumatic Stress Disorder symptoms at baseline, 6, 12, and 18 months.

RESULTS

ART adherence was relatively high at baseline, and youth reported elevated rates of depression/anxiety and trauma symptoms. There were no differential treatment effects on adherence, but depression/anxiety improved over time. Youth with lower depression/anxiety at baseline seemed to benefit more from TI-CBTe than usual care, whereas women with high baseline distress seemed to benefit more from usual care. Youth were less likely to score in high Post-Traumatic Stress Disorder symptom categories at the follow-up, with no differential treatment effects.

CONCLUSIONS

TI-CBTe did not outperform usual care on ART adherence, possibly reflecting relatively high adherence at baseline, simplified medication regimens over time, a strong comparison condition, or because youth assigned to TI-CBTe returned to their support groups after the intervention. TI-CBTe was more effective for youth with lower depression/anxiety symptoms, whereas youth with high distress benefitted more from the support groups. TI-CBTe was feasible and acceptable, and young adults living with HIV were able to deliver a mental health intervention with fidelity. The powerful nature of the comparison group, ongoing support groups, points to the potential value of locally crafted interventions in low-resource settings.

Factors influencing the integration of evidence-based task-strengthening strategies for hypertension control within HIV clinics in Nigeria

Background

Evidence-based task-strengthening strategies for hypertension (HTN) control (TASSH) are not readily available for patients living with HIV in sub-Saharan Africa where the dual burden of HTN and HIV remains high. We are conducting a cluster randomized controlled trial comparing the effectiveness of practice facilitation versus a self-directed control (i.e., receipt of TASSH with no practice facilitation) in reducing blood pressure and increasing the adoption of task-strengthening strategies for HTN control within HIV clinics in Nigeria. Prior to implementing the trial, we conducted formative research to identify factors that may influence the integration of TASSH within HIV clinics in Nigeria.

Methods

This mixed-methods study was conducted with purposively selected healthcare providers at 29 HIV clinics, followed by a 1-day stakeholder meeting with 19 representatives of HIV clinics. We collected quantitative practice assessment data using two instruments: (a) an adapted Service Availability and Readiness Assessment (SARA) tool to assess the capacity of the clinic to manage NCDs and (b) Implementation Climate Scale to assess the degree to which there is a strategic organizational climate supportive of the evidence-based practice implementation. The quantitative data were analyzed using descriptive statistics and measures of scale reliability. We also used the Consolidated Framework for Implementation Research (CFIR), to thematically analyze qualitative data generated and relevant to the aims of this study.

Results

Across the 29 clinics surveyed, the focus on TASSH (mean=1.77 (SD=0.59)) and educational support (mean=1.32 (SD=0.68)) subscales demonstrated the highest mean score, with good–excellent internal consistency reliability (Cronbach’s alphas ranging from 0.84 to 0.96). Within the five CFIR domains explored, the major facilitators of the intervention included relative advantage of TASSH compared to current practice, compatibility with clinic organizational structures, support of patients’ needs, and intervention alignment with national guidelines. Barriers included the perceived complexity of TASSH, weak referral network and patient tracking mechanism within the clinics, and limited resources and diagnostic equipment for HTN.

Conclusion

Optimizing healthcare workers’ implementation of evidence-based TASSH within HIV clinics requires attention to both the implementation climate and contextual factors likely to influence adoption and long-term sustainability. These findings have implications for the development of effective practice facilitation strategies to further improve the delivery and integration of TASSH within HIV clinics in Nigeria.

Trial registration

ClinicalTrials.gov, NCT04704336

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00289-z.

A stepped-wedge randomized trial investigating the effect of the Leadership and Organizational Change for Implementation (LOCI) intervention on implementation and transformational leadership, and implementation climate

BACKGROUND

This study evaluates the Leadership and Organizational Change for Implementation (LOCI) strategy and its effect on implementation leadership, transformational leadership, and implementation climate.

METHODS

A stepped wedge cluster randomized study design enrolling 47 first-level leaders from child- and adult-specialized mental health clinics within Norwegian health trusts across three cohorts. All therapists (n = 790) received training in screening of trauma exposure and posttraumatic stress, and a subgroup of therapists (n = 248) received training in evidence-based treatment methods for posttraumatic stress disorder (PTSD). First-level leaders and therapists completed surveys at baseline, 4, 8-, 12-, 16-, and 20-months assessing leadership and implementation climate. General linear mixed-effects models were used to investigate whether the LOCI strategy would lead to greater therapist-rated scores on implementation leadership, transformational leadership, and implementation climate.

RESULTS

After introducing the LOCI strategy, there was a significant increase in therapist-rated implementation and transformational leadership and implementation climate. The increase was sustained at all measurement time points compared to non-LOCI conditions, which demonstrated a steady decrease in scores before LOCI.

CONCLUSIONS

The LOCI strategy can develop better transformational and implementation leadership skills and contribute to a more positive implementation climate, which may enhance successful EBP implementation. Thus, LOCI can help leaders create an organizational context conducive for effective EBP implementation.

TRIAL REGISTRATION

Retrospectively registered: ClinicalTrials NCT03719651 , 25th of October 2018. The trial protocol can be accessed from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6417075/ .

Scaling Evidence-Based Interventions to Improve the Cardiovascular Health of People With Serious Mental Illness

People with serious mental illnesses (SMIs) experience excess mortality, driven in large part by high rates of cardiovascular disease (CVD), with all cardiovascular disease risk factors elevated. Interventions designed to improve the cardiovascular health of people with SMI have been shown to lead to clinically significant improvements in clinical trials; however, the uptake of these interventions into real-life clinical settings remains limited. Implementation strategies, which constitute the "how to" component of changing healthcare practice, are critical to supporting the scale-up of evidence-based interventions that can improve the cardiovascular health of people with SMI. And yet, implementation strategies are often poorly described and rarely justified theoretically in the literature, limiting the ability of researchers and practitioners to tease apart why, what, how, and when implementation strategies lead to improvement. In this Perspective, we describe the implementation strategies that the Johns Hopkins ALACRITY Center for Health and Longevity in Mental Illness is using to scale-up three evidenced-based interventions related to: (1) weight loss; (2) tobacco smoking cessation treatment; and (3) hypertension, dyslipidemia, and diabetes care for people with SMI. Building on concepts from the literature on complex health interventions, we focus on considerations related to the core function of an intervention (i.e., or basic purposes of the change process that the health intervention seeks to facilitate) vs. the form (i.e., implementation strategies or specific activities taken to carry out core functions that are customized to local contexts). By clearly delineating how implementation strategies are operationalized to support the interventions' core functions across these three studies, we aim to build and improve the future evidence base of how to adapt, implement, and evaluate interventions to improve the cardiovascular health of people with SMI.

Implementation of the Wolverine Mental Health Program, Part 3: Sustainment Phase

Sustaining the implementation of an evidence-based practice (EBP) is the ultimate goal of often years of significant personnel and financial investment. Some conceptualize sustainment as a distinct phase following an active implementation period where the contextual factors, processes, and supports are bolstered to ensure continued EBP delivery. This study provides an overview of the sustainment strategies deployed to embed cognitive-behavioral therapy (CBT) in a Midwestern residential treatment facility serving youth with complex mental health needs. Seven key strategies and their outcomes are described: use of CBT teams, new hire orientation plans, monthly campaigns, change in job descriptions and performance evaluations, development of a behavioral reinforcement system for youth, and a pathway to CBT certification. This study provides a window into how one might sustain an EBP by addressing barriers unique to this phase of work.

Conceptualizing outcomes for use with the Consolidated Framework for Implementation Research (CFIR): the CFIR Outcomes Addendum

BACKGROUND

The challenges of implementing evidence-based innovations (EBIs) are widely recognized among practitioners and researchers. Context, broadly defined as everything outside the EBI, includes the dynamic and diverse array of forces working for or against implementation efforts. The Consolidated Framework for Implementation Research (CFIR) is one of the most widely used frameworks to guide assessment of contextual determinants of implementation. The original 2009 article invited critique in recognition for the need for the framework to evolve. As implementation science has matured, gaps in the CFIR have been identified and updates are needed. Our team is developing the CFIR 2.0 based on a literature review and follow-up survey with authors. We propose an Outcomes Addendum to the CFIR to address recommendations from these sources to include outcomes in the framework.

MAIN TEXT

We conducted a literature review and surveyed corresponding authors of included articles to identify recommendations for the CFIR. There were recommendations to add both implementation and innovation outcomes from these sources. Based on these recommendations, we make conceptual distinctions between (1) anticipated implementation outcomes and actual implementation outcomes, (2) implementation outcomes and innovation outcomes, and (3) CFIR-based implementation determinants and innovation determinants.

CONCLUSION

An Outcomes Addendum to the CFIR is proposed. Our goal is to offer clear conceptual distinctions between types of outcomes for use with the CFIR, and perhaps other determinant implementation frameworks as well. These distinctions can help bring clarity as researchers consider which outcomes are most appropriate to evaluate in their research. We hope that sharing this in advance will generate feedback and debate about the merits of our proposed addendum.

Measuring implementation climate: psychometric properties of the Implementation Climate Scale (ICS) in Norwegian mental health care services

BackgroundEmployees' perceptions of organizational climate for implementation of new methods are important in assessing and planning for implementation efforts. More specifically, feedback from employees' points to which implementation strategies to select, adopt, and tailor in building positive climate for implementation of new evidence-based practices within the organization. Implementation climate can be measured with the Implementation Climate Scale (ICS). The purpose of this study was to investigate the psychometric properties of the Norwegian version of the ICS in outpatient mental health clinics.MethodsThe ICS was administered to 383 clinicians within 47 different child and adult mental health clinics across the country. We conducted confirmatory factor analysis to assess the psychometric functioning of the ICS. Cronbach's alpha was examined to assess internal consistency. We also examined criterion related validity of the scale by comparing it with an alternative measure of implementation climate (concurrent validity) and by examining correlations with clinicians' intentions to use evidence-based practices.ResultsResults supported the 6-factor structure and the internal consistency reliability of the ICS. One exception was poor functioning of the Reward scale. Concurrent validity was stronger at the group than at the individual level, and assessment of associations with clinicians' intentions to use evidence- based practices showed positive correlations.ConclusionsThe Norwegian version of the ICS is a promising tool for assessing implementation climate which can provide organizations with specific feedback concerning which aspects of the implementation climate to attend to. Due to poor functioning of the Reward scale, adaptations and further testing of this is recommended.

"Change Doesn't Happen by Itself": A Thematic Analysis of First-Level Leaders' Experiences Participating in the Leadership and Organizational Change for Implementation (LOCI) Strategy

The Leadership and Organizational Change for Implementation (LOCI) strategy is a multifaceted implementation strategy that aims to support successful evidence-based practice (EBP) implementation by fostering effective general leadership, implementation leadership, and implementation climate. How implementation strategies are experienced by participants is important for their utilization and effectiveness in supporting EBP implementation. The current study is the first in-depth qualitative study exploring first-level leaders' experiences of participating in the LOCI strategy. Data were collected as part of a trial where Norwegian child and adult mental health outpatient clinics implemented EBPs for posttraumatic stress disorder (PTSD). Eleven first-level leaders from adult and child clinics participated in semi-structured interviews after completing the LOCI strategy. Data were analyzed through reflexive thematic analysis. The analysis generated four themes related to leaders' experiences of participating in the LOCI strategy: (1) structuring the EBP implementation, (2) taking responsibility for the EBP implementation, (3) interacting with others about the EBP implementation, and (4) becoming aware of EBP implementation and their own leadership. Most participants experienced the LOCI strategy as beneficial for implementing EBPs for PTSD in their clinic. The strategy succeeded in raising awareness of leadership for EBP implementation, and simultaneously provided participants with tools and support for leading the implementation in their clinic. Two participants experienced LOCI as less beneficial than the others. Our results support the strategy's potential to engage and empower first-level leaders to get involved in implementation processes and point to important challenges for future research on implementation strategies.

Predicting a Rapid Transition to Telehealth-Delivered Parent-Child Interaction Therapy Amid COVID-19: A Mixed Methods Study

The sudden onset of COVID-19 forced mental health therapists to rapidly transition to telehealth services. While some therapists and organizations were able to achieve an expeditious transition, others struggled. Using the Exploration, Preparation, Implementation, and Sustainment (EPIS) framework, which outlines key phases that guide the implementation process, the current mixed methods study examined what factors predicted the transition to internet-based Parent-Child Interaction Therapy (iPCIT), a telehealth-delivered evidence-based practice (EBP). We investigated two areas related to the transition: (1) if PCIT therapists transitioned to provide iPCIT and (2) if they made this transition quickly. In Fall 2019, 324 therapists completed a survey about implementing PCIT. After stay-at-home orders, 223 of those therapists completed a follow-up survey about their transition to telehealth, organizational characteristics, their caseloads, and telehealth training. The majority of therapists (82%) transitioned to provide iPCIT, with 48% making the transition in less than a week. Open-ended responses indicated that therapists who did not transition-faced challenges related to limited client resources, a lack of training, and organizational delays. Qualitative findings informed predictors for two logistic regression models that are statistical models that predict the probability of an event occurring, with criterion variables (1) whether therapists transitioned to provide iPCIT and (2) whether they transitioned in less than a week. Results showed that caseload in Fall 2019 and receipt of iPCIT training were associated with iPCIT transition. Organizational setting, resiliency, and baseline caseload predicted rapid transition to iPCIT. Implications regarding supporting the implementation of telehealth delivery of EBPs are discussed.

Intervening for sustainable change: Tailoring strategies to align with values and principles of communities

This paper presents a rationale for tailoring implementation strategies within a values-driven implementation approach. Values-driven implementation seeks to organize implementers around clarifying statements of their shared values in ways that harmonize implementation dynamics related to individual and group mental models, relationships among implementers, and the implementation climate. The proposed approach to tailoring strategies is informed by systems theory and emphasizes the need to focus on both tangible events and behaviors, as well deeper patterns, structures, relationships, and mental models, in order to increase the likelihood of sustaining implementation efforts and improving outcomes for people and communities. We offer for consideration three specific sets of context determinants that are under-represented in the implementation literature and that emerge as especially relevant within a systems approach to identifying and successfully tailoring implementation strategies in the implementation setting including relationships, mental models, and implementation climate.

Organizational factors associated with community therapists’ self-efficacy in EBP delivery: The interplay between sustainment leadership, sustainment climate, and psychological safety

Background

Inner context organizational factors proximally shape therapist experiences with evidence-based practice (EBP) implementation and may influence therapist self-efficacy, which has been linked to sustained use of EBPs in community mental health settings. Research has primarily focused on constructs such as implementation leadership and climate. However, the effects of such factors may depend upon other inner context dimensions, such as psychological safety. Psychologically safe environments are conducive to taking risks, speaking up about problems, and requesting feedback and may promote therapist self-efficacy during implementation. This study examines whether organizational sustainment leadership and sustainment climate relate to therapist EBP self-efficacy only under conditions of psychological safety.

Methods

Data were collected from 410 clinicians in 85 programs during the sustainment phase of a system-driven implementation of multiple EBPs in children's mental health services. Therapists reported on their organization's sustainment leadership, sustainment climate, psychological safety, and their own self-efficacy in delivering specific EBPs. Multilevel regression analyses were conducted to account for nested data structure.

Results

Among program-level variables, sustainment leadership and psychological safety both significantly predicted therapist self-efficacy. However, there were no significant interactions between program-level sustainment climate and psychological safety. Exploratory post-hoc analyses revealed a significant interaction between program-level sustainment leadership and therapist-level perceptions of psychological safety such that that the conditional effect of psychological safety on EBP self-efficacy was significant at high levels of sustainment leadership, but not at low or average levels.

Conclusion

We noted independent links between sustainment leadership, organizational psychological safety and therapists feelings of confidence and mastery with EBPs. Therapists’ individual perceptions of psychological safety were linked to self-efficacy only in programs with high sustainment leadership. Thus, sustainment leadership and psychological safety may both represent implementation intervention targets, but it may not be critical to assess for perceptions of psychological safety before deploying organizational leadership strategies. Plain language abstract Therapist self-efficacy is a therapist's belief that they are capable, knowledgeable, and skilled enough to deliver evidence-based practices (EBPs), and is thought to promote improved clinical and implementation outcomes, such as therapists’ sustained use of EBPs. Conditions within community mental health organizations may influence therapists’ sense of EBP self-efficacy. Leaders’ support and expectations for EBP implementation, and collective staff perceptions about the organization's climate to support EBPs are linked to positive therapist attitudes and EBP adoption. However, less is known about how these implementation-specific organizational factors associated with therapist EBP self-efficacy in the long-term, and how this may depend on general workplace conditions. Specifically, psychologically safe environments – where therapists feel safe taking risks such as asking questions, admitting mistakes, and trying new skills – may be needed to promote self-efficacy when therapists are tasked with learning and using complex multi-component EBP innovations. The current study tested the prediction that leader-driven and program-wide focus on EBP sustainment may promote therapist EBP self-efficacy only in organizations where conditions for learning are psychologically safe. Our findings confirmed that fostering strong sustainment focused leadership and psychologically safe environments may each be important for increasing therapists’ EBP self-efficacy. The model results suggested that individual therapist perceptions of psychological safety were more strongly related to EBP self-efficacy in programs with greater implementation leadership. Findings suggest the importance of increasing EBP leadership behavior to fully potentiate other facilitating conditions for therapist learning in the sustainment phase of EBP implementation initiatives.

Assessing Causal Pathways and Targets of Implementation Variability for EBP use (Project ACTIVE): a study protocol

Background

Advancing causal implementation theory is critical for designing tailored implementation strategies that target specific mechanisms associated with evidence-based practice (EBP) use. This study will test the generalizability of a conceptual model that integrates organizational constructs and behavioral theory to predict clinician use of cognitive-behavioral therapy (CBT) techniques in community mental health centers. CBT is a leading psychosocial EBP for psychiatric disorders that remains underused despite substantial efforts to increase its implementation.

Methods

We will leverage ongoing CBT implementation efforts in two large public health systems (Philadelphia and Texas) to recruit 300 mental health clinicians and 600 of their clients across 40 organizations. Our primary implementation outcomes of interest are clinician intentions to use CBT and direct observation of clinician use of CBT. As CBT comprises discrete components that vary in complexity and acceptability, we will measure clinician use of six discrete components of CBT. After finishing their CBT training, participating clinicians will complete measures of organizational and behavior change constructs delineated in the model. Clinicians also will be observed twice via audio recording delivering CBT with a client. Within 48 h of each observation, theorized moderators of the intention-behavior gap will be collected via survey. A subset of clinicians who report high intentions to use CBT but demonstrate low use will be purposively recruited to complete semi-structured interviews assessing reasons for the intention-behavior gap. Multilevel path analysis will test the extent to which intentions and determinants of intention predict the use of each discrete CBT component. We also will test the extent to which theorized determinants of intention that include psychological, organizational, and contextual factors explain variation in intention and moderate the association between intentions and CBT use.

Discussion

Project ACTIVE will advance implementation theory, currently in its infancy, by testing the generalizability of a promising causal model of implementation. These results will inform the development of implementation strategies targeting modifiable factors that explain substantial variance in intention and implementation that can be applied broadly across EBPs.