1
|
Williams NJ, Ehrhart MG, Aarons GA, Esp S, Sklar M, Carandang K, Vega NR, Brookman-Frazee L, Marcus SC. Improving measurement-based care implementation in youth mental health through organizational leadership and climate: a mechanistic analysis within a randomized trial. Implement Sci 2024; 19:29. [PMID: 38549122 PMCID: PMC10976812 DOI: 10.1186/s13012-024-01356-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Accepted: 02/27/2024] [Indexed: 04/02/2024] Open
Abstract
BACKGROUND Theory and correlational research indicate organizational leadership and climate are important for successful implementation of evidence-based practices (EBPs) in healthcare settings; however, experimental evidence is lacking. We addressed this gap using data from the WISDOM (Working to Implement and Sustain Digital Outcome Measures) hybrid type III effectiveness-implementation trial. Primary outcomes from WISDOM indicated the Leadership and Organizational Change for Implementation (LOCI) strategy improved fidelity to measurement-based care (MBC) in youth mental health services. In this study, we tested LOCI's hypothesized mechanisms of change, namely: (1) LOCI will improve implementation and transformational leadership, which in turn will (2) mediate LOCI's effect on implementation climate, which in turn will (3) mediate LOCI's effect on MBC fidelity. METHODS Twenty-one outpatient mental health clinics serving youth were randomly assigned to LOCI plus MBC training and technical assistance or MBC training and technical assistance only. Clinicians rated their leaders' implementation leadership, transformational leadership, and clinic implementation climate for MBC at five time points (baseline, 4-, 8-, 12-, and 18-months post-baseline). MBC fidelity was assessed using electronic metadata for youth outpatients who initiated treatment in the 12 months following MBC training. Hypotheses were tested using longitudinal mixed-effects models and multilevel mediation analyses. RESULTS LOCI significantly improved implementation leadership and implementation climate from baseline to follow-up at 4-, 8-, 12-, and 18-month post-baseline (all ps < .01), producing large effects (range of ds = 0.76 to 1.34). LOCI's effects on transformational leadership were small at 4 months (d = 0.31, p = .019) and nonsignificant thereafter (ps > .05). LOCI's improvement of clinic implementation climate from baseline to 12 months was mediated by improvement in implementation leadership from baseline to 4 months (proportion mediated [pm] = 0.82, p = .004). Transformational leadership did not mediate LOCI's effect on implementation climate (p = 0.136). Improvement in clinic implementation climate from baseline to 12 months mediated LOCI's effect on MBC fidelity during the same period (pm = 0.71, p = .045). CONCLUSIONS LOCI improved MBC fidelity in youth mental health services by improving clinic implementation climate, which was itself improved by increased implementation leadership. Fidelity to EBPs in healthcare settings can be improved by developing organizational leaders and strong implementation climates. TRIAL REGISTRATION ClinicalTrials.gov identifier: NCT04096274. Registered September 18, 2019.
Collapse
Affiliation(s)
- Nathaniel J Williams
- Institute for the Study of Behavioral Health and Addiction, Boise State University, Boise, ID, USA.
- School of Social Work, Boise State University, Boise, ID, 83725, USA.
| | - Mark G Ehrhart
- Department of Psychology, University of Central Florida, Orlando, FL, USA
| | - Gregory A Aarons
- Department of Psychiatry, University of California, San Diego, CA, USA
| | - Susan Esp
- Institute for the Study of Behavioral Health and Addiction, Boise State University, Boise, ID, USA
- School of Social Work, Boise State University, Boise, ID, 83725, USA
| | - Marisa Sklar
- Department of Psychiatry, University of California, San Diego, CA, USA
| | | | - Nallely R Vega
- School of Social Work, Boise State University, Boise, ID, 83725, USA
| | | | - Steven C Marcus
- School of Social Policy and Practice, University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
2
|
Zhang X, Peng M, He M, Du M, Jiang M, Cui M, Cai Y, Yan Q, Wang Y. Climates and associated factors for evidence-based practice implementation among nurses: a cross-sectional study. BMC Nurs 2024; 23:62. [PMID: 38254125 PMCID: PMC10801976 DOI: 10.1186/s12912-023-01694-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/13/2023] [Accepted: 12/28/2023] [Indexed: 01/24/2024] Open
Abstract
BACKGROUND The organizational climate that fosters and supports the implementation of evidence is a key factor influencing the effective implementation of evidence-based practice (EBP). Nurses, being the largest group of medical staff, play a crucial role in EBP implementation. The perception of the climate for EBP implementation among nurses in their organizations is unknown, especially among Chinese nurses. AIMS To clarify the implementation climate of EBP among Chinese nurses and identify the factors associated with the implementation and development of evidence-based nursing practices. METHODS This study employed a descriptive cross-sectional study design. From March 2023 to April 2023, a sample of nurses (n = 1260) from two Tertiary care hospitals in central China were selected and surveyed by self-designed social-demographic questionnaire and Implementation Climate Scale. Multiple linear stepwise regression analysis was conducted to determine the predictors of implementation climate. RESULTS The nurses achieved a mean ICS score of 59.10 ± 11.22, with mean scores exceeding 3 points for each sub-dimension and item. In the results of multiple linear regression, income satisfaction, implementation of evidence-based nursing practice project(s) within the unit, experience of evidence-based nursing practice during working life, and specific training or courses in evidence-based nursing practice were predictors of ICS. Of these, income satisfaction was the most significant predictor. These factors could explain 17.5% of the total variance in implementation climate. CONCLUSION The climate for implementing EBP in Chinese nursing organizations was relatively strong. Nursing managers can enhance the climate for implementing EBP in their organizations by actively improving salary and enhancing EBP-related trainings and practices. RELEVANCE TO CLINICAL PRACTICE Understanding nurses' perceptions of the EBP implementation climate in their organizations can help to identify specific barriers and facilitators to the development of EBP and facilitate its successful implementation. PATIENT OR PUBLIC CONTRIBUTION Clinical nurses were involved in data collection and completed the questionnaires on EBP implementation climate.
Collapse
Affiliation(s)
- Xinyue Zhang
- Nursing Department, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, 430030, Wuhan, Hubei, China
- School of Nursing, Tongji Medical College, Huazhong University of Science and Technology, 430030, Wuhan, Hubei, China
| | - Mengting Peng
- Nursing Department, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, 430030, Wuhan, Hubei, China
- School of Nursing, Tongji Medical College, Huazhong University of Science and Technology, 430030, Wuhan, Hubei, China
| | - Mei He
- Nursing Department, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, 430030, Wuhan, Hubei, China
| | - Meijie Du
- Nursing Department, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, 430030, Wuhan, Hubei, China
| | - Mengyao Jiang
- Nursing Department, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, 430030, Wuhan, Hubei, China
| | - Mengying Cui
- Nursing Department, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, 430030, Wuhan, Hubei, China
- School of Nursing, Tongji Medical College, Huazhong University of Science and Technology, 430030, Wuhan, Hubei, China
| | - Yue Cai
- Nursing Department, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, 430030, Wuhan, Hubei, China
- School of Nursing, Tongji Medical College, Huazhong University of Science and Technology, 430030, Wuhan, Hubei, China
| | - Qi Yan
- Nursing Department, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, 430030, Wuhan, Hubei, China
- School of Nursing, Tongji Medical College, Huazhong University of Science and Technology, 430030, Wuhan, Hubei, China
| | - Ying Wang
- Nursing Department, Tongji Hospital, Tongji Medical College, Huazhong University of Science and Technology, 430030, Wuhan, Hubei, China.
| |
Collapse
|
3
|
Sansbury GM, Pence BW, Zimba C, Yanguela J, Landrum K, Matewere M, Mbota M, Malava JK, Tikhiwa H, Morrison AM, Akiba CF, Gaynes BN, Udedi M, Hosseinipour MC, Stockton MA. Improving integrated depression and non-communicable disease care in Malawi through engaged leadership and supportive implementation climate. BMC Health Serv Res 2023; 23:1413. [PMID: 38098079 PMCID: PMC10722817 DOI: 10.1186/s12913-023-10344-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/09/2023] [Accepted: 11/17/2023] [Indexed: 12/17/2023] Open
Abstract
BACKGROUND Low- and middle-income countries often lack access to mental health services, leading to calls for integration within other primary care systems. In sub-Saharan Africa, integration of depression treatment in non-communicable disease (NCD) settings is feasible, acceptable, and effective. However, leadership and implementation climate challenges often hinder effective integration and quality of services. The aim of this study was to identify discrete leadership strategies that facilitate overcoming barriers to the integration of depression care in NCD clinics in Malawi and to understand how clinic leadership shapes the implementation climate. METHODS We conducted 39 in-depth interviews with the District Medical Officer, the NCD coordinator, one NCD provider, and the research assistant from each of the ten Malawian NCD clinics (note one District Medical Officer served two clinics). Based on semi-structured interview guides, participants were asked their perspectives on the impact of leadership and implementation climate on overcoming barriers to integrating depression care into existing NCD services. Thematic analysis used both inductive and deductive approaches to identify emerging themes and compare among participant type. RESULTS The results revealed how engaged leadership can fuel a positive implementation climate where clinics had heightened capacity to overcome implementation barriers. Effective leaders were approachable and engaged in daily operations of the clinic and problem-solving. They held direct involvement with and mentorship during the intervention, providing assistance in patient screening and consultation with treatment plans. Different levels of leadership utilized their respective standings and power dynamics to influence provider attitudes and perceptions surrounding the intervention. Leaders acted by informing providers about the intervention source and educating them on the importance of mental healthcare, as it was often undervalued. Lastly, they prioritized teamwork and collective ownership for the intervention, increasing provider responsibility. CONCLUSION Training that prioritizes leadership visibility and open communication will facilitate ongoing Malawi Ministry of Health efforts to scale up evidence-based depression treatment within NCD clinics. This proves useful where extensive and external monitoring may be limited. Ultimately, these results can inform successful strategies to close implementation gaps to achieve integration of mental health services in low-resource settings through improved leadership and implementation climate. TRIAL REGISTRATION These findings are reported from ClinicalTrials.gov, NCT03711786. Registered on 18/10/2018. https://clinicaltrials.gov/ct2/show/NCT03711786 .
Collapse
Affiliation(s)
- Griffin M Sansbury
- Tidziwe Centre, University of North Carolina Project-Malawi, Private Bag A-104, Lilongwe, Malawi.
| | - Brian W Pence
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, 135 Dauer Drive, 2101 McGavran-Greenberg Hall, CB #7435, Chapel Hill, NC, 27599-7440, USA
| | - Chifundo Zimba
- Tidziwe Centre, University of North Carolina Project-Malawi, Private Bag A-104, Lilongwe, Malawi
| | - Juan Yanguela
- Department of Health Policy and Management, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, 135 Dauer Drive, 1101 McGavran-Greenberg Hall, CB #7411, Chapel Hill, NC, 27599-7411, USA
| | - Kelsey Landrum
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, 135 Dauer Drive, 2101 McGavran-Greenberg Hall, CB #7435, Chapel Hill, NC, 27599-7440, USA
| | - Maureen Matewere
- Tidziwe Centre, University of North Carolina Project-Malawi, Private Bag A-104, Lilongwe, Malawi
| | - MacDonald Mbota
- Tidziwe Centre, University of North Carolina Project-Malawi, Private Bag A-104, Lilongwe, Malawi
| | - Jullita K Malava
- Malawi Epidemiology and Intervention Research Unit (MEIRU), P.O. Box 46, Chilumba, Karonga District, Malawi
| | - Harriet Tikhiwa
- Tidziwe Centre, University of North Carolina Project-Malawi, Private Bag A-104, Lilongwe, Malawi
| | - Abigail M Morrison
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, 135 Dauer Drive, 2101 McGavran-Greenberg Hall, CB #7435, Chapel Hill, NC, 27599-7440, USA
| | - Christopher F Akiba
- RTI International, 3040 East Cornwallis Road, P.O. Box 12194, Research Triangle Park, NC, 27709-2194, USA
| | - Bradley N Gaynes
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, 135 Dauer Drive, 2101 McGavran-Greenberg Hall, CB #7435, Chapel Hill, NC, 27599-7440, USA
- Division of Global Mental Health, Department of Psychiatry, University of North Carolina at Chapel Hill School of Medicine, 101 Manning Dr #1, Chapel Hill, NC, 27514, USA
| | - Michael Udedi
- Malawi Ministry of Health and Population, Non-Communicable Diseases and Mental Health Clinical Services, P.O. Box 30377, Lilongwe, 3, Malawi
| | - Mina C Hosseinipour
- Tidziwe Centre, University of North Carolina Project-Malawi, Private Bag A-104, Lilongwe, Malawi
- Department of Medicine, University of North Carolina at Chapel Hill School of Medicine, 101 Manning Dr #1, Chapel Hill, NC, 27514, USA
| | - Melissa A Stockton
- Tidziwe Centre, University of North Carolina Project-Malawi, Private Bag A-104, Lilongwe, Malawi
- Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, 135 Dauer Drive, 2101 McGavran-Greenberg Hall, CB #7435, Chapel Hill, NC, 27599-7440, USA
| |
Collapse
|
4
|
Egeland KM, Borge RH, Peters N, Bækkelund H, Braathu N, Sklar M, Aarons GA, Skar AMS. Individual-level associations between implementation leadership, climate, and anticipated outcomes: a time-lagged mediation analysis. Implement Sci Commun 2023; 4:75. [PMID: 37434244 DOI: 10.1186/s43058-023-00459-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Accepted: 06/17/2023] [Indexed: 07/13/2023] Open
Abstract
BACKGROUND Leaders can improve implementation outcomes by developing an organizational climate conducive to the implementation of evidence-based practices (EBP). This study tested the lagged associations between individual-level perceptions of implementation leadership, implementation climate, and three anticipated implementation outcomes, that is EBP acceptability, appropriateness, and feasibility. METHODS Screening tools and treatment methods for posttraumatic stress disorder were implemented in 43 Norwegian mental health services. A sample of 494 child and adult mental health care professionals (M = 43 years, 78% female) completed surveys addressing perceptions of first-level leaders' (n = 47) implementation leadership and their clinics' implementation climate. Single-level structural equation models estimating both direct, indirect, and total effects were used to investigate whether perceived implementation climate mediated the association between perceived implementation leadership and perceived acceptability, appropriateness, and feasibility of screening tools and treatment methods. RESULTS Regarding the treatment methods, implementation leadership was associated with therapists' perceptions of acceptability, appropriateness, and feasibility. Implementation climate also mediated between implementation leadership and the outcomes. Regarding the screening tools, implementation leadership was not associated with the outcomes. However, implementation climate mediated between implementation leadership and therapists' perceptions of acceptability and feasibility, but not appropriateness. Analyses with the implementation climate subscales showed stronger associations for therapists' perceptions of the treatment methods than of screening tools. CONCLUSIONS Leaders may promote positive implementation outcomes, both directly and through implementation climate. With regard to the effect sizes and explained variance, results indicated that both implementation leadership and implementation climate were more strongly associated with the therapists' perceptions of the treatment methods, implemented by one group of therapists, than the screening tools, implemented by all therapists. This may imply that implementation leadership and climate may have stronger effects for smaller implementation teams within a larger system than for system-wide implementations or when the clinical interventions being implemented are more complex rather than simple ones. TRIAL REGISTRATION ClinicalTrials NCT03719651, 25 October 2018.
Collapse
Affiliation(s)
- Karina Myhren Egeland
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484, Oslo, Norway.
| | - Randi Hovden Borge
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484, Oslo, Norway
- National Institute of Occupational Health, Gydas vei 8, 0363, Oslo, Norway
| | - Nadina Peters
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484, Oslo, Norway
| | - Harald Bækkelund
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484, Oslo, Norway
| | - Nora Braathu
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484, Oslo, Norway
| | - Marisa Sklar
- Department of Psychiatry, University of California, San Diego, 9500 Gilman Drive (0812), La Jolla, CA, 92093-0812, USA
- Diego ACTRI Dissemination and Implementation Science Center, UC San, 9500 Gilman Drive, La Jolla, CA, 92093, USA
- Child and Adolescent Services Research Center, 3665 Kearny Villa Rd., Suite 200N, San Diego, CA, 92123, USA
| | - Gregory A Aarons
- Department of Psychiatry, University of California, San Diego, 9500 Gilman Drive (0812), La Jolla, CA, 92093-0812, USA
- Diego ACTRI Dissemination and Implementation Science Center, UC San, 9500 Gilman Drive, La Jolla, CA, 92093, USA
- Child and Adolescent Services Research Center, 3665 Kearny Villa Rd., Suite 200N, San Diego, CA, 92123, USA
| | - Ane-Marthe Solheim Skar
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484, Oslo, Norway
| |
Collapse
|
5
|
Lin YJ, Ranusch A, Seagull FJ, Sussman JB, Barnes GD. Dynamic interplay between available resources and implementation climate across phases of implementation: a qualitative study of a VA national population health tool. Implement Sci Commun 2023; 4:74. [PMID: 37386501 DOI: 10.1186/s43058-023-00460-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/27/2022] [Accepted: 06/21/2023] [Indexed: 07/01/2023] Open
Abstract
BACKGROUND Available resources within an organization can determine the implementation success of an intervention. However, few studies have investigated how the required resources change over the phases of implementation. Using stakeholder interviews, we examined the changes in and interactions between available resources and implementation climate in the implementation and sustainment phases of a national implementation effort for a population health tool. METHODS We conducted a secondary analysis of the interviews with 20 anticoagulation professionals at 17 clinical sites in the Veterans Health Administration health system about their experiences with a population health dashboard for anticoagulant management. Interview transcripts were coded using constructs from the Consolidated Framework for Implementation Research (CFIR) and according to the phase of implementation (pre-implementation, implementation, and sustainment) as defined by the VA Quality Enhancement Research Initiative (QUERI) Roadmap. We analyzed the factors that may determine successful implementation by examining the co-occurrence patterns between available resources and implementation climate across different implementation phases. To illustrate the variations in these determinants across phases, we aggregated and scored coded statements using a previously published CFIR scoring system (- 2 to + 2). Key relationships between available resources and implementation climate were identified and summarized using thematic analysis. RESULTS The resources necessary to support the successful implementation of an intervention are not static; both the quantity and types of resources shift based on the phases of the intervention. Furthermore, increased resource availability does not guarantee the sustainment of intervention success. Users need different types of support beyond the technical aspects of an intervention, and this support varies over time. Specifically, available resources in the form of technological support and social/emotional support help users establish trust in a new technological-based intervention during the implementation phase. Resources that foster and maintain collaboration between users and other stakeholders help them stay motivated during sustainment. CONCLUSIONS Our findings highlight the dynamic nature of available resources and their impacts on the implementation climate across different phases of implementation. A better understanding of the dynamics of available resources over time from the users' perspectives will allow the adaptation of resources to better meet the needs of the intervention stakeholders.
Collapse
Affiliation(s)
- Ying-Jen Lin
- Center for Bioethics and Social Sciences in Medicine, University of Michigan, Ann Arbor, MI, USA
| | - Allison Ranusch
- Center for Clinical Management Research, Veterans Affairs Ann Arbor Healthcare System, Ann Arbor, MI, USA
| | - F Jacob Seagull
- Center for Bioethics and Social Sciences in Medicine, University of Michigan, Ann Arbor, MI, USA
| | - Jeremy B Sussman
- Center for Clinical Management Research, Veterans Affairs Ann Arbor Healthcare System, Ann Arbor, MI, USA
- Institute for Healthcare Policy and Innovation, University of Michigan, Ann Arbor, MI, USA
- Division of General Medicine, Department of Internal Medicine, University of Michigan, Ann Arbor, MI, USA
| | - Geoffrey D Barnes
- Center for Bioethics and Social Sciences in Medicine, University of Michigan, Ann Arbor, MI, USA.
- Institute for Healthcare Policy and Innovation, University of Michigan, Ann Arbor, MI, USA.
- Frankel Cardiovascular Center, Department of Internal Medicine, University of Michigan, 2800 Plymouth Rd, B14 G214, Ann Arbor, MI, 48109-2800, USA.
| |
Collapse
|
6
|
Rosen CS, Kaplan AN, Nelson DB, La Bash H, Chard KM, Eftekhari A, Kehle-Forbes S, Wiltsey Stirman S, Sayer NA. Implementation context and burnout among Department of Veterans Affairs psychotherapists prior to and during the COVID-19 pandemic. J Affect Disord 2023; 320:517-524. [PMID: 36191645 PMCID: PMC9523596 DOI: 10.1016/j.jad.2022.09.141] [Citation(s) in RCA: 7] [Impact Index Per Article: 7.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/13/2022] [Revised: 09/22/2022] [Accepted: 09/27/2022] [Indexed: 02/02/2023]
Abstract
BACKGROUND The first goal of this study was to assess longitudinal changes in burnout among psychotherapists prior to (T1) and during the COVID-19 pandemic (T2). The second objective was to assess the effects of job demands, job resources (including organizational support for evidence-based psychotherapies, or EBPs) and pandemic-related stress (T2 only) on burnout. METHOD Psychotherapists providing EBPs for posttraumatic stress disorder in U.S. Department of Veterans Affairs (VA) facilities completed surveys assessing burnout, job resources, and job demands prior to (T1; n = 346) and during (T2; n = 193) the COVID-19 pandemic. RESULTS Burnout prevalence increased from 40 % at T1 to 56 % at T2 (p < .001). At T1, stronger implementation climate and implementation leadership (p < .001) and provision of only cognitive processing therapy (rather than use of prolonged exposure therapy or both treatments; p < .05) reduced burnout risk. Risk factors for burnout at T2 included T1 burnout, pandemic-related stress, less control over when and how to deliver EBPs, being female, and being a psychologist rather than social worker (p < .02). Implementation leadership did not reduce risk of burnout at T2. LIMITATIONS This study involved staff not directly involved in treating COVID-19, in a healthcare system poised to transition to telehealth delivery. CONCLUSION Organizational support for using EBPs reduced burnout risk prior to but not during the pandemic. Pandemic related stress rather than increased work demands contributed to elevated burnout during the pandemic. A comprehensive approach to reducing burnout must address the effects of both work demands and personal stressors.
Collapse
Affiliation(s)
- Craig S. Rosen
- National Center for Posttraumatic Stress Disorder Dissemination & Training Division, VA Palo Alto Health Care System, Menlo Park, CA, USA,Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA, USA,Corresponding author at: VAPAHCS (334-PTSD), 795 Willow Road, Menlo Park, CA 94025, USA
| | - Adam N. Kaplan
- Center for Care Delivery and Outcomes Research, Minneapolis VA Health Care System, Minneapolis, MN, USA,Department of Medicine, University of Minnesota, Minneapolis, MN, USA
| | - David B. Nelson
- Center for Care Delivery and Outcomes Research, Minneapolis VA Health Care System, Minneapolis, MN, USA,Department of Medicine, University of Minnesota, Minneapolis, MN, USA
| | - Heidi La Bash
- National Center for Posttraumatic Stress Disorder Dissemination & Training Division, VA Palo Alto Health Care System, Menlo Park, CA, USA
| | - Kathleen M. Chard
- Cincinnati VA Medical Center, Cincinnati, OH, USA,University of Cincinnati College of Medicine, Cincinnati, OH, USA
| | - Afsoon Eftekhari
- National Center for Posttraumatic Stress Disorder Dissemination & Training Division, VA Palo Alto Health Care System, Menlo Park, CA, USA
| | - Shannon Kehle-Forbes
- Center for Care Delivery and Outcomes Research, Minneapolis VA Health Care System, Minneapolis, MN, USA,National Center for PTSD Women's Health Sciences Division, Boston MA, USA
| | - Shannon Wiltsey Stirman
- National Center for Posttraumatic Stress Disorder Dissemination & Training Division, VA Palo Alto Health Care System, Menlo Park, CA, USA,Department of Psychiatry and Behavioral Sciences, Stanford University School of Medicine, Stanford, CA, USA
| | - Nina A. Sayer
- Center for Care Delivery and Outcomes Research, Minneapolis VA Health Care System, Minneapolis, MN, USA,Department of Medicine, University of Minnesota, Minneapolis, MN, USA,Department of Psychiatry, University of Minnesota, Minneapolis, MN, USA
| |
Collapse
|
7
|
Williams NJ, Becker-Haimes EM, Schriger SH, Beidas RS. Linking organizational climate for evidence-based practice implementation to observed clinician behavior in patient encounters: a lagged analysis. Implement Sci Commun 2022; 3:64. [PMID: 35690845 PMCID: PMC9188232 DOI: 10.1186/s43058-022-00309-y] [Citation(s) in RCA: 9] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/26/2022] [Accepted: 05/20/2022] [Indexed: 11/10/2022] Open
Abstract
Background Theory and empirical research suggest organizational climate for evidence-based practice (EBP) implementation may be an important and malleable target to improve clinician use of EBPs in healthcare; however, this work has been criticized for overreliance on self-report measures of implementation outcomes and cross-sectional designs. This study combines data from two studies spanning 7 years to test the hypothesis that higher levels of organizational EBP implementation climate prospectively predicts improved clinician adherence to an EBP, cognitive behavioral therapy (CBT), as rated by expert observers. Methods Biennial assessments of EBP implementation climate collected in 10 community mental health agencies in Philadelphia as part of a systemwide evaluation (time 1) were linked to subsequent observer ratings of clinician adherence to CBT in clinical encounters with 108 youth (time 2). Experts rated clinician adherence to CBT using the Therapy Process Observation Coding System which generated two primary outcomes (a) maximum CBT adherence per session (i.e., highest rated CBT intervention per session; depth of delivery) and (b) average CBT adherence per session (i.e., mean rating across all CBT interventions used; depth and breadth of delivery). Results On average, time 2 clinician adherence observations occurred 19.8 months (SD = 10.15) after time 1 organizational climate assessments. Adjusting for organization, clinician, and client covariates, a one standard deviation increase in organizational EBP implementation climate at time 1 predicted a 0.63-point increase in clinicians’ maximum CBT adherence per session at time 2 (p = 0.000), representing a large effect size (d = 0.93; 95% CI = 0.63–1.24) when comparing organizations in the upper (k = 3) versus lower tertiles (k = 3) of EBP implementation climate. Higher levels of time 1 organizational EBP implementation climate also predicted higher time 2 average CBT adherence per session (b = 0.23, p < 0.001, d = 0.72). Length of time between assessments of climate and adherence did not moderate these relationships. Conclusions Organizational EBP implementation climate is a promising predictor of clinicians’ subsequent observed adherence to CBT. Implementation strategies that target this antecedent may improve the delivery of EBPs in healthcare settings.
Collapse
Affiliation(s)
- Nathaniel J Williams
- School of Social Work, Boise State University, 1910 University Drive, Boise, ID, 83625, USA.
| | - Emily M Becker-Haimes
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, USA.,Hall Mercer Community Mental Health, University of Pennsylvania Health System, Philadelphia, USA
| | - Simone H Schriger
- Department of Psychology, University of Pennsylvania, Philadelphia, USA
| | - Rinad S Beidas
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, USA.,Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, USA.,Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, USA.,Penn Implementation Science Center at the Leonard Davis Institute of Health Economics (PISCE@LDI), University of Pennsylvania, Philadelphia, USA.,Penn Medicine Nudge Unit, University of Pennsylvania Health System, Philadelphia, USA.,Center for Health Incentives and Behavioral Economics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, USA
| |
Collapse
|
8
|
Skar AMS, Braathu N, Peters N, Bækkelund H, Endsjø M, Babaii A, Borge RH, Wentzel-Larsen T, Ehrhart MG, Sklar M, Brown CH, Aarons GA, Egeland KM. A stepped-wedge randomized trial investigating the effect of the Leadership and Organizational Change for Implementation (LOCI) intervention on implementation and transformational leadership, and implementation climate. BMC Health Serv Res 2022; 22:298. [PMID: 35246135 PMCID: PMC8895588 DOI: 10.1186/s12913-022-07539-9] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/16/2021] [Accepted: 01/24/2022] [Indexed: 11/12/2022] Open
Abstract
BACKGROUND This study evaluates the Leadership and Organizational Change for Implementation (LOCI) strategy and its effect on implementation leadership, transformational leadership, and implementation climate. METHODS A stepped wedge cluster randomized study design enrolling 47 first-level leaders from child- and adult-specialized mental health clinics within Norwegian health trusts across three cohorts. All therapists (n = 790) received training in screening of trauma exposure and posttraumatic stress, and a subgroup of therapists (n = 248) received training in evidence-based treatment methods for posttraumatic stress disorder (PTSD). First-level leaders and therapists completed surveys at baseline, 4, 8-, 12-, 16-, and 20-months assessing leadership and implementation climate. General linear mixed-effects models were used to investigate whether the LOCI strategy would lead to greater therapist-rated scores on implementation leadership, transformational leadership, and implementation climate. RESULTS After introducing the LOCI strategy, there was a significant increase in therapist-rated implementation and transformational leadership and implementation climate. The increase was sustained at all measurement time points compared to non-LOCI conditions, which demonstrated a steady decrease in scores before LOCI. CONCLUSIONS The LOCI strategy can develop better transformational and implementation leadership skills and contribute to a more positive implementation climate, which may enhance successful EBP implementation. Thus, LOCI can help leaders create an organizational context conducive for effective EBP implementation. TRIAL REGISTRATION Retrospectively registered: ClinicalTrials NCT03719651 , 25th of October 2018. The trial protocol can be accessed from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6417075/ .
Collapse
Affiliation(s)
- Ane-Marthe Solheim Skar
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484, Oslo, Norway.
| | - Nora Braathu
- grid.504188.00000 0004 0460 5461Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484 Oslo, Norway
| | - Nadina Peters
- grid.504188.00000 0004 0460 5461Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484 Oslo, Norway
| | - Harald Bækkelund
- grid.504188.00000 0004 0460 5461Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484 Oslo, Norway
| | - Mathilde Endsjø
- grid.504188.00000 0004 0460 5461Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484 Oslo, Norway
| | - Aida Babaii
- grid.504188.00000 0004 0460 5461Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484 Oslo, Norway
| | - Randi Hovden Borge
- grid.504188.00000 0004 0460 5461Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484 Oslo, Norway
| | - Tore Wentzel-Larsen
- grid.504188.00000 0004 0460 5461Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484 Oslo, Norway ,Centre for Child and Adolescent Mental Health, Eastern and Southern Norway, Gullhaugveien 1, 0484 Oslo, Norway
| | - Mark G. Ehrhart
- grid.170430.10000 0001 2159 2859Department of Psychology, University of Central Florida, 4111 Pictor Lane, Orlando, FL 32816-1390 USA
| | - Marisa Sklar
- grid.266100.30000 0001 2107 4242Department of Psychiatry, University of California, San Diego, 9500 Gilman Drive (0812), La Jolla, San Diego, CA 92093-0812 USA ,grid.267102.00000000104485736Child and Adolescent Services Research Center, 3665 Kearny Villa Rd., Suite 200N, San Diego, CA 92123 USA
| | - C. Hendricks Brown
- grid.16753.360000 0001 2299 3507Feinberg School of Medicine, Northwestern University, 750 North Lake Shore Drive, Chicago, IL 60611 USA
| | - Gregory A. Aarons
- grid.266100.30000 0001 2107 4242Department of Psychiatry, University of California, San Diego, 9500 Gilman Drive (0812), La Jolla, San Diego, CA 92093-0812 USA ,grid.267102.00000000104485736Child and Adolescent Services Research Center, 3665 Kearny Villa Rd., Suite 200N, San Diego, CA 92123 USA
| | - Karina M. Egeland
- grid.504188.00000 0004 0460 5461Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484 Oslo, Norway
| |
Collapse
|
9
|
Braathu N, Laukvik EH, Egeland KM, Skar AMS. Validation of the Norwegian versions of the Implementation Leadership Scale (ILS) and Multifactor Leadership Questionnaire (MLQ) in a mental health care setting. BMC Psychol 2022; 10:25. [PMID: 35135616 PMCID: PMC8822706 DOI: 10.1186/s40359-022-00725-8] [Citation(s) in RCA: 2] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/05/2021] [Accepted: 01/24/2022] [Indexed: 12/25/2022] Open
Abstract
Background The implementation of evidence-based practices (EBPs) is of crucial importance in health care institutions and requires effective management from leaders. However, there is a lack of assessment tools sufficient to evaluate the degree to which the employees´ rate how well their leaders are at implementing EBPs. This emphasises the need for validated and widely used scales relevant for EBPs. Methods The current study evaluated the psychometric properties of the Multifactor Leadership Questionnaire (MLQ) and Implementation Leadership Scale (ILS) in a Norwegian mental health care setting. Results Results from confirmatory factor analyses indicate that the MLQ and ILS are valid instruments for measuring general and implementation leadership. The scales demonstrate good convergent validity. In addition, attitudes towards EBPs did not seem to be associated with the ILS and MLQ, further supporting the applicability of the instruments. Conclusions The two scales demonstrate good psychometric properties in a Norwegian mental health care setting, which suggests that the MLQ and ILS are valid and reliable tools for measuring leadership in an implementation setting. More research is greatly needed to disentangle the link between perceived leadership and objective measures of successful implementation of EBPs. Trial registration NSD 690,133, NSD 60,059/3/OOS.
Collapse
Affiliation(s)
- Nora Braathu
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484, Oslo, Norway.
| | - Erlend Høen Laukvik
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484, Oslo, Norway
| | - Karina M Egeland
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484, Oslo, Norway
| | - Ane-Marthe Solheim Skar
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1-3, 0484, Oslo, Norway
| |
Collapse
|
10
|
Peters N, Borge RH, Skar AMS, Egeland KM. Measuring implementation climate: psychometric properties of the Implementation Climate Scale (ICS) in Norwegian mental health care services. BMC Health Serv Res 2022; 22:23. [PMID: 34983526 PMCID: PMC8725247 DOI: 10.1186/s12913-021-07441-w] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/06/2021] [Accepted: 12/22/2021] [Indexed: 12/02/2022] Open
Abstract
BackgroundEmployees' perceptions of organizational climate for implementation of new methods are important in assessing and planning for implementation efforts. More specifically, feedback from employees' points to which implementation strategies to select, adopt, and tailor in building positive climate for implementation of new evidence-based practices within the organization. Implementation climate can be measured with the Implementation Climate Scale (ICS). The purpose of this study was to investigate the psychometric properties of the Norwegian version of the ICS in outpatient mental health clinics.MethodsThe ICS was administered to 383 clinicians within 47 different child and adult mental health clinics across the country. We conducted confirmatory factor analysis to assess the psychometric functioning of the ICS. Cronbach's alpha was examined to assess internal consistency. We also examined criterion related validity of the scale by comparing it with an alternative measure of implementation climate (concurrent validity) and by examining correlations with clinicians' intentions to use evidence-based practices.ResultsResults supported the 6-factor structure and the internal consistency reliability of the ICS. One exception was poor functioning of the Reward scale. Concurrent validity was stronger at the group than at the individual level, and assessment of associations with clinicians' intentions to use evidence- based practices showed positive correlations.ConclusionsThe Norwegian version of the ICS is a promising tool for assessing implementation climate which can provide organizations with specific feedback concerning which aspects of the implementation climate to attend to. Due to poor functioning of the Reward scale, adaptations and further testing of this is recommended.
Collapse
Affiliation(s)
- Nadina Peters
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484, Oslo, Norway.
| | - Randi Hovden Borge
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484, Oslo, Norway
| | - Ane- Marthe Solheim Skar
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484, Oslo, Norway
| | - Karina M Egeland
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484, Oslo, Norway
| |
Collapse
|
11
|
Williams NJ, Wolk CB, Becker-Haimes EM, Beidas RS. Testing a theory of strategic implementation leadership, implementation climate, and clinicians' use of evidence-based practice: a 5-year panel analysis. Implement Sci 2020; 15:10. [PMID: 32033575 PMCID: PMC7006179 DOI: 10.1186/s13012-020-0970-7] [Citation(s) in RCA: 51] [Impact Index Per Article: 12.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/31/2019] [Accepted: 01/31/2020] [Indexed: 01/15/2023] Open
Abstract
BACKGROUND Implementation theory suggests that first-level leaders, sometimes referred to as middle managers, can increase clinicians' use of evidence-based practice (EBP) in healthcare settings by enacting specific leadership behaviors (i.e., proactive, knowledgeable, supportive, perseverant with regard to implementation) that develop an EBP implementation climate within the organization; however, longitudinal and quasi-experimental studies are needed to test this hypothesis. METHODS Using data collected at three waves over a 5-year period from a panel of 30 outpatient children's mental health clinics employing 496 clinicians, we conducted a quasi-experimental difference-in-differences study to test whether within-organization change in implementation leadership predicted within-organization change in EBP implementation climate, and whether change in EBP implementation climate predicted within-organization change in clinicians' use of EBP. At each wave, clinicians reported on their first-level leaders' implementation leadership, their organization's EBP implementation climate, and their use of both EBP and non-EBP psychotherapy techniques for childhood psychiatric disorders. Hypotheses were tested using econometric two-way fixed effects regression models at the organization level which controlled for all stable organizational characteristics, population trends in the outcomes over time, and time-varying covariates. RESULTS Organizations that improved from low to high levels of implementation leadership experienced significantly greater increases in their level of EBP implementation climate (d = .92, p = .017) and within-organization increases in implementation leadership accounted for 11% of the variance in improvement in EBP implementation climate beyond all other covariates. In turn, organizations that improved from low to high levels of EBP implementation climate experienced significantly greater increases in their clinicians' average EBP use (d = .55, p = .007) and within-organization improvement in EBP implementation climate accounted for 14% of the variance in increased clinician EBP use. Mediation analyses indicated that improvement in implementation leadership had a significant indirect effect on clinicians' EBP use via improvement in EBP implementation climate (d = .26, 95% CI [.02 to .59]). CONCLUSIONS When first-level leaders increase their frequency of implementation leadership behaviors, organizational EBP implementation climate improves, which in turn contributes to increased EBP use by clinicians. Trials are needed to test strategies that target this implementation leadership-EBP implementation climate mechanism.
Collapse
Affiliation(s)
- Nathaniel J Williams
- School of Social Work, Boise State University, Boise, ID, USA. .,Institute for the Study of Behavioral Health and Addiction, Boise State University, Boise, ID, USA. .,School of Social Work, Boise State University, Room 711, 1910 University Drive, Boise, ID, 83725, USA.
| | - Courtney Benjamin Wolk
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
| | - Emily M Becker-Haimes
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.,Hall Mercer Community Mental Health Center, Pennsylvania Hospital, Philadelphia, PA, USA
| | - Rinad S Beidas
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.,Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.,Department of Medicine, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA.,Penn Implementation Science Center at the Leonard Davis Institute of Health Economics (PISCE@LDI), University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
12
|
Dorsey S, Gray CL, Wasonga AI, Amanya C, Weiner BJ, Belden CM, Martin P, Meza RD, Weinhold AK, Soi C, Murray LK, Lucid L, Turner EL, Mildon R, Whetten K. Advancing successful implementation of task-shifted mental health care in low-resource settings (BASIC): protocol for a stepped wedge cluster randomized trial. BMC Psychiatry 2020; 20:10. [PMID: 31914959 PMCID: PMC6947833 DOI: 10.1186/s12888-019-2364-4] [Citation(s) in RCA: 24] [Impact Index Per Article: 6.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/28/2019] [Accepted: 11/15/2019] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND The mental health treatment gap-the difference between those with mental health need and those who receive treatment-is high in low- and middle-income countries. Task-shifting has been used to address the shortage of mental health professionals, with a growing body of research demonstrating the effectiveness of mental health interventions delivered through task-shifting. However, very little research has focused on how to embed, support, and sustain task-shifting in government-funded systems with potential for scale up. The goal of the Building and Sustaining Interventions for Children (BASIC) study is to examine implementation policies and practices that predict adoption, fidelity, and sustainment of a mental health intervention in the education sector via teacher delivery and the health sector via community health volunteer delivery. METHODS BASIC is a Hybrid Type II Implementation-Effectiveness trial. The study design is a stepped wedge, cluster randomized trial involving 7 sequences of 40 schools and 40 communities surrounding the schools. Enrollment consists of 120 teachers, 120 community health volunteers, up to 80 site leaders, and up to 1280 youth and one of their primary guardians. The evidence-based mental health intervention is a locally adapted version of Trauma-focused Cognitive Behavioral Therapy, called Pamoja Tunaweza. Lay counselors are trained and supervised in Pamoja Tunaweza by local trainers who are experienced in delivering the intervention and who participated in a Train-the-Trainer model of skills transfer. After the first sequence completes implementation, in-depth interviews are conducted with initial implementing sites' counselors and leaders. Findings are used to inform delivery of implementation facilitation for subsequent sequences' sites. We use a mixed methods approach including qualitative comparative analysis to identify necessary and sufficient implementation policies and practices that predict 3 implementation outcomes of interest: adoption, fidelity, and sustainment. We also examine child mental health outcomes and cost of the intervention in both the education and health sectors. DISCUSSION The BASIC study will provide knowledge about how implementation of task-shifted mental health care can be supported in government systems that already serve children and adolescents. Knowledge about implementation policies and practices from BASIC can advance the science of implementation in low-resource contexts. TRIAL REGISTRATION Trial Registration: ClinicalTrials.gov Identifier: NCT03243396. Registered 9th August 2017, https://clinicaltrials.gov/ct2/show/NCT03243396.
Collapse
Affiliation(s)
- Shannon Dorsey
- Department of Psychology, University of Washington Guthrie Hall 119A, Box 351525, Seattle, WA, 98195, USA.
| | - Christine L Gray
- Center for Health Policy and Inequalities Research, Duke Global Health Institute, Duke University, Campus Box 90392, Durham, NC, 27710, USA
| | | | - Cyrilla Amanya
- Research Department, Ace Africa Kenya, P.O. Box 1185, Bungoma, 50200, Kenya
| | - Bryan J Weiner
- Department of Global Health, University of Washington, Harris Hydraulics Laboratory, 1510 San Juan Road, Seattle, WA, 98195, USA
- Department of Health Services, School of Public Health, University of Washington, Box 357965, Seattle, WA, 98195, USA
| | - C Micha Belden
- Center for Health Policy and Inequalities Research, Duke Global Health Institute, Duke University, Campus Box 90392, Durham, NC, 27710, USA
| | - Prerna Martin
- Department of Psychology, University of Washington Guthrie Hall 119A, Box 351525, Seattle, WA, 98195, USA
| | - Rosemary D Meza
- Department of Psychology, University of Washington Guthrie Hall 119A, Box 351525, Seattle, WA, 98195, USA
| | - Andrew K Weinhold
- Center for Health Policy and Inequalities Research, Duke Global Health Institute, Duke University, Campus Box 90392, Durham, NC, 27710, USA
| | - Caroline Soi
- Department of Global Health, University of Washington, Harris Hydraulics Laboratory, 1510 San Juan Road, Seattle, WA, 98195, USA
| | - Laura K Murray
- Department of Mental Health, Johns Hopkins Bloomberg School of Public Health, 624 N. Broadway, 8th floor, Baltimore, MD, 21205, USA
| | - Leah Lucid
- Department of Psychology, University of Washington Guthrie Hall 119A, Box 351525, Seattle, WA, 98195, USA
| | - Elizabeth L Turner
- Department of Biostatistics and Bioinformatics, Duke University School of Medicine, Duke University, Durham, NC, 27710, USA
- Duke Global Health Institute, Duke University, Campus Box 90519, Durham, NC, 27708, USA
| | - Robyn Mildon
- Centre for Evidence and Implementation, 33 Lincoln Square South, Carlton, Victoria, 3053, Australia
| | - Kathryn Whetten
- Center for Health Policy and Inequalities Research, Duke Global Health Institute, Duke University, Campus Box 90392, Durham, NC, 27710, USA
- Terry Sanford Institute of Public Policy, Duke University, Box 90239, Durham, NC, 27708, USA
| |
Collapse
|
13
|
Bunger AC, Birken SA, Hoffman JA, MacDowell H, Choy-Brown M, Magier E. Elucidating the influence of supervisors' roles on implementation climate. Implement Sci 2019; 14:93. [PMID: 31653254 PMCID: PMC6815002 DOI: 10.1186/s13012-019-0939-6] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/13/2019] [Accepted: 09/11/2019] [Indexed: 11/20/2022] Open
Abstract
Background Supervisors play an essential role in implementation by diffusing and synthesizing information, selling implementation, and translating top management’s project plans to frontline workers. Theory and emerging evidence suggest that through these roles, supervisors shape implementation climate—i.e., the degree to which innovations are expected, supported, and rewarded. However, it is unclear exactly how supervisors carry out each of these roles in ways that contribute to implementation climate—this represents a gap in the understanding of the causal mechanisms that link supervisors’ behavior with implementation climate. This study examined how supervisors’ performance of each of these roles influences three core implementation climate domains (expectations, supports, and rewards). Materials and methods A sequenced behavioral health screening, assessment, and referral intervention was implemented within a county-based child welfare agency. We conducted 6 focus groups with supervisors and frontline workers from implementing work units 6 months post-implementation (n = 51) and 1 year later (n = 40) (12 groups total). Participants were asked about implementation determinants, including supervision and implementation context. We audio-recorded, transcribed, and analyzed focus groups using an open coding process during which the importance of the supervisors’ roles emerged as a major theme. We further analyzed this code using concepts and definitions related to middle managers’ roles and implementation climate. Results In this work setting, supervisors (1) diffused information about the intervention proactively, and in response to workers’ questions, (2) synthesized information by tailoring it to workers’ individual needs, (3) translated top managements’ project plans into day-to-day tasks through close monitoring and reminders, and (4) justified implementation. All four of these roles appeared to shape the implementation climate by conveying strong expectations for implementation. Three roles (diffusing, synthesizing, and mediating) influenced climate by supporting workers during implementation. Only one role (diffusing) influenced climate by conveying rewards. Conclusions Supervisors shaped implementation climate by carrying out four roles (diffusing, synthesizing, mediating, and selling). Findings suggest that the interaction of these roles convey expectations and support for implementation (two implementation climate domains). Our study advances the causal theory explaining how supervisors’ behavior shapes the implementation climate, which can inform implementation practice.
Collapse
Affiliation(s)
- Alicia C Bunger
- College of Social Work, The Ohio State University, 1947 College Road, Columbus, OH, 43210, USA.
| | - Sarah A Birken
- Department of Health Policy and Management, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, 1105C McGavran-Greenberg Hall, Campus Box 7411, Chapel Hill, NC, 27599, USA
| | - Jill A Hoffman
- School of Social Work, Portland State University, 1800 SW 6th Avenue, Suite 600, Portland, OR, 97201, USA
| | - Hannah MacDowell
- Bureau of Maternal, Child and Family Health, Ohio Department of Health, 246 North High Street, Columbus, OH, 43215, USA
| | - Mimi Choy-Brown
- School of Social Work, University of Minnesota, Peters Hall, 1404 Gortner Ave, Saint Paul, MN, 55108, USA
| | - Erica Magier
- College of Social Work, The Ohio State University, 1947 College Road, Columbus, OH, 43210, USA
| |
Collapse
|
14
|
Ehrhart MG, Torres EM, Hwang J, Sklar M, Aarons GA. Validation of the Implementation Climate Scale (ICS) in substance use disorder treatment organizations. Subst Abuse Treat Prev Policy 2019; 14:35. [PMID: 31443666 PMCID: PMC6708170 DOI: 10.1186/s13011-019-0222-5] [Citation(s) in RCA: 16] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 04/08/2019] [Accepted: 08/15/2019] [Indexed: 01/25/2023]
Abstract
Background One critical factor in the implementation of evidence-based practice (EBP) in substance use disorder treatment organizations is an inner organizational context that clearly supports implementation efforts. The Implementation Climate Scale (ICS) has been developed to allow researchers and organizations to assess climate for EBP implementation in health and allied health service organizations. The ICS consists of 18 items and measures six dimensions of implementation climate: focus on EBP, educational support for EBP, recognition for EBP, rewards for EBP, selection for EBP, and selection for openness. The ICS was initially developed in a mental health context; thus, the goal of this study was to provide initial validation of the ICS in substance use disorder (SUD) treatment settings. Methods Confirmatory factor analysis (CFA) was used to assess the psychometric functioning of the ICS using survey data from 326 providers in 65 teams in SUD treatment programs. Cronbach’s alpha was examined to assess internal consistency of the ICS, and individual and team level construct-based validity was examined by comparing its correlations with service climate, molar climate, and organizational change. Results We found evidence for the reliability, factor structure, and validity of the ICS in SUD services. The psychometric functioning of the ICS in SUD treatment settings was comparable to that found in mental health contexts. Conclusions The ICS is a brief and pragmatic tool for researchers to better understand a critical antecedent for implementation effectiveness in SUD treatment and for organizational leaders in SUD treatment organizations to evaluate the extent to which providers perceive that their organization supports EBP implementation.
Collapse
Affiliation(s)
- Mark G Ehrhart
- Department of Psychology, University of Central Florida, Orlando, FL, USA
| | - Elisa M Torres
- Department of Psychology, George Mason University, Fairfax, VA, USA
| | - Joyce Hwang
- Department of Psychology, San Diego State University, San Diego, CA, USA
| | - Marisa Sklar
- Department of Psychiatry, University of California, La Jolla, San Diego, CA, USA.,Child and Adolescent Services Research Center, San Diego, CA, USA
| | - Gregory A Aarons
- Department of Psychiatry, University of California, La Jolla, San Diego, CA, USA. .,Child and Adolescent Services Research Center, San Diego, CA, USA.
| |
Collapse
|
15
|
Egeland KM, Skar AMS, Endsjø M, Laukvik EH, Bækkelund H, Babaii A, Granly LB, Husebø GK, Borge RH, Ehrhart MG, Sklar M, Brown CH, Aarons GA. Testing the leadership and organizational change for implementation (LOCI) intervention in Norwegian mental health clinics: a stepped-wedge cluster randomized design study protocol. Implement Sci 2019; 14:28. [PMID: 30866973 PMCID: PMC6417075 DOI: 10.1186/s13012-019-0873-7] [Citation(s) in RCA: 21] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/01/2019] [Accepted: 02/19/2019] [Indexed: 11/29/2022] Open
Abstract
BACKGROUND Alignment across levels of leadership within an organization is needed for successful implementation of evidence-based practice. The leadership and organizational change for implementation (LOCI) intervention is a multi-faceted multilevel implementation strategy focusing on enhancing first-level general and implementation leadership while also engaging with organization upper management to develop an organizational climate for implementation. The aim of the project is to evaluate the effectiveness of LOCI in supporting the implementation of evidence-based treatment for PTSD in child- and adult-specialized mental health clinics in health trusts in Norway. METHODS The study design is a stepped-wedge cluster randomized trial with enrollment of clinics in three cohorts. Executives, clinic leaders, and therapists will be asked to complete surveys assessing leadership and implementation climate. Surveys will be completed at baseline, 4, 8, 12, 16, and 20 months. Results from surveys will be shared with executives and clinic leaders to inform the subsequent creation of tailored leadership and climate development plans for enhanced implementation. Patients will complete surveys measuring traumatic events and post-traumatic stress symptoms during the therapy process. Therapy sessions will be audio or video recorded and scored for fidelity as part of training. DISCUSSION This study aims to provide knowledge on how to improve leadership and organizational climate to enhance effective implementation of evidence-based treatments in mental health services. TRIAL REGISTRATION The study has been registrated in ClinicalTrials with ID NCT03719651 .
Collapse
Affiliation(s)
- Karina M. Egeland
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484 Oslo, Norway
| | - Ane-Marthe Solheim Skar
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484 Oslo, Norway
| | - Mathilde Endsjø
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484 Oslo, Norway
| | - Erlend Høen Laukvik
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484 Oslo, Norway
| | - Harald Bækkelund
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484 Oslo, Norway
| | - Aida Babaii
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484 Oslo, Norway
| | - Lene Beate Granly
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484 Oslo, Norway
| | - Gry Kristina Husebø
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484 Oslo, Norway
| | - Randi Hovden Borge
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484 Oslo, Norway
| | - Mark G. Ehrhart
- Department of Psychology, San Diego State University, 5500 Campanile Drive, San Diego, CA 92182-4611 USA
- Department of Psychology, University of Central Florida, 4111 Pictor Lane, Orlando, FL 32816-1390 USA
| | - Marisa Sklar
- Department of Psychiatry, University of California, San Diego, 9500 Gilman Drive (0812), La Jolla, San Diego, CA 92093-0812 USA
- Child and Adolescent Services Research Center, 3665 Kearny Villa Rd., Suite 200N, San Diego, CA 92123 USA
| | - C. Hendricks Brown
- Feinberg School of Medicine, Northwestern University, 750 North Lake Shore Drive, Chicago, IL 60611 USA
| | - Gregory A. Aarons
- Department of Psychiatry, University of California, San Diego, 9500 Gilman Drive (0812), La Jolla, San Diego, CA 92093-0812 USA
- Child and Adolescent Services Research Center, 3665 Kearny Villa Rd., Suite 200N, San Diego, CA 92123 USA
| |
Collapse
|
16
|
Birken S, Clary A, Tabriz AA, Turner K, Meza R, Zizzi A, Larson M, Walker J, Charns M. Middle managers' role in implementing evidence-based practices in healthcare: a systematic review. Implement Sci 2018; 13:149. [PMID: 30541596 PMCID: PMC6292008 DOI: 10.1186/s13012-018-0843-5] [Citation(s) in RCA: 70] [Impact Index Per Article: 11.7] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/27/2018] [Accepted: 11/28/2018] [Indexed: 12/01/2022] Open
Abstract
BACKGROUND Middle managers are in a unique position to promote the implementation of evidence-based practices (EBPs) in healthcare organizations, yet knowledge of middle managers' role in implementation and determinants (e.g., individual-, organizational-, and system-level factors) which influence their role remains fractured, spanning decades and disciplines. To synthesize understanding, we undertook a systematic review of studies of middle managers' role in healthcare EBP implementation and determinants of that role. METHODS We searched MEDLINE/PubMed and Business Source Complete (Ebsco) for literature on middle managers' role in healthcare EBP implementation and its determinants. We abstracted data from records that met inclusion criteria (i.e., written in English, peer-reviewed, and reporting either a protocol or results of an empirical study) into a matrix for analysis. We summarized categorical variables using descriptive statistics. To analyze qualitative data, we used a priori codes and then allowed additional themes to emerge. RESULTS One hundred five records, spanning across several countries and healthcare settings and relating to a range of EBPs, met our inclusion criteria. Studies of middle managers' role in healthcare EBP implementation and its determinants substantially increased from 1996 to 2015. Results from included studies suggest that middle managers shape implementation climate in addition to fulfilling the four roles hypothesized in extant theory of middle managers' role in implementation. However, extant studies offered little understanding of determinants of middle managers' role. CONCLUSIONS Our findings suggest that middle managers may play an important role in facilitating EBP implementation. Included studies offered little understanding regarding the relative importance of various roles, potential moderators of the relationship between middle managers' roles and EBP implementation, or determinants of middle managers' role in EBP implementation. Future studies should seek to understand determinants and moderators of middle managers' role. Clearer understanding may facilitate the translation of evidence into practice.
Collapse
Affiliation(s)
- Sarah Birken
- Department of Health Policy and Management, Gillings School of Global Public Health, The University of North Carolina at Chapel Hill, 1103E McGavran-Greenberg, 135 Dauer Drive, Campus Box 7411, Chapel Hill, NC 27599-7411 USA
| | - Alecia Clary
- Department of Health Policy and Management, Gillings School of Global Public Health, The University of North Carolina at Chapel Hill, 1103E McGavran-Greenberg, 135 Dauer Drive, Campus Box 7411, Chapel Hill, NC 27599-7411 USA
| | - Amir Alishahi Tabriz
- Department of Health Policy and Management, Gillings School of Global Public Health, The University of North Carolina at Chapel Hill, 1103E McGavran-Greenberg, 135 Dauer Drive, Campus Box 7411, Chapel Hill, NC 27599-7411 USA
| | - Kea Turner
- Department of Health Policy and Management, Gillings School of Global Public Health, The University of North Carolina at Chapel Hill, 1103E McGavran-Greenberg, 135 Dauer Drive, Campus Box 7411, Chapel Hill, NC 27599-7411 USA
| | - Rosemary Meza
- Department of Psychology, University of Washington, Guthrie Hall, Room 119A, UW Box 351525, Seattle, WA 98195 USA
| | - Alexandra Zizzi
- Department of Health Policy and Management, Gillings School of Global Public Health, The University of North Carolina at Chapel Hill, 1103E McGavran-Greenberg, 135 Dauer Drive, Campus Box 7411, Chapel Hill, NC 27599-7411 USA
| | - Madeline Larson
- Department of Educational Psychology, University of Minnesota, 104 Burton Hall, 178 Pillsbury Dr. S.E, Minneapolis, MN 55455 USA
| | - Jennifer Walker
- The University of North Carolina at Chapel Hill, Health Sciences Library, 335 S. Columbia Street, Campus Box CB 7585, Chapel Hill, NC 27599-7411 USA
| | - Martin Charns
- Center for Healthcare Organization and Implementation Research (CHOIR), VA Boston Healthcare System, 150 South Huntington Avenue (152M), Jamaica Plain Campus, Building 9, Boston, MA 02130 USA
- Department of Health Law, Policy & Management, School of Public Health, Boston University, 715 Albany St. Talbot Building, Boston, MA 02118 USA
| |
Collapse
|
17
|
Turner K, Trogdon JG, Weinberger M, Stover AM, Ferreri S, Farley JF, Ray N, Patti M, Renfro C, Shea CM. Testing the organizational theory of innovation implementation effectiveness in a community pharmacy medication management program: a hurdle regression analysis. Implement Sci 2018; 13:105. [PMID: 30064454 PMCID: PMC6069858 DOI: 10.1186/s13012-018-0799-5] [Citation(s) in RCA: 20] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/07/2018] [Accepted: 07/23/2018] [Indexed: 12/02/2022] Open
Abstract
BACKGROUND Many state Medicaid programs are implementing pharmacist-led medication management programs to improve outcomes for high-risk beneficiaries. There are a limited number of studies examining implementation of these programs, making it difficult to assess why program outcomes might vary across organizations. To address this, we tested the applicability of the organizational theory of innovation implementation effectiveness to examine implementation of a community pharmacy Medicaid medication management program. METHODS We used a hurdle regression model to examine whether organizational determinants, such as implementation climate and innovation-values fit, were associated with effective implementation. We defined effective implementation in two ways: implementation versus non-implementation and program reach (i.e., the proportion of the target population that received the intervention). Data sources included an implementation survey administered to participating community pharmacies and administrative data. RESULTS The findings suggest that implementation climate is positively and significantly associated with implementation versus non-implementation (AME = 2.65, p < 0.001) and with program reach (AME = 5.05, p = 0.001). Similarly, the results suggest that innovation-values fit is positively and significantly associated with implementation (AME = 2.17, p = 0.037) and program reach (AME = 11.79, p < 0.001). Some structural characteristics, such as having a clinical pharmacist on staff, were significant predictors of implementation and program reach whereas other characteristics, such as pharmacy type or prescription volume, were not. CONCLUSIONS Our study supported the use of the organizational theory of innovation implementation effectiveness to identify organizational determinants that are associated with effective implementation (e.g., implementation climate and innovation-values fit). Unlike broader environmental factors or structural characteristics (e.g., pharmacy type), implementation climate and innovation-values fit are modifiable factors and can be targeted through intervention-a finding that is important for community pharmacy practice. Additional research is needed to determine what implementation strategies can be used by community pharmacy leaders and practitioners to develop a positive implementation climate and innovation-values fit for medication management programs.
Collapse
Affiliation(s)
- Kea Turner
- Department of Health Policy and Management, Gillings School of Global Public Health, The University of North Carolina at Chapel Hill, 135 Dauer Drive, Chapel Hill, NC 27599-7411 USA
| | - Justin G. Trogdon
- Department of Health Policy and Management, Gillings School of Global Public Health, The University of North Carolina at Chapel Hill, 135 Dauer Drive, Chapel Hill, NC 27599-7411 USA
| | - Morris Weinberger
- Department of Health Policy and Management, Gillings School of Global Public Health, The University of North Carolina at Chapel Hill, 135 Dauer Drive, Chapel Hill, NC 27599-7411 USA
| | - Angela M. Stover
- Department of Health Policy and Management, Gillings School of Global Public Health, The University of North Carolina at Chapel Hill, 135 Dauer Drive, Chapel Hill, NC 27599-7411 USA
| | - Stefanie Ferreri
- Division of Practice Advancement and Clinical Education, Eshelman School of Pharmacy, The University of North Carolina at Chapel Hill, 115B Beard Hall, Chapel Hill, NC 27599-7411 USA
| | - Joel F. Farley
- Department of Pharmaceutical Care and Health Systems, College of Pharmacy, University of Minnesota, 308 Harvard Street SE, Minneapolis, MN 55455 USA
| | - Neepa Ray
- Center for Medication Optimization through Practice and Policy, Eshelman School of Pharmacy, The University of North Carolina at Chapel Hill, 2400 Kerr Hall, Chapel Hill, NC 27599-7411 USA
| | - Michael Patti
- Division of Practice Advancement and Clinical Education, Eshelman School of Pharmacy, The University of North Carolina at Chapel Hill, 115B Beard Hall, Chapel Hill, NC 27599-7411 USA
| | - Chelsea Renfro
- Department of Clinical Pharmacy and Translational Science, University of Tennessee Health Science Center, 881 Madison Avenue, Memphis, TN 38163 USA
| | - Christopher M. Shea
- Department of Health Policy and Management, Gillings School of Global Public Health, The University of North Carolina at Chapel Hill, 1103E McGavran-Greenberg, 135 Dauer Drive, Chapel Hill, NC 27599-7411 USA
| |
Collapse
|
18
|
Williams NJ, Ehrhart MG, Aarons GA, Marcus SC, Beidas RS. Linking molar organizational climate and strategic implementation climate to clinicians' use of evidence-based psychotherapy techniques: cross-sectional and lagged analyses from a 2-year observational study. Implement Sci 2018; 13:85. [PMID: 29940989 PMCID: PMC6019309 DOI: 10.1186/s13012-018-0781-2] [Citation(s) in RCA: 47] [Impact Index Per Article: 7.8] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/04/2017] [Accepted: 06/18/2018] [Indexed: 11/23/2022] Open
Abstract
Background Behavioral health organizations are characterized by multiple organizational climates, including molar climate, which encompasses clinicians’ shared perceptions of how the work environment impacts their personal well-being, and strategic implementation climate, which includes clinicians’ shared perceptions of the extent to which evidence-based practice implementation is expected, supported, and rewarded by the organization. Theory suggests these climates have joint, cross-level effects on clinicians’ implementation of evidence-based practice and that these effects may be long term (i.e., up to 2 years); however, no empirical studies have tested these relationships. We hypothesize that molar climate moderates implementation climate’s concurrent and long-term relationships with clinicians’ use of evidence-based practice such that strategic implementation climate will have its most positive effects when it is accompanied by a positive molar climate. Methods Hypotheses were tested using data collected from 235 clinicians in 20 behavioral health organizations. At baseline, clinicians reported on molar climate and implementation climate. At baseline and at a 2-year follow-up, all clinicians who were present in the organizations reported on their use of cognitive-behavioral psychotherapy techniques, an evidence-based practice for youth psychiatric disorders. Two-level mixed-effects regression models tested whether baseline molar climate and implementation climate interacted in predicting clinicians’ evidence-based practice use at baseline and at 2-year follow-up. Results In organizations with more positive molar climates at baseline, higher levels of implementation climate predicted increased evidence-based practice use among clinicians who were present at baseline and among clinicians who were present in the organizations at 2-year follow-up; however, in organizations with less positive molar climates, implementation climate was not related to clinicians’ use of evidence-based practice at either time point. Conclusions Optimizing clinicians’ implementation of evidence-based practice in behavioral health requires attention to both molar climate and strategic implementation climate. Strategies that focus exclusively on implementation climate may not be effective levers for behavior change if the organization does not also engender a positive molar climate. These findings have implications for the development of implementation theory and effective implementation strategies.
Collapse
Affiliation(s)
- Nathaniel J Williams
- School of Social Work, Boise State University, 1910 University Drive, Boise, ID, 83725, USA.
| | - Mark G Ehrhart
- Department of Psychology, University of Central Florida, Orlando, FL, USA
| | - Gregory A Aarons
- Department of Psychiatry, University of California, San Diego, CA, USA
| | - Steven C Marcus
- School of Social Policy and Practice, University of Pennsylvania, Philadelphia, PA, USA
| | - Rinad S Beidas
- Department of Psychiatry, University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
19
|
Dorsey S, Pullmann MD, Kerns SEU, Jungbluth N, Meza R, Thompson K, Berliner L. The Juggling Act of Supervision in Community Mental Health: Implications for Supporting Evidence-Based Treatment. Adm Policy Ment Health 2017; 44:838-52. [PMID: 28315076 DOI: 10.1007/s10488-017-0796-z] [Citation(s) in RCA: 38] [Impact Index Per Article: 6.3] [Reference Citation Analysis] [What about the content of this article? (0)] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 02/01/2023]
Abstract
Supervisors are an underutilized resource for supporting evidence-based treatments (EBTs) in community mental health. Little is known about how EBT-trained supervisors use supervision time. Primary aims were to describe supervision (e.g., modality, frequency), examine functions of individual supervision, and examine factors associated with time allocation to supervision functions. Results from 56 supervisors and 207 clinicians from 25 organizations indicate high prevalence of individual supervision, often alongside group and informal supervision. Individual supervision serves a wide range of functions, with substantial variation at the supervisor-level. Implementation climate was the strongest predictor of time allocation to clinical and EBT-relevant functions.
Collapse
|
20
|
Lyon AR, Cook CR, Brown EC, Locke J, Davis C, Ehrhart M, Aarons GA. Assessing organizational implementation context in the education sector: confirmatory factor analysis of measures of implementation leadership, climate, and citizenship. Implement Sci 2018; 13:5. [PMID: 29310673 PMCID: PMC5759223 DOI: 10.1186/s13012-017-0705-6] [Citation(s) in RCA: 67] [Impact Index Per Article: 11.2] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2017] [Accepted: 12/27/2017] [Indexed: 12/03/2022] Open
Abstract
Background A substantial literature has established the role of the inner organizational setting on the implementation of evidence-based practices in community contexts, but very little of this research has been extended to the education sector, one of the most common settings for the delivery of mental and behavioral health services to children and adolescents. The current study examined the factor structure, psychometric properties, and interrelations of an adapted set of pragmatic organizational instruments measuring key aspects of the organizational implementation context in schools: (1) strategic implementation leadership, (2) strategic implementation climate, and (3) implementation citizenship behavior. Method The Implementation Leadership Scale (ILS), Implementation Climate Scale (ICS), and Implementation Citizenship Behavior Scale (ICBS) were adapted by a research team that included the original scale authors and experts in the implementation of evidence-based practices in schools. These instruments were then administered to a geographically representative sample (n = 196) of school-based mental/behavioral health consultants to assess the reliability and structural validity via a series of confirmatory factor analyses. Results Overall, the original factor structures for the ILS, ICS, and ICBS were confirmed in the current sample. The one exception was poor functioning of the Rewards subscale of the ICS, which was removed in the final ICS model. Correlations among the revised measures, evaluated as part of an overarching model of the organizational implementation context, indicated both unique and shared variance. Conclusions The current analyses suggest strong applicability of the revised instruments to implementation of evidence-based mental and behavioral practices in the education sector. The one poorly functioning subscale (Rewards on the ICS) was attributed to typical educational policies that do not allow for individual financial incentives to personnel. Potential directions for future expansion, revision, and application of the instruments in schools are discussed.
Collapse
Affiliation(s)
- Aaron R Lyon
- University of Washington, 6200 NE 74th Street, Suite 100, Seattle, WA, 98115, USA.
| | - Clayton R Cook
- University of Minnesota, 250 Education Sciences Bldg, 56 East River Road, Minneapolis, MN, 55455, USA
| | - Eric C Brown
- Department of Public Health Sciences, Miller School of Medicine, University of Miami, 1120 NW 14th Street, Office 104, Miami, FL, 33136, USA
| | - Jill Locke
- University of Washington, 6200 NE 74th Street, Suite 100, Seattle, WA, 98115, USA
| | - Chayna Davis
- University of Washington, 6200 NE 74th Street, Suite 100, Seattle, WA, 98115, USA
| | - Mark Ehrhart
- University of Central Florida, 4111 Pictor Lane, Orlando, FL, 32816-1390, USA
| | - Gregory A Aarons
- University of California San Diego, 9500 Gilman Drive (0812), La Jolla, CA, 92093, USA.,Child and Adolescent Services Research Center, 9500 Gilman Drive (0812), La Jolla, CA, 92093, USA
| |
Collapse
|
21
|
Ehrhart MG, Torres EM, Wright LA, Martinez SY, Aarons GA. Validating the Implementation Climate Scale (ICS) in child welfare organizations. Child Abuse Negl 2016; 53:17-26. [PMID: 26563643 PMCID: PMC4818155 DOI: 10.1016/j.chiabu.2015.10.017] [Citation(s) in RCA: 24] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [What about the content of this article? (0)] [Affiliation(s)] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/01/2015] [Revised: 10/02/2015] [Accepted: 10/19/2015] [Indexed: 06/05/2023]
Abstract
There is increasing emphasis on the use of evidence-based practices (EBPs) in child welfare settings and growing recognition of the importance of the organizational environment, and the organization's climate in particular, for how employees perceive and support EBP implementation. Recently, Ehrhart, Aarons, and Farahnak (2014) reported on the development and validation of a measure of EBP implementation climate, the Implementation Climate Scale (ICS), in a sample of mental health clinicians. The ICS consists of 18 items and measures six critical dimensions of implementation climate: focus on EBP, educational support for EBP, recognition for EBP, rewards for EBP, selection or EBP, and selection for openness. The goal of the current study is to extend this work by providing evidence for the factor structure, reliability, and validity of the ICS in a sample of child welfare service providers. Survey data were collected from 215 child welfare providers across three states, 12 organizations, and 43 teams. Confirmatory factor analysis demonstrated good fit to the six-factor model and the alpha reliabilities for the overall measure and its subscales was acceptable. In addition, there was general support for the invariance of the factor structure across the child welfare and mental health sectors. In conclusion, this study provides evidence for the factor structure, reliability, and validity of the ICS measure for use in child welfare service organizations.
Collapse
Affiliation(s)
- Mark G. Ehrhart
- Department of Psychology, San Diego State University, San Diego, CA USA
| | - Elisa M. Torres
- Department of Psychiatry, University of California, San Diego, La Jolla, CA USA
- Child and Adolescent Services Research Center, San Diego, CA USA
| | - Lisa A. Wright
- Department of Psychology, San Diego State University, San Diego, CA USA
| | | | - Gregory A. Aarons
- Department of Psychiatry, University of California, San Diego, La Jolla, CA USA
- Child and Adolescent Services Research Center, San Diego, CA USA
| |
Collapse
|