1
|
Azizian Kia S, Wittkampf L, van Lankeren J, Janse P. Motives of Therapists for Using Routine Outcome Monitoring (ROM) and How it is Used by Them in Clinical Practice: Two Qualitative Studies. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2024:10.1007/s10488-024-01374-2. [PMID: 38589737 DOI: 10.1007/s10488-024-01374-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/29/2024] [Indexed: 04/10/2024]
Abstract
Despite its demonstrated value, many mental health institutions struggle to implement progress feedback effectively. There is also insufficient information about how therapists utilize progress feedback. To gain more insight, two qualitative studies were conducted. The first study compared the attitudes and motives of therapists who used and those who did not use progress feedback. The second study examined how psychologists incorporated progress feedback into their practice. In total, 23 therapists were interviewed, and the data were analyzed using thematic analysis. The first study found that almost all the therapists had a positive attitude about progress feedback. Those who did not use it indicated reasons such as a heavy workload and patient-related factors, and they also lacked sufficient information about the potential benefits of progress feedback. The second study revealed four major ways in which the therapists utilized feedback, namely: supporting actions to discuss progress feedback, discussing progress feedback with patients, modifications in the ongoing treatment, and peer consultation. However, discussions during peer consultations about using progress feedback for patients who were not benefitting from treatment and how the treatment might be adjusted accordingly were often lacking. In conclusion, it is crucial that training and education be provided to therapists on how to use progress feedback effectively. Having regular discussions about progress feedback during peer consultations could also facilitate its integration into clinical practice.
Collapse
Affiliation(s)
| | | | | | - Pauline Janse
- Pro Persona Research, Wolfheze, The Netherlands.
- Behavioural Science Institute, Radboud University, Nijmegen, The Netherlands.
| |
Collapse
|
2
|
Williams NJ, Ehrhart MG, Aarons GA, Esp S, Sklar M, Carandang K, Vega NR, Brookman-Frazee L, Marcus SC. Improving measurement-based care implementation in youth mental health through organizational leadership and climate: a mechanistic analysis within a randomized trial. Implement Sci 2024; 19:29. [PMID: 38549122 PMCID: PMC10976812 DOI: 10.1186/s13012-024-01356-w] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/16/2023] [Accepted: 02/27/2024] [Indexed: 04/02/2024] Open
Abstract
BACKGROUND Theory and correlational research indicate organizational leadership and climate are important for successful implementation of evidence-based practices (EBPs) in healthcare settings; however, experimental evidence is lacking. We addressed this gap using data from the WISDOM (Working to Implement and Sustain Digital Outcome Measures) hybrid type III effectiveness-implementation trial. Primary outcomes from WISDOM indicated the Leadership and Organizational Change for Implementation (LOCI) strategy improved fidelity to measurement-based care (MBC) in youth mental health services. In this study, we tested LOCI's hypothesized mechanisms of change, namely: (1) LOCI will improve implementation and transformational leadership, which in turn will (2) mediate LOCI's effect on implementation climate, which in turn will (3) mediate LOCI's effect on MBC fidelity. METHODS Twenty-one outpatient mental health clinics serving youth were randomly assigned to LOCI plus MBC training and technical assistance or MBC training and technical assistance only. Clinicians rated their leaders' implementation leadership, transformational leadership, and clinic implementation climate for MBC at five time points (baseline, 4-, 8-, 12-, and 18-months post-baseline). MBC fidelity was assessed using electronic metadata for youth outpatients who initiated treatment in the 12 months following MBC training. Hypotheses were tested using longitudinal mixed-effects models and multilevel mediation analyses. RESULTS LOCI significantly improved implementation leadership and implementation climate from baseline to follow-up at 4-, 8-, 12-, and 18-month post-baseline (all ps < .01), producing large effects (range of ds = 0.76 to 1.34). LOCI's effects on transformational leadership were small at 4 months (d = 0.31, p = .019) and nonsignificant thereafter (ps > .05). LOCI's improvement of clinic implementation climate from baseline to 12 months was mediated by improvement in implementation leadership from baseline to 4 months (proportion mediated [pm] = 0.82, p = .004). Transformational leadership did not mediate LOCI's effect on implementation climate (p = 0.136). Improvement in clinic implementation climate from baseline to 12 months mediated LOCI's effect on MBC fidelity during the same period (pm = 0.71, p = .045). CONCLUSIONS LOCI improved MBC fidelity in youth mental health services by improving clinic implementation climate, which was itself improved by increased implementation leadership. Fidelity to EBPs in healthcare settings can be improved by developing organizational leaders and strong implementation climates. TRIAL REGISTRATION ClinicalTrials.gov identifier: NCT04096274. Registered September 18, 2019.
Collapse
Affiliation(s)
- Nathaniel J Williams
- Institute for the Study of Behavioral Health and Addiction, Boise State University, Boise, ID, USA.
- School of Social Work, Boise State University, Boise, ID, 83725, USA.
| | - Mark G Ehrhart
- Department of Psychology, University of Central Florida, Orlando, FL, USA
| | - Gregory A Aarons
- Department of Psychiatry, University of California, San Diego, CA, USA
| | - Susan Esp
- Institute for the Study of Behavioral Health and Addiction, Boise State University, Boise, ID, USA
- School of Social Work, Boise State University, Boise, ID, 83725, USA
| | - Marisa Sklar
- Department of Psychiatry, University of California, San Diego, CA, USA
| | | | - Nallely R Vega
- School of Social Work, Boise State University, Boise, ID, 83725, USA
| | | | - Steven C Marcus
- School of Social Policy and Practice, University of Pennsylvania, Philadelphia, PA, USA
| |
Collapse
|
3
|
Jensen-Doss A, Casline E, Woodard GS, Phillips DA, Lane E, Palafu T, Waye C, Ramirez V, Cheron DM, Okamura K. Provider Use of, Attitudes Towards, and Self-efficacy with Key Measurement-based Care Practices in Youth Mental Health Treatment: A Multi-site Examination. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2024:10.1007/s10488-024-01354-6. [PMID: 38334881 DOI: 10.1007/s10488-024-01354-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 02/02/2024] [Indexed: 02/10/2024]
Abstract
Measurement-based care (MBC) is an underutilized evidence-based practice, and current implementation efforts demonstrate limited success in increasing MBC use. A better understanding of MBC implementation determinants is needed to improve these efforts, particularly from studies examining the full range of MBC practices and that span multiple samples of diverse providers using different MBC systems. This study addressed these limitations by conducting a multi-site survey examining MBC predictors and use in youth treatment. Participants were 159 clinicians and care coordinators working in youth mental health care settings across the United States. Participants were drawn from three program evaluations of MBC implementation. Providers completed measures assessing use of five MBC practices (administering measures, viewing feedback, reviewing feedback in supervision, sharing feedback with clients in session, and using feedback to plan treatment), MBC self-efficacy, and MBC attitudes. Despite expectations that MBC should be standard care for all clients, providers reported only administering measures to 40-60% of clients on average, with practices related to the use of feedback falling in the 1-39% range. Higher MBC self-efficacy and more positive views of MBC practicality predicted higher MBC use, although other attitude measures were not significant predictors. Effects of predictors were not moderated by site, suggesting consistent predictors across implementation settings. Implications of study findings for future research and for the implementation of MBC are discussed.
Collapse
Affiliation(s)
| | - Elizabeth Casline
- Center for Dissemination and Implementation Science, Northwestern University Feinberg School of Medicine, Chicago, IL, US
| | - Grace S Woodard
- Department of Psychology, University of Miami, Coral Gables, FL, USA
| | | | - Elizabeth Lane
- Department of Psychology, University of Miami, Coral Gables, FL, USA
| | - Tessa Palafu
- The Baker Center for Children and Families, Boston, MA, USA
| | - Catherine Waye
- The Baker Center for Children and Families, Boston, MA, USA
| | | | | | - Kelsie Okamura
- The Baker Center for Children and Families, Boston, MA, USA
| |
Collapse
|
4
|
Sklar M, Ehrhart MG, Ramirez N, Carandang K, Kuhn N, Day A, Aarons GA, Williams NJ. Implementation leadership and implementation climate in context: A single organization intrinsic case study for implementation of digital measurement-based care. IMPLEMENTATION RESEARCH AND PRACTICE 2024; 5:26334895241236680. [PMID: 38550748 PMCID: PMC10976481 DOI: 10.1177/26334895241236680] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/28/2024] Open
Abstract
Background Although studies have demonstrated that implementation leadership and climate are important constructs in predicting evidence-based practice (EBP) implementation, concrete descriptions of how they operate during organizational implementation efforts are lacking. This case study fills that gap through an in-depth description of an organization with effective implementation leadership that successfully built a strong implementation climate. This case study provides an illustration of implementation leadership and climate in tangible, replicable terms to assist managers, practitioners, and researchers in addressing the organizational context in their own implementation projects. Method A single organization, intrinsic case study was employed to paint a multifaceted picture of how one organization leveraged implementation leadership to strengthen a climate for the successful implementation of digital measurement-based care. The case was drawn from a cluster-randomized trial designed to test the effects of a leadership-focused implementation strategy on youth-level fidelity and clinical outcomes of digital measurement-based care. Following the completion of the trial, case study activities commenced. Descriptive summaries of multiple data sources (including quantitative data on implementation leadership and climate, coaching call and organizational alignment meeting recordings and notes, and development plans) were produced and revised iteratively until consensus was reached. Leadership actions were analyzed for corresponding dimensions of implementation leadership and climate. Results Specific actions organizational leaders took, as well as the timing specific strategies were enacted, to create a climate for implementation are presented, along with lessons learned from this experience. Conclusion This case study offers concrete steps organizational leaders took to create a consistent and aligned message that the implementation of a specific EBP was a top priority in the agency. The general approach taken to create an implementation climate provides several lessons for leaders, especially for EBPs that have broad implications across an organization.
Collapse
Affiliation(s)
- Marisa Sklar
- Department of Psychiatry, University of California, San Diego, La Jolla, CA, USA
- UC San Diego ACTRI Dissemination and Implementation Science Center, La Jolla, CA, USA
- Child and Adolescent Services Research Center, San Diego, CA, USA
| | - Mark G. Ehrhart
- Department of Psychology, University of Central Florida, Orlando, FL, USA
| | - Nallely Ramirez
- Institute for the Study of Behavioral Health and Addiction, Boise State University, Boise, ID, USA
| | | | | | - Ana Day
- Oregon Community Programs, Eugene, OR, USA
| | - Gregory A. Aarons
- Department of Psychiatry, University of California, San Diego, La Jolla, CA, USA
- UC San Diego ACTRI Dissemination and Implementation Science Center, La Jolla, CA, USA
- Child and Adolescent Services Research Center, San Diego, CA, USA
| | - Nathaniel J. Williams
- Institute for the Study of Behavioral Health and Addiction, Boise State University, Boise, ID, USA
- School of Social Work, Boise State University, Boise, ID, USA
| |
Collapse
|
5
|
Williams NJ, Marcus SC, Ehrhart MG, Sklar M, Esp SM, Carandang K, Vega N, Gomes AE, Brookman-Frazee L, Aarons GA. Randomized Trial of an Organizational Implementation Strategy to Improve Measurement-Based Care Fidelity and Youth Outcomes in Community Mental Health. J Am Acad Child Adolesc Psychiatry 2023:S0890-8567(23)02239-6. [PMID: 38070868 DOI: 10.1016/j.jaac.2023.11.010] [Citation(s) in RCA: 2] [Impact Index Per Article: 2.0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/02/2023] [Revised: 10/26/2023] [Accepted: 11/30/2023] [Indexed: 12/28/2023]
Abstract
OBJECTIVE Measurement-based care (MBC), which collects session-by-session symptom data from patients and provides clinicians with feedback on treatment response, is a highly generalizable evidence-based practice with significant potential to improve the outcomes of mental health treatment in youth when implemented with fidelity; however, it is rarely used in community settings. This study tested whether an implementation strategy targeting organizational leadership and organizational implementation climate could improve MBC fidelity and clinical outcomes for youth in outpatient mental health clinics. METHOD In a cluster randomized trial, 21 clinics were assigned to the Leadership and Organizational Change for Implementation strategy plus training and technical assistance in MBC (k = 11, n = 117) or training and technical assistance only (k = 10, n = 117). Primary outcomes of MBC fidelity (assessed via electronic metadata) and youth symptom improvement (assessed via caregiver-reported change on the Shortform Assessment for Children Total Problem Score) were collected for consecutively enrolled youths (ages 4-18 years) who initiated treatment in the 12 months following MBC training. Outcomes of each youth were assessed for 6 months following baseline. RESULTS A total of 234 youths were enrolled and included in intent-to-treat analyses. At baseline, there were no significant differences by condition in clinic, clinician, or youth characteristics. Youths in clinics using the Leadership and Organizational Change for Implementation strategy experienced significantly higher MBC fidelity compared with youths in control clinics (23.1% vs 3.4%, p = .014), and exhibited significantly greater reductions in symptoms from baseline to 6 months (d = 0.31, 95% CI: 0.04-0.58, p = .023). CONCLUSION Implementation strategies targeting organizational leadership and focused implementation climate can improve fidelity to evidence-based practices and clinical outcomes of youth mental health services. CLINICAL TRIAL REGISTRATION INFORMATION Working to Implement and Sustain Digital Outcome Measures (WISDOM); https://clinicaltrials.gov/; NCT04096274.
Collapse
|
6
|
Xiao R, Bonafide CP, Williams NJ, Cidav Z, Landrigan CP, Faerber J, Makeneni S, Wolk CB, Schondelmeyer AC, Brady PW, Beidas RS, Schisterman EF. Eliminating Monitor Overuse (EMO) type III effectiveness-deimplementation cluster-randomized trial: Statistical analysis plan. Contemp Clin Trials Commun 2023; 36:101219. [PMID: 37842322 PMCID: PMC10568304 DOI: 10.1016/j.conctc.2023.101219] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/15/2023] [Revised: 09/19/2023] [Accepted: 10/01/2023] [Indexed: 10/17/2023] Open
Abstract
Background Deimplementing overused health interventions is essential to maximizing quality and value while minimizing harm, waste, and inefficiencies. Three national guidelines discourage continuous pulse oximetry (SpO2) monitoring in children who are not receiving supplemental oxygen, but the guideline-discordant practice remains prevalent, making it a prime target for deimplementation. This paper details the statistical analysis plan for the Eliminating Monitor Overuse (EMO) SpO2 trial, which compares the effect of two competing deimplementation strategies (unlearning only vs. unlearning plus substitution) on the sustainment of deimplementation of SpO2 monitoring in children with bronchiolitis who are in room air. Methods The EMO Trial is a hybrid type 3 effectiveness-deimplementation trial with a longitudinal cluster-randomized design, conducted in Pediatric Research in Inpatient Settings Network hospitals. The primary outcome is deimplementation sustainment, analyzed as a longitudinal difference-in-differences comparison between study arms. This analysis will use generalized hierarchical mixed-effects models for longitudinal clustering outcomes. Secondary outcomes include the length of hospital stay and oxygen supplementation duration, modeled using linear mixed-effects regressions. Using the well-established counterfactual approach, we will also perform a mediation analysis of hospital-level mechanistic measures on the association between the deimplementation strategy and the sustainment outcome. Discussion We anticipate that the EMO Trial will advance the science of deimplementation by providing new insights into the processes, mechanisms, and likelihood of sustained practice change using rigorously designed deimplementation strategies. This pre-specified statistical analysis plan will mitigate reporting bias and support data-driven approaches. Trial registration ClinicalTrials.gov NCT05132322. Registered on 24 November 2021.
Collapse
Affiliation(s)
- Rui Xiao
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, Blockley Hall, 423 Guardian Drive, Philadelphia, PA, 19104-6021, USA
| | - Christopher P. Bonafide
- Section of Hospital Medicine and Clinical Futures, Children's Hospital of Philadelphia, 3500 Civic Center Blvd, The Hub, Philadelphia, PA, 19104, USA
- Department of Pediatrics, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, USA
- Penn Implementation Science Center at the Leonard Davis Institute of Health Economics (PISCE@LDI), University of Pennsylvania, Philadelphia, PA, USA
| | - Nathaniel J. Williams
- School of Social Work, Boise State University, 1910 W. University Drive, Boise, ID, 83725, USA
| | - Zuleyha Cidav
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, 3535 Market St, Philadelphia, Pennsylvania, 19104, USA
- Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA, USA
| | - Christopher P. Landrigan
- Division of General Pediatrics, Boston Children's Hospital, Enders 1, 300 Longwood Ave, Boston, MA, 02115, USA
- Department of Pediatrics, Harvard Medical School, Boston, MA, USA
| | - Jennifer Faerber
- Data Science and Biostatistics Unit, Roberts Center for Pediatric Research, Children's Hospital of Philadelphia, 2716 South Street, Philadelphia, PA, 19146, USA
| | - Spandana Makeneni
- Data Science and Biostatistics Unit, Roberts Center for Pediatric Research, Children's Hospital of Philadelphia, 2716 South Street, Philadelphia, PA, 19146, USA
| | - Courtney Benjamin Wolk
- Department of Psychiatry, Perelman School of Medicine, University of Pennsylvania, 3553 Market Street, 3rd Floor, Philadelphia, PA, 19104, USA
| | - Amanda C. Schondelmeyer
- Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, USA
- Division of Hospital Medicine, James M. Anderson Center for Health Systems Excellence, Cincinnati Children's Hospital Medical Center, 3333 Burnet Ave ML 9016, Cincinnati, OH, 45229, USA
| | - Patrick W. Brady
- Department of Pediatrics, University of Cincinnati College of Medicine, Cincinnati, OH, USA
- Division of Hospital Medicine, James M. Anderson Center for Health Systems Excellence, Cincinnati Children's Hospital Medical Center, 3333 Burnet Ave ML 9016, Cincinnati, OH, 45229, USA
| | - Rinad S. Beidas
- Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, 625 North Michigan Avenue, Chicago, IL, 60661, USA
| | - Enrique F. Schisterman
- Department of Biostatistics, Epidemiology, and Informatics, Perelman School of Medicine, Blockley Hall, 423 Guardian Drive, Philadelphia, PA, 19104-6021, USA
| |
Collapse
|
7
|
Cho E, Cook JR, Hawley KM. A Structural Model of Organization and Clinician Factors Associated with Standardized Measure Use in a National Survey of Youth Mental Health Providers. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2023; 50:876-887. [PMID: 37458956 PMCID: PMC11056910 DOI: 10.1007/s10488-023-01286-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/02/2023] [Indexed: 10/01/2023]
Abstract
Standardized assessment measures are important for accurate diagnosis of mental health problems and for treatment planning and evaluation. However, little is known about youth mental health providers' typical use of standardized measures across disciplines and outside the context of evidence-based practice initiatives. A multidisciplinary national survey examined the frequency with which 674 youth mental health providers administer standardized and unstandardized measures, and the extent to which organizational (i.e., implementation climate, rigid hierarchical organizational structure) and provider (i.e., attitudes toward standardized assessment measures, highest degree, practice setting) characteristics are associated with standardized measure use. Providers used unstandardized measures far more frequently than standardized measures. Providers' perceptions (a) that standardized measures are practical or feasible, (b) that their organization supports and values evidence-based practices, and (c) that their organization has a rigid hierarchical structure predicted greater use of standardized measures. Working in schools predicted less frequent SMU, while working in higher education and other professional settings predicted more frequent SMU. Standardized measures were not routinely used in this community-based sample. A rigid hierarchical organizational structure may be conducive to more frequent administration of standardized measures, but it is unclear whether such providers actually utilize these measures for clinical decision-making. Alternative strategies to promote standardized measure use may include promoting organizational cultures that value empirical data and encouraging use of standardized measures and training providers to use pragmatic standardized measures for clinical decision making.
Collapse
Affiliation(s)
- Evelyn Cho
- University of Missouri, Columbia, USA
- Harvard University, Cambridge, USA
| | - Jonathan R Cook
- University of Missouri, Columbia, USA
- Pacific Anxiety Group, Belmont, USA
| | | |
Collapse
|
8
|
Egeland KM, Borge RH, Peters N, Bækkelund H, Braathu N, Sklar M, Aarons GA, Skar AMS. Individual-level associations between implementation leadership, climate, and anticipated outcomes: a time-lagged mediation analysis. Implement Sci Commun 2023; 4:75. [PMID: 37434244 DOI: 10.1186/s43058-023-00459-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/21/2022] [Accepted: 06/17/2023] [Indexed: 07/13/2023] Open
Abstract
BACKGROUND Leaders can improve implementation outcomes by developing an organizational climate conducive to the implementation of evidence-based practices (EBP). This study tested the lagged associations between individual-level perceptions of implementation leadership, implementation climate, and three anticipated implementation outcomes, that is EBP acceptability, appropriateness, and feasibility. METHODS Screening tools and treatment methods for posttraumatic stress disorder were implemented in 43 Norwegian mental health services. A sample of 494 child and adult mental health care professionals (M = 43 years, 78% female) completed surveys addressing perceptions of first-level leaders' (n = 47) implementation leadership and their clinics' implementation climate. Single-level structural equation models estimating both direct, indirect, and total effects were used to investigate whether perceived implementation climate mediated the association between perceived implementation leadership and perceived acceptability, appropriateness, and feasibility of screening tools and treatment methods. RESULTS Regarding the treatment methods, implementation leadership was associated with therapists' perceptions of acceptability, appropriateness, and feasibility. Implementation climate also mediated between implementation leadership and the outcomes. Regarding the screening tools, implementation leadership was not associated with the outcomes. However, implementation climate mediated between implementation leadership and therapists' perceptions of acceptability and feasibility, but not appropriateness. Analyses with the implementation climate subscales showed stronger associations for therapists' perceptions of the treatment methods than of screening tools. CONCLUSIONS Leaders may promote positive implementation outcomes, both directly and through implementation climate. With regard to the effect sizes and explained variance, results indicated that both implementation leadership and implementation climate were more strongly associated with the therapists' perceptions of the treatment methods, implemented by one group of therapists, than the screening tools, implemented by all therapists. This may imply that implementation leadership and climate may have stronger effects for smaller implementation teams within a larger system than for system-wide implementations or when the clinical interventions being implemented are more complex rather than simple ones. TRIAL REGISTRATION ClinicalTrials NCT03719651, 25 October 2018.
Collapse
Affiliation(s)
- Karina Myhren Egeland
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484, Oslo, Norway.
| | - Randi Hovden Borge
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484, Oslo, Norway
- National Institute of Occupational Health, Gydas vei 8, 0363, Oslo, Norway
| | - Nadina Peters
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484, Oslo, Norway
| | - Harald Bækkelund
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484, Oslo, Norway
| | - Nora Braathu
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484, Oslo, Norway
| | - Marisa Sklar
- Department of Psychiatry, University of California, San Diego, 9500 Gilman Drive (0812), La Jolla, CA, 92093-0812, USA
- Diego ACTRI Dissemination and Implementation Science Center, UC San, 9500 Gilman Drive, La Jolla, CA, 92093, USA
- Child and Adolescent Services Research Center, 3665 Kearny Villa Rd., Suite 200N, San Diego, CA, 92123, USA
| | - Gregory A Aarons
- Department of Psychiatry, University of California, San Diego, 9500 Gilman Drive (0812), La Jolla, CA, 92093-0812, USA
- Diego ACTRI Dissemination and Implementation Science Center, UC San, 9500 Gilman Drive, La Jolla, CA, 92093, USA
- Child and Adolescent Services Research Center, 3665 Kearny Villa Rd., Suite 200N, San Diego, CA, 92123, USA
| | - Ane-Marthe Solheim Skar
- Norwegian Centre for Violence and Traumatic Stress Studies (NKVTS), Gullhaugveien 1, 0484, Oslo, Norway
| |
Collapse
|
9
|
Choy-Brown M, Williams NJ, Ramirez N, Esp S. Psychometric evaluation of a pragmatic measure of clinical supervision as an implementation strategy. Implement Sci Commun 2023; 4:39. [PMID: 37024945 PMCID: PMC10080877 DOI: 10.1186/s43058-023-00419-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/25/2022] [Accepted: 03/16/2023] [Indexed: 04/08/2023] Open
Abstract
BACKGROUND Valid and reliable measurement of implementation strategies is essential to advancing implementation science; however, this area lags behind the measurement of implementation outcomes and determinants. Clinical supervision is a promising and highly feasible implementation strategy in behavioral healthcare for which pragmatic measures are lacking. This research aimed to develop and psychometrically evaluate a pragmatic measure of clinical supervision conceptualized in terms of two broadly applicable, discrete clinical supervision techniques shown to improve providers' implementation of evidence-based psychosocial interventions-(1) audit and feedback and (2) active learning. METHODS Items were generated based on a systematic review of the literature and administered to a sample of 154 outpatient mental health clinicians serving youth and 181 community-based mental health providers serving adults. Scores were evaluated for evidence of reliability, structural validity, construct-related validity, and measurement invariance across the two samples. RESULTS In sample 1, confirmatory factor analysis (CFA) supported the hypothesized two-factor structure of scores on the Evidence-Based Clinical Supervision Strategies (EBCSS) scale (χ2=5.89, df=4, p=0.208; RMSEA=0.055, CFI=0.988, SRMR=0.033). In sample 2, CFA replicated the EBCSS factor structure and provided discriminant validity evidence relative to an established supervisory alliance measure (χ2=36.12, df=30, p=0.204; RMSEA=0.034; CFI=0.990; SRMR=0.031). Construct-related validity evidence was provided by theoretically concordant associations between EBCSS subscale scores and agency climate for evidence-based practice implementation in sample 1 (d= .47 and .55) as well as measures of the supervision process in sample 2. Multiple group CFA supported the configural, metric, and partial scalar invariance of scores on the EBCSS across the two samples. CONCLUSIONS Scores on the EBCSS provide a valid basis for inferences regarding the extent to which behavioral health providers experience audit and feedback and active learning as part of their clinical supervision in both clinic- and community-based behavioral health settings. TRIAL REGISTRATION ClinicalTrials.gov NCT04096274 . Registered on 19 September 2019.
Collapse
Affiliation(s)
- Mimi Choy-Brown
- University of Minnesota, Twin Cities, 1404 Gortner Avenue, St. Paul, MN 55108 USA
| | - Nathaniel J. Williams
- Boise State University, 1910 University Drive, Education Suite 717, Boise, ID 83725-1940 USA
| | - Nallely Ramirez
- Boise State University, 1910 University Drive, Education Suite 717, Boise, ID 83725-1940 USA
| | - Susan Esp
- Boise State University, 1910 University Drive, Education Suite 717, Boise, ID 83725-1940 USA
| |
Collapse
|