1
|
Crable EL, Sklar M, Kandah A, Samuels HC, Ehrhart MG, Aalsma MC, Hulvershorn L, Willging CE, Aarons GA. Utility of the Leadership and Organizational Change for Implementation-Systems Level (LOCI-SL) strategy for a statewide substance use treatment implementation effort. JOURNAL OF SUBSTANCE USE AND ADDICTION TREATMENT 2024; 164:209433. [PMID: 38852821 DOI: 10.1016/j.josat.2024.209433] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Revised: 03/14/2024] [Accepted: 05/21/2024] [Indexed: 06/11/2024]
Abstract
INTRODUCTION Multi-level and cross-context implementation strategies are needed to support health systems, healthcare delivery organizations, and providers to adopt evidence-based practice (EBP) for substance use disorder (SUD) treatment. However, misalignment between state oversight agencies and healthcare organizations about which services to prioritize and which outcomes are reasonable to expect can hinder implementation success and widespread access to high-quality care. This study investigated the utility of the Leadership and Organizational Change for Implementation-System Level (LOCI-SL) strategy for supporting statewide EBP implementation for SUD treatment. METHODS Nine community mental health centers (CMHCs) contracted by a state agency participated in a combined motivational-enhancement therapy and cognitive behavioral therapy (MET/CBT) implementation effort. Five of the CMHCs also received the LOCI-SL strategy to obtain ongoing implementation support. We conducted 21 individual interviews and three small group interviews with 30 participants across CMHCs and state health agencies to investigate the utility of LOCI-SL in supporting their EBP implementation efforts. Deductive thematic analysis was guided by the Exploration, Preparation, Implementation, Sustainment Framework. RESULTS Five themes described CMHCs' LOCI-SL and broader contextual experiences implementing EBPs: (1) LOCI-SL supported executives in Preparation phase activities that holistically considered organizational needs and capacity to implement and sustain EBPs; (2) LOCI-SL facilitated trust and communication processes across Preparation, Implementation, and Sustainment phases to improve EBP uptake; (3) LOCI-SL increased CMHCs' use of implementation climate strengthening activities throughout the Implementation phase; (4) state contracts did not emphasize quality and thus were not sufficient bridging factors to enforce EBP fidelity during Implementation; and, (5) limited funding and low Medicaid reimbursement rates hindered EBP use throughout the Implementation and Sustainment phases. CONCLUSIONS LOCI-SL was viewed as a favorable and useful implementation strategy for supporting statewide adoption of EBPs. However, outer context barriers, including limited financial investments in the treatment system, impeded implementation and sustainment efforts. While previous research suggests that contracts are viable alignment-promoting bridging factors, this study demonstrates the importance of articulating implementation outcome expectations to aid state-contracted organizations in achieving EBP implementation success. This study also highlights the need for multi-level implementation strategies to effectively align implementation expectations between outer- and inner-context entities.
Collapse
Affiliation(s)
- Erika L Crable
- Department of Psychiatry, University of California San Diego, La Jolla, CA, USA; Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, La Jolla, CA, USA; ACTRI Dissemination and Implementation Science Center, University of California San Diego, La Jolla, USA; Child and Adolescent Services Research Center, San Diego, CA, USA.
| | - Marisa Sklar
- Department of Psychiatry, University of California San Diego, La Jolla, CA, USA; ACTRI Dissemination and Implementation Science Center, University of California San Diego, La Jolla, USA; Child and Adolescent Services Research Center, San Diego, CA, USA
| | - Alexandra Kandah
- Department of Psychology, University of Central Florida, Orlando, FL, USA
| | | | - Mark G Ehrhart
- Department of Psychology, University of Central Florida, Orlando, FL, USA
| | - Matthew C Aalsma
- Adolescent Behavioral Health Research Program, Indiana University School of Medicine, Indianapolis, IN, USA; Indiana Clinical and Translational Sciences Institute, Indianapolis, IN, USA
| | - Leslie Hulvershorn
- Adolescent Behavioral Health Research Program, Indiana University School of Medicine, Indianapolis, IN, USA; Indiana Clinical and Translational Sciences Institute, Indianapolis, IN, USA
| | - Cathleen E Willging
- Southwest Center, Pacific Institute for Research and Evaluation, Albuquerque, NM, USA
| | - Gregory A Aarons
- Department of Psychiatry, University of California San Diego, La Jolla, CA, USA; ACTRI Dissemination and Implementation Science Center, University of California San Diego, La Jolla, USA; Child and Adolescent Services Research Center, San Diego, CA, USA
| |
Collapse
|
2
|
Layman DM, Kammer J, Leckman-Westin E, Hogan M, Goldstein Grumet J, Labouliere CD, Stanley B, Carruthers J, Finnerty M. The Relationship Between Suicidal Behaviors and Zero Suicide Organizational Best Practices in Outpatient Mental Health Clinics. Psychiatr Serv 2021; 72:1118-1125. [PMID: 33730886 PMCID: PMC8853689 DOI: 10.1176/appi.ps.202000525] [Citation(s) in RCA: 14] [Impact Index Per Article: 4.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
Abstract
OBJECTIVE This study tested the hypothesis that fidelity of clinics to Zero Suicide (ZS) organizational practices is inversely related to suicidal behaviors of patients under clinical care. METHODS Using cross-sectional analyses, the authors examined the fidelity of 110 outpatient mental health clinics to ZS organizational best practices and suicidal behaviors of clinic patients in the year before a large-scale Zero Suicide implementation. Fidelity to ZS organizational best practices was assessed over a 1-year period with an adapted version of the ZS Organizational Self-Study instrument (17 items self-rated on a Likert scale of 1-5). Suicidal behaviors of patients were identified by extracting information on suicide attempts and deaths from a mandated statewide incident-reporting system database. Clinics were dichotomized into any or no suicide incidents during the year of observation. Logistic regression analyses were used to adjust for clinic census and population type (majority child or adult). RESULTS The clinics (N=110) served 30,257 patients per week. Clinics' total average fidelity score was 3.1±0.6 (range=1.41-4.12). For each point increase in fidelity, clinics had a significantly reduced likelihood of having a suicide incident (adjusted odds ratio=0.31, 95% confidence interval=0.14-0.69). Exploratory analysis identified significant differences for seven of 17 ZS organizational practices, with the largest effect sizes for suicide-specific quality improvement policies and activities (η2=0.097) and lethal means reduction (η2=0.073). CONCLUSIONS These findings support an association between clinics' use of ZS organizational best practices and lower suicidal behaviors of patients under their care. Findings also support the validity of the ZS Organizational Self-Study instrument.
Collapse
Affiliation(s)
- Deborah M Layman
- New York State Office of Mental Health, New York City (Layman, Finnerty) and Albany (Kammer, Leckman-Westin, Carruthers); Department of Psychiatry, Case Western Reserve University School of Medicine, Cleveland (Hogan); Education Development Center, Waltham, Massachusetts (Goldstein Grumet); New York State Psychiatric Institute and Columbia University, New York City (Labouliere, Stanley)
| | - Jamie Kammer
- New York State Office of Mental Health, New York City (Layman, Finnerty) and Albany (Kammer, Leckman-Westin, Carruthers); Department of Psychiatry, Case Western Reserve University School of Medicine, Cleveland (Hogan); Education Development Center, Waltham, Massachusetts (Goldstein Grumet); New York State Psychiatric Institute and Columbia University, New York City (Labouliere, Stanley)
| | - Emily Leckman-Westin
- New York State Office of Mental Health, New York City (Layman, Finnerty) and Albany (Kammer, Leckman-Westin, Carruthers); Department of Psychiatry, Case Western Reserve University School of Medicine, Cleveland (Hogan); Education Development Center, Waltham, Massachusetts (Goldstein Grumet); New York State Psychiatric Institute and Columbia University, New York City (Labouliere, Stanley)
| | - Mike Hogan
- New York State Office of Mental Health, New York City (Layman, Finnerty) and Albany (Kammer, Leckman-Westin, Carruthers); Department of Psychiatry, Case Western Reserve University School of Medicine, Cleveland (Hogan); Education Development Center, Waltham, Massachusetts (Goldstein Grumet); New York State Psychiatric Institute and Columbia University, New York City (Labouliere, Stanley)
| | - Julie Goldstein Grumet
- New York State Office of Mental Health, New York City (Layman, Finnerty) and Albany (Kammer, Leckman-Westin, Carruthers); Department of Psychiatry, Case Western Reserve University School of Medicine, Cleveland (Hogan); Education Development Center, Waltham, Massachusetts (Goldstein Grumet); New York State Psychiatric Institute and Columbia University, New York City (Labouliere, Stanley)
| | - Christa D Labouliere
- New York State Office of Mental Health, New York City (Layman, Finnerty) and Albany (Kammer, Leckman-Westin, Carruthers); Department of Psychiatry, Case Western Reserve University School of Medicine, Cleveland (Hogan); Education Development Center, Waltham, Massachusetts (Goldstein Grumet); New York State Psychiatric Institute and Columbia University, New York City (Labouliere, Stanley)
| | - Barbara Stanley
- New York State Office of Mental Health, New York City (Layman, Finnerty) and Albany (Kammer, Leckman-Westin, Carruthers); Department of Psychiatry, Case Western Reserve University School of Medicine, Cleveland (Hogan); Education Development Center, Waltham, Massachusetts (Goldstein Grumet); New York State Psychiatric Institute and Columbia University, New York City (Labouliere, Stanley)
| | - Jay Carruthers
- New York State Office of Mental Health, New York City (Layman, Finnerty) and Albany (Kammer, Leckman-Westin, Carruthers); Department of Psychiatry, Case Western Reserve University School of Medicine, Cleveland (Hogan); Education Development Center, Waltham, Massachusetts (Goldstein Grumet); New York State Psychiatric Institute and Columbia University, New York City (Labouliere, Stanley)
| | - Molly Finnerty
- New York State Office of Mental Health, New York City (Layman, Finnerty) and Albany (Kammer, Leckman-Westin, Carruthers); Department of Psychiatry, Case Western Reserve University School of Medicine, Cleveland (Hogan); Education Development Center, Waltham, Massachusetts (Goldstein Grumet); New York State Psychiatric Institute and Columbia University, New York City (Labouliere, Stanley)
| |
Collapse
|
3
|
Selick A, Langill G, Cheng C, Addington D, Archie S, Butt S, Durbin J. Feasibility and acceptability of a volunteer peer fidelity assessment model in early psychosis intervention programmes in Ontario: Results from a pilot study. Early Interv Psychiatry 2021; 15:480-485. [PMID: 32232930 DOI: 10.1111/eip.12961] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/11/2019] [Revised: 01/14/2020] [Accepted: 03/15/2020] [Indexed: 10/24/2022]
Abstract
AIM Fidelity monitoring can support high-quality service delivery but is resource-intensive to implement. A fidelity assessment model utilizing volunteer assessors was trialled as a low-cost strategy for conducting fidelity assessments. This article reports on the acceptability and feasibility of this model. METHODS Twenty volunteer assessors were trained to conduct fidelity assessments in nine Early Psychosis Intervention programmes across Ontario, Canada. Assessments were conducted using the First-Episode Psychosis Services Fidelity Scale based on a 2-day site visit, during which assessors interviewed staff, clients and families; reviewed charts; observed a team meeting and reviewed programme materials. The model was evaluated based on assessor focus groups, programme interviews, consensus meeting data and time-tracking logs. General inductive analysis was used to code and synthesize qualitative data. Quantitative data were aggregated and summarized. RESULTS Participant feedback was positive and indicated that use of peer assessors and the in-person site visit added value to the process. The model was perceived to provide valuable information to support internal quality improvement efforts. Assessors reported direct benefits from participating, including networking and learning opportunities. Key challenges were the high time demand on assessors and turnover in the assessor team. CONCLUSIONS The volunteer peer fidelity model was perceived to be a valuable improvement process by participants, but the high cost and reliance on ongoing volunteerism makes its sustainability uncertain. Next steps may include exploring remote assessment strategies or direct payments, although these strategies risk reducing the acceptability, and therefore uptake, of the assessment.
Collapse
Affiliation(s)
- Avra Selick
- Provincial System Support Program, Centre for Addiction and Mental Health, Toronto, Ontario, Canada.,Institute for Health Policy Management and Evaluation, University of Toronto, Toronto, Ontario, Canada
| | - Gordon Langill
- Canadian Mental Health Association, Haliburton Kawartha Pine Ridge Branch, Ontario, Canada
| | - Chiachen Cheng
- Department of Psychiatry, Northern School of Medicine, Thunder Bay, Ontario, Canada
| | - Donald Addington
- Department of Psychiatry, University of Calgary, Calgary, Alberta, Canada
| | - Suzanne Archie
- Department of Psychiatry and Behavioural Neurosciences, McMaster University, Hamilton, Ontario, Canada
| | - Shannel Butt
- Canadian Mental Health Association, Elgin Branch, St Thomas, Ontario, Canada
| | - Janet Durbin
- Provincial System Support Program, Centre for Addiction and Mental Health, Toronto, Ontario, Canada.,Department of Psychiatry, University of Toronto, Toronto, Ontario, Canada
| |
Collapse
|
4
|
Bowser DM, Henry BF, McCollister KE. Cost analysis in implementation studies of evidence-based practices for mental health and substance use disorders: a systematic review. Implement Sci 2021; 16:26. [PMID: 33706780 PMCID: PMC7953634 DOI: 10.1186/s13012-021-01094-3] [Citation(s) in RCA: 20] [Impact Index Per Article: 6.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/21/2020] [Accepted: 02/22/2021] [Indexed: 12/02/2022] Open
Abstract
Background This study is a systematic literature review of cost analyses conducted within implementation studies on behavioral health services. Cost analysis of implementing evidence-based practices (EBP) has become important within implementation science and is critical for bridging the research to practice gap to improve access to quality healthcare services. Costing studies in this area are rare but necessary since cost can be a barrier to implementation and sustainment of EBP. Methods We followed Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology and applied the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist. Key search terms included: (1) economics, (2) implementation, (3) EBP, and (4) behavioral health. Terms were searched within article title and abstracts in: EconLit, SocINDEX, Medline, and PsychINFO. A total of 464 abstracts were screened independently by two authors and reduced to 37 articles using inclusion and exclusion criteria. After a full-text review, 18 articles were included. Results Findings were used to classify costs into direct implementation, direct services, and indirect implementation. While all studies included phases of implementation as part of their design, only five studies examined resources across multiple phases of an implementation framework. Most studies reported direct service costs associated with adopting a new practice, usually summarized as total EBP cost, cost per client, cost per clinician, and/or cost per agency. For studies with detailed analysis, there were eleven direct cost categories represented. For five studies that reported costs per child served, direct implementation costs varied from $886 to $9470 per child, while indirect implementation costs ranged from $897 to $3805 per child. Conclusions This is the first systematic literature review to examine costs of implementing EBP in behavioral healthcare settings. Since 2000, 18 studies were identified that included a cost analysis. Given a wide variation in the study designs and economic methods, comparison across studies was challenging, which is a major limitation in the field, as it becomes difficult to replicate studies or to estimate future costs to inform policy decisions related to budgeting. We recommend future economic implementation studies to consider standard economic costing methods capturing costs across implementation framework phases to support comparisons and replicability.
Collapse
Affiliation(s)
- Diana M Bowser
- Heller School for Social Policy and Management, Brandeis University, 415 South St, Waltham, MA, 02453, USA
| | - Brandy F Henry
- Heller School for Social Policy and Management, Brandeis University, 415 South St, Waltham, MA, 02453, USA. .,School of Social Work, Columbia University, 1255 Amsterdam Ave, New York, NY, 10027, USA.
| | - Kathryn E McCollister
- Department of Public Health Sciences, University of Miami Miller School of Medicine, 1120 NW 14th St., CRB 1019, Miami, FL, 33136, USA
| |
Collapse
|
5
|
Bond GR, Drake RE. Assessing the Fidelity of Evidence-Based Practices: History and Current Status of a Standardized Measurement Methodology. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2019; 47:874-884. [PMID: 31691055 DOI: 10.1007/s10488-019-00991-6] [Citation(s) in RCA: 56] [Impact Index Per Article: 11.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/27/2022]
Abstract
Evidence-based practices are effective only when implemented faithfully. This paper explicates the history, standardization, and methods for developing and validating measures of fidelity. We overviewed the past 20 years of developing fidelity measures, summarized standardization of the development procedures, and described needed psychometric assessments. Fidelity assessment has become the sine qua non of implementation, technical assistance, and research on evidence-based practices. Researchers have established standardized procedures for scale development and psychometric testing. Widescale use of fidelity measurement remains challenging. The implementation of evidence-based practice and the development and validation of fidelity measures are interdependent. International improvements of mental health care will require attention to both.
Collapse
Affiliation(s)
- Gary R Bond
- Westat, Rivermill Commercial Center, 85 Mechanic Street, Suite C3-1, Lebanon, NH, 03766, USA.
| | - Robert E Drake
- Westat, Rivermill Commercial Center, 85 Mechanic Street, Suite C3-1, Lebanon, NH, 03766, USA
| |
Collapse
|