1
|
Pickard KE, Hendrix NM, Greenfield ES, Yohannes M. Using Causative Methods to Determine System-Level Factors Driving the Uptake and Use of Evidence-Based Practices in a Public Early Intervention System. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2024; 51:554-566. [PMID: 38431889 DOI: 10.1007/s10488-024-01346-6] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/18/2024] [Indexed: 03/05/2024]
Abstract
Part C Early Intervention (EI) systems are an entry point to services for autistic toddlers and can be leveraged to facilitate access to autism evidence-based practices (EBPs). However, EI systems are complex and limited research has examined how an EI system's infrastructure (i.e. system-level factors) impacts the adoption and implementation of EBPs. To address this gap, 36 EI providers and 9 EI administrators completed a semi-structured interview or focus group about factors impacting the implementation of autism EBPs. Qualitative analysis included a combination of grounded theory and causative coding. Analyses were refined by input from providers, administrators, and family stakeholders in the form of round tables and presentations at the state's interagency coordinating council. Primary themes centered on: (1) the costs associated with independent contracting structures; (2) operational demands; (3) workforce stability; (4) communication consistency; and (5) implementation supports for EBP implementation. Causative coding helped to demonstrate the perceived relationships between these factors and underscored the important role of incentivization structures, collaboration opportunities, and championing in supporting the use of EBPs within a system that primarily uses independent contracting structures. The current study extends previous research by demonstrating how several system-level factors are perceived to play a role in the adoption and implementation of EBPs by independently contracted EI providers. These findings underscore the need for implementation strategies, such as incentivization strategies and social network building, to increase providers' implementation of autism EBPs within EI systems.
Collapse
Affiliation(s)
- Katherine E Pickard
- Department of Pediatrics, Division of Autism and Related Disabilities, Emory University School of Medicine, Atlanta, USA.
- Marcus Autism Center, Children's Healthcare of Atlanta, Atlanta, USA.
| | - Nicole M Hendrix
- Department of Pediatrics, Division of Autism and Related Disabilities, Emory University School of Medicine, Atlanta, USA
- Marcus Autism Center, Children's Healthcare of Atlanta, Atlanta, USA
| | - Elizabeth S Greenfield
- Department of Pediatrics, Division of Autism and Related Disabilities, Emory University School of Medicine, Atlanta, USA
- Marcus Autism Center, Children's Healthcare of Atlanta, Atlanta, USA
| | - Millena Yohannes
- Department of Pediatrics, Division of Autism and Related Disabilities, Emory University School of Medicine, Atlanta, USA
- Marcus Autism Center, Children's Healthcare of Atlanta, Atlanta, USA
| |
Collapse
|
2
|
Aarons GA, Sklar M, Ehrhart MG, Roesch S, Moullin JC, Carandang K. Randomized trial of the leadership and organizational change for implementation (LOCI) strategy in substance use treatment clinics. JOURNAL OF SUBSTANCE USE AND ADDICTION TREATMENT 2024:209437. [PMID: 38866139 DOI: 10.1016/j.josat.2024.209437] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/07/2024] [Revised: 04/28/2024] [Accepted: 05/21/2024] [Indexed: 06/14/2024]
Abstract
INTRODUCTION Evidence-based practice (EBP) implementation represents a strategic change that requires alignment of leadership and support throughout organizations. Leadership and Organizational Change for Implementation (LOCI) is a multifaceted implementation strategy that aims to improve implementation leadership and climate within organizations through iterative cycles of leadership and climate assessment and feedback, leadership training and coaching, and strategic planning with upper-level leaders. This study tested the effects of LOCI on transformational and implementation leadership, implementation climate, implementation citizenship behavior, and EBP reach. METHODS A multiple cohort, cluster randomized trial tests the effect of LOCI in 60 clinics across nine behavioral health organizations in California and Arizona, USA. The study randomized clinics within organizations to either LOCI or a leadership training webinar control condition in three consecutive cohorts. Repeated web-based surveys of direct service providers (nLOCI = 201, nControl = 179) assessed leadership, implementation climate, and implementation citizenship over time. Multilevel autoregressive modeling was the primary statistical analysis such that providers (level-1) were nested within clinics (level-2). The study predicted between-condition differences at 4-, 8-, and 12-month follow-up assessments. Provider engagement in a fidelity monitoring process assessed reach of motivational interviewing (i.e., number of sessions recorded/submitted for fidelity coding). An independent sample t-test explored between condition differences in motivational interviewing reach. RESULTS Results indicated between condition differences at 4 months for implementation leadership, implementation climate, and implementation citizenship behavior such that greater improvements were evidenced in the LOCI condition compared to the control condition. Reach of MI was significantly greater in the LOCI vs control condition such that LOCI providers were significantly more likely to engage in the fidelity monitoring process (chi-square (1, n = 370) = 5.59, p = .018). CONCLUSIONS LOCI was developed based on organizational theories of strategic leadership and climate to affect organizational change processes that communicate that innovation implementation is expected, supported, and recognized as a value of the organization. The LOCI implementation strategy resulted in more positive hypothesized outcomes compared to the control condition. Organizational change strategies have utility for implementing health innovations in complex, multilevel contexts and for greater sustainment of facilitative leader behaviors, strategic implementation climate, and improved implementation outcomes. TRIAL REGISTRATION This study is registered with Clinicaltrials.gov gov (NCT03042832, 2 February 2017; retrospectively registered).
Collapse
Affiliation(s)
- Gregory A Aarons
- University of California San Diego, Department of Psychiatry, 9500 Gilman Drive (0812), La Jolla, CA 92093-0812, United States of America; UC San Diego Altman Clinical and Translational Research Institute Dissemination and Implementation Science Center, 9500 Gilman Drive, La Jolla, CA 92093, United States of America; Child and Adolescent Services Research Center, 3665 Kearny Villa Rd., Suite 200N, San Diego, CA 92123, United States of America.
| | - Marisa Sklar
- University of California San Diego, Department of Psychiatry, 9500 Gilman Drive (0812), La Jolla, CA 92093-0812, United States of America; UC San Diego Altman Clinical and Translational Research Institute Dissemination and Implementation Science Center, 9500 Gilman Drive, La Jolla, CA 92093, United States of America; Child and Adolescent Services Research Center, 3665 Kearny Villa Rd., Suite 200N, San Diego, CA 92123, United States of America.
| | - Mark G Ehrhart
- University of Central Florida, Department of Psychology, United States of America.
| | - Scott Roesch
- San Diego State University, Department of Psychology, 5500 Campanile Drive, San Diego, CA 92182-4611, United States of America.
| | - Joanna C Moullin
- Curtin University, School of Population Health, Kent Street, Bentley, Western Australia 6102, Australia
| | - Kristine Carandang
- University of California San Diego, Department of Psychiatry, 9500 Gilman Drive (0812), La Jolla, CA 92093-0812, United States of America; Child and Adolescent Services Research Center, 3665 Kearny Villa Rd., Suite 200N, San Diego, CA 92123, United States of America
| |
Collapse
|
3
|
Crable EL, Sklar M, Kandah A, Samuels HC, Ehrhart MG, Aalsma MC, Hulvershorn L, Willging CE, Aarons GA. Utility of the Leadership and Organizational Change for Implementation-Systems Level (LOCI-SL) strategy for a statewide substance use treatment implementation effort. JOURNAL OF SUBSTANCE USE AND ADDICTION TREATMENT 2024; 164:209433. [PMID: 38852821 DOI: 10.1016/j.josat.2024.209433] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/17/2024] [Revised: 03/14/2024] [Accepted: 05/21/2024] [Indexed: 06/11/2024]
Abstract
INTRODUCTION Multi-level and cross-context implementation strategies are needed to support health systems, healthcare delivery organizations, and providers to adopt evidence-based practice (EBP) for substance use disorder (SUD) treatment. However, misalignment between state oversight agencies and healthcare organizations about which services to prioritize and which outcomes are reasonable to expect can hinder implementation success and widespread access to high-quality care. This study investigated the utility of the Leadership and Organizational Change for Implementation-System Level (LOCI-SL) strategy for supporting statewide EBP implementation for SUD treatment. METHODS Nine community mental health centers (CMHCs) contracted by a state agency participated in a combined motivational-enhancement therapy and cognitive behavioral therapy (MET/CBT) implementation effort. Five of the CMHCs also received the LOCI-SL strategy to obtain ongoing implementation support. We conducted 21 individual interviews and three small group interviews with 30 participants across CMHCs and state health agencies to investigate the utility of LOCI-SL in supporting their EBP implementation efforts. Deductive thematic analysis was guided by the Exploration, Preparation, Implementation, Sustainment Framework. RESULTS Five themes described CMHCs' LOCI-SL and broader contextual experiences implementing EBPs: (1) LOCI-SL supported executives in Preparation phase activities that holistically considered organizational needs and capacity to implement and sustain EBPs; (2) LOCI-SL facilitated trust and communication processes across Preparation, Implementation, and Sustainment phases to improve EBP uptake; (3) LOCI-SL increased CMHCs' use of implementation climate strengthening activities throughout the Implementation phase; (4) state contracts did not emphasize quality and thus were not sufficient bridging factors to enforce EBP fidelity during Implementation; and, (5) limited funding and low Medicaid reimbursement rates hindered EBP use throughout the Implementation and Sustainment phases. CONCLUSIONS LOCI-SL was viewed as a favorable and useful implementation strategy for supporting statewide adoption of EBPs. However, outer context barriers, including limited financial investments in the treatment system, impeded implementation and sustainment efforts. While previous research suggests that contracts are viable alignment-promoting bridging factors, this study demonstrates the importance of articulating implementation outcome expectations to aid state-contracted organizations in achieving EBP implementation success. This study also highlights the need for multi-level implementation strategies to effectively align implementation expectations between outer- and inner-context entities.
Collapse
Affiliation(s)
- Erika L Crable
- Department of Psychiatry, University of California San Diego, La Jolla, CA, USA; Herbert Wertheim School of Public Health and Human Longevity Science, University of California San Diego, La Jolla, CA, USA; ACTRI Dissemination and Implementation Science Center, University of California San Diego, La Jolla, USA; Child and Adolescent Services Research Center, San Diego, CA, USA.
| | - Marisa Sklar
- Department of Psychiatry, University of California San Diego, La Jolla, CA, USA; ACTRI Dissemination and Implementation Science Center, University of California San Diego, La Jolla, USA; Child and Adolescent Services Research Center, San Diego, CA, USA
| | - Alexandra Kandah
- Department of Psychology, University of Central Florida, Orlando, FL, USA
| | | | - Mark G Ehrhart
- Department of Psychology, University of Central Florida, Orlando, FL, USA
| | - Matthew C Aalsma
- Adolescent Behavioral Health Research Program, Indiana University School of Medicine, Indianapolis, IN, USA; Indiana Clinical and Translational Sciences Institute, Indianapolis, IN, USA
| | - Leslie Hulvershorn
- Adolescent Behavioral Health Research Program, Indiana University School of Medicine, Indianapolis, IN, USA; Indiana Clinical and Translational Sciences Institute, Indianapolis, IN, USA
| | - Cathleen E Willging
- Southwest Center, Pacific Institute for Research and Evaluation, Albuquerque, NM, USA
| | - Gregory A Aarons
- Department of Psychiatry, University of California San Diego, La Jolla, CA, USA; ACTRI Dissemination and Implementation Science Center, University of California San Diego, La Jolla, USA; Child and Adolescent Services Research Center, San Diego, CA, USA
| |
Collapse
|
4
|
Shattuck D, Willging CE, Peterson J, Ramos MM. Outer-context determinants on the implementation of school-based interventions for LGBTQ+ adolescents. IMPLEMENTATION RESEARCH AND PRACTICE 2024; 5:26334895241249417. [PMID: 38666140 PMCID: PMC11044576 DOI: 10.1177/26334895241249417] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/28/2024] Open
Abstract
Background Schools are critical venues for supporting LGBTQ+ youth well-being. Implementing LGBTQ-supportive practices can decrease experiences of stigmatization, discrimination, and victimization that lead to adverse mental health outcomes like anxiety, depression, and suicidality. However, schools are also subject to a wide range of outer-context pressures that may influence their priorities and implementation of LGBTQ-supportive practices. We assessed the role of emergent outer-context determinants in the context of a 5-year cluster randomized controlled trial to study the implementation of LGBTQ-supportive evidence-informed practices (EIPs) in New Mexico high schools. Method Using an iterative coding approach, we analyzed qualitative data from annual interviews with school professionals involved in EIP implementation efforts. Results The analysis yielded three categories of outer-context determinants that created challenges and opportunities for implementation: (a) social barriers related to heterocentrism, cisgenderism, and religious conservatism; (b) local, state, and national policy and political discourse; and (c) crisis events. Conclusions By exploring the implications of outer-context determinants for the uptake of LGBTQ-supportive practices, we demonstrate that these elements are dynamic-not simply reducible to barriers or facilitators-and that assessing outer-context determinants shaping implementation environments is crucial for addressing LGBTQ health equity.
Collapse
Affiliation(s)
- Daniel Shattuck
- Pacific Institute for Research and Evaluation (PIRE)—Southwest Center, Albuquerque, NM, USA
| | - Cathleen E. Willging
- Pacific Institute for Research and Evaluation (PIRE)—Southwest Center, Albuquerque, NM, USA
| | - Jeffery Peterson
- School of Public and Community Health Services, University of Montana, Missoula, Montana, USA
| | - Mary M. Ramos
- Department of Pediatrics, University of New Mexico School of Medicine, Albuquerque, NM, USA
| |
Collapse
|
5
|
Wortham WK, Rodwin AH, Purtle J, Munson MR, Raghavan R. Revisiting the policy ecology framework for implementation of evidence-based practices in mental health settings. Implement Sci 2023; 18:58. [PMID: 37936123 PMCID: PMC10629012 DOI: 10.1186/s13012-023-01309-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2023] [Accepted: 10/01/2023] [Indexed: 11/09/2023] Open
Abstract
BACKGROUND Over the past three decades, policy actors and actions have been highly influential in supporting the implementation of evidence-based practices (EBPs) in mental health settings. An early examination of these actions resulted in the Policy Ecology Framework (PEF), which was originally developed as a tactical primer for state and local mental health regulators in the field of child mental health. However, the policy landscape for implementation has evolved significantly since the original PEF was published. An interrogation of the strategies originally proposed in the PEF is necessary to provide an updated menu of strategies to improve our understanding of the mechanisms of policy action and promote system improvement. OBJECTIVES This paper builds upon the original PEF to address changes in the policy landscape for the implementation of mental health EBPs between 2009 and 2022. We review the current state of policy strategies that support the implementation of EBPs in mental health care and outline key areas for policy-oriented implementation research. Our review identifies policy strategies at federal, state, agency, and organizational levels, and highlights developments in the social context in which EBPs are implemented. Furthermore, our review is organized around some key changes that occurred across each PEF domain that span organizational, agency, political, and social contexts along with subdomains within each area. DISCUSSION We present an updated menu of policy strategies to support the implementation of EBPs in mental health settings. This updated menu of strategies considers the broad range of conceptual developments and changes in the policy landscape. These developments have occurred across the organizational, agency, political, and social contexts and are important for policymakers to consider in the context of supporting the implementation of EBPs. The updated PEF expands and enhances the specification of policy levers currently available, and identifies policy targets that are underdeveloped (e.g., de-implementation and sustainment) but are becoming visible opportunities for policy to support system improvement. The updated PEF clarifies current policy efforts within the field of implementation science in health to conceptualize and better operationalize the role of policy in the implementation of EBPs.
Collapse
Affiliation(s)
- Whitney K Wortham
- Silver School of Social Work, New York University, 1 Washington Square North, New York, NY, 10003, USA.
| | - Aaron H Rodwin
- Silver School of Social Work, New York University, 1 Washington Square North, New York, NY, 10003, USA
| | - Jonathan Purtle
- Department of Public Health Policy & Management, Global Center for Implementation Science, School of Global Public Health, New York University, 708 Broadway, New York, NY, 10003, USA
| | - Michelle R Munson
- Silver School of Social Work, New York University, 1 Washington Square North, New York, NY, 10003, USA
| | - Ramesh Raghavan
- Silver School of Social Work, New York University, 1 Washington Square North, New York, NY, 10003, USA
| |
Collapse
|
6
|
Naar S, Fernandez MI, Todd L, Green SKS, Budhwani H, Carcone A, Coyle K, Aarons GA, MacDonell K, Harper GW. Understanding implementation completion of tailored motivational interviewing in multidisciplinary adolescent HIV clinics. IMPLEMENTATION RESEARCH AND PRACTICE 2023; 4:26334895231164585. [PMID: 37091536 PMCID: PMC10068499 DOI: 10.1177/26334895231164585] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/25/2023] Open
Abstract
Background Understanding the barriers and facilitators of implementation completion is critical to determining why some implementation efforts fail and some succeed. Such studies provide the foundation for developing further strategies to support implementation completion when scaling up evidence-based practices (EBPs) such as Motivational Interviewing. Method This mixed-methods study utilized the Exploration, Preparation, Implementation, and Sustainment framework in an iterative analytic design to compare adolescent HIV clinics that demonstrated either high or low implementation completion in the context of a hybrid Type III trial of tailored motivational interviewing. Ten clinics were assigned to one of three completion categories (high, medium, and low) based on percentage of staff who adhered to three components of implementation strategies. Comparative analysis of staff qualitative interviews compared and contrasted the three high-completion clinics with the three low-completion clinics. Results Results suggested several factors that distinguished high-completion clinics compared to low-completion clinics including optimism, problem-solving barriers, leadership, and staff stress and turnover. Conclusions Implementation strategies targeting these factors can be added to EBP implementation packages to improve implementation success. Plain Language Summary While studies have begun to address adherence to intervention techniques, this is one of the first studies to address organizational adherence to implementation strategies. Youth HIV providers from different disciplines completed interviews about critical factors in both the inner and outer context that can support or hinder an organization's adherence to implementation strategies. Compared to less adherent clinics, more adherent clinics reported more optimism, problem-solving, and leadership strengths and less staff stress and turnover. Implementation strategies addressing these factors could be added to implementation packages to improve implementation success.
Collapse
Affiliation(s)
- Sylvie Naar
- Center for Translational Behavioral Science, Florida State University, Tallahassee, FL, USA
| | - M. Isabel Fernandez
- Kiran C. Patel College of Osteopathic Medicine, Nova Southeastern
University, Fort Lauderdale, FL, USA
| | - Lisa Todd
- Department of Family Medicine and Public Health Sciences, Wayne State University School of
Medicine, Detroit, MI, USA
| | - Sara K. Shaw Green
- Center for Translational Behavioral Science, Florida State University, Tallahassee, FL, USA
| | - Henna Budhwani
- Department of Health Care Organization and Policy, School of Public
Health, University of Alabama at Birmingham (UAB), Birmingham, AL, USA
| | - April Carcone
- Department of Family Medicine and Public Health Sciences, Wayne State University School of
Medicine, Detroit, MI, USA
| | | | - Gregory A. Aarons
- Department of Psychiatry, UC San Diego, La Jolla, CA, USA
- ACTRI Dissemination and Implementation Science Center, UC San Diego,
La Jolla, CA, USA
- Child and Adolescent Services Research
Center, San Diego, CA, USA
| | - Karen MacDonell
- Department of Family Medicine and Public Health Sciences, Wayne State University School of
Medicine, Detroit, MI, USA
| | - Gary W. Harper
- Department of Health Behavior and Health Education, School of Public
Health, University of Michigan School of Public
Health, Ann Arbor, MI, USA
| |
Collapse
|
7
|
A M, K LBC, E S, S C, P F. A protocol for a multi-site cohort study to evaluate child and adolescent mental health service transformation in England using the i-THRIVE model. PLoS One 2023; 18:e0265782. [PMID: 37155627 PMCID: PMC10166497 DOI: 10.1371/journal.pone.0265782] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/03/2022] [Accepted: 03/14/2023] [Indexed: 05/10/2023] Open
Abstract
The National i-THRIVE Programme seeks to evaluate the impact of the NHS England-funded whole system transformation on child and adolescent mental health services (CAMHS). This article reports on the design for a model of implementation that has been applied in CAMHS across over 70 areas in England using the 'THRIVE' needs-based principles of care. The implementation protocol in which this model, 'i-THRIVE' (implementing-THRIVE), will be used to evaluate the effectiveness of the THRIVE intervention is reported, together with the evaluation protocol for the process of implementation. To evaluate the effectiveness of i-THRIVE to improve care for children and young people's mental health, a cohort study design will be conducted. N = 10 CAMHS sites that adopt the i-THRIVE model from the start of the NHS England-funded CAMHS transformation will be compared to N = 10 'comparator sites' that choose to use different transformation approaches within the same timeframe. Sites will be matched on population size, urbanicity, funding, level of deprivation and expected prevalence of mental health care needs. To evaluate the process of implementation, a mixed-methods approach will be conducted to explore the moderating effects of context, fidelity, dose, pathway structure and reach on clinical and service level outcomes. This study addresses a unique opportunity to inform the ongoing national transformation of CAMHS with evidence about a popular new model for delivering children and young people's mental health care, as well as a new implementation approach to support whole system transformation. If the outcomes reflect benefit from i-THRIVE, this study has the potential to guide significant improvements in CAMHS by providing a more integrated, needs-led service model that increases access and involvement of patients with services and in the care they receive.
Collapse
Affiliation(s)
- Moore A
- The Anna Freud National Centre for Children and Families, London, United Kingdom
- Psychoanalysis Unit, Division of Psychology and Language Sciences, University College London, London, United Kingdom
- Department of Psychiatry, University of Cambridge, Cambridge, United Kingdom
| | - Lindley Baron-Cohen K
- Psychoanalysis Unit, Division of Psychology and Language Sciences, University College London, London, United Kingdom
| | - Simes E
- The Anna Freud National Centre for Children and Families, London, United Kingdom
- Psychoanalysis Unit, Division of Psychology and Language Sciences, University College London, London, United Kingdom
| | - Chen S
- Department of Psychiatry, University of Cambridge, Cambridge, United Kingdom
| | - Fonagy P
- The Anna Freud National Centre for Children and Families, London, United Kingdom
- Psychoanalysis Unit, Division of Psychology and Language Sciences, University College London, London, United Kingdom
| |
Collapse
|
8
|
Crable EL, Lengnick-Hall R, Stadnick NA, Moullin JC, Aarons GA. Where is "policy" in dissemination and implementation science? Recommendations to advance theories, models, and frameworks: EPIS as a case example. Implement Sci 2022; 17:80. [PMID: 36503520 PMCID: PMC9742035 DOI: 10.1186/s13012-022-01256-x] [Citation(s) in RCA: 28] [Impact Index Per Article: 14.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 08/25/2022] [Accepted: 12/01/2022] [Indexed: 12/14/2022] Open
Abstract
BACKGROUND Implementation science aims to accelerate the public health impact of evidence-based interventions. However, implementation science has had too little focus on the role of health policy - and its inseparable politics, polity structures, and policymakers - in the implementation and sustainment of evidence-based healthcare. Policies can serve as determinants, implementation strategies, the evidence-based "thing" to be implemented, or another variable in the causal pathway to healthcare access, quality, and patient outcomes. Research describing the roles of policy in dissemination and implementation (D&I) efforts is needed to resolve persistent knowledge gaps about policymakers' evidence use, how evidence-based policies are implemented and sustained, and methods to de-implement policies that are ineffective or cause harm. Few D&I theories, models, or frameworks (TMF) explicitly guide researchers in conceptualizing where, how, and when policy should be empirically investigated. We conducted and reflected on the results of a scoping review to identify gaps of existing Exploration, Preparation, Implementation, and Sustainment (EPIS) framework-guided policy D&I studies. We argue that rather than creating new TMF, researchers should optimize existing TMF to examine policy's role in D&I. We describe six recommendations to help researchers optimize existing D&I TMF. Recommendations are applied to EPIS, as one example for advancing TMF for policy D&I. RECOMMENDATIONS (1) Specify dimensions of a policy's function (policy goals, type, contexts, capital exchanged). (2) Specify dimensions of a policy's form (origin, structure, dynamism, outcomes). (3) Identify and define the nonlinear phases of policy D&I across outer and inner contexts. (4) Describe the temporal roles that stakeholders play in policy D&I over time. (5) Consider policy-relevant outer and inner context adaptations. (6) Identify and describe bridging factors necessary for policy D&I success. CONCLUSION Researchers should use TMF to meaningfully conceptualize policy's role in D&I efforts to accelerate the public health impact of evidence-based policies or practices and de-implement ineffective and harmful policies. Applying these six recommendations to existing D&I TMF advances existing theoretical knowledge, especially EPIS application, rather than introducing new models. Using these recommendations will sensitize researchers to help them investigate the multifaceted roles policy can play within a causal pathway leading to D&I success.
Collapse
Affiliation(s)
- Erika L Crable
- Department of Psychiatry, University of California, La Jolla, San Diego, CA, 92093, USA.
- Child and Adolescent Services Research Center, San Diego, CA, USA.
- UC San Diego Altman Clinical and Translational Research Institute Dissemination and Implementation Science Center, La Jolla, San Diego, CA, USA.
| | | | - Nicole A Stadnick
- Department of Psychiatry, University of California, La Jolla, San Diego, CA, 92093, USA
- Child and Adolescent Services Research Center, San Diego, CA, USA
- UC San Diego Altman Clinical and Translational Research Institute Dissemination and Implementation Science Center, La Jolla, San Diego, CA, USA
| | - Joanna C Moullin
- Faculty of Health Sciences, enAble Institute, Curtin University, Perth, WA, Australia
| | - Gregory A Aarons
- Department of Psychiatry, University of California, La Jolla, San Diego, CA, 92093, USA
- Child and Adolescent Services Research Center, San Diego, CA, USA
- UC San Diego Altman Clinical and Translational Research Institute Dissemination and Implementation Science Center, La Jolla, San Diego, CA, USA
| |
Collapse
|
9
|
Bacon C, Malone S, Prewitt K, Hackett R, Hastings M, Dexter S, Luke DA. Assessing the sustainability capacity of evidence-based programs in community and health settings. FRONTIERS IN HEALTH SERVICES 2022; 2:1004167. [PMID: 36925881 PMCID: PMC10012779 DOI: 10.3389/frhs.2022.1004167] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Subscribe] [Scholar Register] [Received: 07/27/2022] [Accepted: 11/09/2022] [Indexed: 12/05/2022]
Abstract
Background Within many public health settings, there remain large challenges to sustaining evidence-based practices. The Program Sustainability Assessment Tool has been developed and validated to measure sustainability capacity of public health, social service, and educational programs. This paper describes how this tool was utilized between January 2014 and January 2019. We describe characteristics of programs that are associated with increased capacity for sustainability and ultimately describe the utility of the PSAT in sustainability research and practice. Methods The PSAT is comprised of 8 subscales, measuring sustainability capacity in eight distinct conceptual domains. Each subscale is made up of five items, all assessed on a 7-point Likert scale. Data were obtained from persons who used the PSAT on the online website (https://sustaintool.org/), from 2014 to 2019. In addition to the PSAT scale, participants were asked about four program-level characteristics. The resulting dataset includes 5,706 individual assessments reporting on 2,892 programs. Results The mean overall PSAT score was 4.73, with the lowest and highest scoring subscales being funding stability and program adaptation, respectively. Internal consistency for each subscale was excellent (average Cronbach's alpha = 0.90, ranging from 0.85 to 0.94). Confirmatory factor analysis highlighted good to excellent fit of the PSAT measurement model (eight distinct conceptual domains) to the observed data, with a comparative fit index of 0.902, root mean square error of approximation equal to 0.054, and standardized root mean square residual of 0.054. Overall sustainability capacity was significantly related to program size (F = 25.6; p < 0.001). Specifically, smaller programs (with staff sizes of ten or below) consistently reported lower program sustainability capacity. Capacity was not associated with program age and did not vary significantly by program level. Discussion The PSAT maintained its excellent reliability when tested with a large and diverse sample over time. Initial criterion validity was explored through the assessment of program characteristics, including program type and program size. The data collected reinforces the ability of the PSAT to assess sustainability capacity for a wide variety of public health and social programs.
Collapse
Affiliation(s)
- Caren Bacon
- Center for Public Health Systems Science, Brown School, Washington University in St. Louis, St. Louis, MO, United States
| | - Sara Malone
- Center for Public Health Systems Science, Brown School, Washington University in St. Louis, St. Louis, MO, United States
- Division of Public Health Sciences, Department of Surgery, Washington University in St. Louis School of Medicine, St. Louis, MO, United States
| | - Kim Prewitt
- Center for Public Health Systems Science, Brown School, Washington University in St. Louis, St. Louis, MO, United States
| | - Rachel Hackett
- Center for Public Health Systems Science, Brown School, Washington University in St. Louis, St. Louis, MO, United States
| | - Molly Hastings
- Center for Public Health Systems Science, Brown School, Washington University in St. Louis, St. Louis, MO, United States
| | - Sarah Dexter
- Center for Public Health Systems Science, Brown School, Washington University in St. Louis, St. Louis, MO, United States
| | - Douglas A Luke
- Center for Public Health Systems Science, Brown School, Washington University in St. Louis, St. Louis, MO, United States
| |
Collapse
|
10
|
Cho E, Tugendrajch SK, McMillen JC, Proctor EK, Hawley KM. Implementation of Evidence-Based Practices within Treatment-As-Usual and Evidence-Based Practice Initiatives. ADMINISTRATION AND POLICY IN MENTAL HEALTH AND MENTAL HEALTH SERVICES RESEARCH 2022; 49:757-784. [PMID: 35501585 PMCID: PMC11003240 DOI: 10.1007/s10488-022-01197-z] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 04/03/2022] [Indexed: 11/25/2022]
Abstract
Publicly funded initiatives are underway to improve implementation of evidence-based practices (EBP) in youth mental health services. However, we know little about the success of these initiatives or about EBP implementation independent of such initiatives. We examined EBP implementation in a treatment as usual (TAU) state and in six states with publicly funded EBP initiatives (EBPIs). In Study 1, we examined providers' use of practices derived from the evidence base (PDEB) and their predictors among 780 providers in a TAU state. In Study 2, we conducted a systematic review of implementation strategies, outcomes, and predictors of EBP use in six state funded EBPIs. Study 1 suggests TAU providers use PDEB alongside practices without consistent research support; provider racial/ethnic minority status, learning theory orientation, and manual use predict greater PDEB use. Study 2 indicates EBPIs employ multiple recommended implementation strategies with variable outcomes across studies and measurement approaches. Predictors of EBP use in EBPIs also varied, though training, setting, and youth age were consistent predictors across studies. While sample differences and inconsistent measurement across studies made direct comparisons somewhat tenuous, rates of PDEB use in the TAU sample appeared similar to those in publicly funded EBPIs. However, two states reported comparisons with TAU samples and found higher EBP implementation under EBPI. Different predictors impacted EBP use in TAU versus EBPIs. Our findings highlight the need for improved evaluation of EBPIs including clear reporting standards for outcomes and more consistent, standardized measurement of EBP use in order to better understand and improve EBPIs.
Collapse
Affiliation(s)
- E Cho
- Harvard University, 33 Kirkland St, Cambridge, MA, 02138, USA
| | - S K Tugendrajch
- University of Missouri, 200 South 7th Street, Columbia, MO, 65211, USA
| | - J C McMillen
- University of Chicago, 969 E. 60th Street, Chicago, IL, 60637, USA
| | - E K Proctor
- Washington University in St. Louis, One Brookings Drive, St. Louis, MO, 63130, USA
| | - K M Hawley
- University of Missouri, 204C McAlester Hall, Columbia, MO, 65211, USA.
| |
Collapse
|
11
|
Woodfield MJ, Cargo T, Merry SN, Hetrick SE. Barriers to Clinician Implementation of Parent-Child Interaction Therapy (PCIT) in New Zealand and Australia: What Role for Time-Out? INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2021; 18:13116. [PMID: 34948725 PMCID: PMC8700887 DOI: 10.3390/ijerph182413116] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 11/19/2021] [Revised: 12/01/2021] [Accepted: 12/08/2021] [Indexed: 12/22/2022]
Abstract
BACKGROUND Parent-Child Interaction Therapy (PCIT) is an effective parent training approach for a commonly occurring and disabling condition, namely conduct problems in young children. Yet, despite ongoing efforts to train clinicians in PCIT, the intervention is not widely available in New Zealand and Australia. METHODS We undertook a cross-sectional online survey of clinicians in New Zealand and Australia who had completed at least the 40-h initial PCIT training, to understand the barriers they encountered in their implementation efforts, and the extent to which attitudes toward time-out influenced implementation. The overall response rate was 47.5% (NZ: 60%; Australia: 31.4%). RESULTS Responses suggested that participants generally viewed PCIT as both acceptable and effective. Australian participants reported seeing significantly more clients for PCIT per week than those in NZ (Medians 0 and 2, respectively; χ2(1) = 14.08, p < 0.001) and tended to view PCIT as more effective in treating disruptive and oppositional behaviour (95% CI: -0.70, -0.13, p = 0.005). Participants currently seeing PCIT clients described it as more enjoyable to implement than those not using PCIT (95% CI: -0.85, -0.10, p = 0.01). Thirty-eight percent of participants indicated that they adapt or tailor the standardised protocol, primarily by adding in content relating to emotion regulation, and removing content relating to time-out. Participants generally felt that they had fewer skills, less knowledge, and less confidence relating to the Parent-Directed Interaction phase of PCIT (which involves time-out), compared with the Child-Directed Interaction phase. CONCLUSION While we had hypothesised that time-out represented an intra-intervention component that detracted from implementation success, results suggested that clinician concern over the use of time-out was present but not prominent. Rather, the lack of access to suitable equipment (i.e., one-way mirror and ear-piece) and difficulties associated with clients attending clinic-based sessions were barriers most commonly reported by clinicians. We suggest that future research might consider whether and how PCIT might be "re-implemented" by already-trained clinicians, moving beyond simply training more clinicians in the approach.
Collapse
Affiliation(s)
- Melanie J. Woodfield
- The Werry Centre, Department of Psychological Medicine, University of Auckland, Auckland 1023, New Zealand; (T.C.); (S.N.M.); (S.E.H.)
- Auckland District Health Board, Auckland 1023, New Zealand
| | - Tania Cargo
- The Werry Centre, Department of Psychological Medicine, University of Auckland, Auckland 1023, New Zealand; (T.C.); (S.N.M.); (S.E.H.)
- Department of Psychology, University of Auckland, Auckland 1023, New Zealand
| | - Sally N. Merry
- The Werry Centre, Department of Psychological Medicine, University of Auckland, Auckland 1023, New Zealand; (T.C.); (S.N.M.); (S.E.H.)
| | - Sarah E. Hetrick
- The Werry Centre, Department of Psychological Medicine, University of Auckland, Auckland 1023, New Zealand; (T.C.); (S.N.M.); (S.E.H.)
- Centre for Youth Mental Health, University of Melbourne, Parkville, VIC 3010, Australia
| |
Collapse
|