1
|
Hazlett AD, Benzer JK, Montejos K, Pittman DL, Creech SK, Claborn KR, Acosta J, Chinman M. Organizational Capacity for Sexual Assault Prevention Within a U.S. Army Installation. Mil Med 2024:usae332. [PMID: 38943537 DOI: 10.1093/milmed/usae332] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/25/2024] [Revised: 03/20/2024] [Accepted: 06/18/2024] [Indexed: 07/01/2024] Open
Abstract
INTRODUCTION Preventing military sexual assault (SA) is a DoD priority. Building prevention capacity could strengthen the impact of prevention programs and improve outcomes. Capacity was conceptualized as implementation knowledge and skills using the Getting To Outcomes (GTO) process and organizational-level capacities using the Prevention Evaluation Framework, a framework that applies best practices in prevention and implementation science to prevention at the program and organizational level. The present study assesses implementation knowledge and skills currently possessed by SA prevention personnel in one U.S. Army installation with the goal of identifying preexisting proficiencies and potential gaps. The study uses the Prevention Evaluation Framework organizational-level domains to identify organizational capacities that could be improved. MATERIALS AND METHODS Fifteen SA prevention program personnel were recruited for individual 1-hour interviews from one U.S. Army installation. A semi-structured interview guide was developed by the study team to assess knowledge and skills within the GTO framework. Applied thematic analysis guided data coding and analysis. RESULTS Participants reported a range of proficiencies and gaps in capacity for SA prevention. Within the prevention workforce capacity, proficiencies were skill in tailoring training, and an existing train-the-trainer model. Gaps were training to facilitate difficult conversations, and an inability to tailor trainings to audience needs. Two proficiencies related to the comprehensive approach were having a victim advocate present during trainings, and providing training across leadership levels. The final proficiency involved data. Existing data sources were viewed as useful for identifying the nature and extent of local problems. However, participants reported a lack of confidence and skill in selecting appropriate outcome measures and collecting and using primary data. Gaps in leadership capacity were cultural norms, lack of trust, leader investment, logistical difficulty working with leaders, and comfort engaging with leaders. Gaps in collaborative relationships were the lack of informal networking within the SA prevention area to include civilians. CONCLUSIONS The GTO-specific knowledge and skills of SA prevention personnel at one U.S. Army installation revealed proficiencies and gaps in organizational capacities influencing the prevention mission. Findings indicate that GTO support could be useful for improving the quality of program activities. However, results also indicate that GTO support would be more effective if they were paired with sufficient prevention infrastructure at the organizational level.
Collapse
Affiliation(s)
- Abigail D Hazlett
- Moody College of Communication, University of Texas at Austin, Austin, TX 78712-0115, USA
| | - Justin K Benzer
- Department of Psychiatry and Behavioral Sciences, Dell Medical School, Austin, TX 78712, USA
- VA VISN 17 Center of Excellence for Research on Returning Veterans, Veterans Health Administration, Waco, TX 76711, USA
| | - Kendra Montejos
- Education Leadership and Policy, University of Texas at Austin, Austin, TX 78712, USA
| | | | - Suzannah K Creech
- Department of Psychiatry and Behavioral Sciences, Dell Medical School, Austin, TX 78712, USA
- VA VISN 17 Center of Excellence for Research on Returning Veterans, Veterans Health Administration, Waco, TX 76711, USA
| | - Kasey R Claborn
- Steve Hicks School of Social Work, University of Texas at Austin, Austin, TX 78712, USA
| | | | - Matthew Chinman
- RAND Corporation, Arlington, VA 22202, USA
- Veteran Affairs Pittsburgh Healthcare System, Veterans Health Administration, Pittsburgh, PA 15240, USA
| |
Collapse
|
2
|
Kilbourne A, Chinman M, Rogal S, Almirall D. Adaptive Designs in Implementation Science and Practice: Their Promise and the Need for Greater Understanding and Improved Communication. Annu Rev Public Health 2024; 45:69-88. [PMID: 37931183 PMCID: PMC11070446 DOI: 10.1146/annurev-publhealth-060222-014438] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/08/2023]
Abstract
The promise of adaptation and adaptive designs in implementation science has been hindered by the lack of clarity and precision in defining what it means to adapt, especially regarding the distinction between adaptive study designs and adaptive implementation strategies. To ensure a common language for science and practice, authors reviewed the implementation science literature and found that the term adaptive was used to describe interventions, implementation strategies, and trial designs. To provide clarity and offer recommendations for reporting and strengthening study design, we propose a taxonomy that describes fixed versus adaptive implementation strategies and implementation trial designs. To improve impact, (a) futureimplementation studies should prespecify implementation strategy core functions that in turn can be taught to and replicated by health system/community partners, (b) funders should support exploratory studies that refine and specify implementation strategies, and (c) investigators should systematically address design requirements and ethical considerations (e.g., randomization, blinding/masking) with health system/community partners.
Collapse
Affiliation(s)
- Amy Kilbourne
- Quality Enhancement Research Initiative, U.S. Department of Veterans Affairs, Washington, District of Columbia, USA
- Department of Learning Health Sciences, University of Michigan Medical School, Ann Arbor, Michigan, USA;
| | - Matthew Chinman
- RAND Corporation, Pittsburgh, Pennsylvania, USA
- Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, Pennsylvania, USA
- Mental Illness Research, Education, and Clinical Center, VA Pittsburgh Healthcare System, Pittsburgh, Pennsylvania, USA
| | - Shari Rogal
- Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, Pennsylvania, USA
- Departments of Medicine and Surgery, University of Pittsburgh, Pittsburgh, Pennsylvania, USA
| | - Daniel Almirall
- Institute for Social Research and Department of Statistics, University of Michigan, Ann Arbor, Michigan, USA
| |
Collapse
|
3
|
Garavito GAA, Moniz T, Mansilla C, Iqbal S, Dobrogowska R, Bennin F, Talwar S, Khalid AF, Vindrola-Padros C. Activities used by evidence networks to promote evidence-informed decision-making in the health sector- a rapid evidence review. BMC Health Serv Res 2024; 24:261. [PMID: 38418985 PMCID: PMC10903073 DOI: 10.1186/s12913-024-10744-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/08/2023] [Accepted: 02/18/2024] [Indexed: 03/02/2024] Open
Abstract
BACKGROUND Evidence networks facilitate the exchange of information and foster international relationships among researchers and stakeholders. These networks are instrumental in enabling the integration of scientific evidence into decision-making processes. While there is a global emphasis on evidence-based decision-making at policy and organisational levels, there exists a significant gap in our understanding of the most effective activities to exchange scientific knowledge and use it in practice. The objective of this rapid review was to explore the strategies employed by evidence networks to facilitate the translation of evidence into decision-making processes. This review makes a contribution to global health policymaking by mapping the landscape of knowledge translation in this context and identifying the evidence translation activities that evidence networks have found effective. METHODS The review was guided by standardised techniques for conducting rapid evidence reviews. Document searching was based on a phased approach, commencing with a comprehensive initial search strategy and progressively refining it with each subsequent search iterations. The Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) statement was followed. RESULTS The review identified 143 articles, after screening 1135 articles. Out of these, 35 articles were included in the review. The studies encompassed a diverse range of countries, with the majority originating from the United States (n = 14), followed by Canada (n = 5), Sweden (n = 2), and various other single locations (n = 14). These studies presented a varied set of implementation strategies such as research-related activities, the creation of teams/task forces/partnerships, meetings/consultations, mobilising/working with communities, influencing policy, activity evaluation, training, trust-building, and regular meetings, as well as community-academic-policymaker engagement. CONCLUSIONS Evidence networks play a crucial role in developing, sharing, and implementing high-quality research for policy. These networks face challenges like coordinating diverse stakeholders, international collaboration, language barriers, research consistency, knowledge dissemination, capacity building, evaluation, and funding. To enhance their impact, sharing network efforts with wider audiences, including local, national, and international agencies, is essential for evidence-based decision-making to shape evidence-informed policies and programmes effectively.
Collapse
Affiliation(s)
- Germán Andrés Alarcón Garavito
- Rapid Research Evaluation and Appraisal Lab (RREAL) - Department of Targeted Intervention, University College London, London, UK
| | - Thomas Moniz
- Rapid Research Evaluation and Appraisal Lab (RREAL) - Department of Targeted Intervention, University College London, London, UK
| | - Cristián Mansilla
- McMaster Health Forum, McMaster University, Toronto, Ontario, Canada
| | - Syka Iqbal
- Rapid Research Evaluation and Appraisal Lab (RREAL) - Department of Targeted Intervention, University College London, London, UK
| | - Rozalia Dobrogowska
- Rapid Research Evaluation and Appraisal Lab (RREAL) - Department of Targeted Intervention, University College London, London, UK
| | - Fiona Bennin
- Rapid Research Evaluation and Appraisal Lab (RREAL) - Department of Targeted Intervention, University College London, London, UK
| | - Shivangi Talwar
- Rapid Research Evaluation and Appraisal Lab (RREAL) - Department of Targeted Intervention, University College London, London, UK
- Division of Psychiatry, University College London, London, UK
| | | | - Cecilia Vindrola-Padros
- Rapid Research Evaluation and Appraisal Lab (RREAL) - Department of Targeted Intervention, University College London, London, UK.
| |
Collapse
|
4
|
Lambdin BH, Bluthenthal RN, Garner BR, Wenger LD, Browne EN, Morris T, Ongais L, Megerian CE, Kral AH. Organize and mobilize for implementation effectiveness to improve overdose education and naloxone distribution from syringe services programs: a randomized controlled trial. Implement Sci 2024; 19:22. [PMID: 38419058 PMCID: PMC10900734 DOI: 10.1186/s13012-024-01354-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/20/2023] [Accepted: 02/14/2024] [Indexed: 03/02/2024] Open
Abstract
BACKGROUND The United States (US) continues to face decades-long increases in opioid overdose fatalities. As an opioid overdose reversal medication, naloxone can dramatically reduce opioid overdose mortality rates when distributed to people likely to experience or witness an opioid overdose and packaged with education on its use, known as overdose education and naloxone distribution (OEND). Syringe services programs (SSPs) are ideal venues for OEND with staff who are culturally competent in providing services for people who are at risk of experiencing or observing an opioid overdose. We carried out a randomized controlled trial of SSPs to understand the effectiveness of the organize and mobilize for implementation effectiveness (OMIE) approach at improving OEND implementation effectiveness within SSPs. METHODS Using simple randomization, 105 SSPs were enrolled into the trial and assigned to one of two study arms - (1) dissemination of OEND best practice recommendations (Control SSPs) or the OMIE approach along with dissemination of the OEND best practice recommendations (i.e., OMIE SSPs). OMIE SSPs could participate in 60-min OMIE sessions once a month for up to 12 months. At 12-month post-baseline, 102 of 105 SSPs (97%) responded to the follow-up survey. RESULTS The median number of sessions completed by OMIE SSPs was 10. Comparing OMIE SSPs to control SSPs, we observed significant increases in the number of participants receiving naloxone (incidence rate ratio: 2.15; 95% CI: 1.42, 3.25; p < 0.01) and the rate of naloxone doses distributed per SSP participant (adjusted incidence rate ratio: 1.97; 95% CI: 1.18, 3.30; p = 0.01). We observed no statistically significant difference in the number of adopted best practices between conditions (difference in means 0.2, 95% CI: - 0.7, 1.0; p = 0.68). We also observed a threshold effect where SSPs receiving a higher OMIE dose had greater effect sizes with regard to the number of people given naloxone and the number of naloxone doses distributed. CONCLUSIONS In conclusion, the multifaceted OMIE approach was effective at increasing naloxone distribution from SSPs, despite substantial external shocks during the trial. These findings have major implications for addressing the overdose crisis, which has continued unabated for decades. TRIAL REGISTRATION ClinicalTrials.gov, NCT03924505 . Registered 19 April 2019.
Collapse
Affiliation(s)
- Barrot H Lambdin
- RTI International, 2150 Shattuck Avenue, 8Th Floor, Berkeley, CA, 94704, USA.
- Department of Epidemiology and Biostatistics, University of California San Francisco, San Francisco, CA, USA.
- Department of Global Health, University of Washington, Seattle, WA, USA.
| | - Ricky N Bluthenthal
- Keck Medicine, Department of Population and Public Health Sciences, University of Southern California, 1975 Zonal Ave, Los Angeles, CA, 90033, USA
| | - Bryan R Garner
- Department of Internal Medicine, College of Medicine, The Ohio State University, 370 W. 9Th Avenue, Columbus, OH, 43210, USA
| | - Lynn D Wenger
- RTI International, 2150 Shattuck Avenue, 8Th Floor, Berkeley, CA, 94704, USA
| | - Erica N Browne
- RTI International, 2150 Shattuck Avenue, 8Th Floor, Berkeley, CA, 94704, USA
| | - Terry Morris
- RTI International, 2150 Shattuck Avenue, 8Th Floor, Berkeley, CA, 94704, USA
| | - Lee Ongais
- San Francisco AIDS Foundation, 1035 Market Street, 4Th Floor, San Francisco, CA, 94103, USA
| | - Cariné E Megerian
- RTI International, 2150 Shattuck Avenue, 8Th Floor, Berkeley, CA, 94704, USA
| | - Alex H Kral
- RTI International, 2150 Shattuck Avenue, 8Th Floor, Berkeley, CA, 94704, USA
| |
Collapse
|
5
|
Chinman M, Acosta J, Bush-Mecenas S, Smucker S, Farris C, Fortson B, Imm P, Lamont A, Maguire T, Martin L, Wandersman A, Watson A, Wicker A, Tharp A. Improving Sexual Assault and Sexual Harassment Prevention from the Bottom-up: a Pilot of Getting To Outcomes in the US Military. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2023; 24:1352-1364. [PMID: 37642815 PMCID: PMC10575806 DOI: 10.1007/s11121-023-01577-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/17/2023] [Indexed: 08/31/2023]
Abstract
While the Department of Defense (DoD) has given increased attention and priority to preventing sexual assault and sexual harassment (SA/SH), it remains a problem. To build its prevention capacity, DoD piloted Getting To Outcomes® (GTO®) from 2019 to 2022 at 10 military installations. GTO is an evidence-based planning and implementation support that has been used in many civilian contexts but has only recently been adapted for military SA/SH. The purpose of this study was to describe GTO use, identify its benefits and challenges, and discuss lessons the GTO effort yielded for prevention more broadly using a framework of organizational and program-level capacities needed for successful prevention in the military context, called the Prevention Evaluation Framework (PEF). GTO was piloted with 10 military installations ("sites") representing all Military Services, plus the Coast Guard and National Guard. GTO is comprised of a written guide, training, and ongoing coaching. The pilot's goal was for each site to use GTO to implement a SA/SH prevention program twice. Participants from each site were interviewed and data was collected on GTO steps completed, whether GTO spurred new evaluation activities and collaborations, and the degree of leadership support for GTO. Most sites completed all GTO steps at least once. Interviews showed that DoD participants believe GTO improved prevention understanding, planning, and evaluation capacity; strengthened confidence in chosen programs; and helped sites tailor programs to the military context. Barriers were the complexity of GTO, DoD personnel turnover, and the disruption that the COVID pandemic caused in sexual assault prevention program delivery. Many respondents were unsure if they would continue all of GTO after the coaching ended, but many believed they would continue at least some parts. According to the PEF, the GTO pilot revealed several additional prevention system gaps (e.g., need for leadership support) and changes needed to GTO (e.g., stronger leader and champion engagement), to support quality prevention. The military and other large organizations will need to focus on these issues to ensure prevention implementation and evaluation are conducted with quality.
Collapse
Affiliation(s)
- Matthew Chinman
- RAND Corporation, 4570 Fifth Avenue, Pittsburgh, PA 15213 USA
| | - Joie Acosta
- RAND Corporation, 4570 Fifth Avenue, Pittsburgh, PA 15213 USA
| | | | - Sierra Smucker
- RAND Corporation, 4570 Fifth Avenue, Pittsburgh, PA 15213 USA
| | - Coreen Farris
- RAND Corporation, 4570 Fifth Avenue, Pittsburgh, PA 15213 USA
| | - Beverly Fortson
- Department of Defense, Office of Force Resiliency, Washington, DC USA
| | | | | | - Thomas Maguire
- Department of Defense, Office of Force Resiliency, Washington, DC USA
| | - Laurie Martin
- RAND Corporation, 4570 Fifth Avenue, Pittsburgh, PA 15213 USA
| | | | | | - Amanda Wicker
- RAND Corporation, 4570 Fifth Avenue, Pittsburgh, PA 15213 USA
| | - Andra Tharp
- Department of Defense, Office of Force Resiliency, Washington, DC USA
| |
Collapse
|
6
|
Trivedi KK, Schaffzin JK, Deloney VM, Aureden K, Carrico R, Garcia-Houchins S, Garrett JH, Glowicz J, Lee GM, Maragakis LL, Moody J, Pettis AM, Saint S, Schweizer ML, Yokoe DS, Berenholtz S. Implementing strategies to prevent infections in acute-care settings. Infect Control Hosp Epidemiol 2023; 44:1232-1246. [PMID: 37431239 PMCID: PMC10527889 DOI: 10.1017/ice.2023.103] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 07/12/2023]
Abstract
This document introduces and explains common implementation concepts and frameworks relevant to healthcare epidemiology and infection prevention and control and can serve as a stand-alone guide or be paired with the "SHEA/IDSA/APIC Compendium of Strategies to Prevent Healthcare-Associated Infections in Acute Care Hospitals: 2022 Updates," which contain technical implementation guidance for specific healthcare-associated infections. This Compendium article focuses on broad behavioral and socio-adaptive concepts and suggests ways that infection prevention and control teams, healthcare epidemiologists, infection preventionists, and specialty groups may utilize them to deliver high-quality care. Implementation concepts, frameworks, and models can help bridge the "knowing-doing" gap, a term used to describe why practices in healthcare may diverge from those recommended according to evidence. It aims to guide the reader to think about implementation and to find resources suited for a specific setting and circumstances by describing strategies for implementation, including determinants and measurement, as well as the conceptual models and frameworks: 4Es, Behavior Change Wheel, CUSP, European and Mixed Methods, Getting to Outcomes, Model for Improvement, RE-AIM, REP, and Theoretical Domains.
Collapse
Affiliation(s)
| | - Joshua K. Schaffzin
- Children’s Hospital of Eastern Ontario, University of Ottawa, Ottawa, Ontario, Canada
| | - Valerie M. Deloney
- Society for Healthcare Epidemiology of America (SHEA), Arlington, Virginia
| | | | - Ruth Carrico
- Division of Infectious Diseases, University of Louisville School of Medicine, Louisville, Kentucky
| | | | - J. Hudson Garrett
- Division of Infectious Diseases, University of Louisville School of Medicine, Louisville, Kentucky
| | - Janet Glowicz
- Centers for Disease Control and Prevention, Atlanta, Georgia
| | - Grace M. Lee
- Stanford Children’s Health, Stanford, California
| | | | - Julia Moody
- Clinical Services Group, HCA Healthcare, Nashville, Tennessee
| | | | - Sanjay Saint
- VA Ann Arbor Healthcare System and University of Michigan, Ann Arbor, Michigan
| | | | - Deborah S. Yokoe
- University of California San Francisco School of Medicine, UCSF Medical Center, San Francisco, California
| | - Sean Berenholtz
- Clinical Services Group, HCA Healthcare, Nashville, Tennessee
| |
Collapse
|
7
|
Shearer AL, Acosta J, Ebener P, Simonson J, Chinman M. Scaling Up Implementation Support for Violence Prevention and Resilience Promotion in the Air Force. HEALTH EDUCATION & BEHAVIOR 2023; 50:328-338. [PMID: 36448341 DOI: 10.1177/10901981221136841] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/02/2022]
Abstract
Closing the gap between research and practice requires that organizations can consistently incorporate new ideas and best practices. The Department of the Air Force (DAF) Integrated Resilience Directorate is leading a coordinated effort to increase the adoption of evidence-based violence prevention and resilience promotion programs across the entire Department. To support this effort, DAF is using Getting To Outcomes (GTO), an evidence-based implementation support that helps organizations plan, implement, and self-evaluate programs. Thus, the DAF is not only scaling up prevention programs but is also embarking on the largest scale-up of GTO to date. The study team trained personnel from every Air Force installation across the world to use GTO for their programs. Quantitative findings from training questionnaires and ratings of implementation plans as well as qualitative results from resilience personnel interviews suggest some lessons learned for scaling up implementation support. This study builds on established implementation science frameworks for scaling up interventions by identifying critical tasks and unique supports needed to scale up evidence-based prevention. Results suggest GTO helped DAF scale-up prevention across the Department, and that establishing leadership buy-in, simplifying evidence-based program selection and adaptation, monitoring implementation and outcomes, and creating dedicated prevention practitioner roles are critical tasks to support scale-up of evidence-based prevention. Unique supports needed to scale-up evidence-based prevention include multitiered learning systems; integrated tools that improve access to data and evolving evidence; prevention personnel with skills in program management, adaptation, and evaluation; timely crosscutting data; continuous learning to support sustainability; and leadership buy-in.
Collapse
Affiliation(s)
| | | | | | - Jordan Simonson
- Headquarters Air Force, A1Z-Integrated Resilience, Washington, DC, USA
| | | |
Collapse
|
8
|
Rawal S, Osae SP, Cobran EK, Albert A, Young HN. Pharmacists' naloxone services beyond community pharmacy settings: A systematic review. Res Social Adm Pharm 2023; 19:243-265. [PMID: 36156267 DOI: 10.1016/j.sapharm.2022.09.002] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/18/2022] [Revised: 08/30/2022] [Accepted: 09/02/2022] [Indexed: 01/25/2023]
Abstract
BACKGROUND Pharmacists' provision of naloxone services in community pharmacy settings is well-recognized. Recently, studies describing pharmacists' naloxone services in settings other than community pharmacies have emerged in the literature. There is a need to synthesize evidence from these studies to evaluate the scope and impact of pharmacists' naloxone services beyond community pharmacy settings. OBJECTIVES The objectives of this systematic review were to a) identify pharmacists' naloxone services and their outcomes, and b) examine knowledge, attitudes, and barriers (KAB) related to naloxone service provision in non-community pharmacy settings. METHODS Eligible studies were identified using PubMed, Web of Science, and CINAHL. Inclusion criteria were as follows: peer-reviewed empirical research conducted in the U.S. from January 2010 through February 2022; published in English; and addressed a) pharmacists' naloxone services and/or b) KAB related to the implementation of naloxone services. PRISMA guidelines were used to report this study. RESULTS Seventy-six studies were identified. The majority were non-randomized and observational; only two used a randomized controlled (RCT) design. Most studies were conducted in veterans affairs (30%) and academic medical centers (21%). Sample sizes ranged from n = 10 to 217,469, and the majority reported sample sizes <100. Pharmacists' naloxone services involved clinical staff education, utilization of screening tools to identify at-risk patients, naloxone prescribing and overdose education and naloxone dispensing (OEND). Outcomes of implementing naloxone services included improved naloxone knowledge, positive attitudes, increased OEND, and overdose reversals. Pharmacists cited inadequate training, time constraints, reimbursement issues, and stigma as barriers that hindered naloxone service implementation. CONCLUSION This systematic review found robust evidence regarding pharmacist-based naloxone services beyond community pharmacy settings. Future programs should use targeted approaches to help pharmacists overcome barriers and enhance naloxone services. Additional research is needed to evaluate pharmacist naloxone services by using rigorous methodologies (e.g., larger sample sizes, RCT designs).
Collapse
Affiliation(s)
- Smita Rawal
- Clinical and Administrative Pharmacy, University of Georgia, Athens, GA, USA.
| | - Sharmon P Osae
- Clinical and Administrative Pharmacy, University of Georgia, Albany, GA, USA
| | - Ewan K Cobran
- Department of Quantitative Health Science, Mayo Clinic College of Medicine and Sciences, Scottsdale, AZ, USA
| | - Alexis Albert
- College of Pharmacy, University of Georgia, Athens, GA, USA
| | - Henry N Young
- Clinical and Administrative Pharmacy, University of Georgia, Athens, GA, USA
| |
Collapse
|
9
|
Yakovchenko V, Rogal SS, Goodrich DE, Lamorte C, Neely B, Merante M, Gibson S, Scott D, McCurdy H, Nobbe A, Morgan TR, Chinman MJ. Getting to implementation: Adaptation of an implementation playbook. Front Public Health 2023; 10:980958. [PMID: 36684876 PMCID: PMC9853037 DOI: 10.3389/fpubh.2022.980958] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/29/2022] [Accepted: 12/12/2022] [Indexed: 01/09/2023] Open
Abstract
Introduction Implementation strategies supporting the translation of evidence into practice need to be tailored and adapted for maximum effectiveness, yet the field of adapting implementation strategies remains nascent. We aimed to adapt "Getting To Outcomes"® (GTO), a 10-step implementation playbook designed to help community-based organizations plan and evaluate behavioral health programs, into "Getting To Implementation" (GTI) to support the selection, tailoring, and use of implementation strategies in health care settings. Methods Our embedded evaluation team partnered with operations, external facilitators, and site implementers to employ participatory methods to co-design and adapt GTO for Veterans Health Administration (VA) outpatient cirrhosis care improvement. The Framework for Reporting Adaptations and Modifications to Evidenced-based Implementation Strategies (FRAME-IS) guided documentation and analysis of changes made pre- and post-implementation of GTI at 12 VA medical centers. Data from multiple sources (interviews, observation, content analysis, and fidelity tracking) were triangulated and analyzed using rapid techniques over a 3-year period. Results Adaptations during pre-implementation were planned, proactive, and focused on context and content to improve acceptability, appropriateness, and feasibility of the GTI playbook. Modifications during and after implementation were unplanned and reactive, concentrating on adoption, fidelity, and sustainability. All changes were collaboratively developed, fidelity consistent at the level of the facilitator and/or implementer. Conclusion GTO was initially adapted to GTI to support health care teams' selection and use of implementation strategies for improving guideline-concordant medical care. GTI required ongoing modification, particularly in steps regarding team building, context assessment, strategy selection, and sustainability due to difficulties with step clarity and progression. This work also highlights the challenges in pragmatic approaches to collecting and synthesizing implementation, fidelity, and adaptation data. Trial registration This study was registered on ClinicalTrials.gov (Identifier: NCT04178096).
Collapse
Affiliation(s)
- Vera Yakovchenko
- Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, PA, United States
| | - Shari S. Rogal
- Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, PA, United States
- Division of Gastroenterology, Hepatology, and Nutrition, Department of Medicine, University of Pittsburgh, Pittsburgh, PA, United States
- Department of Surgery, University of Pittsburgh, Pittsburgh, PA, United States
| | - David E. Goodrich
- Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, PA, United States
| | - Carolyn Lamorte
- Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, PA, United States
| | - Brittney Neely
- Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, PA, United States
| | - Monica Merante
- Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, PA, United States
| | - Sandra Gibson
- Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, PA, United States
- Division of Gastroenterology, Hepatology, and Nutrition, Department of Medicine, University of Pittsburgh, Pittsburgh, PA, United States
| | - Dawn Scott
- Department of Medicine, Central Texas Veterans Healthcare System, Temple, TX, United States
| | - Heather McCurdy
- VA Ann Arbor Healthcare System, Ann Arbor, MI, United States
| | - Anna Nobbe
- Digestive Disease Section, Cincinnati VA Medical Center, Cincinnati, OH, United States
| | - Timothy R. Morgan
- Gastroenterology Section, VA Long Beach Healthcare System, Long Beach, CA, United States
- Division of Gastroenterology, Department of Medicine, University of California, Irvine, Irvine, CA, United States
| | - Matthew J. Chinman
- Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, PA, United States
- RAND Corporation, Pittsburgh, PA, United States
| |
Collapse
|
10
|
Aldridge WA, Roppolo RH, Brown J, Bumbarger BK, Boothroyd RI. Mechanisms of change in external implementation support: A conceptual model and case examples to guide research and practice. IMPLEMENTATION RESEARCH AND PRACTICE 2023; 4:26334895231179761. [PMID: 37790181 PMCID: PMC10291867 DOI: 10.1177/26334895231179761] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/05/2023] Open
Abstract
Background External implementation support (EIS) is a well-recognized feature of implementation science and practice, often under related terms such as technical assistance and implementation facilitation. Existing models of EIS have gaps related to addressing practice outcomes at both individual and organizational levels, connecting practice activities to intended outcomes, or grounding in well-established theories of behavior and organization change. Moreover, there have been calls to clarify the mechanisms of change through which EIS influences related outcomes. Method In this article, we theorize about mechanisms of change within EIS. Our theorizing process aligns with the approach advocated by Kislov et al. We aim to consolidate prior EIS literature, combining related constructs from previous empirical and conceptual work while drawing on our extensive EIS experience to develop a higher-order, midrange theory of change. Results Our theory of change is empirically and practically informed, conceptually situated within an established grand theory of change, and guided by eight practice principles and social cognitive theory. The theory of change proposes 10 core practice components as mechanisms of change within EIS. When used according to underlying theory and principles, they are believed to contribute to favorable practice outcomes at individual, team, organizational, and system levels. The model offers flexibility by recognizing the need for sequential support processes and the demand to practice in dynamic and responsive ways. Case examples are presented to illustrate major themes and patterns of the model in action. Conclusions The proposed model is intended to support prospective EIS studies by conceptualizing discernable practice components with hypothesized relationships to proximal and distal practice outcomes. The model can be behaviorally operationalized to compliment and extend competency-based approaches to implementation support practitioner (ISP) training and coaching. Over time, the model should be refined based on new empirical findings and contributions from ISPs across the field.
Collapse
Affiliation(s)
- William A. Aldridge
- The Impact Center at Frank Porter Graham Child Development Institute, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | - Rebecca H. Roppolo
- The Impact Center at Frank Porter Graham Child Development Institute, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| | | | - Brian K. Bumbarger
- Griffith Criminology Institute, Griffith University, Mount Gravatt, Queensland, Australia
| | - Renée I. Boothroyd
- The Impact Center at Frank Porter Graham Child Development Institute, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
| |
Collapse
|
11
|
Scott VC, Jillani Z, Malpert A, Kolodny-Goetz J, Wandersman A. A scoping review of the evaluation and effectiveness of technical assistance. Implement Sci Commun 2022; 3:70. [PMID: 35765107 PMCID: PMC9238031 DOI: 10.1186/s43058-022-00314-1] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/08/2022] [Accepted: 05/25/2022] [Indexed: 11/10/2022] Open
Abstract
Background Although the benefits of evidence-based practices (EBPs) for advancing community outcomes are well-recognized, challenges with the uptake of EBPs are considerable. Technical assistance (TA) is a core capacity building strategy that has been widely used to support EBP implementation and other community development and improvement efforts. Yet despite growing reliance on TA, no reviews have systematically examined the evaluation of TA across varying implementation contexts and capacity building aims. This study draws on two decades of peer-reviewed publications to summarize the evidence on the evaluation and effectiveness of TA. Methods Guided by Arksey and O’Malley’s six-stage methodological framework, we used a scoping review methodology to map research on TA evaluation. We included peer-reviewed articles published in English between 2000 and 2020. Our search involved five databases: Business Source Complete, Cumulative Index to Nursing and Allied Health Literature (CINAHL), Education Resources Information Center (ERIC), PsycInfo, and PubMed. Results A total of 125 evaluation research studies met the study criteria. Findings indicate that publications have increased over the last two decades, signaling a growth in the recognition and reporting of TA. Technical assistance is being implemented across diverse settings, often serving socially vulnerable and under-resourced populations. Most evaluation research studies involved summative evaluations, with TA outcomes mostly reported at the organizational level. Only 5% of the studies examined sustainability of TA outcomes. This review also demonstrates that there is a lack of consistent standards regarding the definition of TA and the level of reporting across relevant TA evaluation categories (e.g., cadence of contact, and directionality). Conclusions Advances in the science and practice of TA hinge on understanding what aspects of TA are effective and when, how, and for whom these aspects of TA are effective. Addressing these core questions requires (i) a standard definition for TA; (ii) more robust and rigorous evaluation research designs that involve comparison groups and assessment of direct, indirect, and longitudinal outcomes; (iii) increased use of reliable and objective TA measures; and (iv) development of reporting standards. We view this scoping review as a foundation for improving the state of the science and practice of evaluating TA. Supplementary Information The online version contains supplementary material available at 10.1186/s43058-022-00314-1.
Collapse
|
12
|
Measuring capacity to use evidence-based interventions in community-based organizations: A comprehensive, scoping review. J Clin Transl Sci 2022; 6:e92. [PMID: 36003212 PMCID: PMC9389281 DOI: 10.1017/cts.2022.426] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/04/2022] [Revised: 06/30/2022] [Accepted: 07/06/2022] [Indexed: 11/14/2022] Open
Abstract
Introduction: Community-based organizations (CBOs) are well-positioned to incorporate research evidence, local expertise, and contextual factors to address health inequities. However, insufficient capacity limits use of evidence-based interventions (EBIs) in these settings. Capacity-building implementation strategies are popular, but a lack of standard models and validated measures hinders progress in the field. To advance the literature, we conducted a comprehensive scoping review. Methods: With a reference librarian, we executed a comprehensive search strategy of PubMed/Medline, Web of Science Core Collection, and EBSCO Global Health. We included articles that addressed implementation science, capacity-building, and CBOs. Of 5527 articles, 99 met our inclusion criteria, and we extracted data using a double-coding process Results: Of the 99 articles, 47% defined capacity explicitly, 31% defined it indirectly, and 21% did not define it. Common concepts in definitions were skills, knowledge/expertise, and resources. Of the 57 articles with quantitative analysis, 48 (82%) measured capacity, and 11 (23%) offered psychometric data for the capacity measures. Of the 99 studies, 40% focused exclusively on populations experiencing inequities and 22% included those populations to some extent. The bulk of the studies came from high-income countries. Conclusions: Implementation scientists should 1) be explicit about models and definitions of capacity and strategies for building capacity, 2) specify expected multi-level implementation outcomes, 3) develop and use validated measures for quantitative work, and 4) integrate equity considerations into the conceptualization and measurement of capacity-building efforts. With these refinements, we can ensure that the necessary supports reach CBO practitioners and critical partners for addressing health inequities.
Collapse
|
13
|
Getting To Implementation (GTI)-Teach: A seven-step approach for teaching the fundamentals of implementation science. J Clin Transl Sci 2022; 6:e100. [PMID: 36106128 PMCID: PMC9428668 DOI: 10.1017/cts.2022.420] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/18/2022] [Revised: 05/31/2022] [Accepted: 06/13/2022] [Indexed: 11/07/2022] Open
Abstract
Introduction: Implementation Science (IS) is a complex and rapidly evolving discipline, posing challenges for educators. We developed, implemented, and evaluated a novel, pragmatic approach to teach IS. Methods: Getting To Implementation (GTI)-Teach was developed as a seven-step educational model to guide students through the process of developing, conducting, and sustaining an IS research project. During the four-week online course, students applied the steps to self-selected implementation problems. Students were invited to complete two online post-course surveys to assess course satisfaction and self-reported changes in IS knowledge and relevance of GTI-Teach Steps to their work. Results were summarized using descriptive statistics; self-reported post-course changes in IS knowledge were compared using paired t-tests. Results: GTI-Teach was developed to include seven Steps: 1. Define the implementation problem; 2. Conceptualize the problem; 3. Prioritize implementation barriers and facilitators; 4. Select and tailor implementation strategies; 5. Design an implementation study; 6. Evaluate implementation; 7. Sustain implementation. Thirteen students, ranging in experience from medical students to full professors, enrolled in and completed the first GTI-Teach course. Of the seven students (54%) completing an end-of course survey, six (86%) were very satisfied with the course. Ten students (77%) responded to the tailored, 6-month post-course follow-up survey. They retrospectively reported a significant increase in their knowledge across all steps of GTI-Teach (1.3–1.8 points on a 5-point Likert scale) and rated each of the Steps as highly relevant to their work. Conclusions: GTI-Teach is a seven-step model for teaching IS fundamentals that students reported increased their knowledge and was relevant to their work.
Collapse
|
14
|
House LD, Tevendale H, Brittain A, Burley K, Fuller TR, Mueller T, Romero L, Venugopalan B, Koumans EH. Implementation of Community‑Wide Initiatives Designed to Reduce Teen Pregnancy: Measuring Progress in a 5‑Year Project in 10 Communities. SEXUALITY RESEARCH & SOCIAL POLICY : JOURNAL OF NSRC : SR & SP 2022; 19:496-508. [PMID: 37664490 PMCID: PMC10473242 DOI: 10.1007/s13178-021-00565-y] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Accepted: 03/11/2021] [Indexed: 09/05/2023]
Abstract
Introduction Community-wide initiatives (CWI) to prevent teen pregnancy were implemented in 10 communities in the USA. The CWI supported the implementation of evidence-based teen pregnancy interventions (EBIs) and implementation of best practices for adolescent reproductive health care. Implementation was supported through mobilizing communities, educating stakeholders, and strategies to promote health equity. Methods We assessed indicators of progress of the CWI for the following five project components (data collected from 2010 to 2015): community mobilization, stakeholder education, working with diverse communities, evidence-based interventions, and increasing access to clinical services and the potential contributions of training and technical assistance. Results Communities engaged multiple stakeholder groups to contribute to planning, community outreach and education, and partnership development and used multiple dissemination methods to share information on adolescent reproductive health needs and teen pregnancy prevention strategies. The amount of training and technical assistance from state- and community-based organizations was associated with increased numbers of youth receiving EBIs and increased provision of contraceptives. The number of health centers implementing best practices for adolescent reproductive health services increased; conducting sexual health assessments, offering hormonal contraception or IUD, and offering quick start of IUDs were associated with increases in long-acting reversible contraception utilization. Conclusions These findings demonstrate that scaled prevention efforts can occur with adequate support including training and technical assistance and community awareness and engagement in the process. Policy Implications The findings raise important questions for understanding what factors contribute to successful community-wide implementation of EBIs and health center best practices for contraceptive access and whether these lead to reductions in teen pregnancies in highly impacted communities.
Collapse
Affiliation(s)
- L. Duane House
- Division of Reproductive Health, Centers for Disease Control and Prevention, Atlanta, GA, USA
| | - Heather Tevendale
- Division of Reproductive Health, Centers for Disease Control and Prevention, Atlanta, GA, USA
| | - Anna Brittain
- Division of Reproductive Health, Centers for Disease Control and Prevention, Atlanta, GA, USA
| | - Kim Burley
- Insignia Federal Group, Atlanta, GA, USA
| | - Taleria R. Fuller
- Division of Reproductive Health, Centers for Disease Control and Prevention, Atlanta, GA, USA
| | - Trish Mueller
- Division of Reproductive Health, Centers for Disease Control and Prevention, Atlanta, GA, USA
| | - Lisa Romero
- Division of Reproductive Health, Centers for Disease Control and Prevention, Atlanta, GA, USA
| | | | - Emilia H. Koumans
- Division of Reproductive Health, Centers for Disease Control and Prevention, Atlanta, GA, USA
| |
Collapse
|
15
|
Chinman M, Acosta J, Ebener P, Shearer A. "What We Have Here, Is a Failure to [Replicate]": Ways to Solve a Replication Crisis in Implementation Science. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2021; 23:739-750. [PMID: 34312769 DOI: 10.1007/s11121-021-01286-9] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 07/14/2021] [Indexed: 10/20/2022]
Abstract
Adapting the classic line from the 1967 film Cool Hand Luke, the title is meant to convey that implementation science (IS), like other fields, has not been embracing replication studies, which is a key component to the open science movement. The purpose of this article is to review what is known about replication of implementation trials and identify the gaps and next steps to continue increasing the transparency, openness, and replicability of implementation research. After presenting an overview of study replication and how it is a key component of open science, the article will examine how replication of implementation studies has (or more accurately has not) been approached in IS. As will be discussed, replication in IS shares some challenges with studies that attempt to replicate interventions, but also presents unique challenges. This article discusses different types of replications (e.g., direct vs. conceptual) and how they can benefit the field of IS. The article then presents a specific example of an implementation strategy called Getting To Outcomes© to describe how to design a replication study and interpret the results. The article ends with multiple options implementation scientists could consider to improve the likelihood and quality of replication studies. The discussion also envisions how implementation science can enable researchers and practitioners to work together in real-world contexts to encourage wide replication of implementation studies and advance the goal of improving public health.
Collapse
Affiliation(s)
- Matthew Chinman
- RAND Corporation, 4570 Fifth Ave #600, Pittsburgh, PA, 15213, USA.
| | - Joie Acosta
- RAND Corporation, 4570 Fifth Ave #600, Pittsburgh, PA, 15213, USA
| | - Patricia Ebener
- RAND Corporation, 4570 Fifth Ave #600, Pittsburgh, PA, 15213, USA
| | - Amy Shearer
- RAND Corporation, 4570 Fifth Ave #600, Pittsburgh, PA, 15213, USA
| |
Collapse
|
16
|
Zimba CC, Akiba CF, Matewere M, Thom A, Udedi M, Masiye JK, Kulisewa K, Go VFL, Hosseinipour MC, Gaynes BN, Pence BW. Facilitators, barriers and potential solutions to the integration of depression and non-communicable diseases (NCDs) care in Malawi: a qualitative study with service providers. Int J Ment Health Syst 2021; 15:59. [PMID: 34116699 PMCID: PMC8196431 DOI: 10.1186/s13033-021-00480-0] [Citation(s) in RCA: 8] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/07/2020] [Accepted: 05/25/2021] [Indexed: 11/24/2022] Open
Abstract
Background Integration of depression services into infectious disease care is feasible, acceptable, and effective in sub-Saharan African settings. However, while the region shifts focus to include chronic diseases, additional information is required to integrate depression services into chronic disease settings. We assessed service providers’ views on the concept of integrating depression care into non-communicable diseases’ (NCD) clinics in Malawi. The aim of this analysis was to better understand barriers, facilitators, and solutions to integrating depression into NCD services. Methods Between June and August 2018, we conducted nineteen in-depth interviews with providers. Providers were recruited from 10 public hospitals located within the central region of Malawi (i.e., 2 per clinic, with the exception of one clinic where only one provider was interviewed because of scheduling challenges). Using a semi structured interview guide, we asked participants questions related to their understanding of depression and its management at their clinic. We used thematic analysis allowing for both inductive and deductive approach. Themes that emerged related to facilitators, barriers and suggested solutions to integrate depression assessment and care into NCD clinics. We used CFIR constructs to categorize the facilitators and barriers. Results Almost all providers knew what depression is and its associated signs and symptoms. Almost all facilities had an NCD-dedicated room and reported that integrating depression into NCD care was feasible. Facilitators of service integration included readiness to integrate services by the NCD providers, availability of antidepressants at the clinic. Barriers to service integration included limited knowledge and lack of training regarding depression care, inadequacy of both human and material resources, high workload experienced by the providers and lack of physical space for some depression services especially counseling. Suggested solutions were training of NCD staff on depression assessment and care, engaging hospital leaders to create an NCD and depression care integration policy, integrating depression information into existing documents, increasing staff, and reorganizing clinic flow. Conclusion Findings of this study suggest a need for innovative implementation science solutions such as reorganizing clinic flow to increase the quality and duration of the patient-provider interaction, as well as ongoing trainings and supervisions to increase clinical knowledge. Trial registration This study reports finding of part of the formative phase of “The Sub-Saharan Africa Regional Partnership (SHARP) for Mental Health Capacity Building—A Clinic-Randomized Trial of Strategies to Integrate Depression Care in Malawi” registered as NCT03711786
Collapse
Affiliation(s)
| | - Christopher F Akiba
- Gillings School of Global and Public Health, University of North Carolina At Chapel Hill, Chapel Hill, NC, USA
| | | | - Annie Thom
- Malawi Ministry of Health, Lilongwe, Malawi
| | | | | | | | | | | | | | | |
Collapse
|
17
|
Rogal SS, Yakovchenko V, Morgan T, Bajaj JS, Gonzalez R, Park A, Beste L, Miech EJ, Lamorte C, Neely B, Gibson S, Malone PS, Chartier M, Taddei T, Garcia-Tsao G, Powell BJ, Dominitz JA, Ross D, Chinman MJ. Getting to implementation: a protocol for a Hybrid III stepped wedge cluster randomized evaluation of using data-driven implementation strategies to improve cirrhosis care for Veterans. Implement Sci 2020; 15:92. [PMID: 33087156 PMCID: PMC7579930 DOI: 10.1186/s13012-020-01050-7] [Citation(s) in RCA: 13] [Impact Index Per Article: 3.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/05/2020] [Accepted: 10/05/2020] [Indexed: 02/07/2023] Open
Abstract
BACKGROUND Cirrhosis is a rapidly increasing cause of global mortality. To improve cirrhosis care, the Veterans Health Administration (VHA) developed the Hepatic Innovation Team (HIT) Collaborative to support VA Medical Centers (VAMCs) to deliver evidence-based cirrhosis care. This randomized HIT program evaluation aims to develop and assess a novel approach for choosing and applying implementation strategies to improve the quality of cirrhosis care. METHODS Evaluation aims are to (1) empirically determine which combinations of implementation strategies are associated with successful implementation of evidence-based practices (EBPs) for Veterans with cirrhosis, (2) manualize these "data-driven" implementation strategies, and (3) assess the effectiveness of data-driven implementation strategies in increasing cirrhosis EBP uptake. Aim 1 will include an online survey of all VAMCs' use of 73 implementations strategies to improve cirrhosis care, as defined by the Expert Recommendations for Implementing Change taxonomy. Traditional statistical as well as configurational comparative methods will both be employed to determine which combinations of implementation strategies are associated with site-level adherence to EBPs for cirrhosis. In aim 2, semi-structured interviews with high-performing VAMCs will be conducted to operationalize successful implementation strategies for cirrhosis care. These data will be used to inform the creation of a step-by-step guide to tailoring and applying the implementation strategies identified in aim 1. In aim 3, this manualized implementation intervention will be assessed using a hybrid type III stepped-wedge cluster randomized design. This evaluation will be conducted in 12 VAMCs, with four VAMCs crossing from control to intervention every 6 months, in order to assess the effectiveness of using data-driven implementation strategies to improve guideline-concordant cirrhosis care. DISCUSSION Successful completion of this innovative evaluation will establish the feasibility of using early evaluation data to inform a manualized, user-friendly implementation intervention for VAMCs with opportunities to improve care. This evaluation will provide implementation support tools that can be applied to enhance the implementation of other evidence-based practices. TRIAL REGISTRATION This project was registered at ClinicalTrials.Gov ( NCT04178096 ) on 4/29/20.
Collapse
Affiliation(s)
- Shari S Rogal
- Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Research Office Building (151R), University Drive C, Pittsburgh, PA, 15240, USA. .,Departments of Medicine and Surgery, University of Pittsburgh, Pittsburgh, PA, USA.
| | - Vera Yakovchenko
- Center for Healthcare Organization and Implementation Research, Edith Nourse Rogers Memorial VA Hospital, Bedford, MA, USA
| | - Timothy Morgan
- Gastroenterology Section, VA Long Beach Healthcare System, Long Beach, CA, USA.,Division of Gastroenterology, Department of Medicine, University of California, Irvine, CA, USA
| | - Jasmohan S Bajaj
- Division of Gastroenterology, Hepatology, and Nutrition, Virginia Commonwealth University, Richmond, VA, USA.,Division of Gastroenterology, Hunter Holmes McGuire VA Medical Center, Richmond, VA, USA
| | - Rachel Gonzalez
- Department of Veterans Affairs, Sierra Pacific Veterans Integrated Service Network, Pharmacy Benefits Management, Mather, CA, USA
| | - Angela Park
- Office of Healthcare Transformation, Veterans Engineering Resource Center, Washington, DC, USA
| | - Lauren Beste
- Division of General Internal Medicine, Department of Medicine, VA Puget Sound Healthcare System, Seattle, WA, USA.,Division of General Internal Medicine, University of Washington, Seattle, WA, USA
| | - Edward J Miech
- Department of Veterans Affairs, Roudebush VA Medical Center, HSR&D Center for Health Information & Communication, VA PRIS-M QUERI, Indianapolis, IN, USA
| | - Carolyn Lamorte
- Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Research Office Building (151R), University Drive C, Pittsburgh, PA, 15240, USA
| | - Brittney Neely
- Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Research Office Building (151R), University Drive C, Pittsburgh, PA, 15240, USA
| | - Sandra Gibson
- Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Research Office Building (151R), University Drive C, Pittsburgh, PA, 15240, USA
| | | | - Maggie Chartier
- HIV, Hepatitis and Related Conditions Programs, Office of Specialty Care Services, Veterans Health Administration, Washington, DC, USA
| | - Tamar Taddei
- VA Connecticut Healthcare System, West Haven, CT, USA.,Department of Medicine, Yale University, West Haven, CT, USA
| | - Guadalupe Garcia-Tsao
- VA Connecticut Healthcare System, West Haven, CT, USA.,Department of Medicine, Yale University, West Haven, CT, USA
| | - Byron J Powell
- Brown School, Washington University in St. Louis, St. Louis, MO, USA
| | - Jason A Dominitz
- Gastroenterology Section, VA Puget Sound Health Care System, Seattle, WA, USA.,Department of Medicine, University of Washington, Seattle, WA, USA
| | - David Ross
- HIV, Hepatitis and Related Conditions Programs, Office of Specialty Care Services, Veterans Health Administration, Washington, DC, USA
| | - Matthew J Chinman
- Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Research Office Building (151R), University Drive C, Pittsburgh, PA, 15240, USA.,RAND Corporation, Pittsburgh, PA, USA
| |
Collapse
|
18
|
Cost analysis of a randomized trial of Getting to Outcomes implementation support for a teen pregnancy prevention program offered in Boys and Girls Clubs in Alabama and Georgia. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2020; 21:1114-1125. [PMID: 32880842 DOI: 10.1007/s11121-020-01162-y] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/23/2022]
Abstract
Implementation support can improve outcomes of evidence-based programs (EBP) for adolescents, but with a cost. To assist in determining whether this cost is worthwhile, this study estimated the cost of adding Getting To Outcomes© (GTO) implementation support to a teen pregnancy and sexually transmitted infection prevention EBP called Making Proud Choices (MPC) in 32 Boys and Girls Clubs (BGCs) in Alabama and Georgia. Enhancing Quality Interventions Promoting Healthy Sexuality (EQUIPS) was a 2-year, cluster-randomized controlled trial comparing MPC with MPC + GTO. We used micro-costing to estimate costs and captured MPC and GTO time from activity logs completed by GTO staff. Key resource use and cost components were compared between the randomized groups, years, and states (to capture different community site circumstances) using 2-sample t tests. There were no significant differences between randomized groups in attendees per site, resource use, or costs for either year. However, there were significant differences between states. Adding GTO to MPC increased the societal costs per attendee from $67 to $144 (2015 US dollars) in Georgia and from $106 to $314 in Alabama. The higher Alabama cost was due to longer travel distances and to more BGC staff time spent on GTO in that state. GTO also improved adherence, classroom delivery, and condom-use intentions more in Alabama youth. Thus, Alabama's GTO-related BGC staff time costs may be better estimates of effective GTO. If teen childbearing costs taxpayers approximately $20,000 per teen birth, adding GTO to MPC would be worthwhile to society if it prevented one more teen birth per 140 attendees than MPC alone.Trial registration. ClinicalTrials.gov , NCT01818791. Registered March 26, 2013, https://clinicaltrials.gov/ct2/show/NCT01818791?term=NCT01818791&draw=2&rank=1.
Collapse
|
19
|
Kavanagh S, Shiell A, Hawe P, Garvey K. Resources, relationships, and systems thinking should inform the way community health promotion is funded. CRITICAL PUBLIC HEALTH 2020. [DOI: 10.1080/09581596.2020.1813255] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/03/2023]
Affiliation(s)
- Shane Kavanagh
- School of Health and Social Development, Faculty of Health, Deakin University, Geelong, Australia
| | - Alan Shiell
- Department of Public Health, La Trobe University, Melbourne, Australia
| | - Penelope Hawe
- Menzies Centre for Health Policy, Sydney School of Public Health, University of Sydney, Sydney, Australia
| | - Kate Garvey
- Manager, Partnership Development, Public Health Services, Tasmanian Department of Health, Hobart, Australia and Adjunct Associate Professor, Department of Public Health, La Trobe University, Melbourne, Australia
| |
Collapse
|
20
|
Cannon JS, Gilbert M, Ebener P, Malone PS, Reardon CM, Acosta J, Chinman M. Influence of an Implementation Support Intervention on Barriers and Facilitators to Delivery of a Substance Use Prevention Program. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2020; 20:1200-1210. [PMID: 31473932 DOI: 10.1007/s11121-019-01037-x] [Citation(s) in RCA: 14] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/29/2022]
Abstract
Implementation support interventions have helped organizations implement programs with quality and obtain intended outcomes. For example, a recent randomized controlled trial called Preparing to Run Effective Programs (PREP) showed that an implementation support intervention called Getting To Outcomes (GTO) improved implementation of an evidence-based substance use prevention program (CHOICE) run in community-based settings. However, more information is needed on how these interventions affect organizational barriers and facilitators of implementation. This paper aims to identify differences in implementation facilitators and barriers in sites conducting a substance use prevention program with and without GTO. PREP is a cluster-randomized controlled trial testing GTO, a two-year implementation support intervention, in Boys & Girls Clubs. The trial compares 15 Boys & Girls Club sites implementing CHOICE (control group), a five-session evidence-based alcohol and drug prevention program, with 14 Boys & Girls Club sites implementing CHOICE supported by GTO (intervention group). All sites received CHOICE training. Intervention sites also received GTO manuals, training, and onsite technical assistance to help practitioners complete implementation best practices specified by GTO (i.e., GTO steps). During the first year, technical assistance providers helped the intervention group adopt, plan, and deliver CHOICE, and then evaluate and make quality improvements to CHOICE implementation using feedback reports summarizing their data. Following the second year of CHOICE and GTO implementation, all sites participated in semi-structured interviews to identify barriers and facilitators to CHOICE implementation using the Consolidated Framework for Implementation Research (CFIR). This paper assesses the extent to which these facilitators and barriers differed between intervention and control group. Intervention sites had significantly higher average ratings than control sites for two constructs from the CFIR process domain: planning and reflecting and evaluating. At the same time, intervention sites had significantly lower ratings on the culture and available resources constructs. Findings suggest that strong planning, evaluation, and reflection-likely improved with GTO support-can facilitate implementation even in the face of perceptions of a less desirable implementation climate. These findings highlight that implementation support, such as GTO, is likely to help low-resourced community-based organizations improve program delivery through a focus on implementation processes. TRIAL REGISTRATION: This project is registered at ClinicalTrials.gov with number NCT02135991 (URL: https://clinicaltrials.gov/show/NCT02135991). The trial was first registered May 12, 2014.
Collapse
Affiliation(s)
- Jill S Cannon
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA.
| | - Marylou Gilbert
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA
| | - Patricia Ebener
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA
| | | | - Caitlin M Reardon
- Center for Clinical Management Research, VA Ann Arbor Healthcare System, Ann Arbor, MI, USA
| | - Joie Acosta
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA
| | - Matthew Chinman
- RAND Corporation, 1776 Main Street, Santa Monica, CA, 90401, USA
| |
Collapse
|
21
|
Strompolis M, Cain JM, Wilson A, Aldridge WA, Armstrong JM, Srivastav A. Community capacity coach: Embedded support to implement evidenced-based prevention. JOURNAL OF COMMUNITY PSYCHOLOGY 2020; 48:1132-1146. [PMID: 32442336 DOI: 10.1002/jcop.22375] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 05/30/2019] [Revised: 04/28/2020] [Accepted: 04/29/2020] [Indexed: 06/11/2023]
Abstract
Given the pervasive nature of child abuse and neglect, multilevel public health approaches are necessary. Implementation science can help in understanding the most effective ways to build infrastructure and support delivery of such approaches. In this theoretical paper, we describe the implementation of the Positive Parenting Program (Triple P), an evidence-based population-level parenting program in South Carolina. While implementation science has informed how to move population-level efforts to scale, we discuss challenges that persist in practice, such as when there is a need for multiple stakeholders to understand, support, and apply implementation best practices in a systematic and consistent way. To address this challenge, we introduce the role of a Community Capacity Coach, who lives in the local community and works towards the implementation of Triple P. The Coach is responsible for bridging gaps between the local community and statewide support systems. We detail the ways in which the Coach's role aligns with key intermediary functions, and how the Coach is embedded within the larger Triple P statewide support system. We then discuss the assessment of the Coach's impact; and conclude with future directions and next steps for this role within Triple P South Carolina.
Collapse
Affiliation(s)
- Melissa Strompolis
- Children's Trust of South Carolina, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | - J Montana Cain
- Children's Trust of South Carolina, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | - Abygail Wilson
- Children's Trust of South Carolina, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | - William A Aldridge
- The Impact Center at FPG Child Development Institute, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | - Jenna M Armstrong
- The Impact Center at FPG Child Development Institute, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| | - Aditi Srivastav
- Children's Trust of South Carolina, University of North Carolina at Chapel Hill, Chapel Hill, North Carolina
| |
Collapse
|
22
|
Acosta J, Chinman M, Ebener PA, Malone PS, Cannon JS, D'Amico EJ. Sustaining an Evidence-Based Program Over Time: Moderators of Sustainability and the Role of the Getting to Outcomes® Implementation Support Intervention. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2020; 21:807-819. [PMID: 32323166 DOI: 10.1007/s11121-020-01118-2] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.8] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/12/2023]
Abstract
Problematic rates of alcohol, e-cigarette, and other drug use among US adolescents highlight the need for effective implementation of evidence-based programs (EBPs), yet schools and community organizations have great difficulty implementing and sustaining EBPs. Although a growing number of studies show that implementation support interventions can improve EBP implementation, the literature on how to improve sustainability through implementation support is limited. This randomized controlled trial advances the literature by testing the effects of one such implementation intervention-Getting To Outcomes (GTO)-on sustainability of CHOICE, an after-school EBP for preventing substance use among middle-school students. CHOICE implementation was tracked for 2 years after GTO support ended across 29 Boys and Girls Club sites in the greater Los Angeles area. Predictors of sustainability were identified for a set of key tasks targeted by the GTO approach (e.g., goal setting, evaluation, collectively called "GTO performance") and for CHOICE fidelity using a series of path models. One year after GTO support ended, we found no differences between GTO and control sites on CHOICE fidelity. GTO performance was also similar between groups; however, GTO sites were superior in conducting evaluation. Better GTO performance predicted better CHOICE fidelity. Two years after GTO support ended, GTO sites were significantly more likely to sustain CHOICE implementation when compared with control sites. This study suggests that using an implementation support intervention like GTO can help low-resource settings continue to sustain their EBP implementation to help them get the most out of their investment. ClinicalTrials.gov Identifier: NCT02135991.
Collapse
|
23
|
Quetsch LB, Herschell AD, Kogan JN, Gavin JG, Hale G, Stein BD. Community-based behavioral health administrator perspectives on sustainability of Dialectical Behavior Therapy: a qualitative evaluation. Borderline Personal Disord Emot Dysregul 2020; 7:5. [PMID: 32161650 PMCID: PMC7047370 DOI: 10.1186/s40479-020-0120-5] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/12/2019] [Accepted: 02/11/2020] [Indexed: 11/24/2022] Open
Abstract
BACKGROUND Substantial resources have been invested in evidence-based practice (EBP) implementation in community settings; however, research suggests that EBPs do not always sustain over time. METHOD This qualitative study explored the perspectives of 13 community behavioral health agency leaders regarding the sustainability of an EBP 25 to 28 months following the original training period. Administrators from 10 agencies were interviewed to understand the complexities of the implementation process, sustainability of Dialectical Behavior Therapy, and their recommendations to enhance implementation and sustainability. RESULTS A content analysis revealed five emergent themes: treatment model opinions, resource concerns, staff selection/ turnover, population characteristics, and recommendations for future implementation. CONCLUSIONS These themes likely would be helpful in informing the design of future implementation and sustainability initiatives sensitive to the challenges of integrating EBPs in community settings.
Collapse
Affiliation(s)
| | - Amy D Herschell
- 2Community Care Behavioral Health Organization, UPMC Insurance Services Division, Pittsburgh, USA
| | - Jane N Kogan
- 3UPMC Center for High-Value Health Care, UPMC Insurance Services Division, Pittsburgh, USA
| | - James G Gavin
- 2Community Care Behavioral Health Organization, UPMC Insurance Services Division, Pittsburgh, USA
| | | | - Bradley D Stein
- 2Community Care Behavioral Health Organization, UPMC Insurance Services Division, Pittsburgh, USA.,5RAND Corporation, Pittsburgh, USA
| |
Collapse
|
24
|
Cost Analysis of a Randomized Trial of Getting to Outcomes Implementation Support of CHOICE in Boys and Girls Clubs in Southern California. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2019; 21:245-255. [PMID: 31865544 DOI: 10.1007/s11121-019-01082-6] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/25/2022]
Abstract
Costs of supporting prevention program implementation are not well known. This study estimates the societal costs of implementing CHOICE, a voluntary after-school alcohol and other drug prevention program for adolescents, in Boys and Girls Clubs (BGCs) across Southern California with and without an implementation support system called Getting To Outcomes© (GTO). This article uses micro-costing methods to estimate the cost of the CHOICE program and GTO support. Labor and expense data were obtained from logs kept by the BGC staff and by the GTO technical assistance (TA) staff, and staff time was valued based on Bureau of Labor Statistics estimates. From the societal perspective, the cost of implementing CHOICE at BGCs over the 2-year study period was $27 per attendee when CHOICE was offered by itself (all costs incurred by the BGCs) and $177 per attendee when CHOICE was offered with GTO implementation support ($67 cost to the BGCs; $110 to the entity funding GTO). These results were most sensitive to assumptions as to the number of times CHOICE was offered per year. Adding GTO implementation support to CHOICE increased the cost per attendee by approximately $150. For this additional cost, there was evidence that the CHOICE program was offered with more fidelity and offered more often after the 2-year intervention ended. If the long-term benefits of this better and continued implementation are found to exceed these additional costs, GTO could be an attractive structure to support evidence-based substance misuse prevention programs. Trial Registration. This project is registered at ClinicalTrials.gov with number NCT02135991 (URL: https://clinicaltrials.gov/show/NCT02135991). The trial was registered May 12, 2014.
Collapse
|
25
|
Powell BJ, Fernandez ME, Williams NJ, Aarons GA, Beidas RS, Lewis CC, McHugh SM, Weiner BJ. Enhancing the Impact of Implementation Strategies in Healthcare: A Research Agenda. Front Public Health 2019; 7:3. [PMID: 30723713 PMCID: PMC6350272 DOI: 10.3389/fpubh.2019.00003] [Citation(s) in RCA: 350] [Impact Index Per Article: 70.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/16/2018] [Accepted: 01/04/2019] [Indexed: 01/10/2023] Open
Abstract
The field of implementation science was developed to better understand the factors that facilitate or impede implementation and generate evidence for implementation strategies. In this article, we briefly review progress in implementation science, and suggest five priorities for enhancing the impact of implementation strategies. Specifically, we suggest the need to: (1) enhance methods for designing and tailoring implementation strategies; (2) specify and test mechanisms of change; (3) conduct more effectiveness research on discrete, multi-faceted, and tailored implementation strategies; (4) increase economic evaluations of implementation strategies; and (5) improve the tracking and reporting of implementation strategies. We believe that pursuing these priorities will advance implementation science by helping us to understand when, where, why, and how implementation strategies improve implementation effectiveness and subsequent health outcomes.
Collapse
Affiliation(s)
- Byron J Powell
- Department of Health Policy and Management, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States.,Cecil G. Sheps Center for Health Services Research, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States.,Frank Porter Graham Child Development Institute, University of North Carolina at Chapel Hill, Chapel Hill, NC, United States
| | - Maria E Fernandez
- Center for Health Promotion and Prevention Research, School of Public Health, University of Texas Health Science Center at Houston, Houston, TX, United States
| | | | - Gregory A Aarons
- Department of Psychiatry, University of California, San Diego, La Jolla, CA, United States
| | - Rinad S Beidas
- Department of Psychiatry, Center for Mental Health, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States.,Department of Medical Ethics and Health Policy, Perelman School of Medicine, University of Pennsylvania, Philadelphia, PA, United States.,Leonard Davis Institute of Health Economics, University of Pennsylvania, Philadelphia, PA, United States
| | - Cara C Lewis
- MacColl Center for Healthcare Innovation, Kaiser Permanente Washington Health Research Institute, Seattle, WA, United States
| | - Sheena M McHugh
- School of Public Health, University College Cork, Cork, Ireland
| | - Bryan J Weiner
- Department of Global Health, Department of Health Services, University of Washington, Seattle, WA, United States
| |
Collapse
|
26
|
Chinman M, Gellad WF, McCarthy S, Gordon AJ, Rogal S, Mor MK, Hausmann LRM. Protocol for evaluating the nationwide implementation of the VA Stratification Tool for Opioid Risk Management (STORM). Implement Sci 2019; 14:5. [PMID: 30658658 PMCID: PMC6339438 DOI: 10.1186/s13012-019-0852-z] [Citation(s) in RCA: 13] [Impact Index Per Article: 2.6] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/18/2018] [Accepted: 01/07/2019] [Indexed: 12/11/2022] Open
Abstract
BACKGROUND Mitigating the risks of adverse outcomes from opioids is critical. Thus, the Veterans Affairs (VA) Healthcare System developed the Stratification Tool for Opioid Risk Management (STORM), a dashboard to assist clinicians with opioid risk evaluation and mitigation. Updated daily, STORM calculates a "risk score" of adverse outcomes (e.g., suicide-related events, overdoses, overdose death) from variables in the VA medical record for all patients with an opioid prescription and displays this information along with documentation of recommended risk mitigation strategies and non-opioid pain treatments. In March 2018, the VA issued a policy notice requiring VA Medical Centers (VAMCs) to complete case reviews for patients whom STORM identifies as very high-risk (i.e., top 1% of STORM risk scores). Half of VAMCs were randomly assigned notices that also stated that additional support and oversight would be required for VAMCs that failed to meet an established percentage of case reviews. Using a stepped-wedge cluster randomized design, VAMCs will be further randomized to conduct case reviews for an expanded pool of patients (top 5% of STORM risk scores vs. 1%) starting either 9 or 15 months after the notice was released, creating four natural arms. VA commissioned an evaluation to understand the implementation strategies and factors associated with case review completion rates, whose protocol is described in this report. METHODS This mixed-method study will include an online survey of all VAMCs to identify implementation strategies and interviews at a subset of facilities to identify implementation barriers and facilitators. The survey is based on the Expert Recommendations for Implementing Change (ERIC) project, which engaged experts to create consensus on 73 implementation strategies. We will use regression models to compare the number and types of implementation strategies across arms and their association with case review completion rates. Using questions from the Consolidated Framework for Implementation Research, we will interview stakeholders at 40 VAMCs with the highest and lowest adherence to opioid therapy guidelines. DISCUSSION By identifying which implementation strategies, barriers, and facilitators influence case reviews to reduce opioid-related adverse outcomes, this unique implementation evaluation will enable the VA to improve the design of future opioid safety initiatives. TRIAL REGISTRATION This project is registered at the ISRCTN Registry with number ISRCTN16012111 . The trial was first registered on 5/3/2017.
Collapse
Affiliation(s)
- Matthew Chinman
- Veterans Integrated Service Network 4 Mental Illness Research, Education and Clinical Center, VA Pittsburgh, Pittsburgh, PA USA
- Center for Health Equity Research and Promotion, Veterans Affairs Pittsburgh, Pittsburgh, PA USA
- RAND Corporation, Pittsburgh, PA USA
- VA Pittsburgh Healthcare System, Research Office Building (151R), University Drive C, Pittsburgh, PA 15240 USA
| | - Walid F. Gellad
- Center for Health Equity Research and Promotion, Veterans Affairs Pittsburgh, Pittsburgh, PA USA
- Department of Medicine, University of Pittsburgh, Pittsburgh, PA USA
| | - Sharon McCarthy
- Veterans Integrated Service Network 4 Mental Illness Research, Education and Clinical Center, VA Pittsburgh, Pittsburgh, PA USA
- Center for Health Equity Research and Promotion, Veterans Affairs Pittsburgh, Pittsburgh, PA USA
| | - Adam J. Gordon
- Program for Addiction Research, Clinical Care, Knowledge, and Advocacy, University of Utah School of Medicine, Salt Lake City, UT USA
- Informatics, Decision-Enhancement, and Analytic Sciences Center, VA Salt Lake City Health Care System, Salt Lake City, UT USA
| | - Shari Rogal
- Center for Health Equity Research and Promotion, Veterans Affairs Pittsburgh, Pittsburgh, PA USA
- Department of Medicine, University of Pittsburgh, Pittsburgh, PA USA
- Department of Surgery, University of Pittsburgh School of Medicine, Pittsburgh, PA USA
| | - Maria K. Mor
- Center for Health Equity Research and Promotion, Veterans Affairs Pittsburgh, Pittsburgh, PA USA
| | - Leslie R. M. Hausmann
- Center for Health Equity Research and Promotion, Veterans Affairs Pittsburgh, Pittsburgh, PA USA
- Department of Medicine, University of Pittsburgh, Pittsburgh, PA USA
| |
Collapse
|
27
|
Chinman M, Ebener P, Malone PS, Cannon J, D'Amico EJ, Acosta J. Testing implementation support for evidence-based programs in community settings: a replication cluster-randomized trial of Getting To Outcomes®. Implement Sci 2018; 13:131. [PMID: 30348227 PMCID: PMC6196461 DOI: 10.1186/s13012-018-0825-7] [Citation(s) in RCA: 22] [Impact Index Per Article: 3.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/13/2018] [Accepted: 10/15/2018] [Indexed: 11/10/2022] Open
Abstract
Background Community organizations can have difficulty implementing evidence-based prevention programs. More research is needed on implementation support interventions designed to help these organizations implement programs with quality. Methods Preparing to Run Effective Programs (PREP) is a randomized controlled trial testing Getting To Outcomes (GTO), a 2-year implementation support intervention. It compares 15 Boys and Girls Club sites implementing CHOICE (control group), a five-session evidence-based alcohol and drug prevention program, with 14 similar sites implementing CHOICE supported by GTO (intervention group). PREP replicates a previous GTO study that had the same design, but featured a teen pregnancy prevention program instead. All sites received typical CHOICE training. Fourteen intervention sites received GTO manuals, training, and onsite technical assistance to help practitioners complete implementation best practices specified by GTO (i.e., GTO steps). During the first year, technical assistance providers helped the intervention group adopt, plan, and deliver CHOICE. Then, this group was trained on evaluation and quality improvement steps of GTO using feedback reports summarizing their own data, which yielded revised plans for subsequent implementation of CHOICE. This paper presents results regarding GTO’s impact on CHOICE fidelity (adherence, quality of delivery, dosage) and the proximal outcomes of the youth participants (aged 10–14)—attitudes and intentions regarding cigarettes, alcohol, and marijuana use. Fidelity was assessed at all sites by observer ratings and attendance logs. Proximal outcomes were assessed via survey at baseline, 3, and 6 months. Results After 1 year, fidelity and proximal outcomes were similar between Intervention and control groups. After 2 years (which included GTO quality improvement activities that took place between years 1 and 2), intervention sites had higher ratings of CHOICE adherence and quality of delivery (dosage remained similar). Proximal outcomes did not differ between groups in either year, although there was universally high endorsement of prosocial responses to those outcomes from the start. Conclusions Findings suggest that systematic implementation support provided by GTO can help community organizations achieve better fidelity. Findings replicate the implementation results from a previous GTO study using the same design, but with a different evidence-based program and different fidelity measures. Although proximal outcomes did not change, in large part due to ceiling effects, the implementation findings suggest GTO can support a variety of programs. Trial registration This project is registered at ClinicalTrials.gov with number NCT02135991. The trial was first registered on May 12, 2014.
Collapse
Affiliation(s)
- Matthew Chinman
- RAND Corporation, 4570 Fifth Avenue, Pittsburgh, PA, 15213, USA.
| | - Patricia Ebener
- RAND Corporation, 4570 Fifth Avenue, Pittsburgh, PA, 15213, USA
| | | | - Jill Cannon
- RAND Corporation, 4570 Fifth Avenue, Pittsburgh, PA, 15213, USA
| | | | - Joie Acosta
- RAND Corporation, 4570 Fifth Avenue, Pittsburgh, PA, 15213, USA
| |
Collapse
|
28
|
Chinman M, Acosta J, Ebener P, Malone PS, Slaughter ME. A Cluster-Randomized Trial of Getting To Outcomes' Impact on Sexual Health Outcomes in Community-Based Settings. PREVENTION SCIENCE : THE OFFICIAL JOURNAL OF THE SOCIETY FOR PREVENTION RESEARCH 2018; 19:437-448. [PMID: 28971273 PMCID: PMC5880746 DOI: 10.1007/s11121-017-0845-6] [Citation(s) in RCA: 11] [Impact Index Per Article: 1.8] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/01/2022]
Abstract
The USA has high teen pregnancy rates compared to other developed nations. Many community-based organizations need assistance conducting evidence-based teen pregnancy prevention programs (EBPs) appropriately. This study evaluated the impact of an implementation support intervention called Getting To Outcomes (GTO) designed to help such organizations. This cluster randomized controlled trial compared 16 Boys and Girls Clubs (BGCs) implementing a teen pregnancy prevention EBP called Making Proud Choices for two years, with 16 BGCs implementing MPC augmented with GTO training, tools, and technical assistance. Participating middle school youth were compared on proximal outcomes (knowledge, attitudes, and intentions about sex and condoms from baseline to post) and sexual behaviors (frequency of sex and condom use, from baseline to 6-month follow-up). In year 1, there were no significant effects of GTO for any proximal outcome. After GTO-stimulated quality improvement in year 2, the GTO group improved significantly more on condom attitudes and use intentions. Frequency of sex and condom use did not differ between the two groups in either year; however, base rates of these behaviors in the sample were very low. Findings suggest that in typical community-based settings, detailed manuals and training common to structured EBPs may be sufficient to yield some improvement in key proximal outcomes, but that more systematic implementation support is needed to achieve greater improvement in these outcomes. Using GTO with many communities, as currently supported by various federal agencies, could yield public health impact via improvements in condom attitudes and use intentions.
Collapse
Affiliation(s)
- Matthew Chinman
- RAND Corporation, 4570 Fifth Avenue, Pittsburgh, PA, 15213, USA.
| | - Joie Acosta
- RAND Corporation, 4570 Fifth Avenue, Pittsburgh, PA, 15213, USA
| | - Patricia Ebener
- RAND Corporation, 4570 Fifth Avenue, Pittsburgh, PA, 15213, USA
| | | | | |
Collapse
|
29
|
Scarrow G, Angus D, Holmes BJ. Reviewer training to assess knowledge translation in funding applications is long overdue. Res Integr Peer Rev 2017; 2:13. [PMID: 29451528 PMCID: PMC5803621 DOI: 10.1186/s41073-017-0037-8] [Citation(s) in RCA: 4] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/23/2016] [Accepted: 05/30/2017] [Indexed: 11/19/2022] Open
Abstract
Background Health research funding agencies are placing a growing focus on knowledge translation (KT) plans, also known as dissemination and implementation (D&I) plans, in grant applications to decrease the gap between what we know from research and what we do in practice, policy, and further research. Historically, review panels have focused on the scientific excellence of applications to determine which should be funded; however, relevance to societal health priorities, the facilitation of evidence-informed practice and policy, or realizing commercialization opportunities all require a different lens. Discussion While experts in their respective fields, grant reviewers may lack the competencies to rigorously assess the KT components of applications. Funders of health research—including health charities, non-profit agencies, governments, and foundations—have an obligation to ensure that these components of funding applications are as rigorously evaluated as the scientific components. In this paper, we discuss the need for a more rigorous evaluation of knowledge translation potential by review panels and propose how this may be addressed. Conclusion We propose that reviewer training supported in various ways including guidelines and KT expertise on review panels and modalities such as online and face-to-face training will result in the rigorous assessment of all components of funding applications, thus increasing the relevance and use of funded research evidence. An unintended but highly welcome consequence of such training could be higher quality D&I or KT plans in subsequent funding applications from trained reviewers.
Collapse
Affiliation(s)
- Gayle Scarrow
- Michael Smith Foundation for Health Research, Vancouver, British Columbia Canada
| | - Donna Angus
- Michael Smith Foundation for Health Research, Vancouver, British Columbia Canada
| | - Bev J Holmes
- Michael Smith Foundation for Health Research, Vancouver, British Columbia Canada
| |
Collapse
|
30
|
A systematic review of peer-supported interventions for health promotion and disease prevention. Prev Med 2017; 101:156-170. [PMID: 28601621 DOI: 10.1016/j.ypmed.2017.06.008] [Citation(s) in RCA: 62] [Impact Index Per Article: 8.9] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 12/01/2016] [Revised: 05/08/2017] [Accepted: 06/05/2017] [Indexed: 01/01/2023]
Abstract
UNLABELLED Prior research has examined peer programs with respect to specific peer roles (e.g.; peer support) or specific health/wellness domains (e.g.; exercise/diet), or have aggregated effects across roles and domains. We sought to conduct a systematic review that categorizes and assesses the effects of peer interventions to promote health and wellness by peer role, intervention type, and outcomes. We use evidence mapping to visually catalog and synthesize the existing research. We searched PubMed and WorldCat databases (2005 to 2015) and New York Academy of Medicine Grey Literature Report (1999 to 2016) for English-language randomized control trials. We extracted study design, study participants, type of intervention(s), peer role(s), outcomes assessed and measures used, and effects from 116 randomized controlled trials. Maps were created to provide a visual display of the evidence by intervention type, peer role, outcome type, and significant vs null or negative effects. There are more null than positive effects across peer interventions, with notable exceptions: group-based interventions that use peers as educators or group facilitators commonly improve knowledge, attitudes, beliefs, and perceptions; peer educators also commonly improved social health/connectedness and engagement. Dyadic peer support influenced behavior change and peer counseling shows promising effects on physical health outcomes. Programs seeking to use peers in public health campaigns can use evidence maps to identify interventions that have previously demonstrated beneficial effects. Those seeking to produce health outcomes may benefit from identifying the mechanisms by which they expect their program to produce these effects and associated proximal outcomes for future evaluations. PROSPERO REGISTRATION NUMBER Although we attempted to register our protocol with PROSPERO, we did not meet eligibility criteria because we were past the data collection phase. The full PROSPERO-aligned protocol is available from the authors.
Collapse
|
31
|
Leviton LC. Generalizing about Public Health Interventions: A Mixed-Methods Approach to External Validity. Annu Rev Public Health 2017; 38:371-391. [DOI: 10.1146/annurev-publhealth-031816-044509] [Citation(s) in RCA: 28] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/09/2022]
Abstract
Public health researchers and practitioners are calling for greater focus on external validity, the ability to generalize findings of evidence-based interventions (EBIs) beyond the limited number of studies testing effectiveness. For public health, the goal is applicability: to translate, disseminate, and implement EBIs for an impact on population health. This article is a review of methods and how they might be combined to better assess external validity. The methods include (a) better description of EBIs and their contexts; (b) combining of statistical tools and logic to draw inferences about study samples; (c) sharper definition of the theory behind the intervention and core intervention components; and (d) more systematic consultation of practitioners. For population impact, studies should focus on context features that are likely to be both important (based on program theory) and frequently encountered by practitioners. Mixed-method programs of research will allow public health to expand causal generalizations.
Collapse
Affiliation(s)
- Laura C. Leviton
- The Robert Wood Johnson Foundation, Princeton, New Jersey 08543-2316
| |
Collapse
|
32
|
Chinman M, McCarthy S, Hannah G, Byrne TH, Smelson DA. Using Getting To Outcomes to facilitate the use of an evidence-based practice in VA homeless programs: a cluster-randomized trial of an implementation support strategy. Implement Sci 2017; 12:34. [PMID: 28279207 PMCID: PMC5345223 DOI: 10.1186/s13012-017-0565-0] [Citation(s) in RCA: 17] [Impact Index Per Article: 2.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2016] [Accepted: 03/01/2017] [Indexed: 11/24/2022] Open
Abstract
BACKGROUND Incorporating evidence-based integrated treatment for dual disorders into typical care settings has been challenging, especially among those serving Veterans who are homeless. This paper presents an evaluation of an effort to incorporate an evidence-based, dual disorder treatment called Maintaining Independence and Sobriety Through Systems Integration, Outreach, and Networking-Veterans Edition (MISSION-Vet) into case management teams serving Veterans who are homeless, using an implementation strategy called Getting To Outcomes (GTO). METHODS This Hybrid Type III, cluster-randomized controlled trial assessed the impact of GTO over and above MISSION-Vet Implementation as Usual (IU). Both conditions received standard MISSION-Vet training and manuals. The GTO group received an implementation manual, training, technical assistance, and data feedback. The study occurred in teams at three large VA Medical Centers over 2 years. Within each team, existing sub-teams (case managers and Veterans they serve) were the clusters randomly assigned. The trial assessed MISSION-Vet services delivered and collected via administrative data and implementation barriers and facilitators, via semi-structured interview. RESULTS No case managers in the IU group initiated MISSION-Vet while 68% in the GTO group did. Seven percent of Veterans with case managers in the GTO group received at least one MISSION-Vet session. Most case managers appreciated the MISSION-Vet materials and felt the GTO planning meetings supported using MISSION-Vet. Case manager interviews also showed that MISSION-Vet could be confusing; there was little involvement from leadership after their initial agreement to participate; the data feedback system had a number of difficulties; and case managers did not have the resources to implement all aspects of MISSION-Vet. CONCLUSIONS This project shows that GTO-like support can help launch new practices but that multiple implementation facilitators are needed for successful execution of a complex evidence-based program like MISSION-Vet. TRIAL REGISTRATION ClinicalTrials.gov NCT01430741.
Collapse
Affiliation(s)
- Matthew Chinman
- VISN 4 Mental Illness Research and Clinical Center and the Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, PA USA
- RAND Corporation, Pittsburgh, PA USA
- VA National Center on Homelessness Among Veterans, Philadelphia, PA USA
| | - Sharon McCarthy
- VISN 4 Mental Illness Research and Clinical Center and the Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, PA USA
| | - Gordon Hannah
- VISN 4 Mental Illness Research and Clinical Center and the Center for Health Equity Research and Promotion, VA Pittsburgh Healthcare System, Pittsburgh, PA USA
| | - Thomas Hugh Byrne
- VA National Center on Homelessness Among Veterans, Philadelphia, PA USA
- Boston University School of Social Work, Boston, MA USA
- VA Center for Healthcare Organization and Implementation Research, Boston, MA USA
| | - David A. Smelson
- VA National Center on Homelessness Among Veterans, Philadelphia, PA USA
- VA Center for Healthcare Organization and Implementation Research, Boston, MA USA
- Department of Psychiatry, University of Massachusetts Medical School, Worcester, MA USA
| |
Collapse
|
33
|
Mueller T, Tevendale HD, Fuller TR, House LD, Romero LM, Brittain A, Varanasi B. Teen Pregnancy Prevention: Implementation of a Multicomponent, Community-Wide Approach. J Adolesc Health 2017; 60:S9-S17. [PMID: 28235440 PMCID: PMC6521958 DOI: 10.1016/j.jadohealth.2016.11.002] [Citation(s) in RCA: 27] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 03/23/2016] [Revised: 09/22/2016] [Accepted: 11/03/2016] [Indexed: 11/17/2022]
Abstract
This article provides an overview and description of implementation activities of the multicomponent, community-wide initiatives of the Teenage Pregnancy Prevention Program initiated in 2010 by the Office of Adolescent Health and the Centers for Disease Control and Prevention. The community-wide initiatives applied the Interactive Systems Framework for dissemination and implementation through training and technical assistance on the key elements of the initiative: implementation of evidence-based teen pregnancy prevention (TPP) interventions; enhancing quality of and access to youth-friendly reproductive health services; educating stakeholders about TPP; working with youth in communities most at risk of teen pregnancy; and mobilizing the community to garner support. Of nearly 12,000 hours of training and technical assistance provided, the majority was for selecting, implementing, and evaluating an evidence-based TPP program. Real-world implementation of a community-wide approach to TPP takes time and effort. This report describes implementation within each of the components and shares lessons learned during planning and implementation phases of the initiative.
Collapse
Affiliation(s)
- Trisha Mueller
- Division of Reproductive Health, National Center for Chronic Disease Prevention and Health Promotion, Centers for Disease Control and Prevention, Atlanta, Georgia.
| | - Heather D Tevendale
- Division of Reproductive Health, National Center for Chronic Disease Prevention and Health Promotion, Centers for Disease Control and Prevention, Atlanta, Georgia
| | - Taleria R Fuller
- Division of Reproductive Health, National Center for Chronic Disease Prevention and Health Promotion, Centers for Disease Control and Prevention, Atlanta, Georgia
| | - L Duane House
- Division of Reproductive Health, National Center for Chronic Disease Prevention and Health Promotion, Centers for Disease Control and Prevention, Atlanta, Georgia
| | - Lisa M Romero
- Division of Reproductive Health, National Center for Chronic Disease Prevention and Health Promotion, Centers for Disease Control and Prevention, Atlanta, Georgia
| | - Anna Brittain
- Division of Reproductive Health, National Center for Chronic Disease Prevention and Health Promotion, Centers for Disease Control and Prevention, Atlanta, Georgia
| | | |
Collapse
|