1
|
McKay VR, Zamantakis A, Pachicano AM, Merle JL, Purrier MR, Swan M, Li DH, Mustanski B, Smith JD, Hirschhorn LR, Benbow N. Establishing evidence criteria for implementation strategies in the US: a Delphi study for HIV services. Implement Sci 2024; 19:50. [PMID: 39010153 PMCID: PMC11251241 DOI: 10.1186/s13012-024-01379-3] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/21/2024] [Accepted: 06/28/2024] [Indexed: 07/17/2024] Open
Abstract
BACKGROUND There are no criteria specifically for evaluating the quality of implementation research and recommending implementation strategies likely to have impact to practitioners. We describe the development and application of the Best Practices Tool, a set of criteria to evaluate the evidence supporting HIV-specific implementation strategies. METHODS We developed the Best Practices Tool from 2022-2023 in three phases. (1) We developed a draft tool and criteria based on a literature review and key informant interviews. We purposively selected and recruited by email interview participants representing a mix of expertise in HIV service delivery, quality improvement, and implementation science. (2) The tool was then informed and revised through two e-Delphi rounds using a survey delivered online through Qualtrics. The first and second round Delphi surveys consisted of 71 and 52 open and close-ended questions, respectively, asking participants to evaluate, confirm, and make suggestions on different aspects of the rubric. After each survey round, data were analyzed and synthesized as appropriate; and the tool and criteria were revised. (3) We then applied the tool to a set of research studies assessing implementation strategies designed to promote the adoption and uptake of evidence-based HIV interventions to assess reliable application of the tool and criteria. RESULTS Our initial literature review yielded existing tools for evaluating intervention-level evidence. For a strategy-level tool, additions emerged from interviews, for example, a need to consider the context and specification of strategies. Revisions were made after both Delphi rounds resulting in the confirmation of five evaluation domains - research design, implementation outcomes, limitations and rigor, strategy specification, and equity - and four evidence levels - best, promising, more evidence needed, and harmful. For most domains, criteria were specified at each evidence level. After an initial pilot round to develop an application process and provide training, we achieved 98% reliability when applying the criteria to 18 implementation strategies. CONCLUSIONS We developed a tool to evaluate the evidence supporting implementation strategies for HIV services. Although specific to HIV in the US, this tool is adaptable for evaluating strategies in other health areas.
Collapse
Affiliation(s)
- Virginia R McKay
- Center for Public Health Systems Science, Brown School, Washington University in St. Louis, St. Louis, MO, USA.
| | - Alithia Zamantakis
- Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
- Institute for Sexual and Gender Minority Health and Wellbeing, Northwestern University, Chicago, IL, USA
| | - Ana Michaela Pachicano
- Institute for Sexual and Gender Minority Health and Wellbeing, Northwestern University, Chicago, IL, USA
| | - James L Merle
- Department of Population Health Sciences, Division of Health System Innovation and Research, Spencer Fox Eccles School of Medicine, The University of Utah, Salt Lake City, UT, USA
| | - Morgan R Purrier
- Institute for Sexual and Gender Minority Health and Wellbeing, Northwestern University, Chicago, IL, USA
| | - McKenzie Swan
- Center for Public Health Systems Science, Brown School, Washington University in St. Louis, St. Louis, MO, USA
| | - Dennis H Li
- Institute for Sexual and Gender Minority Health and Wellbeing, Northwestern University, Chicago, IL, USA
- Center for Prevention Implementation Methodology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
- Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
- Center for Dissemination and Implementation Science, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | - Brian Mustanski
- Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
- Institute for Sexual and Gender Minority Health and Wellbeing, Northwestern University, Chicago, IL, USA
- Center for Prevention Implementation Methodology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
- Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | - Justin D Smith
- Institute for Sexual and Gender Minority Health and Wellbeing, Northwestern University, Chicago, IL, USA
| | - Lisa R Hirschhorn
- Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| | - Nanette Benbow
- Institute for Sexual and Gender Minority Health and Wellbeing, Northwestern University, Chicago, IL, USA
- Center for Prevention Implementation Methodology, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
- Department of Psychiatry and Behavioral Sciences, Feinberg School of Medicine, Northwestern University, Chicago, IL, USA
| |
Collapse
|
2
|
Mckay V, Zamantakis A, Pachicano AM, Merle J, Purrier M, Swan M, Li D, Mustanski B, Smith JD, Hirschhorn L, Benbow N. Establishing Evidence Criteria for Implementation Strategies: A Delphi Study for HIV Services. RESEARCH SQUARE 2024:rs.3.rs-3979631. [PMID: 38464091 PMCID: PMC10925451 DOI: 10.21203/rs.3.rs-3979631/v1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 03/12/2024]
Abstract
Background There are no criteria specifically for evaluating the quality of implementation research and recommend implementation strategies likely to have impact to practitioners. We describe the development and application of the Best Practices Rubric, a set of criteria to evaluate the evidence supporting implementation strategies, in the context of HIV. Methods We developed the Best Practices Rubric from 2022-2023 in three phases. (1) We purposively selected and recruited by email participants representing a mix of expertise in HIV service delivery, quality improvement, and implementation science. We developed a draft rubric and criteria based on a literature review and key informant interviews. (2) The rubric was then informed and revised through two e-Delphi rounds using a survey delivered online through Qualtrics. The first and second round Delphi surveys consisted of 71 and 52 open and close-ended questions, respectively, asking participants to evaluate, confirm, and make suggestions on different aspects of the rubric. After each survey round, data were analyzed and synthesized as appropriate, and the rubric and criteria were revised. (3) We then applied the rubric to a set of research studies assessing 18 implementation strategies designed to promote the adoption and uptake of pre-exposure prophylaxis, an HIV prevention medication, to assess reliable application of the rubric and criteria. Results Our initial literature review yielded existing rubrics and criteria for evaluating intervention-level evidence. For a strategy-level rubric, additions emerged from interviews, for example, a need to consider the context and specification of strategies. Revisions were made after both Delphi rounds resulting in the confirmation of five evaluation domains - research design, implementation outcomes, limitations and rigor, strategy specification, and equity - and four evidence levels - best practice, promising practice, more evidence needed, and harmful practices. For most domains, criteria were specified at each evidence level. After an initial pilot round to develop an application process and provide training, we achieved 98% reliability when applying the criteria to 18 implementation strategies. Conclusions We developed a rubric to evaluate the evidence supporting implementation strategies for HIV services. Although the rubric is specific to HIV, this tool is adaptable for evaluating strategies in other health areas.
Collapse
|
3
|
Bunting SR, Feinstein BA, Hazra A, Sheth NK, Garber SS. Knowledge of HIV and HIV pre-exposure prophylaxis among medical and pharmacy students: A national, multi-site, cross-sectional study. Prev Med Rep 2021; 24:101590. [PMID: 34976649 PMCID: PMC8683973 DOI: 10.1016/j.pmedr.2021.101590] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/10/2021] [Revised: 09/27/2021] [Accepted: 10/03/2021] [Indexed: 01/19/2023] Open
Abstract
Pre-exposure prophylaxis (PrEP) is a highly effective strategy for preventing HIV. However, prescription of PrEP has not reached the scale that is necessary to meet the public health need of reducing HIV incidence. A factor contributing to this slow scale-up is limited healthcare practitioners' knowledge of PrEP, making PrEP education a priority. We conducted a national, cross-sectional study of medical (allopathic and osteopathic) and pharmacy students regarding knowledge of PrEP and HIV between October 2020 and February 2021. We included 28 items in our knowledge assessment. Analysis sought to identify gaps in knowledge as well as academic and demographic correlates of knowledge. A total of 2,353 students participated in the study (response rate = 17.0%). The overall mean HIV knowledge score was 79.6% correct. Regarding specific items, 68.7% of participants believed HIV treatment was difficult because it required many pills, and 61.1% incorrectly indicated a person with an undetectable HIV viral load could transmit the virus to their sexual partners. Overall mean PrEP knowledge was 84.1%. Approximately one-third of participants did not identify HIV-negative status as a requirement to be a PrEP candidate. Gay/lesbian participants and those who were in the late-phase of training reported higher knowledge of both HIV and PrEP than did heterosexual participants and those in the early-phase of training. This study identifies specific gaps in training on HIV prevention with PrEP that must be improved in health professions education to ensure PrEP reaches its full potential in ending the HIV epidemic.
Collapse
Affiliation(s)
- Samuel R Bunting
- Department of Psychiatry & Behavioral Neuroscience, The University of Chicago Medical Center, Chicago, IL, USA
| | - Brian A Feinstein
- Department of Psychology, College of Health Professions, Rosalind Franklin University of Medicine and Science, North Chicago, IL, USA
| | - Aniruddha Hazra
- Section of Infectious Diseases and Global Health, Department of Medicine, The University of Chicago Medical Center, Chicago, IL, USA
| | - Neeral K Sheth
- Department of Psychiatry and Behavioral Sciences, Rush University Medical Center, Chicago, IL, USA
| | - Sarah S Garber
- College of Pharmacy, Rosalind Franklin University of Medicine and Science, North Chicago, IL, USA
| |
Collapse
|
4
|
Acceptability, Appropriateness, and Preliminary Effects of the PrEP Diffusion Training for Lay HIV Workers: Increased PrEP Knowledge, Decreased Stigma, and Diffusion of Innovation. AIDS Behav 2021; 25:3413-3424. [PMID: 33791880 DOI: 10.1007/s10461-021-03248-2] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/23/2021] [Indexed: 11/27/2022]
Abstract
Training lay health workers is a critical intervention strategy to support HIV pre-exposure prophylaxis (PrEP) implementation. However, few evaluations of such trainings have been published. We conducted multi-time-point surveys to evaluate the effect of a training intervention on knowledge, PrEP stigma, and implementation behavior among lay HIV workers in China. Results indicated high acceptability and appropriateness of the training. PrEP knowledge score increased by 65% from pre- to post-training, and remained high one-month post-training. We observed a significant decrease in PrEP stigma and a positive influence on determinants of implementation behaviors. All lay HIV workers surveyed one-month after the training reported having disseminated PrEP information in social networks; 43% reported integrating PrEP education into routine work. The training is an effective strategy to build lay HIV workers' capacity in PrEP implementation. Longer-term post-training follow up would be ideal to further assess actual PrEP uptake and sustained effects on PrEP implementation.
Collapse
|
5
|
Agovi AMA, Anikpo I, Cvitanovich MJ, Craten KJ, Asuelime EO, Ojha RP. Knowledge needs for implementing HIV pre-exposure prophylaxis among primary care providers in a safety-net health system. Prev Med Rep 2020; 20:101266. [PMID: 33364148 PMCID: PMC7750167 DOI: 10.1016/j.pmedr.2020.101266] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/10/2020] [Accepted: 11/21/2020] [Indexed: 11/17/2022] Open
Abstract
Safety-net health systems are a primary source of care for socioeconomically disadvantaged individuals who may be eligible for HIV pre-exposure prophylaxis (PrEP) and are priority groups under the Ending the HIV Epidemic (EHE) initiative. Nevertheless, little evidence is available about barriers to PrEP implementation in safety-net settings. We aimed to assess the association between PrEP knowledge and prescribing practices, and to ascertain unmet knowledge needs to implement PrEP. In 2019, we surveyed primary care providers (PCPs) in a safety-net health system that serves an EHE priority jurisdiction located in North Texas. Our questionnaire ascertained self-reported prescribing practices, knowledge, and training needs related to PrEP. We used penalized logistic regression to estimate odds ratio (OR) and 95% posterior limits (PL) for the association between provider self-rated knowledge of PrEP and PrEP prescribing. Our study population comprised 62 primary care providers, of whom 61% were female, 60% were non-Hispanic White, 76% were physicians (76%), 57% had ≥ 10 years of practice experience, 45% reported low self-rated PrEP knowledge, and 35% prescribed PrEP in the past year. Providers with low PrEP knowledge had 69% lower odds of prescribing PrEP within the past year (OR = 0.31, 95% PL: 0.12, 0.82). Eligibility for PrEP, side effects and adherence concerns were key unmet knowledge needs. Our findings suggest that low provider PrEP knowledge may be a barrier to PrEP prescribing among safety-net PCPs. Our results provide insight about specific educational needs of PCPs in a safety-net health system, which are amenable to educational intervention.
Collapse
Affiliation(s)
- Afiba Manza-A. Agovi
- Center for Outcomes Research, JPS Health Network, Fort Worth, TX, United States
- Department of Medical Education, TCU and UNTHSC School of Medicine, Fort Worth, TX, United States
- Corresponding author at: Center for Outcomes Research, JPS Health Network, 1500 South Main Street, Fort Worth, TX 76104, United States.
| | - Ifedioranma Anikpo
- Center for Outcomes Research, JPS Health Network, Fort Worth, TX, United States
| | | | - Kevin J. Craten
- Center for Outcomes Research, JPS Health Network, Fort Worth, TX, United States
| | - Eve O. Asuelime
- Healing Wings-Infectious Disease Clinic, JPS Health Network, Fort Worth, TX, United States
| | - Rohit P. Ojha
- Center for Outcomes Research, JPS Health Network, Fort Worth, TX, United States
- Department of Medical Education, TCU and UNTHSC School of Medicine, Fort Worth, TX, United States
| |
Collapse
|