Factors influencing implementation dose and fidelity thereof and related student outcomes of an evidence-based national HIV prevention program

Teachers' Perceptions of the Impact of the COVID-19 Pandemic and Their Implementation of an Evidence-based HIV Prevention Program in the Bahamas

Information on how school-based programs is implemented and sustained during crises is limited. In this study, we assessed the impact of the COVID-19 pandemic on the implementation of a HIV prevention intervention in The Bahamas. Data were collected from 139 Grade 6 teachers in 2021-2022. Teachers attended virtual training and received implementation monitoring from coordinators. On average, teachers taught 26.4 (SD = 9.2) of the 35 core activities, and 7.4 (SD = 2.4) out of 9 sessions. More than half (58.3%) of teachers completed 28 or more core activities; 69.1% covered eight or all nine sessions, which is equivalent to 80% of the HIV intervention curriculum. Almost half of the teachers (43%) reported that the pandemic negatively impacted their ability to teach the program; 72% of teachers maintained that the program remained "very important" during times of crisis. Greater self-efficacy and supports increased implementation fidelity.

Fidelity of implementation of national guidelines on malaria diagnosis for children under-five years in Rivers State, Nigeria

BACKGROUND

Malaria is still a disease of global public health importance and children under-five years of age are the most vulnerable to the disease. Nigeria adopted the "test and treat" strategy in the national malaria guidelines as one of the ways to control malaria transmission. The level of adherence to the guidelines is an important indicator for the success or failure of the country's roadmap to malaria elimination by 2030. This study aimed to assess the fidelity of implementation of the national guidelines on malaria diagnosis for children under-five years and examine its associated moderating factors in health care facilities in Rivers State, Nigeria.

METHODS

This was a descriptive, cross-sectional study conducted in Port Harcourt metropolis. Data were collected from 147 public, formal private and informal private health care facilities. The study used a questionnaire developed based on Carroll's Conceptual Framework for Implementation Fidelity. Frequency, mean and median scores for implementation fidelity and its associated factors were calculated. Associations between fidelity and the measured predictors were examined using Mann Whitney U test, Kruskal Wallis test, and multiple linear regression modelling using robust estimation of errors. Regression results are presented in adjusted coefficient (β) and 95% confidence intervals.

RESULTS

The median (IQR) score fidelity score for all participants was 65% (43.3, 85). Informal private facilities (proprietary patent medicine vendors) had the lowest fidelity scores (47%) compared to formal private (69%) and public health facilities (79%). Intervention complexity had a statistically significant inverse relationship to implementation fidelity (β = - 1.89 [- 3.42, - 0.34]). Increase in participant responsiveness (β = 8.57 [4.83, 12.32]) and the type of malaria test offered at the facility (e.g., RDT vs. no test, β = 16.90 [6.78, 27.03]; microscopy vs. no test, β = 21.88 [13.60, 30.16]) were positively associated with fidelity score.

CONCLUSIONS

This study showed that core elements of the "test and treat" strategy, such as testing all suspected cases with approved diagnostic methods before treatment, are still not fully implemented by health facilities. There is a need for strategies to increase fidelity, especially in the informal private health sector, for malaria elimination programme outcomes to be achieved.

Maintaining Program Fidelity in a Changing World: National Implementation of a School-Based HIV Prevention Program

Large-scale, evidence-based interventions face challenges to program fidelity of implementation. We developed implementation strategies to support teachers implementing an evidence-based HIV prevention program in schools, Focus on Youth in The Caribbean (FOYC) and Caribbean Informed Parents and Children Together (CImPACT) in The Bahamas. We examined the effects of these implementation strategies on teachers' implementation in the subsequent year after the initial implementation during the COVID-19 pandemic. Data were collected from 79 Grade 6 teachers in 24 government elementary schools. Teachers completed training workshops and a pre-implementation questionnaire to record their characteristics and perceptions that might affect their program fidelity. School coordinators and peer mentors provided teachers with monitoring, feedback, and mentoring. In Year 1, teachers on average taught 79.3% of the sessions and 80.8% of core activities; teachers in Year 2 covered 84.2% of sessions and 72.9% of the core activities. Teachers with "good" or "excellent" school coordinators in the second year taught significantly more sessions on average (7.8 vs. 7.0, t = 2.04, P < 0.05) and more core activities (26.3 vs. 23.0, t = 2.41, P < 0.05) than teachers with "satisfactory" coordinators. Teachers who had a "good" or "satisfactory" mentor taught more sessions than teachers who did not have a mentor (7.9 vs. 7.3; t = 2.22; P = 0.03). Two-level mixed-effects model analysis indicated that teachers' program fidelity in Year 1, confidence in the execution of core activities, and school coordinators' performance were significantly associated with Year 2 implementation dose. Implementation of FOYC + CImPACT was significantly associated with improved student outcomes. Teachers maintained high fidelity to a comprehensive HIV prevention program over 2 years during the COVID-19 pandemic. Future program implementers should consider additional implementation support to improve the implementation of school-based programs.

Effects of Teacher Training and Continued Support on the Delivery of an Evidence-Based HIV Prevention Program: Findings From a National Implementation Study in the Bahamas

BACKGROUND

Few studies have investigated the effects of teacher training and continued support on teachers' delivery of evidence-based HIV prevention programs. We examined these factors in a national implementation study of an evidence-based HIV risk reduction intervention for adolescents in the sixth grade in the Bahamas.

METHODS

Data were collected from 126 grade 6 teachers and 3,118 students in 58 government elementary schools in the Bahamas in 2019-2021. This is a Hybrid Type III implementation study guided by the Exploration, Preparation, Implementation, Sustainment (EPIS) model. Teachers attended 2-day training workshops. Trained school coordinators and peer mentors provided biweekly monitoring and mentorship. We used mixed-effects models to assess the effects of teacher training and continued support on implementation fidelity.

RESULTS

Teachers who received training in-person or both in-person and online taught the most core activities (27.0 and 27.2 of 35), versus only online training (21.9) and no training (14.9) (F = 15.27, p < .001). Teachers with an "excellent" or "very good" school coordinator taught more core activities than those with a "satisfactory" coordinator or no coordinator (29.2 vs. 27.8 vs. 19.3 vs. 14.8, F = 29.20, p < .001). Teachers with a "very good" mentor taught more core activities and sessions than those with a "satisfactory" mentor or no mentor (30.4 vs. 25.0 vs. 23.1; F = 7.20; p < .01). Teacher training, implementation monitoring, peer mentoring, teachers' self-efficacy, and school-level support were associated with implementation fidelity, which in turn was associated with improved student outcomes (HIV/AIDS knowledge, preventive reproductive health skills, self-efficacy, and intention to use protection).

CONCLUSION

Teachers receiving in-person training and those having higher-rated school coordinator and mentor support taught a larger number of HIV prevention core activities. Effective teacher training, implementation monitoring, and peer mentoring are critical for improving implementation fidelity and student outcomes.

Adapting an evidence-based mindfulness-based intervention for sheltered youth experiencing homelessness

OBJECTIVES

Youth experiencing homelessness (YEH) face challenges that impact their physical, mental, and social wellbeing, emotion regulation, and coping. Mindfulness reduces stress and improves resilience, emotion regulation, and executive functioning. Mindfulness-based interventions (MBI) teach the practice of mindfulness to foster present-moment attention without judgement and enhance self-observation and self-regulation, resulting in greater awareness of thoughts and emotions and improved interpersonal relationships. One such intervention, .b, has been shown to lower stress among youth. While a pilot study of .b among sheltered youth found the intervention to be feasible, the need for modifications was identified to improve its relevance, accessibility, and incorporate a trauma-informed approach.

METHODS

We used the ADAPT-ITT (Assessment, Decisions, Administration, Production, Topical experts, Integration, Training staff, and Testing) framework to adapt the .b mindfulness intervention to YEH living in an emergency shelter. Nine focus group discussions (n = 56), key informant interviews (n = 8), and beta testing with five youth working group sessions (n = 10) identified needed modifications.

RESULTS

Adaptations to the curriculum and delivery modality were made to approximate the average length of stay in the shelter; integrate trauma-informed care approaches; increase diversity of images by race, ethnicity, age, sexual orientation, and gender identity; and increase the relevance of the audio-visual components.

CONCLUSIONS

Youth and the health and social services providers who care for youth generally liked the core concepts and presentation of the curriculum. Using the ADAPT-ITT framework, minor, yet important, changes were made to increase the relevance, acceptability, and feasibility of the intervention. Next steps are to conduct a randomized attention control pilot study to assess feasibility and acceptability.

Shared Components of Worldwide Successful Sexuality Education Interventions for Adolescents: A Systematic Review of Randomized Trials

A crucial aspect of human development is sexuality which has implications for health, particularly in adolescence, since unfavorable sexual experiences may result in physical and mental problems. Sexuality education interventions (SEI) are one of the most used actions to promote sexual health in adolescents. Nevertheless, there is variability across their components; therefore, key elements for an effective SEI targeted at adolescents (A-SEI) are not well known. Based on this background, this study aims to identify the shared components of successful A-SEI through a systematic review of randomized controlled trials (RCT). This study followed the preferred reporting items for systematic reviews and meta-analyses statement. A search was conducted in CINAHL, PsycInfo, PubMed, and Web of Science between November and December 2021. A total of 21 studies passed the inclusion test after the review of 8318 reports. A total of 18 A-SEIs were identified in these studies. The components analyzed were the intervention's approach, dose, type of intervention, theoretical framework, facilitators' training, and intervention methodology. The results established that components that should be present in the design of an effective A-SEI are behavior change theoretical models, the use of participatory methodology, be targeted at mixed-sex groups, facilitators' training, and at least ten hours of weekly intervention.

Comparing standard versus enhanced implementation of an evidence-based HIV prevention program among Bahamian sixth grade students: findings from nationwide implementation trials

BACKGROUND

Effective implementation strategies are needed to address the challenges encountered by teachers in implementation of evidence-based HV prevention programs in schools. The current study: 1) compares implementation fidelity of Focus on Youth in the Caribbean (FOYC) plus Caribbean Informed Parents and Children Together (CImPACT) intervention using enhanced implementation strategies (including biweekly monitoring/feedback and site-based mentoring) to those using more traditional approach (teacher training only); and 2) evaluates the impact of school coordinators' and mentors' performance on teachers' implementation fidelity and student outcomes.

METHODS

Data from an enhanced implementation trial in 2019-2020, involving 24 government primary schools, 79 teachers, and 2252 students, were compared to data from a standard implementation trial in 2011-2012, involving 35 government primary schools, 110 teachers and 2811 students using mixed-effects modeling and structural equation modeling.

FINDINGS

Teachers in the 2019-2020 trial taught more core activities (28.3 vs. 16.3, t = 10.80, P < 0.001) and sessions (7.2 vs. 4.4, t = 9.14, P < 0.001) than those participating in the 2011-2012 trial. Teachers taught > 80% of the intervention curriculum in 2019-2020 compared to 50% curriculum delivery in 2011-2012. Teachers who had a "very good" or "excellent" school coordinator in their schools taught more core activities than those who had a "satisfactory" school coordinator (30.4 vs. 29.6 vs. 22.3, F = 18.54, P < 0.001). Teachers who worked in a school which had a "very good" mentor, taught more core activities than those teachers who did not have a mentor or had only a "satisfactory" mentor (30.4 vs. 27.6; t = 2.96; p = 0.004). Teachers' confidence in implementing core activities, comfort level with the curriculum, attitudes towards sex education in schools, and perceived principal support were significantly related to increased self-efficacy, which in turn was related to teachers' implementation fidelity. The degree of implementation was significantly associated with improved student outcomes.

IMPLICATIONS/CONCLUSION

An evidence-based HIV prevention intervention can achieve a high degree of implementation when delivered with enhanced implementation strategies and implementation monitoring. Future program implementers should consider the purposeful selection and training of school coordinators and mentors to support low-implementing teachers as a potentially important strategy when attempting to achieve high-quality implementation of school-based interventions.

Individual and contextual factors affect the implementation fidelity of youth-friendly services, northwest Ethiopia: A multilevel analysis

BACKGROUND

The evaluation of all potential determinants of implementation fidelity of Youth-Friendly Services (YFS) is crucial for Ethiopia. Previous studies overlooked investigating the determinants at different levels. Therefore, this study aimed to assess the determinants of implementation fidelity of YFS considering individual and contextual levels.

METHODS

This study was conducted among 1,029 youths, from 11 health centers that are implementing the YFS in Central Gondar Zone. Data were collected by face to face interview and facility observation using a semi-structured questionnaire. A Bivariable multi-level mixed effect modelling was employed to assess the main determinants. Four separate models were fitted to reach the full model. The fitness of the model was assessed using Akaike Information Criterion (AIC) and level of significance was declared at p-values < 0.05. The results of fixed effects were presented as adjusted odds ratio (AOR) at their 95% CI.

RESULTS

Four hundred one (39.0%) of the respondents got the YFS with high level of fidelity. Had high level of involvement in the YFS provision (AOR = 1.35, 95% CI: 1.15, 1.57), knew any peer educator trained in YFS (AOR = 1.60, 95% CI: 1.36, 1.86), and involved as a peer educator (AOR = 1.46, 95% CI: 1.24, 1.71), were the individual level determinants. Whereas, got capacity building training; (AOR = 1.93, 95% CI (1.12, 3.48), got supportive supervision, (AOR 2.85, 95% CI (1.99, 6.37), had a separate waiting room (AOR = 9.84, 95%CI: 2.14, 17.79), and system in place to provide continuous support to staff (AOR = 2.81, 95%CI: 1.25, 6.34) were the contextual level determinants.

CONCLUSIONS

The level of implementation fidelity remains low. Both individual and contextual level determinants affect the implementation fidelity of YFS. Therefore, policy makers, planners, managers and YFS providers could consider both individual and contextual factors to improve the implementation fidelity.

The Effectiveness of Two Implementation Strategies for Improving Teachers' Delivery of an Evidenced-based HIV Prevention Program

BACKGROUND

Effective implementation strategies are needed to enhance the success of evidence-based prevention programs. The current study evaluates the effects of two implementation strategies on teachers' implementation of an evidenced-based HIV intervention.

METHODS

Using our 7-item pre-implementation school screening tool, we identified teachers who were at-risk for not implementing the Focus on Youth HIV-risk reduction intervention curriculum which targets grade six through grade 8 students. After completing a two-day curriculum workshop, 81 low- and moderate-performing teachers were randomly assigned to one of four experimental conditions and were asked to teach the two-month intervention curriculum. This optimization trial examines the impact of two implementation strategies: biweekly monitoring/feedbacks (BMF) and site-based assistance/mentorship (SAM). The primary outcome is implementation fidelity defined as number of core activities taught. Linear mixed-effects model was used to examine the association of the implementation strategies with implementation fidelity.

RESULTS

BMF and SAM were significantly associated with teachers' implementation fidelity. Teachers who received both BFM and SAM taught the greatest numbers of core activities (15 core activities on average), followed by teachers who received either BMF (6.9 activities) or SAM (7.9 activities). Teachers who did not receive BMF or SAM taught the lowest numbers (4.1 activities). Teachers' sustained implementation of FOYC in the prior school year was related to increased implementation fidelity during the optimization trial. Teachers' confidence in implementing five core activities, attitudes toward sex education in schools, and perceived principal support were significantly related to increased self-efficacy, which in turn was related to teachers' fidelity of implementation before the optimization trial.

CONCLUSION

BMF and SAM are effective in promoting teachers' implementation of youth evidence-based interventions. Researchers and future program implementers should consider teacher training, teachers' attitudes toward sex education, perceived principal support, and self-efficacy when attempting to maintain the effects of teacher-delivered interventions in schools.

Supporting South African High School Teachers' Implementation of a Prevention Program via Abridged Consultation: Outcomes and Moderators

Research is lacking on consultation support for school-based evidence-based programs (EBP) intended to prevent youths' risky behaviors in schools in low-resourced settings like high schools surrounding Cape Town, South Africa. Thus, this study's objective was to examine implementation outcomes and moderators of an abridged consultation condition for supporting teachers in better implementing HealthWise, an EBP for preventing youth risky sexual and substance use behaviors. Twenty-one schools with 33 teachers receiving abridged consultation (i.e., three consultation meetings, text message reminders, lesson plans, and support kits) were compared to 26 schools with 41 teachers that did not receive any consultation. Teachers with abridged consultation self-reported delivering more HealthWise content. Moderation analyses found teachers with lower educational degrees, who received abridged consultation reported more student interest in HealthWise. When there was higher school-level risk, teachers who received abridged consultation marginally self-reported adapting HealthWise more. Findings suggest consultation support that is abridged or a lower dose than is typical can be feasible in such a low-resourced, overburdened setting while still being associated with EBP coverage, student interest, and adaptation. However, moderation findings suggest contextual factors should be considered to match teachers/schools to the implementation support that best suits them.

Fidelity of Interventions to Reduce or Prevent Stress and/or Anxiety from Pregnancy up to Two Years Postpartum: A Systematic Review

INTRODUCTION

Intervention fidelity refers to whether an intervention is delivered as intended and can enhance interpretation of trial outcomes. Fidelity of interventions to reduce or prevent stress and anxiety during pregnancy and postpartum has yet to be examined despite inconsistent findings for intervention effects. This study systematically reviews use and/or reporting of intervention fidelity strategies in trials of interventions, delivered to (expectant) parents during pregnancy and postpartum, to reduce or prevent stress and/or anxiety.

METHODS

MEDLINE, Embase, CINAHL, PsychINFO, and Maternity and Infant Care were searched from inception to March 2019. Studies were included if they were randomised controlled trials including pregnant women, expectant fathers and/or partners during pregnancy, and/ or parents within the first two years postpartum. The National Institutes of Health Behavior Change Consortium checklist was used to assess fidelity across five domains (study design, provider training, delivery, receipt, enactment).

RESULTS

Sixteen papers (14 interventions) were identified. Average reported use of fidelity strategies was 'low' (45%), ranging from 17.5 to 76%. Fidelity ratings ranged from 22% for provider training to 54% for study design.

CONCLUSIONS

Low levels of intervention fidelity may explain previous inconsistent effects of stress and anxiety reduction interventions. Important methodological areas for improvement include intervention provider training, fidelity of comparator conditions, and consideration of non-specific treatment effects. Increased methodological rigour in fidelity enhancement and assessment will improve intervention implementation and enhance examination of stress and anxiety reduction and prevention interventions delivered during pregnancy and the postpartum.

Implementation fidelity and acceptability of an intervention to improve vaccination uptake and child health in rural India: a mixed methods evaluation of a pilot cluster randomized controlled trial

Background

The Tika Vaani intervention, an initiative to improve basic health knowledge and empower beneficiaries to improve vaccination uptake and child health for underserved rural populations in India, was assessed in a pilot cluster randomized trial. The intervention was delivered through two strategies: mHealth (using mobile phones to send vaccination reminders and audio-based messages) and community mobilization (face-to-face meetings) in rural Indian villages from January to September 2018. We assessed acceptability and implementation fidelity to determine whether the intervention delivered in the pilot trial can be implemented at a larger scale.

Methods

We adapted the Conceptual Framework for implementation fidelity to assess acceptability and fidelity of the pilot interventions using a mixed methods design. Quantitative data sources include a structured checklist, household surveys, and mobile phone call patterns. Qualitative data came from field observations, intervention records, semi-structured interviews and focus groups with project recipients and implementers. Quantitative analyses assessed whether activities were implemented as planned, using descriptive statistics to describe participant characteristics and the percentage distribution of activities. Qualitative data were analyzed using content analysis and in the light of the implementation fidelity model to explore moderating factors and to determine how well the intervention was received.

Results

Findings demonstrated high (86.7%) implementation fidelity. A total of 94% of the target population benefited from the intervention by participating in a face-to-face group meeting or via mobile phone. The participants felt that the strategies were useful means for obtaining information. The clarity of the intervention theory, the motivation, and commitment of the implementers as well as the periodic meetings of the supervisors largely explain the high level of fidelity obtained. Geographic distance, access to a mobile phone, level of education, and gender norms are contextual factors that contributed to heterogeneity in participation.

Conclusions

Although the intervention was evaluated in the context of a randomized trial that could explain the high level of fidelity obtained, this evaluation provides confirmatory evidence that the results of the study reflect the underlying theory. The mobile platform coupled with community mobilization was well-received by the participants and could be a useful way to improve health knowledge and change behavior.

Trial registration

ISRCTN 44840759 (22 April 2018)

Bahamas National Implementation Project: Proposal for Sustainability of an Evidence-based HIV Prevention Intervention in a School Setting

BACKGROUND

Sustained implementation of school-based prevention programs is low. Effective strategies are needed to enhance both high-level implementation fidelity and sustainability of prevention programs.

OBJECTIVE

This proposed study aims to determine if the provision of either biweekly monitoring and feedback and site-based assistance and mentorship or both to at-risk and moderate-performing teachers with monitoring through an enhanced decision-making platform by the Ministry of Education (MOE) and Ministry of Health (MOH) based on the real-time implementation data will increase national implementation fidelity and result in sustained implementation over time.

METHODS

This study will target government schools including 200 grade 6 teachers in 80 primary schools and 100 junior/middle high school teachers (and their classes) on 12 Bahamian islands. Teacher and school coordinator training will be conducted by the MOE in year 1, followed by an optimization trial among teachers in the capital island. Informed by these results, an implementation intervention will be conducted to train using different levels of educational intensity all at-risk and moderate-performing teachers. Subsequently selected training and implementation strategies will be evaluated for the national implementation of Focus on Youth in the Caribbean and Caribbean Informed Parents and Children Together in years 2 to 5.

RESULTS

It is hypothesized that a more intensive training and supervision program for at-risk and moderate-performing teachers will enhance their implementation fidelity to the average level of the high-performing group (85%), an HIV prevention program delivered at the national level can be implemented with fidelity in grade 6 and sustained over time (monitored annually), and student outcomes will continue to be highly correlated with implementation fidelity and be sustained over time (assessed annually through grade 9). The proposed study is funded by the National Institute of Child Health and Human Development from August 1, 2018, through May 31, 2023.

CONCLUSIONS

The study will explore several theory-driven implementation strategies to increase sustained teacher implementation fidelity and thereby increase the general public health impact of evidence-based interventions. The proposed project has potential to make significant contributions to advancing school-based HIV prevention research and implementation science and serve as a global model for the Fast Track strategy.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

PRR1-10.2196/14816.

Socio-Ecological Factors Associated With Students' Perceived Impact of an Evidence-Based Sexual Health Education Curriculum

BACKGROUND

Although schools often implement evidence-based sexual health education programs to address sexual and reproductive health disparities, multiple factors may influence program effectiveness.

METHODS

Using student-reported perceived impact measures as a proxy for program effectiveness, we employed a socio-ecological approach to examine student, teacher, school, and district factors associated with greater perceived impact of It's Your Game (IYG), an evidence-based middle school sexual health education program. The student sample was 58.7% female, 51.8% Hispanic, mean age 13.2 years, from 73 middle schools. We assessed students' (N = 4531) perceived impact of IYG on healthy decision-making (α = 0.75) and sexual communication (α = 0.71); satisfaction with IYG activities and teacher; and demographics. We assessed teachers' (N = 56) self-efficacy to teach IYG, perceived administrative support, implementation barriers, and demographics. School and district data were abstracted from state records. We used multilevel logistic regression to estimate associations between independent variables and student-reported perceived impact.

RESULTS

In final multivariate models, students' demographics (sex, β = 0.06, SE = 0.015), satisfaction with IYG (β = 0.21, SE = 0.012), and their IYG teacher (β = 0.18, SE = 0.013) (all p = .000) were significantly associated with perceived impact on healthy decision-making. Similar findings resulted for sexual communication. No other variables were significantly associated with perceived impact.

CONCLUSIONS

Helping schools select age-appropriate, culturally relevant programs, and facilitate supportive learning environments may enhance the perceived impact of sexual health education programs.

Reconciling Adaptation and Fidelity: Implications for Scaling Up High Quality Youth Programs

In the field of prevention science, some consider fidelity to manualized protocols to be a hallmark of successful implementation. A growing number of scholars agree that high-quality implementation should also include some adaptations to local context, particularly as prevention programs are scaled up, in order to strengthen their relevance and increase participant engagement. From this perspective, fidelity and adaptation can both be seen as necessary, albeit mutually exclusive, dimensions of implementation quality. In this article, we propose that the relationship between these two constructs may be more complex, particularly when adaptations are consistent with the key principles underlying the program model. Our argument draws on examples from the implementation of a manualized youth voice program (YVP) in two different organizations serving six distinct communities. Through a series of retreats, implementers identified examples of modifications made and grouped them into themes. Results suggest that some adaptations were actually indicators of fidelity to the key principles of YVPs: power-sharing, youth ownership, and engagement in social change. We therefore offer suggestions for re-conceptualizing the fidelity-adaptation debate, highlight implications for measurement and assessment, and illustrate that the de facto treatment of adaptation and fidelity as opposing constructs may limit the diffusion or scaling up of these types of youth programs.

Promoting Tuberculosis Preventive Therapy for People Living with HIV in South Africa: Interventions Hindered by Complicated Clinical Guidelines and Imbalanced Patient-Provider Dynamics

Isoniazid preventive therapy (IPT) reduces the risk of active tuberculosis among people living with HIV, but implementation of IPT in South Africa and elsewhere remains slow. The objective of this study was to examine both nurse perceptions of clinical mentorship and patient perceptions of in-queue health education for promoting IPT uptake in Potchefstroom, South Africa. We measured adoption, fidelity, acceptability, and sustainability of the interventions using both quantitative and qualitative methods. Adoption, fidelity, and acceptability of the interventions were moderately high. However, nurses believed they could not sustain their increased prescriptions of IPT, and though many patients intended to ask nurses about IPT, few did. Most patients attributed their behavior to an imbalance of patient-provider power. National IPT guidelines should be unambiguous and easily implemented after minimal training on patient eligibility and appropriate medication durations, nurse-patient dynamics should empower the patient, and district-level support and monitoring should be implemented.

Program implementer perspectives replicating evidence based sexual reproductive health programs

PURPOSE

Explore factors affecting implementation of evidence based adolescent sexual/reproductive health programs, from the perspectives of program implementers.

METHODS

In-depth interviews were conducted with 18 program implementers delivering six sexual/reproductive health programs in New Jersey. Programs were delivered among 2698 primarily African American and Hispanic adolescents in school and community-based settings. Interview transcripts were coded and analyzed iteratively for themes by trained experts.

RESULTS

Program implementers reported weaknesses in the program curricula content and design, scheduling constraints with partner sites, and questions from adolescents as factors challenging to implementation. Relationship-building (with adolescents and community partners) and answering adolescent questions were identified as strategies to program implementation. Implementers expressed need for flexibility in the curriculum to tailor the program to participant needs. However, implementers felt restricted in tailoring the program because of a perceived need to adhere to the prescribed program.

CONCLUSIONS

Evidence based programs may need to provide more flexibility for implementers to customize programs to student needs. Given the age range of the target audience (ages 10-19), programming should consider the variance of an adolescent's life and assumptions regarding adolescent biology knowledge, modes of communication, and type of sexual relationships. Additionally, facilitators may better meet student needs if they feel less restricted.

Using Intervention Mapping for Program Design and Production of iCHAMPSS: An Online Decision Support System to Increase Adoption, Implementation, and Maintenance of Evidence-Based Sexual Health Programs

In Texas and across the United States, unintended pregnancy, HIV, and sexually transmitted infections (STIs) among adolescents remain serious public health issues. Sexual risk-taking behaviors, including early sexual initiation, contribute to these public health problems. Over 35 sexual health evidence-based programs (EBPs) have been shown to reduce sexual risk behaviors and/or prevent teen pregnancies or STIs. Because more than half of these EBPs are designed for schools, they could reach and impact a considerable number of adolescents if implemented in these settings. Most schools across the U.S. and in Texas, however, do not implement these programs. U.S. school districts face many barriers to the successful dissemination (i.e., adoption, implementation, and maintenance) of sexual health EBPs, including lack of knowledge about EBPs and where to find them, perceived lack of support from school administrators and parents, lack of guidance regarding the adoption process, competing priorities, and lack of specialized training on sexual health. Therefore, this paper describes how we used intervention mapping (Steps 3 and 4, in particular), a systematic design framework that uses theory, empirical evidence, and input from the community to develop CHoosing And Maintaining Effective Programs for Sex Education in Schools (iCHAMPSS), an online decision support system to help school districts adopt, implement, and maintain sexual health EBPs. Guided by this systematic intervention design approach, iCHAMPSS has the potential to increase dissemination of sexual health EBPs in school settings.

Using mixed methods to assess fidelity of delivery and its influencing factors in a complex self-management intervention for people with osteoarthritis and low back pain

OBJECTIVES AND DESIGN

Despite an increasing awareness of the importance of fidelity of delivery within complex behaviour change interventions, it is often poorly assessed. This mixed methods study aimed to establish the fidelity of delivery of a complex self-management intervention and explore the reasons for these findings using a convergent/triangulation design.

SETTING

Feasibility trial of the Self-management of Osteoarthritis and Low back pain through Activity and Skills (SOLAS) intervention (ISRCTN49875385), delivered in primary care physiotherapy.

METHODS AND OUTCOMES

60 SOLAS sessions were delivered across seven sites by nine physiotherapists. Fidelity of delivery of prespecified intervention components was evaluated using (1) audio-recordings (n=60), direct observations (n=24) and self-report checklists (n=60) and (2) individual interviews with physiotherapists (n=9). Quantitatively, fidelity scores were calculated using percentage means and SD of components delivered. Associations between fidelity scores and physiotherapist variables were analysed using Spearman's correlations. Interviews were analysed using thematic analysis to explore potential reasons for fidelity scores. Integration of quantitative and qualitative data occurred at an interpretation level using triangulation.

RESULTS

Quantitatively, fidelity scores were high for all assessment methods; with self-report (92.7%) consistently higher than direct observations (82.7%) or audio-recordings (81.7%). There was significant variation between physiotherapists' individual scores (69.8% - 100%). Both qualitative and quantitative data (from physiotherapist variables) found that physiotherapists' knowledge (Spearman's association at p=0.003) and previous experience (p=0.008) were factors that influenced their fidelity. The qualitative data also postulated participant-level (eg, individual needs) and programme-level factors (eg, resources) as additional elements that influenced fidelity.

CONCLUSION

The intervention was delivered with high fidelity. This study contributes to the limited evidence regarding fidelity assessment methods within complex behaviour change interventions. The findings suggest a combination of quantitative methods is suitable for the assessment of fidelity of delivery. A mixed methods approach provided a more insightful understanding of fidelity and its influencing factors.

TRIAL REGISTRATION NUMBER

ISRCTN49875385; Pre-results.

Multi-year school-based implementation and student outcomes of an evidence-based risk reduction intervention

Background

Intervention effects observed in efficacy trials are rarely replicated when the interventions are broadly disseminated, underscoring the need for more information about factors influencing real-life implementation and program impact. Using data from the ongoing national implementation of an evidence-based HIV prevention program [Focus on Youth in The Caribbean (FOYC)] in The Bahamas, this study examines factors influencing teachers’ patterns of implementation, the impact of teachers’ initial implementation of FOYC, and subsequent delivery of the booster sessions on students’ outcomes.

Methods

Data were collected from the 80 government elementary and 34 middle schools between 2011 and 2014, involving 208 grade 6, 75 grade 7, and 58 grade 8 teachers and 4411 students initially in grade 6 and followed for 3 years. Student outcomes include HIV/AIDS knowledge, reproductive health skills, self-efficacy, and intention to use protection. Data from teachers includes implementation and modification of the curriculum, attitudes towards the prevention program, comfort level with the curriculum, and attendance at training workshops. Structural equation modeling and mixed-effect modeling analyses were applied to examine the impact of teachers’ implementation.

Results

Teachers’ attitudes towards and comfort with the intervention curriculum, and attendance at the curriculum training workshop had a direct effect on teachers’ patterns of implementation, which had a direct effect on student outcomes. Teachers’ attitudes had a direct positive effect on student outcomes. Teachers’ training in interactive teaching methods and longer duration as teachers were positively associated with teachers’ comfort with the curriculum. High-quality implementation in grade 6 was significantly related to student outcomes in grades 6 and 7 post-implementation. Level of implementation of the booster sessions in grades 7 and 8 were likewise significantly related to subsequent student outcomes in both grades.

Conclusions

High-quality initial implementation of a prevention program is significantly related to better program outcomes. Poor subsequent delivery of booster sessions can undermine the positive effects from the initial implementation while strong subsequent delivery of booster sessions can partially overcome poor initial implementation.

Electronic supplementary material

The online version of this article (doi:10.1186/s13012-016-0539-7) contains supplementary material, which is available to authorized users.

Development of a Brief Pre-Implementation Screening Tool to Identify Teachers Who Are at Risk for Not Implementing Intervention Curriculum and High-Implementing Teachers

Few questionnaires have been developed to screen for potentially poor implementers of school-based interventions. This study combines teacher characteristics, perceptions, and teaching/training experiences to develop a short screening tool that can identify potential "low-performing" or "high-performing" teachers pre-implementation. Data were gathered from 208 teachers and 4,411 students who participated in the national implementation of an evidence-based HIV intervention in The Bahamas. Sensitivity and specificity were evaluated for the detection of "low-performing" and "high-performing" teachers. The validity of the screening tool was assessed using receiver operating characteristics analysis. The School Pre-implementation Screening Tool consists of seven predictive factors: duration as teacher, working site, attendance at training workshops, training in interactive teaching, perceived importance of the intervention, comfort in teaching the curriculum, and program priority. The sensitivity and specificity were 74% and 57% in identifying "low-performing" teachers and 81% and 65% with "high-performing" teachers. The screening tool demonstrated an acceptable/good validity (area under the receiver operating characteristics curve was 0.68 for "low-performing teachers" and 0.78 for "high-performing" teachers). Our brief screening tool can facilitate teacher training and recruitment of engaged teachers in implementation of school-based interventions.

The Impact of Teachers' Modifications of an Evidenced-Based HIV Prevention Intervention on Program Outcomes

The degree to which evidence-based program outcomes are affected by modifications is a significant concern in the implementation of interventions. The ongoing national implementation of an evidence-based HIV prevention program targeting grade 6 students in The Bahamas [Focus on Youth in The Caribbean (FOYC)] offers an opportunity to explore factors associated with teachers' modification of FOYC lessons and to examine the impact of types and degrees of modifications on student outcomes. Data were collected in 2012 from 155 teachers and 3646 students in 77 government elementary schools. Results indicate that teachers taught 16 of 30 core activities, 24.5 of 46 total activities and 4.7 of 8 sessions. Over one-half of the teachers made modifications to FOYC core activities; one-fourth of the teachers modified 25 % or more core activities that they taught (heavily modified FOYC). Omitting core activities was the most common content modification, followed by lengthening FOYC lessons with reading, writing assignments or role-play games, and shortening core activities or adding educational videos. Mixed-effects modeling revealed that omitting core activities had negative impacts on all four student outcomes. Shortening core activities and adding videos into lessons had negative impacts on HIV/AIDS knowledge and/or intention to use condom protection. Heavy modifications (>1/4 core activities) were associated with diminished program effectiveness. Heavy modifications and omitting or shortening core activities were negatively related to teachers' level of implementation. We conclude that poorer student outcomes were associated with heavy modifications.

School-based interventions for improving contraceptive use in adolescents

BACKGROUND

Young women, especially adolescents, often lack access to modern contraception. Reasons vary by geography and regional politics and culture. The projected 2015 birth rate in 'developing' regions was 56 per 1000 compared with 17 per 1000 for 'developed' regions.

OBJECTIVES

To identify school-based interventions that improved contraceptive use among adolescents

SEARCH METHODS

Until 6 June 2016, we searched for eligible trials in PubMed, CENTRAL, ERIC, Web of Science, POPLINE, ClinicalTrials.gov and ICTRP.

SELECTION CRITERIA

We considered randomized controlled trials (RCTs) that assigned individuals or clusters. The majority of participants must have been 19 years old or younger.The educational strategy must have occurred primarily in a middle school or high school. The intervention had to emphasize one or more effective methods of contraception. Our primary outcomes were pregnancy and contraceptive use.

DATA COLLECTION AND ANALYSIS

We assessed titles and abstracts identified during the searches. One author extracted and entered the data into RevMan; a second author verified accuracy. We examined studies for methodological quality.For unadjusted dichotomous outcomes, we calculated the Mantel-Haenszel odds ratio (OR) with 95% confidence interval (CI). For cluster randomized trials, we used adjusted measures, e.g. OR, risk ratio, or difference in proportions. For continuous outcomes, we used the adjusted mean difference (MD) or other measures from the models. We did not conduct meta-analysis due to varied interventions and outcome measures.

MAIN RESULTS

The 11 trials included 10 cluster RCTs and an individually randomized trial. The cluster RCTs had sample sizes from 816 to 10,954; the median number of clusters was 24. Most trials were conducted in the USA and UK; one was from Mexico and one from South Africa.We focus here on the trials with moderate quality evidence and an intervention effect. Three addressed preventing pregnancy and HIV/STI through interactive sessions. One trial provided a multifaceted two-year program. Immediately after year one and 12 months after year two, the intervention group was more likely than the standard-curriculum group to report using effective contraception during last sex (reported adjusted ORs 1.62 ± standard error (SE) 0.22) and 1.76 ± SE 0.29), condom use during last sex (reported adjusted ORs 1.91 ± SE 0.27 and 1.68 ± SE 0.25), and less frequent sex without a condom in the past three months (reported ratios of adjusted means 0.50 ± SE 0.31 and 0.63 ± SE 0.23). Another trial compared multifaceted two-year programs on sexual risk reduction and risk avoidance (abstinence-focused) versus usual health education. At 3 months, the risk reduction group was less likely than the usual-education group to report no condom use at last intercourse (reported adjusted OR 0.67, 95% CI 0.47 to 0.96) and sex without a condom in the last three months (reported adjusted OR 0.59, 95% CI 0.36 to 0.95). At 3 and after 15 months, the risk avoidance group was also less likely than the usual-education group to report no condom use at last intercourse (reported adjusted ORs 0.70, 95% CI 0.52 to 0.93; and 0.61, 95% CI 0.45 to 0.85). At the same time points, the risk reduction group had a higher score than the usual-education group for condom knowledge. The third trial provided a peer-led program with eight interactive sessions. At 17 months, the intervention group was less likely than the teacher-led group to report oral contraceptive use during last sex (OR 0.57, 95% CI 0.36 to 0.91). This difference may not have been significant if the investigators had adjusted for the clustering. At 5 and 17 months, the peer-led group had a greater mean increase in knowledge of HIV and pregnancy prevention compared with the control group. An additional trial showed an effect on knowledge only. The group with an emergency contraception (EC) session was more likely than the group without the EC unit to know the time limits for using hormonal EC (pill) and the non-hormonal IUD as EC.

AUTHORS' CONCLUSIONS

Since most trials addressed preventing STI/HIV and pregnancy, they emphasized condom use. However, several studies covered a range of contraceptive methods. The overall quality of evidence was low. Main reasons for downgrading the evidence were having limited information on intervention fidelity, analyzing a subsample rather than all those randomized, and having high losses.

Teachers' patterns of implementation of an evidence-based intervention and their impact on student outcomes: results from a nationwide dissemination over 24-months follow-up

More information is needed about factors influencing real-life implementation and program impact of interventions effective in controlled study conditions. Ongoing national implementation of an evidence-based HIV prevention program targeting grade 6 students in The Bahamas offers the opportunity to examine patterns of implementation and relate them to student outcomes. Data were collected from 208 grade 6 teachers, 75 grade 7 teachers and 4411 grade 6 students followed over 2 years. Mixed-effects modeling analysis examined the association of teachers' patterns of implementation with student outcomes. High quality program implementation in grade 6 (high implementation dosage and fidelity) was significantly related to student outcomes six and 18 months post-intervention. Quality of implementation of the booster session in grade 7 was also significantly related to student outcomes in grade 7. Quality of delivery of the brief booster session a year after initial implementation is important in maintaining or resetting the student outcome trajectory.