Evaluation of a school-based participatory intervention to improve school environments using the Consolidated Framework for Implementation Research

Relationship between the inner setting of CFIR and the delivery of the Healthy School Recognized Campus initiative: a mixed-methods analysis

INTRODUCTION

Healthy School Recognized Campus (HSRC) is a Texas A&M AgriLife Extension initiative that promotes the delivery of multiple evidence-based physical activity and nutrition programs in schools. Simultaneous delivery of programs as part of HSRC can result in critical implementation challenges. The study examines how the inner setting constructs from the Consolidated Framework for Implementation Research (CFIR) impact HSRC program delivery.

METHODS

We surveyed (n = 26) and interviewed (n = 20) HSRC implementers (n = 28) to identify CFIR inner setting constructs related to program acceptability, appropriateness, and feasibility. Using a concurrent mixed-methods design, we coded interviews using the CFIR codebook, administered an inner setting survey, tested for relationships between constructs and implementation outcomes via chi-square tests, and compared quantitative and qualitative results.

RESULTS

Stakeholders at schools that implemented one program vs. more than one program reported no differences in acceptability, appropriateness, or feasibility outcomes (p > .05); however, there was a substantial difference in reported program minutes (1118.4 ± 951.5 vs. 2674.5 ± 1940.8; p = .036). Available resources and leadership engagement were related to HSRC acceptability (r = .41; p = .038 and r = .48; p = .012, respectively) and appropriateness (r = .39; p = .046 and r = 0.63; p = .001, respectively). Qualitative analyses revealed that tangible resources (e.g., curriculum, a garden) enabled implementation, whereas intangible resources (e.g., lack of time) hindered implementation. Participants also stressed the value of buy-in from many different stakeholders. Quantitative results revealed that implementation climate was related to HSRC acceptability (r = .46; p = .018), appropriateness (r = .50; p = .009), and feasibility (r = .55; p = .004). Learning climate was related to HSRC appropriateness (r = .50; p = .009). However, qualitative assessment of implementation climate subconstructs showed mixed perspectives about their relationship with implementation, possibly due to differences in the compatibility/priority of different programs following COVID-19. Networks/communication analysis showed that schools have inner and outer circles of communication that can either benefit or hinder implementation.

CONCLUSION

Few differences were found by the number of programs delivered. Implementation climate (i.e., compatibility, priority) and readiness for implementation (i.e., resources and leadership engagement) were important to HSRC implementation. Strategies that focus on reducing time-related burdens and engaging stakeholders may support HSRC's delivery. Other constructs (e.g., communication, access to knowledge) may be important to the implementation of HSRC but need further exploration.

Exploring the long-term sustainability of school-based nutrition and food programs: What works, where and why?

ISSUE ADDRESSED

Most food and nutrition programs cease within 2 years. Understanding the determinants of program sustainability is crucial to maximise output from funding, whilst allowing sufficient time for program benefits to be achieved. This study applied the Consolidated Framework for Implementation Research (CFIR) to map the barriers and enablers of successful long-term implementation of school-based nutrition and food programs.

METHODS

Qualitative methods with purposive and snowball sampling were used to recruit experts who were identified as being influential in implementing and sustaining long-term (>2 years) school-based food and nutrition programs. Semi-structured interviews with global experts were conducted, transcribed verbatim and coded deductively (by applying the CFIR constructs) and inductively when required. Thematic analysis informed the development of themes.

RESULTS

Interviews were conducted with 11 experts including researchers, government employees, and a consultant of an international agency, from seven countries. Forty-eight deductive codes and eight inductive codes identified six main themes: (1) funding and integrity of its source; (2) political landscape; (3) nutrition policies and their monitoring; (4) involvement of community actors; (5) adaptability of the program and (6) effective program evaluation. Themes related mainly to the 'outer setting' domain of the CFIR.

CONCLUSIONS

The CFIR highlighted pertinent factors that influence the successful long-term implementation of school-based food and nutrition programs. SO WHAT?: The findings suggest that to sustain program implementation beyond its initial funding, relationships across government departments, local organisations and communities, need to be nurtured and prioritised from the outset.

Examining implementation determinants of a culturally grounded, school-based prevention curriculum in rural Hawai'i: A test development and validation study

Background

This study examined the implementation determinants of a culturally grounded, school-based drug prevention curriculum in rural Hawai'i. Test development and validation procedures were used to examine the impact of implementation barriers and facilitators of the curriculum in public or charter middle/intermediate schools on Hawai'i Island.

Method

A five-phase, mixed-methods approach toward test development and validation was used. These phases included item generation (Phase 1), item refinement and selection (Phase 2), item reduction (Phase 3), reliability testing (Phase 4), and validity testing (Phase 5). Educational administrators, teachers, and staff employed by the Hawai'i State Department of Education (HIDOE) participated in the study.

Results

Phases 1 and 2 yielded 50 implementation barriers and 27 implementation facilitators that were evaluated by 204 HIDOE administrators, teachers, and staff. Factor analysis of the barrier items indicated a four-factor solution: (1) Innovation Barriers, (2) HIDOE State-Level Barriers, (3) Teacher-Level Barriers, and (4) Administrator-Level Barriers. Mean comparisons indicated that several barrier and facilitator items differentiated teachers from administrators in the sample.

Conclusions

This study contributes to the implementation measurement literature, specifically in the areas of mental health and substance use. It also highlights the importance of addressing multiple contextual levels in the implementation of culturally focused prevention interventions.

Plain Language Summary Title

Examining Implementation Barriers and Facilitators for School-Based Prevention in Hawai'i. Plain Language Summary Compared with other major ethnic groups, Native Hawaiian and Pacific Islander (NHPI) youth have substantially higher rates of substance use and higher adverse mental and physical health effects related to their use. Despite these disparities, educational and community-based practitioners have long struggled with implementing and sustaining promising substance use interventions for NHPI youth. This study contributes to our understanding of measuring implementation barriers and facilitators for substance use prevention for NHPI youth in rural school settings. It contributes to the field of health disparities and health equity promotion, by addressing calls for research to understand factors affecting successful implementation of prevention programs. This is critical toward achieving health equity for underrepresented and vulnerable populations, such as NHPI and rural youth.

School partner perspectives on the implementation of the Your Voice Your View sexual assault prevention program for high school students

Despite the high risk for sexual assault among adolescents, few sexual assault prevention programs designed for implementation in high schools have sustained rigorous evaluation. The present study sought to better understand the factors that influenced the implementation of Your Voice Your View (YVYV), a four-session sexual assault prevention program for 10th grade students, which includes a teacher "Lunch and Learn" training as well as a 4-week school-specific social norms poster campaign. Following program implementation, eight school partners (i.e., health teachers, guidance counselors, teachers, and principals) participated in an interview to provide feedback on the process of program implementation. The Consolidated Framework for Implementation Research was utilized to examine site-specific determinants of program implementation. Participants discussed the importance of the design quality and packaging of the program, as well as the relative advantage of offering students a violence prevention program led by an outside team, as opposed to teachers in the school. School partners highlighted the importance of intensive preplanning before implementation, clear communication between staff, the utility of engaging a specific champion to coordinate programming, and the utility of offering incentives for participation. Having resources to support implementation, a desire to address sexual violence in the school, and a positive classroom climate in which to administer the small-group sessions were seen as school-specific facilitators of program implementation. These findings can help to support the subsequent implementation of the YVYV program, as well as other sexual assault prevention programs in high schools.

Implementing school-wide sexual assault prevention in middle schools: A qualitative analysis of school stakeholder perspectives

Sexual assault and harassment are significant problems that begin early in the lifespan. The current study sought to understand the contextual factors that influence the implementation of school-wide sexual assault prevention programs in middle schools that focus on fostering community norms change. The Consolidated Framework for Implementation Research (CFIR) is a menu of constructs arranged across 5 domains that assists stakeholders in assessing and identifying site-specific determinants of successful intervention implementation. In the present study, researchers conducted a series of 10 interviews with middle school stakeholders (i.e., principals, guidance counselors, teachers) to document characteristics inside and outside of the school environment (i.e., cultural norms, relative priority, prior interventions, implementation climate) relevant to the implementation of prevention programming, using the CFIR as a guiding framework for analysis. Whereas schools recognized the importance of implementing violence prevention programming, stakeholders reported several other competing demands (i.e., time, resources) that make it difficult to implement rigorous programming without support from an outside agency/team. Community agencies and research teams hoping to implement violence prevention in middle schools can benefit from using stakeholder interviews grounded in the CFIR model to gain a better awareness of the school- and community-specific factors that are likely to influence successful implementation of violence prevention programs in middle schools.

Facilitators and Barriers to Implementation and Sustainability of Nutrition and Physical Activity Interventions in Early Childcare Settings: a Systematic Review

A stronger understanding of the factors influencing implementation of interventions in community-based early childcare settings is needed. The purpose of this systematic review was to synthesize existing research on facilitators and barriers to implementation and sustainability of nutrition and physical activity interventions in early childcare settings targeting 2-5-year-old children, including considerations for equitable implementation. This review adhered to PRISMA 2020 guidelines. Peer-reviewed literature was searched in PubMed, EMBASE, CINAHL, ERIC, and PsycINFO databases up to September 2020. Primary research studies that examined facilitators and barriers (or related synonyms) to the implementation and sustainability of nutrition and physical activity interventions in early childcare settings were eligible for inclusion. The search yielded 8092 records that were screened by four analysts in Covidence software with a final review of 24 studies. Two independent reviewers conducted study selection, data extraction, and quality appraisal (Mixed Methods Appraisal Tool). A "best fit" framework was applied using the Consolidated Framework for Implementation Research (CFIR) constructs to code barriers and facilitators. The most salient constructs were (1) "Available Resources," which was composed of time, staffing, space, and staff trainings; (2) adaptability; and (3) compatibility, the latter two indicating that easily modifiable interventions facilitated a smoother "fit" and were more likely to be successful, given adequate site-level resources. Only nine (28%) reported the use of a theory, model, or framework to guide evaluation; six studies (24%) included factors related to sustainability; and nine studies (38%) conducted their interventions with low-income or minoritized groups. The findings point to the need for intervention evaluations examining nutrition and physical activity to more consistently consider (a) sustainability factors early on in design and adoption phases; (b) use of theory, model, or framework to guide evaluation; and (c) equity-related frameworks and considerations for how equitable implementation.

Somali, Latino, and Hmong Youth Perceptions of School Connectedness

Purpose:

School connectedness positively influences adolescent health outcomes and is a key social determinant of health, yet, contributors to school connectedness for youth from immigrant communities remain poorly defined.

Methods:

This community-based participatory research study uses thematic analysis to identify contributors to Somali, Latino, and Hmong (SLH) adolescents' school connectedness. We conducted nine focus groups with 71 SLH male and female adolescents, the majority aged 13–18 years, in a United States Midwestern metropolitan area.

Results:

SLH students described contributors to their school connectedness that fit within three broad themes: (1) SLH students desire to be known and supported by their teachers as individuals, (2) specific teacher instructional approaches reinforce or undermine SLH student–school connections, and (3) transparency and fairness in school disciplinary practices are especially important for promoting Latino student–school connectedness.

Conclusion:

SLH youth perspectives offer ways for educators to foster increased school connectedness to improve academic and health outcomes among increasingly diverse student populations.

Dissemination and Implementation Science to Advance Health Equity: An Imperative for Systemic Change

Innovations in public health research and evidence-based interventions targeting chronic and infectious diseases are only effective if they reach their target populations. Individuals from low socioeconomic background, racial and ethnic minorities, and sexual/gender minority communities are most susceptible to chronic diseases such as obesity and cancer, and infectious diseases such as HIV and COVID-19. These disparities are driven by social and structural conditions including stigma and discrimination, housing instability and food insecurity, among others. Accordingly, interventions that aim to improve population health must be targeted toward marginalized communities who are often systematically excluded from decision making processes. This article introduces dissemination and implementation science as a key opportunity to advance health equity through integrating measures and metrics that evaluate if an intervention is successful at improving health outcomes in marginalized populations. Implementation science also provides frameworks to help evaluate the key determinants to implementation success which can inform subsequent health outcomes. Examples of how researchers have engaged with community stakeholders are provided, along with strategies in which dissemination has gone beyond traditional practices. Finally, ways in which universities can build capacity for implementation science as a means to address health disparities are provided with the goal of improving the translation of research to practice.

Identifying essential implementation strategies: a mixed methods process evaluation of a multi-strategy policy implementation intervention for schools

BACKGROUND

Physically Active Children in Education (PACE) is composed of eight implementation strategies that improves schools' implementation of a government physical activity policy. A greater understanding of each discrete implementation strategy could inform improvements to PACE for delivery at-scale. This study aimed to: (A) measure the dose delivered, fidelity, adoption and acceptability of each strategy using quantitative data; (B) identify implementation barriers and facilitators using qualitative data; and (C) explore the importance of each strategy by integrating both data sets (mixed methods).

METHODS

This study used data from a cluster randomised noninferiority trial comparing PACE with an adapted version (Adapted PACE) that was delivered with reduced in-person external support to reduce costs and increase scalability. Data were collected from both trials arms for between-group comparison. Descriptive statistics were produced using surveys of principals, in-school champions and teachers; and project records maintained by PACE project officers (objective A). Thematic analysis was performed using in-school champion and project officer interviews (objective B). Both data sets were integrated via a triangulation protocol and findings synthesized in the form of meta-inferences (objective C).

RESULTS

Eleven in-school champions and six project officers completed interviews; 33 principals, 51 in-school champions and 260 teachers completed surveys. Regardless of group allocation, implementation indicators were high for at least one component of each strategy: dose delivered =100%, fidelity ≥95%, adoption ≥83%, acceptability ≥50%; and several implementation barriers and facilitators were identified within three broad categories: external policy landscape, inner organizational structure/context of schools, and intervention characteristics and processes. All strategies were considered important as use varied by school, however support from a school executive and in-school champions' interest were suggested as especially important for optimal implementation.

CONCLUSION

This study highlights the importance of both executive support and in-school champions for successful implementation of school physical activity policies. In particular, identifying and supporting an in-school champion to have high power and high interest is recommended for future implementation strategies. This may reduce the need for intensive external support, thus improving intervention scalability.

Evaluation of a Large-Scale School Wellness Intervention Through the Consolidated Framework for Implementation Research (CFIR): Implications for Dissemination and Sustainability

Background

Numerous studies have tested school-based interventions promoting healthy behaviors in youth, but few have integrated dissemination and implementation (D&I) frameworks. Using D&I frameworks can inform if and how an evidence-based intervention is implemented and maintained and provide strategies to address contextual barriers. Such application is necessary to understand how and why interventions are sustained over time. We evaluated a school wellness initiative called SWITCH® (School Wellness Integration Targeting Child Health) to (1) assess implementation outcomes of adoption, fidelity, and penetration, (2) discern implementation determinants through the Consolidated Framework for Implementation Research (CFIR), and (3) examine differences among inexperienced and experienced schools and influential factors to sustainment.

Methods

A total of 52 schools from Iowa, United States enrolled in the 2019-2020 iteration of SWITCH (22 inexperienced; 30 experienced). The CFIR guided the adaptation of mixed methods data collection and analysis protocols for school settings. Specific attention was focused on (1) fidelity to core elements; (2) adoption of best practices; and (3) penetration of behavior change practices. Determinants were investigated through in-depth qualitative interviews and readiness surveys with implementation leaders. A systematic process was used to score CFIR domains (between -2 and +2) indicating positive or negative influence. Independent t-tests were conducted to capture differences between samples, followed by a cross-case analysis to compare determinants data. Inductive coding yielded themes related to sustainment of SWITCH beyond formal implementation support.

Results

Experienced schools had higher scores on fidelity/compliance (t = -1.86 p = 0.07) and adoption (t = -2.03 p = 0.04). CFIR determinants of innovation source, culture, relative priority, and leadership engagement were positive implementation determinants, whereas tension for change and networks and communications were negative determinants. Distinguishing factors between experienced and inexperienced schools were Readiness for Implementation and Self-efficacy (experienced significantly higher; p < 0.05). Strategies to enhance sustainability were increasing student awareness/advocacy, keeping it simple, and integrating into school culture.

Conclusions

Findings provide specific insights related to SWITCH implementation and sustainability but more generalized insights about the type of support needed to help schools implement and sustain school wellness programming. Tailoring implementation support to both inexperienced and experienced settings will ultimately enhance dissemination and sustainability of evidence-based interventions.