Evaluation of a Large-Scale School Wellness Intervention Through the Consolidated Framework for Implementation Research (CFIR): Implications for Dissemination and Sustainability

An implementation evaluation of a sports-based health intervention for underrepresented middle school youth in Philadelphia

Multicomponent, community-based programs aiming to improve health behaviors in youth are needed but can be challenging to implement. Research is needed to better understand the factors that facilitate and inhibit effective implementation of these programs especially for youth at increased risk of health disparities. This study aimed to identify and explore the implementation determinants and outcomes of a multicomponent health intervention conducted from 2021 to 2022 for middle school students living in underserved communities in Philadelphia, PA, USA. Mixed methods approaches, including self-report surveys and semi-structured qualitative interviews, were administered to 18 members of the implementation staff at the end of the program, including coaches (n = 7), assistant coaches (n = 2), school champions (n = 6), administrative leaders (n = 2), and a school district administrator (n = 1). Survey and interview questions were guided by the Consolidated Framework for Implementation Research (CFIR), and interviews were thematically coded following transcription based on 26 CFIR constructs. Innovation source, evidence strength and quality, cosmopolitanism, and the personal attributes of individuals were key constructs associated with implementation effectiveness. Data revealed three multidimensional themes that highlighted broader challenges influencing implementation: (i) broad consensus, different interpretations, (ii) staffing challenges, and (iii) continuity is key. The need for the program was clearly recognized and overall belief in the purpose of the intervention was strong among key program staff and administration. However, issues including limited engagement with training, staffing turnover, and the rotational programming design hindered implementation. Future projects aiming to implement multicomponent after-school time interventions must ensure a consistent vision among partners and continuity in program delivery.

Exploring the long-term sustainability of school-based nutrition and food programs: What works, where and why?

ISSUE ADDRESSED

Most food and nutrition programs cease within 2 years. Understanding the determinants of program sustainability is crucial to maximise output from funding, whilst allowing sufficient time for program benefits to be achieved. This study applied the Consolidated Framework for Implementation Research (CFIR) to map the barriers and enablers of successful long-term implementation of school-based nutrition and food programs.

METHODS

Qualitative methods with purposive and snowball sampling were used to recruit experts who were identified as being influential in implementing and sustaining long-term (>2 years) school-based food and nutrition programs. Semi-structured interviews with global experts were conducted, transcribed verbatim and coded deductively (by applying the CFIR constructs) and inductively when required. Thematic analysis informed the development of themes.

RESULTS

Interviews were conducted with 11 experts including researchers, government employees, and a consultant of an international agency, from seven countries. Forty-eight deductive codes and eight inductive codes identified six main themes: (1) funding and integrity of its source; (2) political landscape; (3) nutrition policies and their monitoring; (4) involvement of community actors; (5) adaptability of the program and (6) effective program evaluation. Themes related mainly to the 'outer setting' domain of the CFIR.

CONCLUSIONS

The CFIR highlighted pertinent factors that influence the successful long-term implementation of school-based food and nutrition programs. SO WHAT?: The findings suggest that to sustain program implementation beyond its initial funding, relationships across government departments, local organisations and communities, need to be nurtured and prioritised from the outset.

Initial insights into the impact and implementation of Creating Active Schools in Bradford, UK

BACKGROUND

Few whole-school physical activity programmes integrate implementation science frameworks within the design, delivery, and evaluation. As a result, knowledge of the key factors that support implementation at scale is lacking. The Creating Active Schools (CAS) programme was co-designed and is underpinned by the Capability, Opportunity, Motivation and Behaviour (COM-B) model and the Consolidated Framework for Implementation Research (CFIR). The study aims to understand the initial impact and implementation of CAS in Bradford over 9 months using McKay's et al.'s (2019) implementation evaluation roadmap.

METHODS

Focus groups and interviews were conducted with school staff (n = 30, schools = 25), CAS Champions (n = 9), and the CAS strategic lead (n = 1). Qualitative data were analysed both inductively and deductively. The deductive analysis involved coding data into a priori themes based on McKay et al's implementation evaluation roadmap, using a codebook approach to thematic analysis. The inductive analysis included producing initial codes and reviewing themes before finalising.

RESULTS

Identified themes aligned into three categories: (i) key ingredients for successful adoption and implementation of CAS, (ii) CAS implementation: challenges and solutions, and (iv) the perceived effectiveness of CAS at the school level. This included the willingness of schools to adopt and implement whole-school approaches when they are perceived as high quality and aligned with current school values. The programme implementation processes were seen as supportive; schools identified and valued the step-change approach to implementing CAS long-term. Formal and informal communities of practice provided "safe spaces" for cross-school support. Conversely, challenges persisted with gaining broader reach within schools, school staff's self-competence and shifting school culture around physical activity. This resulted in varied uptake between and within schools.

CONCLUSIONS

This study provides novel insights into the implementation of CAS, with outcomes aligning to the adoption, reach, and sustainability. Successful implementation of CAS was underpinned by determinants including acceptability, intervention complexity, school culture and school stakeholders' perceived self-efficacy. The combination of McKay's evaluation roadmap and CFIR establishes a rigorous approach for evaluating activity promotion programmes underpinned by behavioural and implementation science. Resultantly this study offers originality and progression in understanding the implementation and effectiveness of whole-school approaches to physical activity.

Individual and Contextual Factors Associated with Classroom Teachers' Intentions to Implement Classroom Physical Activity

Classroom-based physical activity (CPA) is an evidence-based practice that improves student physical activity outcomes, but national data suggest implementation is insufficient in US classrooms. The purpose of this study was to examine individual and contextual factors associated with elementary school teachers' intentions to implement CPA. We collected input survey data from 181 classroom teachers (10 schools; 98.4% participation among eligible teachers) across three separate cohorts to examine associations between individual and contextual constructs and future CPA implementation intentions. Data were analyzed using multilevel logistic regression. Individual-level characteristics of perceived autonomy for using CPA, perceived relative advantage/compatibility of CPA, and general openness to educational innovations were positively associated with intentions to implement CPA (p < 0.05). Teacher perceptions of contextual factors such as administrator support for CPA were also associated with implementation intentions. This study adds to prior evidence about the importance of theoretically determined constructs for understanding behavioral intentions among front-line implementers such as classroom teachers. Additional research is needed to evaluate interventions designed to change malleable factors, including teachers' perceptions, as well as changing school environments so that teachers perceive more autonomy to use CPA and have the training and resources that build skills for implementation.

Assessing the Impacts of Creating Active Schools on Organisational Culture for Physical Activity

BACKGROUND

National and international guidance recommends whole-school approaches to physical activity, but there are few studies assessing their effectiveness, especially at an organisational level. This study assesses the impact of the Creating Active School's (CAS) programme on organisational changes to physical activity provision.

METHODS

In-school CAS leads completed a 77-item questionnaire assessing school-level organisational change. The questionnaire comprised 19 domains aligned with the CAS framework and COM-B model of behaviour change. Wilcoxon Signed Rank Tests assessed the pre-to-nine-month change.

RESULTS

>70% of schools (n = 53) pre-CAS had inadequate whole-school physical activity provision. After nine months (n = 32), CAS had a significant positive effect on organisational physical activity. The positive change was observed for: whole-school culture and ethos, teachers and wider school staff, academic lessons, physical education (PE) lessons, commute to/from school and stakeholder behaviour.

CONCLUSIONS

This study provides preliminary evidence that CAS is a viable model to facilitate system-level change for physical activity in schools located within deprived areas of a multi-ethnic city. To confirm the results, future studies are required which adopt controlled designs combined with a holistic understanding of implementation determinants and underlying mechanisms.

Factors contributing to the sustained implementation of an early childhood obesity prevention intervention: The INFANT Program

Background

The INFANT Program is an efficacious, group-based program for first-time parents, delivered at three-monthly intervals when INFANT are aged 3-18 months through an existing universal care service in Victoria, Australia. Many lessons have been learnt from its origins as a cluster randomized control trial to its small-scale, community-level implementation. This study aimed to describe factors contributing to its sustained implementation to inform large-scale implementation across Australia.

Methods

This study used a multi-site qualitative exploratory approach. INFANT facilitators trained between 2013 and 2017 were sent an online survey, with optional telephone interviews. The Consolidated Framework for Implementation Research (CFIR) was selected as the underpinning theoretical framework as it offered the opportunity to explore a breadth of possible barriers and enablers across patterns of implementation (never, discontinued, ongoing).

Results

All participants were female (n = 31), the majority were Maternal and Child Health Nurses (48%), representing five regional and nine metro local government areas (LGAs), across all patterns of implementation (never implemented n = 4; discontinued implementation n = 5; ongoing implementation n = 5). All consenting participants were interviewed (n = 11) representing four regional and seven metro LGAs, across all patterns of implementation (never implemented n = 3; discontinued implementation n = 4; ongoing implementation n = 4). The main reason for attending INFANT Program training was to become skilled to implement the program. Mapping identified barriers and enablers to the CFIR revealed the inner and outer settings and implementation process to be of greatest influence. Main differences between LGAs with ongoing and discontinued implementation related to funding availability, organizational management support and endorsement, organizational resourcing and capacity, integration into routine practice and establishing role clarity with partner organizations, and planning for sustained implementation from the start.

Conclusion

This study provides important insights into the barriers and enablers to the sustained implementation of an evidence-based intervention (the INFANT Program) during small scale community-level implementation. The authors therefore contend that the pre-requisite for scale-up of a population health intervention is not just proof of effectiveness but also proof of sustained implementation at the local/organizational level. Study findings have broad transferability given their similarity to those identified for health promotion interventions implemented globally, in healthcare, education and community settings.

A Health Equity Implementation Approach to Child Health Literacy Interventions

Health and behavioural inequalities exist in all populations, including children. As a social determinant of health, health literacy is a crucial driver of equitable health outcomes in children. With the increasing calls for more actions on addressing low health literacy and inequalities, health literacy interventions to improve children's healthy behaviours have emerged as a key strategy to reduce health inequities. However, health literacy interventions face implementation challenges impacting upon potential outcomes, and disparities in the implementation of health literacy interventions also occur. Variation exists in child health literacy intervention target groups, timing, content and formats, and there is a lack of implementation specificity, resulting in a lack of clarity about which intervention strategies are the most effective in improving health literacy, related health behaviours, and associated health outcomes. While actions to facilitate child health intervention implementation exist, to minimise further perpetuation of child health inequities, this perspective calls for a health equity implementation approach to child health literacy interventions.