Scale-up of the Physical Activity 4 Everyone (PA4E1) intervention in secondary schools: 24-month implementation and cost outcomes from a cluster randomised controlled trial

Process Evaluation of a Scaled-Up School-Based Physical Activity Program for Adolescents: Physical Activity 4 Everyone

BACKGROUND

Physical Activity 4 Everyone (PA4E1) is a whole-school physical activity program, with demonstrated efficacy (2012-2014). PA4E1 was adapted (scaled-up) and tested in a scale-up trial (2017-2020). This process evaluation study of the scale-up trial had 2 aims. First, to describe the acceptability, appropriateness, and feasibility of PA4E1 in the scale-up trial, from the perspective of school staff involved in the program management and delivery. Second, to generate themes that may explain school staff assessments of acceptability, appropriateness, and feasibility.

METHODS

Data were collected at various time points throughout the 2-year implementation phase. Online surveys were collected from In-School Champions, Head Physical Education teachers, Principals, and Physical Education teachers (quantitative data). Focus groups and interviews were conducted with In-School Champions, Principals, and Physical Education teachers (qualitative data). Existing published data on website engagement, adaptations, modifications, and the scale-up trial primary outcome (implementation of physical activity practices) were triangulated with the quantitative and qualitative during analysis, to generate themes.

RESULTS

School staff delivering PA4E1 reported it was highly acceptable, appropriate, and feasible. Seven themes were generated relating to acceptability, appropriateness, and feasibility. The themes related to how the program was funded, the delivery modes of implementation support, the identification of easy-wins, the recruitment of the right in-school champion, facilitating principal buy-in, mitigating the impact of school staff turnover, and engaging the whole school.

CONCLUSIONS

Recommendations are made to inform future adaptations for PA4E1 and potentially school-based physical activity programs more generally. The findings may inform future scalability assessments of the suitability of programs for scale-up.

A Hybrid Type III Effectiveness-Implementation Trial to Optimize Medication Safety With Oral Antitumor Therapy in Real-World: The AMBORA Competence and Consultation Center

PURPOSE

Implementation science endeavors to facilitate the translation of evidence-based research into clinical routine. The clinical pharmacological/pharmaceutical care program evaluated in the randomized AMBORA trial on medication safety with oral antitumor therapeutics (OAT) optimizes care delivery and provides significant benefits for patients, treatment teams, and health care systems. Thus, we aimed to investigate the implementation of this care program within the AMBORA Competence and Consultation Center (AMBORA Center).

METHODS

The AMBORA Center within a University Comprehensive Cancer Center offered several services (eg, patient consultations) and was evaluated according to the RE-AIM framework. This multicenter hybrid type III trial focused on implementation outcomes (eg, patient recruitment, referring units, evaluation of services) while concurrently investigating effectiveness (eg, side effects, medication errors). Quantitative and qualitative assessments were combined.

RESULTS

The AMBORA Center conducted over 800 consultations with 420 patients in seven institutions. The primary end point of counseling 70% of patients treated with OAT was not reached. Patients were referred by 15 treatment units compared with 11 units in the AMBORA trial. On the basis of heterogeneous referral rates and characteristics across the institutions, barriers and facilitators of the implementation process were derived. Several survey results (eg, stakeholder interviews, online/paper-based questionnaires) reflected a high appreciation of services by patients and health care professionals. The severity of 60.1% (178 of 296) of detected side effects improved, and 86.3% (297 of 344) of medication errors were resolved.

CONCLUSION

Despite not reaching the primary implementation outcome, the AMBORA Center included more treatment units and demonstrated patient benefit of the AMBORA care program by meeting all effectiveness outcomes. We outlined quantitative and qualitative implementation characteristics to enhance outreach and foster further dissemination of centers to optimize medication safety with OAT.

Interventions to prevent obesity in children aged 12 to 18 years old

BACKGROUND

Prevention of obesity in adolescents is an international public health priority. The prevalence of overweight and obesity is over 25% in North and South America, Australia, most of Europe, and the Gulf region. Interventions that aim to prevent obesity involve strategies that promote healthy diets or 'activity' levels (physical activity, sedentary behaviour and/or sleep) or both, and work by reducing energy intake and/or increasing energy expenditure, respectively. There is uncertainty over which approaches are more effective, and numerous new studies have been published over the last five years since the previous version of this Cochrane Review.

OBJECTIVES

To assess the effects of interventions that aim to prevent obesity in adolescents by modifying dietary intake or 'activity' levels, or a combination of both, on changes in BMI, zBMI score and serious adverse events.

SEARCH METHODS

We used standard, extensive Cochrane search methods. The latest search date was February 2023.

SELECTION CRITERIA

Randomised controlled trials in adolescents (mean age 12 years and above but less than 19 years), comparing diet or 'activity' interventions (or both) to prevent obesity with no intervention, usual care, or with another eligible intervention, in any setting. Studies had to measure outcomes at a minimum of 12 weeks post baseline. We excluded interventions designed primarily to improve sporting performance.

DATA COLLECTION AND ANALYSIS

We used standard Cochrane methods. Our outcomes were BMI, zBMI score and serious adverse events, assessed at short- (12 weeks to < 9 months from baseline), medium- (9 months to < 15 months) and long-term (≥ 15 months) follow-up. We used GRADE to assess the certainty of the evidence for each outcome.

MAIN RESULTS

This review includes 74 studies (83,407 participants); 54 studies (46,358 participants) were included in meta-analyses. Sixty studies were based in high-income countries. The main setting for intervention delivery was schools (57 studies), followed by home (nine studies), the community (five studies) and a primary care setting (three studies). Fifty-one interventions were implemented for less than nine months; the shortest was conducted over one visit and the longest over 28 months. Sixty-two studies declared non-industry funding; five were funded in part by industry. Dietary interventions versus control The evidence is very uncertain about the effects of dietary interventions on body mass index (BMI) at short-term follow-up (mean difference (MD) -0.18, 95% confidence interval (CI) -0.41 to 0.06; 3 studies, 605 participants), medium-term follow-up (MD -0.65, 95% CI -1.18 to -0.11; 3 studies, 900 participants), and standardised BMI (zBMI) at long-term follow-up (MD -0.14, 95% CI -0.38 to 0.10; 2 studies, 1089 participants); all very low-certainty evidence. Compared with control, dietary interventions may have little to no effect on BMI at long-term follow-up (MD -0.30, 95% CI -1.67 to 1.07; 1 study, 44 participants); zBMI at short-term (MD -0.06, 95% CI -0.12 to 0.01; 5 studies, 3154 participants); and zBMI at medium-term (MD 0.02, 95% CI -0.17 to 0.21; 1 study, 112 participants) follow-up; all low-certainty evidence. Dietary interventions may have little to no effect on serious adverse events (two studies, 377 participants; low-certainty evidence). Activity interventions versus control Compared with control, activity interventions do not reduce BMI at short-term follow-up (MD -0.64, 95% CI -1.86 to 0.58; 6 studies, 1780 participants; low-certainty evidence) and probably do not reduce zBMI at medium- (MD 0, 95% CI -0.04 to 0.05; 6 studies, 5335 participants) or long-term (MD -0.05, 95% CI -0.12 to 0.02; 1 study, 985 participants) follow-up; both moderate-certainty evidence. Activity interventions do not reduce zBMI at short-term follow-up (MD 0.02, 95% CI -0.01 to 0.05; 7 studies, 4718 participants; high-certainty evidence), but may reduce BMI slightly at medium-term (MD -0.32, 95% CI -0.53 to -0.11; 3 studies, 2143 participants) and long-term (MD -0.28, 95% CI -0.51 to -0.05; 1 study, 985 participants) follow-up; both low-certainty evidence. Seven studies (5428 participants; low-certainty evidence) reported data on serious adverse events: two reported injuries relating to the exercise component of the intervention and five reported no effect of intervention on reported serious adverse events. Dietary and activity interventions versus control Dietary and activity interventions, compared with control, do not reduce BMI at short-term follow-up (MD 0.03, 95% CI -0.07 to 0.13; 11 studies, 3429 participants; high-certainty evidence), and probably do not reduce BMI at medium-term (MD 0.01, 95% CI -0.09 to 0.11; 8 studies, 5612 participants; moderate-certainty evidence) or long-term (MD 0.06, 95% CI -0.04 to 0.16; 6 studies, 8736 participants; moderate-certainty evidence) follow-up. They may have little to no effect on zBMI in the short term, but the evidence is very uncertain (MD -0.09, 95% CI -0.2 to 0.02; 3 studies, 515 participants; very low-certainty evidence), and they may not reduce zBMI at medium-term (MD -0.05, 95% CI -0.1 to 0.01; 6 studies, 3511 participants; low-certainty evidence) or long-term (MD -0.02, 95% CI -0.05 to 0.01; 7 studies, 8430 participants; low-certainty evidence) follow-up. Four studies (2394 participants) reported data on serious adverse events (very low-certainty evidence): one reported an increase in weight concern in a few adolescents and three reported no effect.

AUTHORS' CONCLUSIONS

The evidence demonstrates that dietary interventions may have little to no effect on obesity in adolescents. There is low-certainty evidence that activity interventions may have a small beneficial effect on BMI at medium- and long-term follow-up. Diet plus activity interventions may result in little to no difference. Importantly, this updated review also suggests that interventions to prevent obesity in this age group may result in little to no difference in serious adverse effects. Limitations of the evidence include inconsistent results across studies, lack of methodological rigour in some studies and small sample sizes. Further research is justified to investigate the effects of diet and activity interventions to prevent childhood obesity in community settings, and in young people with disabilities, since very few ongoing studies are likely to address these. Further randomised trials to address the remaining uncertainty about the effects of diet, activity interventions, or both, to prevent childhood obesity in schools (ideally with zBMI as the measured outcome) would need to have larger samples.

Systems approaches to scaling up: a systematic review and narrative synthesis of evidence for physical activity and other behavioural non-communicable disease risk factors

BACKGROUND

Non-communicable diseases (NCDs) are the leading causes of death worldwide. Systems approaches have potential for creating sustainable outcomes at scale but have rarely been used to support scale up in physical activity/nutrition promotion or NCD prevention more generally. This review aimed to: (i) synthesise evidence on the use of systems approaches in scaling up interventions targeting four behavioural risk factors for NCDs; and (ii) to explore how systems approaches have been conceptualised and used in intervention implementation and scale up.

METHOD

Seven electronic databases were searched for studies published 2016-2021. Eligible studies targeted at least one of four NCD behavioural risk factors (physical inactivity, tobacco use, alcohol consumption, diet), or described evaluation of an intervention planned for or scaled up. Studies were categorised as having a (i) high, (ii) moderate, or (iii) no use of a systems approach. A narrative synthesis of how systems approaches had been operationalised in scale up, following PRISMA guidelines.

RESULTS

Twenty-one intervention studies were included. Only 19% (n = 4) of interventions explicitly used systems thinking to inform intervention design, implementation and scale up (targeting all four risk factors n = 2, diet n = 1, tobacco use n = 1). Five studies ('high use') planned and implemented scale up with an explicit focus on relations between system elements and used system changes to drive impact at scale. Seven studies ('moderate use') considered systems elements impacting scale-up processes or outcomes but did not require achieving system-level changes from the outset. Nine studies ('no use') were designed to work at multiple levels among multiple agencies in an intervention setting, but the complexity of the system and relations between system elements was not articulated. We synthesised reported barriers and facilitators to scaling up, and how studies within each group conceptualised and used systems approaches, and methods, frameworks and principles for scaling up.

CONCLUSION

In physical activity research, and NCD prevention more broadly, the use of systems approaches in scale up remains in its infancy. For researchers, practitioners and policymakers wishing to adopt systems approaches to intervention implementation at scale, guidance is needed on how to communicate and operationalise systems approaches in research and in practice.

TRIAL REGISTRATION

PROSPERO (CRD42021287265).

Effectiveness of Intervention Strategies to Increase Adolescents' Physical Activity and Reduce Sedentary Time in Secondary School Settings, Including Factors Related to Implementation: A Systematic Review and Meta-Analysis

BACKGROUND

Globally, just one in five adolescents meet physical activity guidelines and three-quarters of the school day is spent sitting. It is unclear which types of school-based interventions strategies increase physical activity and reduce sedentary time among adolescents, or how these interventions are implemented influences their effectiveness.

OBJECTIVE

The three aims of our systematic review were to (a) identify intervention strategies used within secondary school settings to improve students' movement behaviours throughout school-based initiatives, delivered at or by the school; (b) determine the overall effect of the interventions (meta-analysis) on physical activity (all intensities), sedentary time, cognitive/academic, physical health and/or psychological outcomes; and (c) describe factors related to intervention implementation.

METHODS

Searches were conducted in MEDLINE complete, EMBASE, CINAHL, SPORTDiscus, APA PsycINFO, and ERIC in January 2023 for studies that (a) included high school-aged adolescents; (b) involved a school-based intervention to increase physical activity and/or decrease sedentary time; and (c) were published in English. Reported effects were pooled in meta-analyses where sufficient data were obtained.

RESULTS

Eighty-five articles, representing 61 interventions, met the inclusion criteria, with 23 unique intervention strategies used. Interventions that involved whole-school approaches (i.e., physical activity sessions, environmental modifications, teacher training, peer support and/or educational resources) were favourably associated with most of the outcomes. The meta-analyses showed: (a) non-significant effects for sedentary time (Standardized mean difference [SMD] = -0.02; 95%CI, -0.14, 0.11), physical activity at all intensities (light: SMD= -0.01; 95%CI, -0.08, 0.05; moderate: SMD = 0.06; 95%CI, -0.09, 0.22; vigorous: SMD = 0.08; 95%CI, -0.02, 0.18; moderate-to-vigorous: SMD = 0.05; 95%CI, -0.01, 0.12) and waist circumference (SMD = 0.09; 95%CI, -0.03, 0.21), and (b) a small statistically significant decrease in body mass index (SMD= -0.09, 95%CI -0.16, -0.0). Factors related to intervention implementation were reported in 51% of the articles.

CONCLUSION

While some intervention approaches demonstrated promise, small or null effects were found in meta-analyses. Future school-based interventions should utilize a whole-school approach designed to increase adolescents' activity across the day. Consistent reporting of implementation will increase understanding of how interventions are adopted, implemented and sustained.

REGISTRATION

PROSPERO (CRD42020169988).

Evaluating the scaling up of an effective implementation intervention (PACE) to increase the delivery of a mandatory physical activity policy in primary schools

BACKGROUND

Physically Active Children in Education (PACE) is an effective implementation intervention for increasing the number of minutes classroom teachers schedule physical activity each week. To date, evaluations of PACE have included a smaller number of schools from only one region in New South Wales Australia. If PACE is to have population-wide benefits we must be able to deliver this support to a larger number of schools across multiple regions. This study aimed to evaluate the scale-up of PACE.

METHODS

An uncontrolled before and after study, with 100 schools from three regions was conducted. Participating schools received PACE for approximately 12 months. We assessed the following outcomes: delivery of the evidence-based intervention (EBI) (i.e. minutes of physical activity scheduled by classroom teachers per week); delivery of the implementation strategies (i.e. reach, dose delivered, adherence and indicators of sustainability); and key determinants of implementation (i.e. acceptability of strategies and cost). Data were collected via project officer records, and principal and teacher surveys. Linear mixed models were used to assess EBI delivery by evaluating the difference in the mean minutes teachers scheduled physical activity per week from baseline to follow-up. Descriptive data were used to assess delivery of the implementation strategies and their perceived acceptability (i.e. PACE). A prospective, trial-based economic evaluation was used to assess cost.

RESULTS

Delivery of the EBI was successful: teachers increas their average minutes of total physical activity scheduled across the school week by 26.8 min (95% CI: 21.2, 32.4, p < 0.001) after receiving PACE. Indicators for delivery of implementation strategies were high: 90% of consenting schools received all strategies and components (reach); 100% of strategies were delivered by the provider (dose); >50% of schools adhered to the majority of strategies (11 of the 14 components); and acceptability was > 50% agreement for all strategies. The incremental cost per additional minute of physical activity scheduled per week was $27 per school (Uncertainty Interval $24, $31).

CONCLUSIONS

PACE can be successfully delivered across multiple regions and to a large number of schools. Given the ongoing and scalable benefits of PACE, it is important that we continue to extend and improve this program while considering ways to reduce the associated cost.

Mapping intervention components from a randomized controlled trial to scale-up of an early life nutrition and movement intervention: The INFANT program

Introduction

Early life parent-focused interventions can effectively improve infant and child nutrition and movement (physical activity and sedentary behavior) as well as parents' health behaviors. Scale-up of such interventions to real-world settings is essential for population-wide benefits. When progressing to scale-up, intervention components may be modified to reflect contextual factors and promote feasibility of scale-up. The INFANT program, an efficacious early life nutrition and movement behavioral intervention began as a randomized controlled trial (RCT), was modified after a small-scale translation, and is currently being scaled-up in Victoria, Australia. This study mapped and compared discrete intervention components of both the original RCT and the scaled-up version of INFANT to examine modifications for scaling up.

Methods

Discrete intervention components, specifically the target behaviors (child-related and parent-related behaviors), delivery features and behavior change techniques (BCTs) from the RCT and the scaled-up program were coded and mapped using established frameworks and taxonomies. Publications and unpublished materials (e.g., facilitator notes, handouts, videos, app) were coded. Coding was performed independently in duplicate, with final coding validated in a meeting with interventionists. Interventionists reported the rationale for modifications made.

Results

The INFANT RCT and scaled-up version targeted the same obesity prevention-related nutrition and movement behaviors. Key modified delivery features at scale-up included reduced number of sessions, a broader range of professionals facilitating groups, the addition of a mobile app for parents replacing hard-copy materials and tangible tools (e.g., pedometers), and broadening of content (e.g., early feeding, updated 24-h movement guidelines). BCTs used across the RCT and scale-up sessions were unchanged. However, the BCTs identified in the between-session support materials were almost double for the scale-up compared with the RCT, primarily due to the reduced number of sessions and the app's capacity to include more content.

Conclusions

INFANT is one of few early life nutrition and movement behavioral interventions being delivered at scale. With INFANT as an example, this study provides critical understanding about what and why intervention components were altered as the RCT was scaled-up. Unpacking these intervention modifications provides important insights for scale-up feasibility, outcome effects, and how to optimize implementation strategies for population-level benefits.

Design and management considerations for control groups in hybrid effectiveness-implementation trials: Narrative review & case studies

Hybrid effectiveness-implementation studies allow researchers to combine study of a clinical intervention's effectiveness with study of its implementation with the aim of accelerating the translation of evidence into practice. However, there currently exists limited guidance on how to design and manage such hybrid studies. This is particularly true for studies that include a comparison/control arm that, by design, receives less implementation support than the intervention arm. Lack of such guidance can present a challenge for researchers both in setting up but also in effectively managing participating sites in such trials. This paper uses a narrative review of the literature (Phase 1 of the research) and comparative case study of three studies (Phase 2 of the research) to identify common themes related to study design and management. Based on these, we comment and reflect on: (1) the balance that needs to be struck between fidelity to the study design and tailoring to emerging requests from participating sites as part of the research process, and (2) the modifications to the implementation strategies being evaluated. Hybrid trial teams should carefully consider the impact of design selection, trial management decisions, and any modifications to implementation processes and/or support on the delivery of a controlled evaluation. The rationale for these choices should be systematically reported to fill the gap in the literature.

Feasibility of a peer-led, after-school physical activity intervention for disadvantaged adolescent females during the COVID-19 pandemic: results from the Girls Active Project (GAP)

Introduction

There is a critical need for interventions that can be feasibly implemented and are effective in successfully engaging adolescent females in physical activity (PA). A theory-based, peer-led, after-school PA intervention, the Girls Active Project (GAP), was codesigned with adolescent females. This study aimed to assess the feasibility of implementing and evaluating the GAP programme.

Setting

One single-sex, female-only, designated disadvantaged postprimary school (students aged 12–18) in Dublin, Ireland.

Methods

Mixed methods were applied with multiple stakeholders over a 12-week trial (March to May 2021). A single-arm study design was used to examine intervention: reach, dose, fidelity, acceptability, compatibility and context. Feasibility of using proposed self-reported outcome measures (moderate-to-vigorous PA levels, self-rated health, life satisfaction, PA self-efficacy and PA enjoyment) was also explored. Due to school closure resulting from the COVID-19 pandemic, the intervention was delivered both online and in person in the school setting.

Results

Eight exercise classes were peer delivered by project leaders (n = 6, students aged 15–17) to intervention recipients (students aged 13-14). Recruitment was low (n = 8, 10% of eligible students, mean age: 13.3 SD: 0.46), yet retention was high (n = 7/8, 88%). Attendance rates were satisfactory (68%), and the intervention was implemented with high fidelity (87%). Data completion rates suggested proposed self-reported outcome measures were deemed appropriate (≥ 95%), except for weight (50%) and height data (80%). Despite COVID-19 hindering intervention implementation, both quantitative and qualitative data suggested that stakeholders were satisfied and perceived the in-person delivered intervention to be compatible with the school setting. Recommended refinements included extending class duration, introducing different rewards, and boosting programme awareness.

Conclusions

Further thought must be given on how to increase recruitment. Overall, the in-person delivered after-school PA programme was well-received by stakeholders and shows promise as an intervention that can be feasibly implemented and evaluated. Suggested improvements to the GAP intervention programme are recommended, before continuing to a more robust evaluation.

Trial registration

10.17605/OSF.IO/75HWJ (prospectively registered, date of registration: 9th December 2020)

Supplementary Information

The online version contains supplementary material available at 10.1186/s40814-022-01149-2.

Economic evaluation of a multi-strategy intervention that improves school-based physical activity policy implementation

BACKGROUND

Internationally, government policies mandating schools to provide students with opportunities to participate in physical activity are poorly implemented. The multi-component Physically Active Children in Education (PACE) intervention effectively assists schools to implement one such policy. We evaluated the value of investment by health service providers tasked with intervention delivery, and explored where adaptations might be targeted to reduce program costs for scale-up.

METHODS

A prospective trial-based economic evaluation of an implementation intervention in 61 primary schools in New South Wales (NSW), Australia. Schools were randomised to the PACE intervention or a wait-list control. PACE strategies included centralised technical assistance, ongoing consultation, principal's mandated change, identifying and preparing in-school champions, educational outreach visits, and provision of educational materials and equipment. Effectiveness was measured as the mean weekly minutes of physical activity implemented by classroom teachers, recorded in a daily log book at baseline and 12-month follow-up. Delivery costs (reported in $AUD, 2018) were evaluated from a public finance perspective. Cost data were used to calculate: total intervention cost, cost per strategy and incremental cost (overall across all schools and as an average per school). Incremental cost-effectiveness ratios (ICERs) were calculated as the incremental cost of delivering PACE divided by the estimated intervention effect.

RESULTS

PACE cost the health service provider a total of $35,692 (95% uncertainty interval [UI] $32,411, $38,331) to deliver; an average cost per school of $1151 (95%UI $1046, $1236). Training in-school champions was the largest contributor: $19,437 total; $627 ($0 to $648) average per school. Educational outreach was the second largest contributor: $4992 total; $161 ($0 to $528) average per school. The ICER was $29 (95%UI $17, $64) for every additional minute of weekly physical activity implemented per school.

CONCLUSION

PACE is a potentially cost-effective intervention for increasing schools implementation of a policy mandate. The investment required by the health service provider makes use of existing funding and infrastructure; the additional cost to assist schools to implement the policy is likely not that much. PACE strategies may be adapted to substantially improve delivery costs.

TRIAL REGISTRATION

Australia New Zealand Clinical Trials Registry ACTRN12617001265369; Prospectively registered 1st September 2017 https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=373520.