Multi-strategy intervention increases school implementation and maintenance of a mandatory physical activity policy: outcomes of a cluster randomised controlled trial

Process Evaluation of a Scaled-Up School-Based Physical Activity Program for Adolescents: Physical Activity 4 Everyone

BACKGROUND

Physical Activity 4 Everyone (PA4E1) is a whole-school physical activity program, with demonstrated efficacy (2012-2014). PA4E1 was adapted (scaled-up) and tested in a scale-up trial (2017-2020). This process evaluation study of the scale-up trial had 2 aims. First, to describe the acceptability, appropriateness, and feasibility of PA4E1 in the scale-up trial, from the perspective of school staff involved in the program management and delivery. Second, to generate themes that may explain school staff assessments of acceptability, appropriateness, and feasibility.

METHODS

Data were collected at various time points throughout the 2-year implementation phase. Online surveys were collected from In-School Champions, Head Physical Education teachers, Principals, and Physical Education teachers (quantitative data). Focus groups and interviews were conducted with In-School Champions, Principals, and Physical Education teachers (qualitative data). Existing published data on website engagement, adaptations, modifications, and the scale-up trial primary outcome (implementation of physical activity practices) were triangulated with the quantitative and qualitative during analysis, to generate themes.

RESULTS

School staff delivering PA4E1 reported it was highly acceptable, appropriate, and feasible. Seven themes were generated relating to acceptability, appropriateness, and feasibility. The themes related to how the program was funded, the delivery modes of implementation support, the identification of easy-wins, the recruitment of the right in-school champion, facilitating principal buy-in, mitigating the impact of school staff turnover, and engaging the whole school.

CONCLUSIONS

Recommendations are made to inform future adaptations for PA4E1 and potentially school-based physical activity programs more generally. The findings may inform future scalability assessments of the suitability of programs for scale-up.

Effect of capacity building interventions on classroom teacher and early childhood educator perceived capabilities, knowledge, and attitudes relating to physical activity and fundamental movement skills: a systematic review and meta-analysis

BACKGROUND

Capacity building may play an important role in improving classroom teachers' and early childhood educators' (ECE) capacity to implement physical activity and FMS interventions. Capacity building is the development of knowledge, skills, and structures to improve the capability of individuals and organisations to achieve effective health promotion. This review aimed to determine the efficacy of capacity building interventions on teachers' and ECEs' perceived capabilities, knowledge, and attitudes relating to physical activity and fundamental movement skills.

METHODS

An exhaustive literature search of six electronic databases was conducted. Controlled, single-group pre-post studies were included if they measured the effect of a capacity building intervention on in-service or pre-service classroom teachers' (primary or secondary) or ECEs' physical activity or fundamental skills related perceived capabilities, knowledge, or attitudes. The effects of interventions were synthesised using random effects meta-analysis. Subgroup analysis and meta-regression was conducted to determine if the effects differed based on study design, type of teacher (ECE vs. primary school), or teacher level (pre-service vs. in-service).

RESULTS

A total of 22 studies reporting on 25 unique samples were included in the meta-analyses. Only studies reporting on ECEs and primary school teachers were identified. Interventions most commonly included training/professional development, resources and toolkits, communities of practice, mentorships, and ongoing support. Results showed that capacity building interventions significantly improved teachers' and ECEs' perceived capabilities (g = 0.614, 95% CI = 0.442, 0.786), knowledge (g = 0.792 95% CI = 0.459, 1.125), and attitudes (g = 0.376 95% CI = 0.181, 0.571). The effects did not differ significantly as a function of any of the moderators examined.

CONCLUSION

Findings from this review provide strong support that capacity building interventions are efficacious at improving teachers' and ECEs' perceived capabilities, knowledge, and attitudes related to promoting physical activity and teaching fundamental movement skills. Pre-service teachers and ECEs should be provided training in physical activity and fundamental movement skills as part of their degrees, and continual professional development and capacity building should be offered to in-service teachers and ECEs to promote physical activity and fundamental movement skills in children.

Research outcomes informing the selection of public health interventions and strategies to implement them: A cross-sectional survey of Australian policy-maker and practitioner preferences

BACKGROUND

A key role of public health policy-makers and practitioners is to ensure beneficial interventions are implemented effectively enough to yield improvements in public health. The use of evidence to guide public health decision-making to achieve this is recommended. However, few studies have examined the relative value, as reported by policy-makers and practitioners, of different broad research outcomes (that is, measures of cost, acceptability, and effectiveness). To guide the conduct of research and better inform public health policy and practice, this study aimed at describing the research outcomes that Australian policy-makers and practitioners consider important for their decision-making when selecting: (a) public health interventions; (b) strategies to support their implementation; and (c) to assess the differences in research outcome preferences between policy-makers and practitioners.

METHOD

An online value-weighting survey was conducted with Australian public health policy-makers and practitioners working in the field of non-communicable disease prevention. Participants were presented with a list of research outcomes and were asked to select up to five they considered most critical to their decision-making. They then allocated 100 points across these - allocating more points to outcomes perceived as more important. Outcome lists were derived from a review and consolidation of evaluation and outcome frameworks in the fields of public health knowledge translation and implementation. We used descriptive statistics to report relative preferences overall and for policy-makers and practitioners separately.

RESULTS

Of the 186 participants; 90 primarily identified as policy-makers and 96 as public health prevention practitioners. Overall, research outcomes of effectiveness, equity, feasibility, and sustainability were identified as the four most important outcomes when considering either interventions or strategies to implement them. Scores were similar for most outcomes between policy-makers and practitioners.

CONCLUSION

For Australian policy-makers and practitioners working in the field of non-communicable disease prevention, outcomes related to effectiveness, equity, feasibility, and sustainability appear particularly important to their decisions about the interventions they select and the strategies they employ to implement them. The findings suggest researchers should seek to meet these information needs and prioritize the inclusion of such outcomes in their research and dissemination activities. The extent to which these outcomes are critical to informing the decision of policy-makers and practitioners working in other jurisdictions or contexts warrants further investigation.

A systematic review to determine the effect of strategies to sustain chronic disease prevention interventions in clinical and community settings: study protocol

BACKGROUND

The primary purpose of this review is to synthesise the effect of strategies aiming to sustain the implementation of evidenced-based interventions (EBIs) targeting key health behaviours associated with chronic disease (i.e. physical inactivity, poor diet, harmful alcohol use, and tobacco smoking) in clinical and community settings. The field of implementation science is bereft of an evidence base of effective sustainment strategies, and as such, this review will provide important evidence to advance the field of sustainability research.

METHODS

This systematic review protocol is reported in accordance with the Preferred Reporting Items for Systematic review and Meta-Analysis (PRISMA) checklist. Methods will follow Cochrane gold-standard review methodology. The search will be undertaken across multiple databases, adapting filters previously developed by the research team, data screening and extraction will be performed in duplicate, strategies will be coded using an adapted sustainability-explicit taxonomy, and evidence will be synthesised using appropriate methods (i.e. meta-analytic following Cochrane or non-meta-analytic following SWiM guidelines). We will include any randomised controlled study that targets any staff or volunteers delivering interventions in clinical or community settings. Studies which report on any objective or subjective measure of the sustainment of a health prevention policy, practice, or programme within any of the eligible settings will be included. Article screening, data extraction, risk of bias, and quality assessment will be performed independently by two review authors. Risk of bias will be assessed using Version 2 of the Cochrane risk-of-bias tool for randomised trials (RoB 2). A random-effect meta-analysis will be conducted to estimate the pooled effect of sustainment strategies separately by setting (i.e. clinical and community). Sub-group analyses will be undertaken to explore possible causes of statistical heterogeneity and may include the following: time period, single or multi-strategy, type of setting, and type of intervention. Differences between sub-groups will be statistically compared.

DISCUSSION/CONCLUSION

This will be the first systematic review to determine the effect of strategies designed to support sustainment on sustaining the implementation of EBIs in clinical and community settings. The findings of this review will directly inform the design of future sustainability-focused implementation trials. Further, these findings will inform the development of a sustainability practice guide for public health practitioners.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42022352333.

More People, More Active, More Often for Heart Health - Taking Action on Physical Activity

Physical inactivity is a leading contributor to increased cardiovascular morbidity and mortality. Almost 500 million new cases of preventable noncommunicable diseases (NCDs) will occur globally between 2020 and 2030 due to physical inactivity, costing just over US$300 billion, or around US$ 27 billion annually (WHO 2022). Active adults can achieve a reduction of up to 35% in risk of death from cardiovascular disease. Physical activity also helps in moderating cardiovascular disease risk factors such as high blood pressure, unhealthy weight and type 2 diabetes. For people with cardiovascular disease, hypertension, type 2 diabetes and many cancers, physical activity is an established and evidence-based part of treatment and management. For children and young people, physical activity affords important health benefits. Physical activity can also achieve important cross-sector goals. Increased walking and cycling can reduce journeys by vehicles, air pollution, and traffic congestion and contribute to increased safety and liveability in cities.

An initial typology of approaches used by policy and practice agencies to achieve sustained implementation of interventions to improve health

BACKGROUND

Scientific investigation of how to sustain the implementation of evidence-based interventions (EBI) is emerging. Sustaining the implementation of EBIs helps ensure their effects on improving health endure. External policy or practice agencies, such as government health departments, are often tasked with supporting individual organisations with sustaining their delivery of EBIs, for example, through financing, training or the provision of other supports. However, to our knowledge, the approaches taken by policy and practice agencies to support the sustainment of EBIs have not been consolidated, categorised and described as a typology.

MAIN BODY

To improve conceptual clarity and support both research and practice, we developed an initial working typology of the practical approaches to sustain implementation of EBIs (i.e. sustainment) in order to improve long term health from the perspective of these agencies. The working typology includes three broad approaches. The first, termed 'Self-Sustainment', is when implementation of the EBI by an organisation (e.g. hospital, clinic, school) is expected to continue (sustain) in the absence of external (agency) support. The second, termed 'Static Sustainment Support', involves the provision of pre-defined external (agency) support to assist organisations to continue implementation of an EBI. The final approach is termed 'Dynamic Sustainment Support', whereby support provided by an external agency is dynamic (continues to be adapted) overtime to assist organisations continue implementation of an intervention which may itself also evolve.

CONCLUSIONS

We describe the contexts and circumstances where each may be most appropriate in achieving sustained implementation and discuss their research and practice implications.

Examining associations between school-level determinants and the implementation of physical activity opportunities

School-based physical activity (PA) opportunities can help students engage in greater amounts of daily PA, meet PA guidelines, and lead to improved health and educational outcomes. However, we do not completely understand the organizational challenges to implementing these opportunities successfully. This exploratory study examined associations between school-level determinants and the implementation of school-based PA opportunities. We analyzed cross-sectional survey data from schools (n = 46) participating in the Healthy Zone School Program (HZSP) (Dallas, Texas, USA) during 2019-2020. Respondents completed an electronic survey that included measures of school-level determinants (e.g. culture, leadership, priority) and the implementation of school-based PA opportunities. We used linear regression models to examine associations between determinants and implementation outcomes (number of PA opportunities delivered, perceived overall success of each PA program/activity used). After adjusting for campus type (i.e. elementary, middle, high, K-12), student race/ethnicity, and percentage of economically disadvantaged students, no constructs were associated with the number of PA opportunities implemented. Linear regression models suggest access to knowledge and information (β = 0.39, P = .012, 95% CI = 0.24-1.44) and implementation climate (β = 0.34, P = .045, 95% CI = 0.02-1.59) were positively associated with the success of school-based PA opportunities. Our findings provide suggestive evidence that access to knowledge and information and a supportive school climate may improve the overall success of PA opportunities provided to students. Future research should examine additional school-level determinants to understand their importance to implementation and inform the development of strategies to improve schools' capacity for implementing PA opportunities successfully.

A bibliographic review of sustainability research output and investment in 10 leading public health journals across three time periods

Objectives

Long-term delivery, or sustainability, of evidence-based interventions is necessary for public health benefits to be realised. However, sustainment of effective interventions is poor. Understanding the evidence-base and identifying potential gaps is necessary to inform where future research efforts are most warranted.

Study design

We undertook a repeat cross-sectional bibliographic review of research published in 10 public health journals across three time periods (2010, 2015 and 2020/2021).

Methods

Studies were eligible if they were a data-based study or review article. Studies were assessed as to whether they focused on sustainability. The percentage of public health research studies assessing sustainability overall and by the three time periods was calculated. The association between time period and the proportion of sustainability articles was assessed using logistic regression. Descriptive statistics were used to summarise study characteristics overall and by time period.

Results

10,588 data-based articles were identified, of which 1.3 % (n = 136) focused on sustainability. There was a statistically significant association between time period and the proportion of sustainability research, with a slight increase across the three time periods: 0.3 % (95 % CI: 0.1 %, 0.7 %) in 2010, 1.4 % (95 % CI: 1.0 %, 1.9 %) in 2015 and 1.6 % (95 % CI: 1.3 %, 1.9 %) in 2020/2021. Most research was descriptive/epidemiological (n = 69, 51 %), few focused on measurement (n = 2, 1.5 %) and none on cost effectiveness. Only one intervention study assessed the effect of specific sustainability strategies.

Conclusions

This bibliographic review highlights the need for more public health research on sustainability, particularly in the areas of measurement, sustainability interventions, and cost effectiveness.

Disseminating health research to public health policy-makers and practitioners: a survey of source, message content and delivery modality preferences

BACKGROUND

Understanding the views of policy-makers and practitioners regarding how best to communicate research evidence is important to support research use in their decision-making.

AIM

To quantify and describe public health policy-makers and practitioners' views regarding the source, content and form of messages describing public health research findings to inform their decision-making. We also sought to examine differences in preferences between public health policy-makers and practitioners.

METHODS

A cross sectional, value-weighting survey of policy-makers and practitioners was conducted. Participants were asked to allocate a proportion of 100 points across different (i) sources of research evidence, (ii) message content and (iii) the form in which evidence is presented. Points were allocated based on their rating of influence, usefulness and preference when making decisions about health policy or practice.

RESULTS

A total of 186 survey responses were received from 90 policy-makers and 96 practitioners. Researchers and government department agencies were the most influential source of research evidence based on mean allocation of points, followed by knowledge brokers, professional peers and associations. Mean point allocation for perceived usefulness of message content was highest for simple summary of key findings and implications, and then evidence-based recommendations and data and statistical summaries. Finally, based on mean scores, policy-makers and practitioners preferred to receive research evidence in the form of peer-reviewed publications, reports, evidence briefs and plain language summaries. There were few differences in scores between policy-makers and practitioners across source, message content or form assessments or those with experience in different behavioural areas.

CONCLUSIONS

The findings should provide a basis for the future development and optimization of dissemination strategies to this important stakeholder group.

Supporting adolescents' participation in muscle-strengthening physical activity: protocol for the 'Resistance Training for Teens' (RT4T) hybrid type III implementation-effectiveness trial

INTRODUCTION

In Australia, only 22% of male and 8% of female adolescents meet the muscle-strengthening physical activity guidelines, and few school-based interventions support participation in resistance training (RT). After promising findings from our effectiveness trial, we conducted a state-wide dissemination of the 'Resistance Training for Teens' (RT4T) intervention from 2015 to 2020. Despite high estimated reach, we found considerable variability in programme delivery and teachers reported numerous barriers to implementation. Supporting schools when they first adopt evidence-based programmes may strengthen programme fidelity, sustainability, and by extension, programme impact. However, the most effective implementation support model for RT4T is unclear.

OBJECTIVE

To compare the effects of three implementation support models on the reach (primary outcome), dose delivered, fidelity, sustainability, impact and cost of RT4T.

METHODS AND ANALYSIS

We will conduct a hybrid type III implementation-effectiveness trial involving grade 9 and 10 (aged 14-16 years) students from 90 secondary schools in New South Wales (NSW), Australia. Schools will be recruited across one cohort in 2023, stratified by school type, socioeconomic status and location, and randomised in a 1:1:1 ratio to receive one of the following levels of implementation support: (1) 'low' (training and resources), (2) 'moderate' (training and resources+external support) or 'high' (training and resources+external support+equipment). Training includes a teacher workshop related to RT4T programme content (theory and practical sessions) and the related resources. Additional support will be provided by trained project officers from five local health districts. Equipment will consist of a pack of semiportable RT equipment (ie, weighted bars, dumbbells, resistance bands and inverted pull up bar stands) valued at ~$A1000 per school. Study outcomes will be assessed at baseline (T0), 6 months (T1) and 18 months (T2). A range of quantitative (teacher logs, observations and teacher surveys) and qualitative (semistructured interviews with teachers) methods will be used to assess primary (reach) and secondary outcomes (dose delivered, fidelity, sustainability, impact and cost of RT4T). Quantitative analyses will use logistic mixed models for dichotomous outcomes, and ordinal or linear mixed effects regression models for continuous outcomes, with alpha levels set at p<0.025 for the outcomes and cost comparisons of the moderate and high support arms against the low support arm.

ETHICS AND DISSEMINATION

Ethics approval has been obtained from the University of Newcastle (H-2021-0418), the NSW Department of Education (SERAP:2022215), Hunter New England Human Research Ethics Committee (2023/ETH00052) and the Catholic Schools Office. The design, conduct and reporting will adhere to the Consolidated Standards of Reporting Trials statement, the Standards for Reporting Implementation Studies statement and the Template for Intervention Description and Replication checklist. Findings will be published in open access peer-reviewed journals, key stakeholders will be provided with a detailed report. We will support ongoing dissemination of RT4T in Australian schools via professional learning for teachers.

TRIAL REGISTRATION NUMBER

ACTRN12622000861752.

A cluster randomised controlled trial to assess the effectiveness of a multi-strategy sustainability intervention on teachers' sustained implementation of classroom physical activity breaks (energisers): study protocol

BACKGROUND

Governments internationally have invested hugely in the implementation and scale-up of school-based physical activity interventions, but have little evidence of how to best sustain these interventions once active implementation support ceases. This study will assess the effectiveness of a multi-strategy sustainability intervention on classroom teachers' sustainment of energisers (short 3-5 min physical activity breaks during class-time) scheduled across the school day from baseline to 12 and 24-month follow-up.

METHODS

A cluster randomised controlled trial will be conducted in 50 primary schools within the Hunter New England, Illawarra Shoalhaven, Murrumbidgee and Northern New South Wales (NSW) Local Health Districts of NSW Australia. Schools will be randomly allocated to receive either usual support or the multi-strategy sustainability intervention that includes: centralised technical assistance from a trained project officer; formal commitment and mandated change obtained from school principals; training in-school champions; reminders for teachers; educational materials provided to teachers; capturing and sharing local knowledge; and engagement of parents, carers and the wider school community. The primary trial outcome will be measured via a teacher logbook to determine the between-group difference in the change in mean minutes of energisers scheduled across the school day at 12 and 24-month follow-up compared to baseline. Analyses will be performed using an intention to treat framework. Linear mixed models will be used to assess intervention effects on the primary outcome at both follow-up periods.

DISCUSSION

This study will be one of the first randomised controlled trials to examine the impact of a multi-strategy sustainability intervention to support schools' sustainment of a physical activity intervention. The proposed research will generate new evidence needed for the partnering organisations to protect their considerable investments to date in physical activity promotion in this setting and will provide seminal evidence for the field globally.

TRIAL REGISTRATION

ACTRN12620000372987 version 1 registered 17th March 2020. Version 3 (current version) updated 4th August 2023.

Improving academic and public health impact of Cochrane public health reviews: what can we learn from bibliographic metrics and author dissemination strategies? A cross-sectional study

BACKGROUND

To facilitate the development of impactful research dissemination strategies, this study aimed to: (i) survey authors of trials included in a sample of Cochrane reviews to describe strategies to disseminate trial findings, and examine their association with academic and policy impacts and (ii) audit academic and policy impact of CPH reviews.

METHODS

Authors of 104 trials within identified Cochrane reviews completed survey items assessing the dissemination strategies. Field weighted citation (FWCI) data extracted from bibliographic databases served as a measure of academic impact of trials and CPH reviews. Policy and practice impacts of trials were assessed during the survey of trial authors using items based on the Payback Framework, and for CPH reviews using 'policy mention' data collected via Altmetric Explorer.

RESULTS

Among the included trials, univariate (but not multivariable) regression models revealed significant associations between the use of dissemination strategies (i.e. posts on social media; workshops with end-users; media-releases) and policy or practice impacts. No significant associations were reported between dissemination strategies and trial FWCI. The mean FWCI of CPH reviews suggest that they are cited 220% more than other reviews in their field.

CONCLUSIONS

Comprehensive dissemination strategies are likely required to maximize the potential the potential impacts of public health research.

A systematic review to determine the effect of strategies to sustain chronic disease prevention interventions in clinical and community settings: study protocol

Background

The primary purpose of this review is to synthesise the effect of strategies aiming to sustain the implementation of evidenced based interventions (EBIs) targeting key health behaviours associated with chronic disease (i.e., physical inactivity, poor diet, harmful alcohol use and tobacco smoking) in clinical and community settings. The field of implementation science is bereft of an evidence base of effective sustainment strategies, and as such this review will provide important evidence to advance the field of sustainability research.

Methods

This systematic review protocol is reported in accordance with the Preferred Reporting Items for Systematic review and Meta-Analysis Protocol (PRISMA-P) checklist (Additional file 1). Methods will follow Cochrane gold-standard review methodology. The search will be undertaken across multiple databases, adapting filters previously developed by the research team; data screening and extraction will be performed in duplicate; strategies will be coded using an adapted sustainability-explicit taxonomy; evidence will be synthesised using appropriate methods (i.e. meta-analytic following Cochrane or non-meta-analytic following SWiM guidelines). We will include any randomised controlled study that targets any staff or volunteers delivering interventions in clinical or community settings. Studies which report on any objective or subjective measure of the sustainment of a health prevention policy, practice, or program within any of the eligible settings will be included. Article screening, data extraction, risk of bias and quality assessment will be performed independently by two review authors. Risk of bias will be assessed using Version 2 of the Cochrane risk-of-bias tool for randomised trials (RoB 2). A random effect meta-analysis will be conducted to estimate the pooled effect of sustainment strategies separately by setting (i.e. clinical and community). Sub-group analyses will be undertaken to explore possible causes of statistical heterogeneity and may include: time period, single or multi strategy, type of setting and type of intervention. Differences between sub-groups will be statistically compared.

Discussion/Conclusion

This will be the first systematic review to determine the effect of strategies designed to support sustainment on sustaining the implementation of EBIs in clinical and community settings. The findings of this review will directly inform the design of future sustainability-focused implementation trials. Further, these findings will inform the development of a sustainability practice guide for public health practitioners.

Registration

This review was prospectively registered with PROSPERO (registration ID: CRD42022352333).

The Effectiveness of a Physical Literacy-Based Intervention for Increasing Physical Activity Levels and Improving Health Indicators in Overweight and Obese Adolescents (CAPACITES 64)

Recently, the concept of Physical Literacy (PL) has emerged as a key concept for promoting active behavior and improving health indicators in adolescents. Overweight and obese adolescents have a low level of Physical Activity (PA), low cardiorespiratory capacity, and high Body Fat percentage (%BF). However, the development of PL in the interest of health improvement has never been studied in overweight and obese adolescents. The objective of this study was to evaluate the impact of an intervention developing PL in overweight and obese adolescents in order to increase their (PA) and improve their health. The study was a prospective, single-arm, non-randomized interventional study. The intervention brings together different actions in PA and dietary education in different adolescent living environments. The study took place over a 9-month period with two data collection times (0; +9 months) and measured Body Mass Index (BMI) and BMI z score, %BF and Skeletal Muscle Mass (%SMM), Moderate-to-Vigorous intensity Physical Activity (MVPA) by accelerometry, CRF, as well as PL by the CAPL-2 tool. Thirteen adolescents (age 11.7 (±1.09) years old) improved their PL scores (+8.3 (±9.3) pts; p ≤ 0.01). BMI z score (-0.3 (±0.3), p ≤ 0.01), their %BF (-3.8 (±4.9); p ≤ 0.01), their CRF (+1.5 (±1.7) mL·min·kg^-1; p ≤ 0.01), and their MVPA (+4.6 (±13.7) min/day; p = 0.36). Initiating multidimensional interventions to develop PL in overweight and obese adolescents may be a promising prospect to enable an increase in their MVPA and improve their long-term health. Longer-term randomized controlled interventional studies are needed to confirm these findings.

Evaluation of the peer leadership for physical literacy intervention: A cluster randomized controlled trial

PURPOSE

The purpose of this research was to develop, implement, and test the efficacy of a theory-driven, evidence-informed peer leadership program for elementary school students (Grade 6 and 7; age 11-12 years) and the Grade 3/4 students with whom they were partnered. The primary outcome was teacher ratings of their Grade 6/7 students' transformational leadership behaviors. Secondary outcomes included: Grade 6/7 students' leadership self-efficacy, as well as Grade 3/4 motivation, perceived competence, general self-concept, fundamental movement skills, school-day physical activity, and program adherence, and program evaluation.

METHODS

We conducted a two-arm cluster randomized controlled trial. In 2019, 6 schools comprising 7 teachers, 132 leaders, and 227 grade 3 and 4 students were randomly allocated to the intervention or waitlist control conditions. Intervention teachers took part in a half-day workshop (January 2019), delivered 7 x 40 minute lessons to Grade 6/7 peer leaders (February and March 2019), and these peer leaders subsequently ran a ten-week physical literacy development program for Grade 3/4 students (2x30 minutes sessions per week). Waitlist-control students followed their usual routines. Assessments were conducted at baseline (January 2019) and immediately post-intervention (June 2019).

RESULTS

The intervention had no significant effect on teacher ratings of their students' transformational leadership (b = 0.201, p = .272) after controlling for baseline and gender. There was no significant condition effect for Grade 6/7 student rated transformation leadership (b = 0.077, p = .569) or leadership self-efficacy (b = 3.747, p = .186) while controlling for baseline and gender. There were null findings for all outcomes related to Grade 3 and 4 students.

DISCUSSION

Adaptions to the delivery mechanism were not effective in increasing leadership skills of older students or components of physical literacy in younger Grade 3/4 students. However, teacher self-reported adherence to the intervention delivery was high.

TRIAL REGISTRATION

This trial was registered on December 19th, 2018 with Clinicaltrials.gov (NCT03783767), https://clinicaltrials.gov/ct2/show/NCT03783767.

Current Practices, Willingness and Perceived Ability to Implement an Injury Prevention Exercise Program Among Post-Primary Physical Education Teachers

BACKGROUND

Musculoskeletal injuries are common in adolescents, and recently schools have been suggested as an opportune location for injury prevention strategies. This study aimed to identify the current practices and perceptions of post-primary PE teachers in Ireland on injury prevention exercise programs (IPEP), which are key to informing potential implementation strategies.

METHODS

Post-primary PE teachers (n = 287) completed an online anonymous survey. Outcome measures included current IPEP practices in PE class, teachers' attitudes toward IPEPs, willingness to implement, and perceived ability to implement an IPEP in PE class. Descriptive statistics were calculated, and Mann-Whitney U tests were used to compare differences between groups.

RESULTS

Results indicated that only 1 in 5 PE teachers currently used an IPEP in class. Of these, no teacher used an existing IPEP exactly as intended, while most teachers were willing to implement an IPEP (80.5%). Those who previously received formal IPEP education or were aware of an existing IPEP had significantly higher perceived ability to implement an IPEP in class (p < .001).

CONCLUSIONS

This study demonstrates that despite a willingness among PE teachers to implement IPEPs in class, few currently do. Thus, post-primary PE class may be an under-utilized setting for adolescent injury prevention and warrants further investigation.

School-level factors associated with the sustainment of weekly physical activity scheduled in Australian elementary schools: an observational study

Background

We aimed to: (1) identify school-level factors associated with the sustainment of weekly physical activity (PA) scheduled in elementary schools following withdrawal of effective implementation support; and (2) determine teacher’s perceived usefulness of suggested strategies for sustaining the scheduling of weekly PA.

Methods

A secondary exploratory analysis was employed of data from the intervention arm (n = 31 schools) of a randomised controlled trial. Self-report survey data from 134 classroom teachers in New South Wales, Australia, collected following withdrawal of initial implementation support (follow-up T1) and six-months following completion of support (follow-up T2) were used. The outcomes of sustainment of weekly overall PA and energisers (short classroom PA breaks) scheduled were measured via teachers’ completion of a daily activity logbook, with results presented as the difference in mean minutes of PA and energisers scheduled at T1 and T2. An adapted version of the Program Sustainability Assessment Tool (PSAT) was used to measure capacity for program sustainability across seven key domains at follow-up T2. Linear mixed regressions were conducted to evaluate associations between school-level sociodemographic characteristics (e.g., school size, remoteness, and type), teacher-reported school factors (i.e., seven adapted PSAT domains) and the sustainment of PA and energisers scheduled across the school week. Perceived usefulness of 14 proposed sustainability strategies was measured via the teacher survey at follow-up T2 and reported descriptively.

Results

No school-level factor was statistically associated with the sustainment of overall weekly PA or energisers scheduled. Teacher-reported factors in two PSAT domains – ‘strategic planning’ and ‘program evaluation’ were statistically negatively associated with the sustainment of weekly energisers scheduled (− 6.74, 95% CI: − 13.02; − 0.47, p = 0.036 and − 6.65, 95% CI: − 12.17; − 1.12, p = 0.019 respectively). The proposed support sustainability strategy – ‘provision of PA equipment packs that enable energisers or integrated lessons’ was perceived useful by the most teachers (85%).

Conclusions

Further research is required to explore additional contextual-specific, and end-user appropriate factors associated with schools’ sustainment of weekly PA scheduled. This will help accurately inform the development of strategies to address these determinants and support the sustainment and long-term benefits of school-based health interventions more broadly.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12889-022-13732-6.

Do the Expert Recommendations for Implementing Change (ERIC) strategies adequately address sustainment?

Background

Sustainability science is an emerging area within implementation science. There is limited evidence regarding strategies to best support the continued delivery and sustained impact of evidence-based interventions (EBIs). To build such evidence, clear definitions, and ways to operationalize strategies specific and/or relevant to sustainment are required. Taxonomies and compilations such as the Expert Recommendations for Implementing Change (ERIC) were developed to describe and organize implementation strategies. This study aimed to adapt, refine, and extend the ERIC compilation to incorporate an explicit focus on sustainment. We also sought to classify the specific phase(s) of implementation when the ERIC strategies could be considered and applied.

Methods

We used a two-phase iterative approach to adapt the ERIC. This involved: (1) adapting through consensus (ERIC strategies were mapped against barriers to sustainment as identified via the literature to identify if existing implementation strategies were sufficient to address sustainment, needed wording changes, or if new strategies were required) and; (2) preliminary application of this sustainment-explicit ERIC glossary (strategies described in published sustainment interventions were coded against the glossary to identify if any further amendments were needed). All team members independently reviewed changes and provided feedback for subsequent iterations until consensus was reached. Following this, and utilizing the same consensus process, the Exploration, Preparation, Implementation and Sustainment (EPIS) Framework was applied to identify when each strategy may be best employed across phases.

Results

Surface level changes were made to the definitions of 41 of the 73 ERIC strategies to explicitly address sustainment. Four additional strategies received deeper changes in their definitions. One new strategy was identified: Communicate with stakeholders the continued impact of the evidence-based practice. Application of the EPIS identified that at least three-quarters of strategies should be considered during preparation and implementation phases as they are likely to impact sustainment.

Conclusion

A sustainment-explicit ERIC glossary is provided to help researchers and practitioners develop, test, or apply strategies to improve the sustainment of EBIs in real-world settings. Whilst most ERIC strategies only needed minor changes, their impact on sustainment needs to be tested empirically which may require significant refinement or additions in the future.

Cluster randomised controlled trial to determine the impact of an activity enabling uniform on primary school student's fitness and physical activity: study protocol for the Active WeAR Everyday (AWARE) study

IntroductionMulticomponent school-based physical activity (PA) interventions can improve students' cardiorespiratory fitness (CRF) and PA. Due to the complex nature of such interventions when delivered at scale their effect sizes markedly reduce. Modifying student school uniforms, so that they are more PA enabling, may be a simple intervention that could enhance student health. The primary aim of this trial is to assess the effectiveness of an activity enabling uniform intervention (shorts, polo shirt and sports shoes) in improving children's CRF.

METHODS AND ANALYSIS

A cluster randomised controlled trial will be conducted in 24 primary schools in New South Wales (NSW), Australia. Schools will be randomly allocated to either intervention or usual practice following baseline data collection. Active WeAR Everyday intervention schools will allow students in grades 4-6 (aged approx. 9-12 years) to wear their existing sports uniform (shorts, polo shirt and sports shoes) every day. To avoid any financial cost to students they will be provided with two additional sports shirts and one pair of shorts. Study outcomes will be assessed at baseline and 9 months postbaseline. The primary outcome is students' CRF measured using the 20 m multistage fitness test. Secondary outcomes include students': mean daily steps and steps/minute measured via accelerometer, quality of life, mental well-being and perceived PA self-efficacy. The acceptability, feasibility and cost of the intervention will be assessed. Analyses will be performed using an intention-to-treat framework. Linear mixed effects regression models will be used to assess intervention effects on the primary outcome at follow-up. Planned exploratory analyses will examine effects by subgroups (eg, gender).

ETHICS AND DISSEMINATION

This study has received approval from Hunter New England Local Health District Human Ethics Committee (2020/ETHO2602) the University of Newcastle, Human Research Ethics Committee (H-2021-0013), NSW Department of Education (SERAP: 2020387) and Catholic School Offices.

TRIAL REGISTRATION NUMBER

ACTRN12621000201875.

Optimising a multi-strategy implementation intervention to improve the delivery of a school physical activity policy at scale: findings from a randomised noninferiority trial

Background

To maximise their potential health benefits, school-based physical activity policies need to be implemented at scale. This paper describes the third in a sequence of trials that sought to optimise an effective strategy (PACE) to assist schools’ implementation of a physical activity policy. Specifically, it aimed to determine the probability that a multi-strategy intervention adapted to reduce in-person contact (Adapted PACE) was “as good as” the original intervention (PACE) in increasing the weekly minutes of structured physical activity implemented by classroom teachers.

Methods

A noninferiority cluster randomised controlled trial was undertaken with 48 primary schools in New South Wales, Australia. Schools were randomised to receive PACE or a model with adaptations made to the delivery modes (Adapted PACE). Teachers’ scheduled minutes of weekly physical activity was assessed at baseline (Oct 2018-Feb 2019) and 12-month follow-up (Oct-Dec 2019). The noninferiority margin was set at − 16.4 minutes based on previous data and decision panel consensus. A linear mixed model analysed within a Bayesian framework was used to explore noninferiority between the two PACE models. A cost minimisation analysis was conducted from the health service provider perspective, using the Australian dollar (AUD).

Results

The posterior estimate for the between group difference at follow-up was − 2.3 minutes (95% credible interval = − 18.02, 14.45 minutes). There was an estimated 96% probability of Adapted PACE being considered noninferior (only 4% of the posterior samples crossed the noninferiority margin of − 16.4 minutes). That is, the minutes of physical activity implemented by teachers at Adapted PACE schools was not meaningfully less than the minutes of physical activity implemented by teachers at PACE schools. The mean total cost was AUD$25,375 (95% uncertainty interval = $21,499, $29,106) for PACE and AUD$16,421 (95% uncertainty interval = $13,974, $19,656) for Adapted PACE; an estimated reduction of AUD$373 (95% uncertainty interval = $173, $560) per school.

Conclusions

It is highly probable that Adapted PACE is noninferior to the original model. It is a cost-efficient alternative also likely to be a more suitable approach to supporting large scale implementation of school physical activity policies.

Trial registration

Retrospectively registered with the Australian New Zealand Clinical Trials Registry (ACTRN12619001229167).

Supplementary Information

The online version contains supplementary material available at 10.1186/s12966-022-01345-6.

Developing an implementation research logic model: using a multiple case study design to establish a worked exemplar

Background

Implementation science frameworks explore, interpret, and evaluate different components of the implementation process. By using a program logic approach, implementation frameworks with different purposes can be combined to detail complex interactions. The Implementation Research Logic Model (IRLM) facilitates the development of causal pathways and mechanisms that enable implementation. Critical elements of the IRLM vary across different study designs, and its applicability to synthesizing findings across settings is also under-explored. The dual purpose of this study is to develop an IRLM from an implementation research study that used case study methodology and to demonstrate the utility of the IRLM to synthesize findings across case sites.

Method

The method used in the exemplar project and the alignment of the IRLM to case study methodology are described. Cases were purposely selected using replication logic and represent organizations that have embedded exercise in routine care for people with cancer or mental illness. Four data sources were selected: semi-structured interviews with purposely selected staff, organizational document review, observations, and a survey using the Program Sustainability Assessment Tool (PSAT). Framework analysis was used, and an IRLM was produced at each case site. Similar elements within the individual IRLM were identified, extracted, and re-produced to synthesize findings across sites and represent the generalized, cross-case findings.

Results

The IRLM was embedded within multiple stages of the study, including data collection, analysis, and reporting transparency. Between 33-44 determinants and 36-44 implementation strategies were identified at sites that informed individual IRLMs. An example of generalized findings describing “intervention adaptability” demonstrated similarities in determinant detail and mechanisms of implementation strategies across sites. However, different strategies were applied to address similar determinants. Dependent and bi-directional relationships operated along the causal pathway that influenced implementation outcomes.

Conclusions

Case study methods help address implementation research priorities, including developing causal pathways and mechanisms. Embedding the IRLM within the case study approach provided structure and added to the transparency and replicability of the study. Identifying the similar elements across sites helped synthesize findings and give a general explanation of the implementation process. Detailing the methods provides an example for replication that can build generalizable knowledge in implementation research.

Supplementary Information

The online version contains supplementary material available at 10.1186/s43058-022-00337-8.

Economic evaluation of a multi-strategy intervention that improves school-based physical activity policy implementation

BACKGROUND

Internationally, government policies mandating schools to provide students with opportunities to participate in physical activity are poorly implemented. The multi-component Physically Active Children in Education (PACE) intervention effectively assists schools to implement one such policy. We evaluated the value of investment by health service providers tasked with intervention delivery, and explored where adaptations might be targeted to reduce program costs for scale-up.

METHODS

A prospective trial-based economic evaluation of an implementation intervention in 61 primary schools in New South Wales (NSW), Australia. Schools were randomised to the PACE intervention or a wait-list control. PACE strategies included centralised technical assistance, ongoing consultation, principal's mandated change, identifying and preparing in-school champions, educational outreach visits, and provision of educational materials and equipment. Effectiveness was measured as the mean weekly minutes of physical activity implemented by classroom teachers, recorded in a daily log book at baseline and 12-month follow-up. Delivery costs (reported in $AUD, 2018) were evaluated from a public finance perspective. Cost data were used to calculate: total intervention cost, cost per strategy and incremental cost (overall across all schools and as an average per school). Incremental cost-effectiveness ratios (ICERs) were calculated as the incremental cost of delivering PACE divided by the estimated intervention effect.

RESULTS

PACE cost the health service provider a total of $35,692 (95% uncertainty interval [UI] $32,411, $38,331) to deliver; an average cost per school of $1151 (95%UI $1046, $1236). Training in-school champions was the largest contributor: $19,437 total; $627 ($0 to $648) average per school. Educational outreach was the second largest contributor: $4992 total; $161 ($0 to $528) average per school. The ICER was $29 (95%UI $17, $64) for every additional minute of weekly physical activity implemented per school.

CONCLUSION

PACE is a potentially cost-effective intervention for increasing schools implementation of a policy mandate. The investment required by the health service provider makes use of existing funding and infrastructure; the additional cost to assist schools to implement the policy is likely not that much. PACE strategies may be adapted to substantially improve delivery costs.

TRIAL REGISTRATION

Australia New Zealand Clinical Trials Registry ACTRN12617001265369; Prospectively registered 1st September 2017 https://www.anzctr.org.au/Trial/Registration/TrialReview.aspx?id=373520.

Identifying essential implementation strategies: a mixed methods process evaluation of a multi-strategy policy implementation intervention for schools

BACKGROUND

Physically Active Children in Education (PACE) is composed of eight implementation strategies that improves schools' implementation of a government physical activity policy. A greater understanding of each discrete implementation strategy could inform improvements to PACE for delivery at-scale. This study aimed to: (A) measure the dose delivered, fidelity, adoption and acceptability of each strategy using quantitative data; (B) identify implementation barriers and facilitators using qualitative data; and (C) explore the importance of each strategy by integrating both data sets (mixed methods).

METHODS

This study used data from a cluster randomised noninferiority trial comparing PACE with an adapted version (Adapted PACE) that was delivered with reduced in-person external support to reduce costs and increase scalability. Data were collected from both trials arms for between-group comparison. Descriptive statistics were produced using surveys of principals, in-school champions and teachers; and project records maintained by PACE project officers (objective A). Thematic analysis was performed using in-school champion and project officer interviews (objective B). Both data sets were integrated via a triangulation protocol and findings synthesized in the form of meta-inferences (objective C).

RESULTS

Eleven in-school champions and six project officers completed interviews; 33 principals, 51 in-school champions and 260 teachers completed surveys. Regardless of group allocation, implementation indicators were high for at least one component of each strategy: dose delivered =100%, fidelity ≥95%, adoption ≥83%, acceptability ≥50%; and several implementation barriers and facilitators were identified within three broad categories: external policy landscape, inner organizational structure/context of schools, and intervention characteristics and processes. All strategies were considered important as use varied by school, however support from a school executive and in-school champions' interest were suggested as especially important for optimal implementation.

CONCLUSION

This study highlights the importance of both executive support and in-school champions for successful implementation of school physical activity policies. In particular, identifying and supporting an in-school champion to have high power and high interest is recommended for future implementation strategies. This may reduce the need for intensive external support, thus improving intervention scalability.

Adaptation and Validation of the Program Sustainability Assessment Tool (PSAT) for Use in the Elementary School Setting

There is a lack of valid and reliable measures of determinants of sustainability specific to public health interventions in the elementary school setting. This study aimed to adapt and evaluate the Program Sustainability Assessment Tool (PSAT) for use in this setting. An expert reference group adapted the PSAT to ensure face validity. Elementary school teachers participating in a multi-component implementation intervention to increase their scheduling of physical activity completed the adapted PSAT. Structural validity was assessed via confirmatory factor analysis. Convergent validity was assessed using linear mixed regression evaluating the associations between scheduling of physical activity and adapted PSAT scores. Cronbach's alpha was used to evaluate internal consistency and intracluster correlation coefficients for interrater reliability. Floor and ceiling effects were also evaluated. Following adaptation and psychometric evaluation, the final measure contained 26 items. Domain Cronbach's alpha ranged from 0.77 to 0.92. Only one domain illustrated acceptable interrater reliability. Evidence for structural validity was mixed and was lacking for convergent validity. There were no floor and ceiling effects. Efforts to adapt and validate the PSAT for the elementary school setting were mixed. Future work to develop and improve measures specific to public health program sustainment that are relevant and psychometrically robust for elementary school settings are needed.

Scale-up of the Physical Activity 4 Everyone (PA4E1) intervention in secondary schools: 24-month implementation and cost outcomes from a cluster randomised controlled trial

Background

Physical Activity 4 Everyone (PA4E1) is an evidence-based program effective at increasing adolescent physical activity (PA) and improving weight status. This study aimed to determine a) the effectiveness of an adapted implementation intervention to scale-up PA4E1 at 24-month follow-up, b) fidelity and reach, and c) the cost and cost-effectiveness of the implementation support intervention.

Methods

A cluster randomised controlled trial using a type III hybrid implementation-effectiveness design in 49 lower socio-economic secondary schools, randomised to a program (n = 24) or control group (n = 25). An adapted implementation intervention consisting of seven strategies was developed to support schools to implement PA4E1 over 24-months. The primary outcome was the proportion of schools implementing at least four of the 7 PA practices, assessed via computer assisted telephone interviews (CATI) with Head Physical Education Teachers. Secondary outcomes included the mean number of PA practices implemented, fidelity and reach, cost and cost-effectiveness. Logistic regression models assessed program effects.

Results

At baseline, no schools implemented four of the 7 PA practices. At 24-months, significantly more schools in the program group (16/23, 69.6%) implemented at least four of the 7 PA practices than the control group (0/25, 0%) (p < 0.001). At 24-months, program schools were implementing an average of 3.6 more practices than control schools (4.1 (1.7) vs. 0.5 (0.8), respectively) (P < 0.001). Fidelity and reach of the implementation intervention were high (> 75%). The total cost of the program was $415,112 AUD (2018) ($17,296 per school; $117.30 per student).

Conclusions

The adapted implementation intervention provides policy makers and researchers with an effective and potentially cost-effective model for scaling-up the delivery of PA4E1 in secondary schools. Further assessment of sustainability is warranted.

Trial registration

Australian New Zealand Clinical Trials Registry ACTRN12617000681358 prospectively registered 12th May 2017.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12966-021-01206-8.