Optimising a multi-strategy implementation intervention to improve the delivery of a school physical activity policy at scale: findings from a randomised noninferiority trial

A cross-sectional study assessing modifications to the delivery of a multi-component implementation strategy (the Get Outside, Get Active program) to improve child physical activity in early childhood education and care services

ISSUE ADDRESSED

The Get Outside, Get Active (GOGA) program is a randomised controlled trial which tested the impact of a multi-component implementation strategy to support early childhood education and care (ECEC) services to replace indoor-only free play with indoor-outdoor-free play. This cross-sectional study aims to describe the extent and nature of modifications made to implementation strategies and Behaviour Change Techniques (BCTs) using the Framework for Reporting Adaptations and Modifications to Evidence-based Implementation Strategies (FRAME-IS) and to describe the fidelity of BCT delivery throughout GOGA.

METHODS

An audit of records was undertaken throughout the intervention delivery period in the intervention arm. GOGA included 14 standard BCTs within six implementation strategies. Modifications and BCT delivery were recorded by Health Promotion Officers via project records. Modifications were categorised according to the FRAME-IS. BCT delivery was recorded using a checklist.

RESULTS

Forty-four ECEC services received the GOGA program. Overall, 60 modifications were recorded. According to FRAME-IS categories, most modifications related to: content; format; pragmatic or practical considerations; tailoring/tweaking/refining in nature; fidelity was inconsistent; the goal was to increase the acceptability, appropriateness, or feasibility of the implementation effort; the rationale was at the practitioner level; and were unplanned/reactive. Overall, 96.4% of standard BCTs were delivered as intended.

CONCLUSIONS

GOGA was delivered with high fidelity to protocol as indicated by the level of BCT delivery. This article details a thorough approach to documenting modifications and provides guidance for future studies. SO WHAT?: This article contributes to the emerging evidence regarding documentation of adaptations and modifications to public health implementation interventions.

Learning Health System to rapidly improve the implementation of a school physical activity policy

BACKGROUND

Learning Health Systems (LHS) - characterised by cycles of evidence generation and application - are increasingly recognised for their potential to improve public health interventions and optimise health impacts; however there is little evidence of their application in the context of public health practice. Here, we describe how an Australian public health unit applied a LHS approach to successfully improve a model of support for implementation of a school-based physical activity policy.

METHODS

This body of work was undertaken in the context of a strong research-practice partnership. Core LHS capabilities included: i) partnerships and stakeholder engagement; ii) workforce development and learning health communities; iii) multi-disciplinary scientific expertise; iv) practice data collection and management system; v) evidence surveillance and synthesis; and vi) governance and organisational processes of decision making. Three cycles of data generation and application were used. Within each cycle, randomised controlled trials conducted in NSW primary schools were used to generate data on the support model's effectiveness for improving schools' implementation of a government physical activity policy, its delivery costs, and process measures such as adoption and acceptability. Each type of data were analysed independently, synthesised, and then presented to a multi-disciplinary team of researchers and practitioners, in consult with stakeholders, leading to collaborative decisions for incremental improvements to the support model.

RESULTS

Cycle 1 tested the first version of the support model (composed of five implementation strategies targeting identified barriers of policy implementation) and showed the model's feasibility and efficacy for improving schools' policy implementation. Data-informed changes were made to enhance impact, including the addition of three implementation strategies to address outstanding barriers. Cycle 2 (now, testing a package of eight implementation strategies) established the model's effectiveness and cost-effectiveness for improving school's policy implementation. Data-informed changes were made to reduce delivery costs, specifically adapting the costliest strategies to reduce in-person contact from external support personnel. Cycle 3 showed that the adaptations minimised the relative cost of delivery without adversely impacting on the effect.

CONCLUSIONS

Through this process, we identified an effective, cost-effective, acceptable and scalable policy implementation support model for service delivery. This provides important information to inform or support LHS approaches for other agencies seeking to optimise the health impact of evidence-based interventions.

An initial typology of approaches used by policy and practice agencies to achieve sustained implementation of interventions to improve health

BACKGROUND

Scientific investigation of how to sustain the implementation of evidence-based interventions (EBI) is emerging. Sustaining the implementation of EBIs helps ensure their effects on improving health endure. External policy or practice agencies, such as government health departments, are often tasked with supporting individual organisations with sustaining their delivery of EBIs, for example, through financing, training or the provision of other supports. However, to our knowledge, the approaches taken by policy and practice agencies to support the sustainment of EBIs have not been consolidated, categorised and described as a typology.

MAIN BODY

To improve conceptual clarity and support both research and practice, we developed an initial working typology of the practical approaches to sustain implementation of EBIs (i.e. sustainment) in order to improve long term health from the perspective of these agencies. The working typology includes three broad approaches. The first, termed 'Self-Sustainment', is when implementation of the EBI by an organisation (e.g. hospital, clinic, school) is expected to continue (sustain) in the absence of external (agency) support. The second, termed 'Static Sustainment Support', involves the provision of pre-defined external (agency) support to assist organisations to continue implementation of an EBI. The final approach is termed 'Dynamic Sustainment Support', whereby support provided by an external agency is dynamic (continues to be adapted) overtime to assist organisations continue implementation of an intervention which may itself also evolve.

CONCLUSIONS

We describe the contexts and circumstances where each may be most appropriate in achieving sustained implementation and discuss their research and practice implications.

Adaptation for sustainability in an implementation trial of team-based collaborative care

Background

Sustaining healthcare interventions once they have been implemented is a pivotal public health endeavor. Achieving sustainability requires context-sensitive adaptations to evidence-based practices (EBPs) or the implementation strategies used to ensure their adoption. For replicability of adaptations beyond the specific setting in question, the underlying logic needs to be clearly described, and adaptations themselves need to be plainly documented. The goal of this project was to describe the process by which implementation facilitation was adapted to improve the uptake of clinical care practices that are consistent with the collaborative chronic care model (CCM).

Method

Quantitative and qualitative data from a prior implementation trial found that CCM-consistent care practices were not fully sustained within outpatient general mental health teams that had received 1 year of implementation facilitation to support uptake. We undertook a multistep consensus process to identify adaptations to implementation facilitation based on these results, with the goal of enhancing the sustainability of CCM-based care in a subsequent trial. The logic for these adaptations, and the resulting adaptations themselves, were documented using two adaptation-oriented implementation frameworks (the iterative decision-making for evaluation of adaptations [IDEA] and the framework for reporting adaptations and modifications to evidence-based implementation strategies [FRAME-IS], respectively).

Results

Three adaptations emerged from this process and were documented using the FRAME-IS: (a) increasing the scope of implementation facilitation within the medical center, (b) having the internal facilitator take a greater role in the implementation process, and (c) shortening the implementation timeframe from 12 to 8 months, while increasing the intensity of facilitation support during that time.

Conclusions

EBP sustainability may require careful adaptation of EBPs or the implementation strategies used to get them into routine practice. Recently developed frameworks such as the IDEA and FRAME-IS may be used to guide decision-making and document resulting adaptations themselves. An ongoing funded study is investigating the utility of the resulting adaptations for improving healthcare.

A cluster randomised controlled trial to assess the effectiveness of a multi-strategy sustainability intervention on teachers' sustained implementation of classroom physical activity breaks (energisers): study protocol

BACKGROUND

Governments internationally have invested hugely in the implementation and scale-up of school-based physical activity interventions, but have little evidence of how to best sustain these interventions once active implementation support ceases. This study will assess the effectiveness of a multi-strategy sustainability intervention on classroom teachers' sustainment of energisers (short 3-5 min physical activity breaks during class-time) scheduled across the school day from baseline to 12 and 24-month follow-up.

METHODS

A cluster randomised controlled trial will be conducted in 50 primary schools within the Hunter New England, Illawarra Shoalhaven, Murrumbidgee and Northern New South Wales (NSW) Local Health Districts of NSW Australia. Schools will be randomly allocated to receive either usual support or the multi-strategy sustainability intervention that includes: centralised technical assistance from a trained project officer; formal commitment and mandated change obtained from school principals; training in-school champions; reminders for teachers; educational materials provided to teachers; capturing and sharing local knowledge; and engagement of parents, carers and the wider school community. The primary trial outcome will be measured via a teacher logbook to determine the between-group difference in the change in mean minutes of energisers scheduled across the school day at 12 and 24-month follow-up compared to baseline. Analyses will be performed using an intention to treat framework. Linear mixed models will be used to assess intervention effects on the primary outcome at both follow-up periods.

DISCUSSION

This study will be one of the first randomised controlled trials to examine the impact of a multi-strategy sustainability intervention to support schools' sustainment of a physical activity intervention. The proposed research will generate new evidence needed for the partnering organisations to protect their considerable investments to date in physical activity promotion in this setting and will provide seminal evidence for the field globally.

TRIAL REGISTRATION

ACTRN12620000372987 version 1 registered 17th March 2020. Version 3 (current version) updated 4th August 2023.

Evaluating the scaling up of an effective implementation intervention (PACE) to increase the delivery of a mandatory physical activity policy in primary schools

BACKGROUND

Physically Active Children in Education (PACE) is an effective implementation intervention for increasing the number of minutes classroom teachers schedule physical activity each week. To date, evaluations of PACE have included a smaller number of schools from only one region in New South Wales Australia. If PACE is to have population-wide benefits we must be able to deliver this support to a larger number of schools across multiple regions. This study aimed to evaluate the scale-up of PACE.

METHODS

An uncontrolled before and after study, with 100 schools from three regions was conducted. Participating schools received PACE for approximately 12 months. We assessed the following outcomes: delivery of the evidence-based intervention (EBI) (i.e. minutes of physical activity scheduled by classroom teachers per week); delivery of the implementation strategies (i.e. reach, dose delivered, adherence and indicators of sustainability); and key determinants of implementation (i.e. acceptability of strategies and cost). Data were collected via project officer records, and principal and teacher surveys. Linear mixed models were used to assess EBI delivery by evaluating the difference in the mean minutes teachers scheduled physical activity per week from baseline to follow-up. Descriptive data were used to assess delivery of the implementation strategies and their perceived acceptability (i.e. PACE). A prospective, trial-based economic evaluation was used to assess cost.

RESULTS

Delivery of the EBI was successful: teachers increas their average minutes of total physical activity scheduled across the school week by 26.8 min (95% CI: 21.2, 32.4, p < 0.001) after receiving PACE. Indicators for delivery of implementation strategies were high: 90% of consenting schools received all strategies and components (reach); 100% of strategies were delivered by the provider (dose); >50% of schools adhered to the majority of strategies (11 of the 14 components); and acceptability was > 50% agreement for all strategies. The incremental cost per additional minute of physical activity scheduled per week was $27 per school (Uncertainty Interval $24, $31).

CONCLUSIONS

PACE can be successfully delivered across multiple regions and to a large number of schools. Given the ongoing and scalable benefits of PACE, it is important that we continue to extend and improve this program while considering ways to reduce the associated cost.

Adaptations to scale-up an early childhood education and care physical activity intervention for real-world availability - Play Active

BACKGROUND

Adaptations for scale-up are ubiquitous but are rarely described in detail. Adaptations may be a key reason for the "scale-up penalty" which is when there is a reduction in intervention effect size following scale-up. The Play Active intervention consists of a physical activity policy for early childhood education and care (ECEC) services, with accompanying implementation support strategies. It was first implemented with 81 ECEC services in Perth, Western Australia, in 2021 - with significant positive changes in physical activity practice uptake. The aim of this paper is to describe the extent, type, fidelity consistency, goals, size, scope, and proposed impact of proposed adaptations to the implementation support strategies for scaling-up Play Active.

METHODS

Proposed adaptations were defined as planned changes, made prior to making the intervention available. The authors created a list of adaptations from a comparison of the Play Active implementation support strategies, before and after adaptation for proposed statewide availability across Western Australia, Queensland and South Australia, Australia. We used the Framework for Reporting Adaptations and Modifications-Enhanced Implementation Strategies (FRAME-IS) to code adaptations to implementation support strategies. Three authors coded each adaptation and rated their size, scope and proposed impact.

RESULTS

Fifty-three adaptations to Play Active were identified. Most (68%) were proposed for the 'content' of implementation strategies, including aspects of their delivery. In practice, this involved changing the delivery mode of implementation support strategies from phone call and email support, to website-based delivery. More than half (56%) of adaptations involved 'adding elements' for scale-up. Most adaptations were 'fidelity consistent' (95%). The main goals for adaptations were related to 'increasing the acceptability, appropriateness, or feasibility' (45%), 'decreasing the costs' (19%) and 'increasing adoption of the evidence-based practice' (19%). Adaptations were small to medium in size, with most proposed to have a positive (87%) or neutral (8%) effect on the effectiveness of the intervention, rather than negative (4%).

CONCLUSIONS

A large number of small, fidelity-consistent, adaptations were proposed for Play Active scale-up. Overall, the process of reporting adaptations was found to be feasible. To understand the impact of these adaptations, it will be important to re-evaluate implementation, effectiveness and process outcomes, at-scale.

Scale-up of the Internet-based Professional Learning to help teachers promote Activity in Youth (iPLAY) intervention: a hybrid type 3 implementation-effectiveness trial

BACKGROUND

Whole-of-school programs have demonstrated success in improving student physical activity levels, but few have progressed beyond efficacy testing to implementation at-scale. The purpose of our study was to evaluate the scale-up of the 'Internet-based Professional Learning to help teachers promote Activity in Youth' (iPLAY) intervention in primary schools using the RE-AIM framework.

METHODS

We conducted a type 3 hybrid implementation-effectiveness study and collected data between April 2016 and June 2021, in New South Wales (NSW), Australia. RE-AIM was operationalised as: (i) Reach: Number and representativeness of students exposed to iPLAY; (ii) Effectiveness: Impact of iPLAY in a sub-sample of students (n = 5,959); (iii) Adoption: Number and representativeness of schools that received iPLAY; (iv) Implementation: Extent to which the three curricular and three non-curricular components of iPLAY were delivered as intended; (v) Maintenance: Extent to which iPLAY was sustained in schools. We conducted 43 semi-structured interviews with teachers (n = 14), leaders (n = 19), and principals (n = 10) from 18 schools (11 from urban and 7 from rural locations) to determine program maintenance.

RESULTS

Reach: iPLAY reached ~ 31,000 students from a variety of socio-economic strata (35% of students were in the bottom quartile, almost half in the middle two quartiles, and 20% in the top quartile).

EFFECTIVENESS

We observed small positive intervention effects for enjoyment of PE/sport (0.12 units, 95% CI: 0.05 to 0.20, d = 0.17), perceptions of need support from teachers (0.26 units, 95% CI: 0.16 to 0.53, d = 0.40), physical activity participation (0.28 units, 95% CI: 0.10 to 0.47, d = 0.14), and subjective well-being (0.82 units, 95% CI: 0.32 to 1.32, d = 0.12) at 24-months. Adoption: 115 schools received iPLAY.

IMPLEMENTATION

Most schools implemented the curricular (59%) and non-curricular (55%) strategies as intended. Maintenance: Based on our qualitative data, changes in teacher practices and school culture resulting from iPLAY were sustained.

CONCLUSIONS

iPLAY had extensive reach and adoption in NSW primary schools. Most of the schools implemented iPLAY as intended and effectiveness data suggest the positive effects observed in our cluster RCT were sustained when the intervention was delivered at-scale.

TRIAL REGISTRATION

ACTRN12621001132831.