Identifying essential implementation strategies: a mixed methods process evaluation of a multi-strategy policy implementation intervention for schools

Learning Health System to rapidly improve the implementation of a school physical activity policy

BACKGROUND

Learning Health Systems (LHS) - characterised by cycles of evidence generation and application - are increasingly recognised for their potential to improve public health interventions and optimise health impacts; however there is little evidence of their application in the context of public health practice. Here, we describe how an Australian public health unit applied a LHS approach to successfully improve a model of support for implementation of a school-based physical activity policy.

METHODS

This body of work was undertaken in the context of a strong research-practice partnership. Core LHS capabilities included: i) partnerships and stakeholder engagement; ii) workforce development and learning health communities; iii) multi-disciplinary scientific expertise; iv) practice data collection and management system; v) evidence surveillance and synthesis; and vi) governance and organisational processes of decision making. Three cycles of data generation and application were used. Within each cycle, randomised controlled trials conducted in NSW primary schools were used to generate data on the support model's effectiveness for improving schools' implementation of a government physical activity policy, its delivery costs, and process measures such as adoption and acceptability. Each type of data were analysed independently, synthesised, and then presented to a multi-disciplinary team of researchers and practitioners, in consult with stakeholders, leading to collaborative decisions for incremental improvements to the support model.

RESULTS

Cycle 1 tested the first version of the support model (composed of five implementation strategies targeting identified barriers of policy implementation) and showed the model's feasibility and efficacy for improving schools' policy implementation. Data-informed changes were made to enhance impact, including the addition of three implementation strategies to address outstanding barriers. Cycle 2 (now, testing a package of eight implementation strategies) established the model's effectiveness and cost-effectiveness for improving school's policy implementation. Data-informed changes were made to reduce delivery costs, specifically adapting the costliest strategies to reduce in-person contact from external support personnel. Cycle 3 showed that the adaptations minimised the relative cost of delivery without adversely impacting on the effect.

CONCLUSIONS

Through this process, we identified an effective, cost-effective, acceptable and scalable policy implementation support model for service delivery. This provides important information to inform or support LHS approaches for other agencies seeking to optimise the health impact of evidence-based interventions.

Optimising a multi-strategy implementation intervention to improve the delivery of a school physical activity policy at scale: findings from a randomised noninferiority trial

Background

To maximise their potential health benefits, school-based physical activity policies need to be implemented at scale. This paper describes the third in a sequence of trials that sought to optimise an effective strategy (PACE) to assist schools’ implementation of a physical activity policy. Specifically, it aimed to determine the probability that a multi-strategy intervention adapted to reduce in-person contact (Adapted PACE) was “as good as” the original intervention (PACE) in increasing the weekly minutes of structured physical activity implemented by classroom teachers.

Methods

A noninferiority cluster randomised controlled trial was undertaken with 48 primary schools in New South Wales, Australia. Schools were randomised to receive PACE or a model with adaptations made to the delivery modes (Adapted PACE). Teachers’ scheduled minutes of weekly physical activity was assessed at baseline (Oct 2018-Feb 2019) and 12-month follow-up (Oct-Dec 2019). The noninferiority margin was set at − 16.4 minutes based on previous data and decision panel consensus. A linear mixed model analysed within a Bayesian framework was used to explore noninferiority between the two PACE models. A cost minimisation analysis was conducted from the health service provider perspective, using the Australian dollar (AUD).

Results

The posterior estimate for the between group difference at follow-up was − 2.3 minutes (95% credible interval = − 18.02, 14.45 minutes). There was an estimated 96% probability of Adapted PACE being considered noninferior (only 4% of the posterior samples crossed the noninferiority margin of − 16.4 minutes). That is, the minutes of physical activity implemented by teachers at Adapted PACE schools was not meaningfully less than the minutes of physical activity implemented by teachers at PACE schools. The mean total cost was AUD$25,375 (95% uncertainty interval = $21,499, $29,106) for PACE and AUD$16,421 (95% uncertainty interval = $13,974, $19,656) for Adapted PACE; an estimated reduction of AUD$373 (95% uncertainty interval = $173, $560) per school.

Conclusions

It is highly probable that Adapted PACE is noninferior to the original model. It is a cost-efficient alternative also likely to be a more suitable approach to supporting large scale implementation of school physical activity policies.

Trial registration

Retrospectively registered with the Australian New Zealand Clinical Trials Registry (ACTRN12619001229167).

Supplementary Information

The online version contains supplementary material available at 10.1186/s12966-022-01345-6.