Characteristics and practices of school-based cluster randomised controlled trials for improving health outcomes in pupils in the UK: a systematic review protocol

Uncovering individualised treatment effects for educational trials

Large-scale Randomised Controlled Trials (RCTs) are widely regarded as "the gold standard" for testing the causal effects of school-based interventions. RCTs typically present the statistical significance of the average treatment effect (ATE), which captures the effect an intervention has had on average for a given population. However, key decisions in child health and education are often about individuals who may be very different from those averages. One way to identify heterogeneous treatment effects across different individuals, not captured by the ATE, is to conduct subgroup analyses. For example, free school meal (FSM) pupils as required for projects funded by the Education Endowment Foundation (EEF) in England. These subgroup analyses, as we demonstrate in 48 EEF-funded RCTs involving over 200,000 students, are usually not standardised across studies and offer flexible degrees of freedom to researchers, potentially leading to mixed, if not misleading, results. Here, we develop and deploy an alternative to ATE and subgroup analysis, a machine-learning and regression-based framework to predict individualised treatment effects (ITEs). ITEs could show where an intervention worked, for which individuals, and to what extent. Our findings have implications for decision-makers in fields like education, healthcare, law, and clinical practices concerning children and adolescents.

Researchers' experiences of the design and conduct challenges associated with parallel-group cluster-randomised trials and views on a novel open-cohort design

BACKGROUND

Two accepted designs exist for parallel-group cluster-randomised trials (CRTs). Closed-cohort designs follow the same individuals over time with a single recruitment period before randomisation, but face challenges in settings with high attrition. (Repeated) cross-sectional designs recruit at one or more timepoints before and/or after randomisation, collecting data from different individuals present in the cluster at these timepoints, but are unsuitable for assessment of individual change over time. An 'open-cohort' design allows individual follow-up with recruitment before and after cluster-randomisation, but little literature exists on acceptability to inform their use in CRTs.

AIM

To document the views and experiences of expert trialists to identify: a) Design and conduct challenges with established parallel-group CRT designs,b) Perceptions of potential benefits and barriers to implementation of open-cohort CRTs,c) Methods for minimising, and investigating the impact of, bias in open-cohort CRTs.

METHODS

Qualitative consultation via two expert workshops including triallists (n = 24) who had worked on CRTs over a range of settings. Workshop transcripts were analysed using Descriptive Thematic Analysis utilising inductive and deductive coding.

RESULTS

Two central organising concepts were developed. Design and conduct challenges with established CRT designs confirmed that current CRT designs are unable to deal with many of the complex research and intervention circumstances found in some trial settings (e.g. care homes). Perceptions of potential benefits and barriers of open cohort designs included themes on: approaches to recruitment; data collection; analysis; minimising/investigating the impact of bias; and how open-cohort designs might address or present CRT design challenges. Open-cohort designs were felt to provide a solution for some of the challenges current CRT designs present in some settings.

CONCLUSIONS

Open-cohort CRT designs hold promise for addressing the challenges associated with standard CRT designs. Research is needed to provide clarity around definition and guidance on application.

Effects of Combined Movement and Storytelling Intervention on Fundamental Motor Skills, Language Development and Physical Activity Level in Children Aged 3 to 6 Years: Study Protocol for a Randomized Controlled Trial

This study protocol aims to analyze and compare the effects of combined movement and storytelling intervention (CMSI) on fundamental motor skills (locomotor skills and object control), language development (language comprehension, language expression, vocabulary and language description), and physical activity levels (light intensity, moderate-to-vigorous intensity and sedentary time) in children aged 3 to 6 years. The sample will consist of 144 children from 12 class groups, randomly assigned to 3 experimental groups (n = 72 children) and 3 control groups (n = 72 children), belonging to 4 class groups of upper-middle-level classes (2 experimental and 2 control; 3 to 4 years), 4 transition level 1 classes (2 experimental and 2 control; 4 to 5 years) and 4 transition level 2 classes (2 experimental and 2 control; 5 to 6 years). The experimental groups will perform CMSI for 3 sessions per week (40 min per session) over 12 weeks (using one motor story per week), while the control groups will not receive any treatment. The main outcome will provide information about fundamental motor skills, language development, and physical activity levels. Our hypothesis indicates that CMSI has the potential to generate significant increases in selected assessments. If this intervention proves to be beneficial, it could contribute to preschool and school curricula.

Minimising Young Children's Anxiety through Schools (MY-CATS): statistical analysis plan for a cluster randomised controlled trial to evaluate the effectiveness and cost-effectiveness of an online parent-led intervention compared with usual school practice for young children identified as at risk for anxiety disorders

BACKGROUND

The Minimising Young Children's Anxiety through Schools (MY-CATS) trial is being conducted to determine whether an online evidence-based parent-guided cognitive behavioural therapy intervention in addition to usual school practice is effective and cost-effective compared with usual school practice in reducing anxiety disorders in children aged 4-7 deemed 'at risk' of anxiety disorders. This update article describes the detailed statistical analysis plan for the MY-CATS trial and reports a review of the underpinning sample size assumptions.

METHODS AND DESIGN

The MY-CATS study is a two-arm, definitive superiority pragmatic parallel group cluster randomised controlled trial in which schools will be randomised 1:1 to receive either the intervention (in addition to usual school practice) or the usual school practice only. This update to the (published) protocol provides a detailed description of the study methods, the statistical principles, the trial population and the planned statistical analyses, including additional analyses comprising instrumental variable regression and mediation analysis.

TRIAL REGISTRATION

ISRCTN82398107 . Prospectively registered on 14 January 2021.

Characteristics and practices of school-based cluster randomised controlled trials for improving health outcomes in pupils in the United Kingdom: a methodological systematic review

Background

Cluster randomised trials (CRTs) are increasingly used to evaluate non-pharmacological interventions for improving child health. Although methodological challenges of CRTs are well documented, the characteristics of school-based CRTs with pupil health outcomes have not been systematically described. Our objective was to describe methodological characteristics of these studies in the United Kingdom (UK).

Methods

MEDLINE was systematically searched from inception to 30^th June 2020. Included studies used the CRT design in schools and measured primary outcomes on pupils. Study characteristics were described using descriptive statistics.

Results

Of 3138 articles identified, 64 were included. CRTs with pupil health outcomes have been increasingly used in the UK school setting since the earliest included paper was published in 1993; 37 (58%) studies were published after 2010. Of the 44 studies that reported information, 93% included state-funded schools. Thirty six (56%) were exclusively in primary schools and 24 (38%) exclusively in secondary schools. Schools were randomised in 56 studies, classrooms in 6 studies, and year groups in 2 studies. Eighty percent of studies used restricted randomisation to balance cluster-level characteristics between trial arms, but few provided justification for their choice of balancing factors. Interventions covered 11 different health areas; 53 (83%) included components that were necessarily administered to entire clusters. The median (interquartile range) number of clusters and pupils recruited was 31.5 (21 to 50) and 1308 (604 to 3201), respectively. In half the studies, at least one cluster dropped out. Only 26 (41%) studies reported the intra-cluster correlation coefficient (ICC) of the primary outcome from the analysis; this was often markedly different to the assumed ICC in the sample size calculation. The median (range) ICC for school clusters was 0.028 (0.0005 to 0.21).

Conclusions

The increasing pool of school-based CRTs examining pupil health outcomes provides methodological knowledge and highlights design challenges. Data from these studies should be used to identify the best school-level characteristics for balancing the randomisation. Better information on the ICC of pupil health outcomes is required to aid the planning of future CRTs. Improved reporting of the recruitment process will help to identify barriers to obtaining representative samples of schools.