A guide to prospective meta-analysis

Quantifying the advantages of conducting a prospective meta-analysis (PMA): a case study of early childhood obesity prevention

Background

For prospective meta-analyses (PMAs), eligible studies are identified, and the PMA hypotheses, selection criteria, and analysis methods are pre-specified before the results of any of the studies are known. This reduces publication bias and selective outcome reporting and provides a unique opportunity for outcome standardisation/harmonisation. We conducted a world-first PMA of four trials investigating interventions to prevent early childhood obesity. The aims of this study were to quantitatively analyse the effects of prospective planning on variations across trials, outcome harmonisation, and the power to detect intervention effects, and to derive recommendations for future PMA.

Methods

We examined intervention design, participant characteristics, and outcomes collected across the four trials included in the EPOCH PMA using their registration records, protocol publications, and variable lists. The outcomes that trials planned to collect prior to inclusion in the PMA were compared to the outcomes that trials collected after PMA inclusion. We analysed the proportion of matching outcome definitions across trials, the number of outcomes per trial, and how collaboration increased the statistical power to detect intervention effects.

Results

The included trials varied in intervention design and participants, this improved external validity and the ability to perform subgroup analyses for the meta-analysis. While individual trials had limited power to detect the main intervention effect (BMI z-score), synthesising data substantially increased statistical power. Prospective planning led to an increase in the number of collected outcome categories (e.g. weight, child’s diet, sleep), and greater outcome harmonisation. Prior to PMA inclusion, only 18% of outcome categories were included in all trials. After PMA inclusion, this increased to 91% of outcome categories. However, while trials mostly collected the same outcome categories after PMA inclusion, some inconsistencies in how the outcomes were measured remained (such as measuring physical activity by hours of outside play versus using an activity monitor).

Conclusion

Prospective planning led to greater outcome harmonisation and greater power to detect intervention effects, while maintaining acceptable variation in trial designs and populations, which improved external validity. Recommendations for future PMA include more detailed harmonisation of outcome measures and careful pre-specification of analyses to avoid research waste by unnecessary over-collection of data.

The Complex Quest of Preventing Obesity in Early Childhood: Describing Challenges and Solutions Through Collaboration and Innovation

Childhood obesity remains a major public health issue and priority area for action. Promisingly, obesity prevention interventions in the first 2000 days of life have shown modest effectiveness in improving health behaviours and healthy weight status in children. Yet, researchers in this field face several challenges. This can lead to research waste and impede progress towards delivering effective, scalable solutions. In this perspective article, we describe some of the key challenges in early childhood obesity prevention and outline innovative and collaborative solutions to overcome these. Combining these solutions will accelerate the generation of high-quality evidence that can be implemented into policy and practice.

Investigating and dealing with publication bias and other reporting biases in meta-analyses of health research: A review

A P value, or the magnitude or direction of results can influence decisions about whether, when, and how research findings are disseminated. Regardless of whether an entire study or a particular study result is unavailable because investigators considered the results to be unfavorable, bias in a meta-analysis may occur when available results differ systematically from missing results. In this article, we summarize the empirical evidence for various reporting biases that lead to study results being unavailable for inclusion in systematic reviews, with a focus on health research. These biases include publication bias and selective nonreporting bias. We describe processes that systematic reviewers can use to minimize the risk of bias due to missing results in meta-analyses of health research, such as comprehensive searches and prospective approaches to meta-analysis. We also outline methods that have been designed for assessing risk of bias due to missing results in meta-analyses of health research, including using tools to assess selective nonreporting of results, ascertaining qualitative signals that suggest not all studies were identified, and generating funnel plots to identify small-study effects, one cause of which is reporting bias. HIGHLIGHTS: Bias in a meta-analysis may occur when available results differ systematically from missing results. Several different tools, plots, and statistical methods have been designed for assessing risk of bias due to missing results in meta-analyses. These include comparison of prespecified analysis plans with completed reports to detect selective nonreporting of results, consideration of qualitative signals that suggest not all studies were identified, and the use of funnel plots to identify small-study effects, for which reporting bias is one of several causes. Information from approaches such as funnel plots and selection models is more difficult to interpret than from less subjective approaches such as detection of incompletely reported results in studies for which prespecified analysis plans were available.

Understanding, comparing and learning from the four EPOCH early childhood obesity prevention interventions: A multi-methods study

BACKGROUND

Childhood obesity is a global problem. Early obesity prevention interventions are complex and differ in effectiveness. Novel frameworks, taxonomies and experience from the Early Prevention of Obesity in CHildren (EPOCH) trials were applied to unpack interventions.

OBJECTIVES

Deconstruct interventions into their components (target behaviours, delivery features and behaviour change techniques [BCTs]). Identify lessons learned and future recommendations for intervention planning, delivery, evaluation and implementation.

METHODS

This multi-methods study deconstructed the four EPOCH interventions into target behaviours, delivery features and BCTs from unpublished and published materials using systematic frameworks. Additionally, semi-structured interviews were conducted with intervention facilitators and principal investigators.

RESULTS

Each trial targeted between 10 and 14 obesity-related behaviours. Key variations in delivery features related to intensity, delivery mode and tailoring. BCTs consistently used across trials included goal-setting, social support, shaping knowledge, role-modelling and credible source. Recommendations from interview analyses include the importance of stakeholder collaboration and consideration of implementation throughout the study process.

CONCLUSIONS

The combination of frameworks, methodologies and interviews used in this study is a major step towards understanding complex early obesity prevention interventions. Future work will link systematic intervention deconstruction with quantitative models to identify which intervention components are most effective and for whom.

Adjuvant or early salvage radiotherapy for the treatment of localised and locally advanced prostate cancer: a prospectively planned systematic review and meta-analysis of aggregate data

BACKGROUND

It is unclear whether adjuvant or early salvage radiotherapy following radical prostatectomy is more appropriate for men who present with localised or locally advanced prostate cancer. We aimed to prospectively plan a systematic review of randomised controlled trials (RCTs) comparing these radiotherapy approaches.

METHODS

We used a prospective framework for adaptive meta-analysis (FAME), starting the review process while eligible trials were ongoing. RCTs were eligible if they aimed to compare immediate adjuvant radiotherapy versus early salvage radiotherapy, following radical prostatectomy in men (age ≥18 years) with intermediate-risk or high-risk, localised or locally advanced prostate cancer. We searched trial registers and conference proceedings until July 8, 2020, to identify eligible RCTs. By establishing the ARTISTIC collaboration with relevant trialists, we were able to anticipate when eligible trial results would emerge, and we developed and registered a protocol with PROSPERO before knowledge of the trial results (CRD42019132669). We used a harmonised definition of event-free survival, as the time from randomisation until the first evidence of either biochemical progression (prostate-specific antigen [PSA] ≥0·4 ng/mL and rising after completion of any postoperative radiotherapy), clinical or radiological progression, initiation of a non-trial treatment, death from prostate cancer, or a PSA level of at least 2·0 ng/mL at any time after randomisation. We predicted when we would have sufficient power to assess whether adjuvant radiotherapy was superior to early salvage radiotherapy. Investigators supplied results for event-free survival, both overall and within predefined patient subgroups. Hazard ratios (HRs) for the effects of radiotherapy timing on event-free survival and subgroup interactions were combined using fixed-effect meta-analysis.

FINDINGS

We identified three eligible trials and were able to obtain updated results for event-free survival for 2153 patients recruited between November, 2007, and December, 2016. Median follow-up ranged from 60 months to 78 months, with a maximum follow-up of 132 months. 1075 patients were randomly assigned to receive adjuvant radiotherapy and 1078 to a policy of early salvage radiotherapy, of whom 421 (39·1%) had commenced treatment at the time of analysis. Patient characteristics were balanced within trials and overall. Median age was similar between trials at 64 or 65 years (with IQRs ranging from 59 to 68 years) across the three trials and most patients (1671 [77·6%]) had a Gleason score of 7. All trials were assessed as having low risk of bias. Based on 270 events, the meta-analysis showed no evidence that event-free survival was improved with adjuvant radiotherapy compared with early salvage radiotherapy (HR 0·95, 95% CI 0·75-1·21; p=0·70), with only a 1 percentage point (95% CI -2 to 3) change in 5-year event-free survival (89% vs 88%). Results were consistent across trials (heterogeneity p=0·18; I²=42%).

INTERPRETATION

This collaborative and prospectively designed systematic review and meta-analysis suggests that adjuvant radiotherapy does not improve event-free survival in men with localised or locally advanced prostate cancer. Until data on long-term outcomes are available, early salvage treatment would seem the preferable treatment policy as it offers the opportunity to spare many men radiotherapy and its associated side-effects.

FUNDING

UK Medical Research Council.

The AEDUCATE Collaboration. Comprehensive antenatal education birth preparation programmes to reduce the rates of caesarean section in nulliparous women. Protocol for an individual participant data prospective meta-analysis

INTRODUCTION

Rates of medical interventions in normal labour and birth are increasing. This prospective meta-analysis (PMA) proposes to assess whether the addition of a comprehensive multicomponent birth preparation programme reduces caesarean section (CS) in nulliparous women compared with standard hospital care. Additionally, do participant characteristics, intervention components or hospital characteristics modify the effectiveness of the programme? METHODS AND ANALYSIS: Population: women with singleton vertex pregnancies, no planned caesarean section (CS) or epidural.Intervention: in addition to hospital-based standard care, a comprehensive antenatal education programme that includes multiple components for birth preparation, addressing the three objectives: preparing women and their birth partner/support person for childbirth through education on physiological/hormonal birth (knowledge and understanding); building women's confidence through psychological preparation (positive mindset) and support their ability to birth without pain relief using evidence-based tools (tools and techniques). The intervention could occur in a hospital-based or community setting.Comparator: standard care alone in hospital-based maternity units.

OUTCOMES

Primary: CS.Secondary: epidural analgesia, mode of birth, perineal trauma, postpartum haemorrhage, newborn resuscitation, psychosocial well-being.Subgroup analysis: parity, model of care, maternal risk status, maternal education, maternal socio-economic status, intervention components.

STUDY DESIGN

An individual participant data (IPD) prospective meta-analysis (PMA) of randomised controlled trials, including cluster design. Each trial is conducted independently but share core protocol elements to contribute data to the PMA. Participating trials are deemed eligible for the PMA if their results are not yet known outside their Data Monitoring Committees.

ETHICS AND DISSEMINATION

Participants in the individual trials will consent to participation, with respective trials receiving ethical approval by their local Human Research Ethics Committees. Individual datasets remain the property of trialists, and can be published prior to the publication of final PMA results. The overall data for meta-analysis will be held, analysed and published by the collaborative group, led by the Cochrane PMA group.

TRIAL REGISTRATION NUMBER

CRD42020103857.

Motor Control Stabilisation Exercise for Patients with Non-Specific Low Back Pain: A Prospective Meta-Analysis with Multilevel Meta-Regressions on Intervention Effects

Low-to-moderate quality meta-analytic evidence shows that motor control stabilisation exercise (MCE) is an effective treatment of non-specific low back pain. A possible approach to overcome the weaknesses of traditional meta-analyses would be that of a prospective meta-analyses. The aim of the present analysis was to generate high-quality evidence to support the view that motor control stabilisation exercises (MCE) lead to a reduction in pain intensity and disability in non-specific low back pain patients when compared to a control group. In this prospective meta-analysis and sensitivity multilevel meta-regression within the MiSpEx-Network, 18 randomized controlled study arms were included. Participants with non-specific low back pain were allocated to an intervention (individualized MCE, 12 weeks) or a control group (no additive exercise intervention). From each study site/arm, outcomes at baseline, 3 weeks, 12 weeks, and 6 months were pooled. The outcomes were current pain (NRS or VAS, 11 points scale), characteristic pain intensity, and subjective disability. A random effects meta-analysis model for continuous outcomes to display standardized mean differences between intervention and control was performed, followed by sensitivity multilevel meta-regressions. Overall, 2391 patients were randomized; 1976 (3 weeks, short-term), 1740 (12 weeks, intermediate), and 1560 (6 months, sustainability) participants were included in the meta-analyses. In the short-term, intermediate and sustainability, moderate-to-high quality evidence indicated that MCE has a larger effect on current pain (SMD = −0.15, −0.15, −0.19), pain intensity (SMD = −0.19, −0.26, −0.26) and disability (SMD = −0.15, −0.27, −0.25) compared with no exercise intervention. Low-quality evidence suggested that those patients with comparably intermediate current pain and older patients may profit the most from MCE. Motor control stabilisation exercise is an effective treatment for non-specific low back pain. Sub-clinical intermediate pain and middle-aged patients may profit the most from this intervention.

Obesity and COVID-19: A Perspective from the European Association for the Study of Obesity on Immunological Perturbations, Therapeutic Challenges, and Opportunities in Obesity

Accumulating evidence suggests that obesity is a major risk factor for the initiation, progression, and outcomes of coronavirus disease 2019 (COVID-19). The European Association for the Study of Obesity (EASO), as a scientific and medical society dedicated to the promotion of health and well-being, is greatly concerned about the concomitant obesity and COVID-19 pandemics and their impact on health and society at large. In this perspective, we will address the inherent immunological perturbations and alterations in the renin-angiotensin-aldosterone system in patients with obesity and COVID-19, and discuss how these impairments may underlie the increased susceptibility and more detrimental outcomes of COVID-19 in people with obesity. Clearly, this has important implications for preventive measures, vaccination, and future therapeutic strategies to combat COVID-19. Furthermore, we will highlight important knowledge gaps and provide suggestions for future research and recommendations for policy actions. Since many new reports on COVID-19 rapidly appear, the present perspective should be seen as a focus for discussion to drive forward further understanding, research initiatives, and clinical management of COVID-19.

Network meta-analyses in reproductive medicine: challenges and opportunities

Network meta-analysis allows researchers to synthesise both direct and indirect evidence, thus enabling simultaneous comparisons of multiple treatments. A relatively recent addition to evidence synthesis in reproductive medicine, this approach has become increasingly popular. Yet, the underlying assumptions of network meta-analyses, which drive the validity of their findings, have been frequently ignored. In this article, we discuss the strengths and limitations of network meta-analyses. In addition, we present an overview of published network meta-analyses in reproductive medicine, summarize their challenges and provide insights into future research opportunities.

Intragastric Botulinum Toxin A Injection Is an Effective Obesity Therapy for Patients with BMI > 40 kg/m2: a Systematic Review and Meta-analysis

We conducted a systematic review and meta-analysis to assess the efficacy of intragastric BTA injection in patients with obesity and morbid obesity. Five RCTs were identified. Intragastric BTA injection was no superior to saline in absolute weight loss or BMI change, but significantly lengthened the gastric emptying time (MD, 15.57; 95% CI, 8.75 to 22.38). In meta-regression analysis, the absolute weight loss was lower in the patients with high baseline BMI. In subgroup analysis, the use of BTA in absolute weight loss was significantly lower among the patients with baseline BMI more than 40 kg/m² (MD, - 5.31; 95% CI, - 6.17 to - 4.45). The benefit of intragastric BTA injection in reduction of absolute weight was observed in patients with baseline BMI more than 40 kg/m².

Fragility index of network meta-analysis with application to smoking cessation data

BACKGROUND AND OBJECTIVES

The network meta-analysis (NMA) is frequently used to synthesize evidence for multiple treatment comparisons, but its complexity may affect the robustness (or fragility) of the results. The fragility index (FI) is recently proposed to assess the fragility of the results from clinical studies and from pairwise meta-analyses. We extend the FI to NMAs with binary outcomes.

METHODS

We define the FI for each treatment comparison in NMAs. It quantifies the minimal number of events necessary to be modified for altering the comparison's statistical significance. We introduce an algorithm to derive the FI and visualizations of the process. A worked example of smoking cessation data is used to illustrate the proposed methods.

RESULTS

Some treatment comparisons had small FIs; their significance (or nonsignificance) could be altered by modifying a few events' status. They were related to various factors, such as P-values, event counts, and sample sizes, in the original NMA. After modifying event status, treatment ranking measures were also changed to different extents.

CONCLUSION

Many NMAs include insufficiently compared treatments, small event counts, or small sample sizes; their results are potentially fragile. The FI offers a useful tool to evaluate treatment comparisons' robustness and reliability.

Interventions commenced by early infancy to prevent childhood obesity-The EPOCH Collaboration: An individual participant data prospective meta-analysis of four randomized controlled trials

BACKGROUND

Childhood obesity is a significant global problem. Childhood obesity prevention interventions may be more effective when started very early in life before metabolic and behavioural patterns are established.

METHODS AND FINDINGS

A prospectively planned, individual participant data meta-analysis of four randomized controlled trials. Participants were first-time mothers of term infants. Trial interventions commenced during pregnancy or early infancy and comprised education and support delivered via group sessions and/or home visits. Control group families accessed existing local well-child health care. The primary outcome was body mass index (BMI) z score at 18 to 24 months; 2196 mother-child dyads were available for analysis. Intervention children had lower BMI z scores at 18 to 24 months than control children (-0.12 adjusted mean; 95% confidence interval, -0.22 to -0.02, P = .017). There was some evidence that the BMI z score reduction was greater in settings with limited well-child health care programmes (interaction P value = .03). Improvements were also detected in television viewing time, feeding practices, and breastfeeding duration.

CONCLUSIONS

Parent-focused intervention programmes that commence by early infancy and which aim to establish a trajectory of healthy lifestyle behaviours produced a modest but statistically significant reduction in BMI z score, which if replicated on a wider scale may have important public health implications.

Using Meta-Analysis and Propensity Score Methods to Assess Treatment Effects Toward Evidence-Based Practice in Extensive Reading

This study aimed to depict the assessment process of treatment effects of extensive reading in a second language (L2) toward the establishment of an evidence-based practice. Although standardized mean differences between treatment and control groups have been applied to interpret the magnitude of treatment effects in observational studies on L2 teaching, individual effect sizes vary according to differences in learners, measures, teaching approaches, and research quality. Prior research on extensive reading has suffered from methodological restrictions, especially due to a lack of appropriate comparison between treatment and control groups. For these reasons, a retrospective meta-analysis including only studies that ensured between-group equivalence was conducted in Study 1 to estimate the effect sizes of extensive reading expected in specific teaching environments. When the focused skill of the one-semester program was reading comprehension, its effect size was predicted as d = 0.55. However, the moderator analysis showed that this treatment effect was overestimated due to selection bias in the analyzed studies and adjusted the effect size from 0.55 to 0.37. In Study 2, propensity score analysis was applied to minimize selection bias attributed to observed confounding variables in the comparison between non-randomized treatment and control groups. Data were collected from 109 Japanese university students of English who received in-class extensive reading for one semester and 115 students who attended another English class as the control group. Various types of matching were attempted, and in consideration of balancing the five covariates that might affect treatment effect estimation, the best solutions were nearest neighborhood matching without replacement, nearest neighborhood matching with replacement, and full matching. The results showed that the average treatment effects of extensive reading on all the participants (d = 0.24–0.44) and on the treated individuals (d = 0.32–0.40) were both consistent with the benchmark established in Study 1. Pedagogical implications and methodological limitations are discussed for decision-making regarding the implementation of L2 teaching practices based on research evidence.

Sensitivity analyses assessing the impact of early stopping on systematic reviews: Recommendations for interpreting guidelines

The CONSORT Statement says that data-driven early stopping of a clinical trial is likely to weaken the inferences that can be drawn from the trial. The GRADE guidelines go further, saying that early stopping is a study limitation that carries the risk of bias, and recommending sensitivity analyses in which trials stopped early are omitted from evidence synthesis. Despite extensive debate in the literature over these issues, the existence of clear recommendations in high profile guidelines makes it inevitable that systematic reviewers will consider sensitivity analyses investigating the impact of early stopping. The purpose of this article is to assess methodologies for conducting such sensitivity analyses, and to make recommendations about how the guidelines should be interpreted. We begin with a clarifying overview of the impacts of early stopping on treatment effect estimation in single studies and meta-analyses. We then warn against naive approaches for conducting sensitivity analyses, including simply omitting trials stopped early from meta-analyses. This approach underestimates treatment effects, which may have serious implications if cost-effectiveness analyses determine whether treatments are made widely available. Instead, we discuss two unbiased approaches to sensitivity analysis, one of which is straightforward but statistically inefficient, and the other of which achieves greater statistical efficiency by making use of recent methodological developments in the analysis of clinical trials. We end with recommendations for interpreting: (a) the CONSORT Statement on reporting of reasons for early stopping, and (b) the GRADE guidelines on sensitivity analyses assessing the impact of early stopping.