Simulation-based power calculations for planning a two-stage individual participant data meta-analysis

Calculating the power of a planned individual participant data meta-analysis to examine prognostic factor effects for a binary outcome

Collecting data for an individual participant data meta-analysis (IPDMA) project can be time consuming and resource intensive and could still have insufficient power to answer the question of interest. Therefore, researchers should consider the power of their planned IPDMA before collecting IPD. Here we propose a method to estimate the power of a planned IPDMA project aiming to synthesise multiple cohort studies to investigate the (unadjusted or adjusted) effects of potential prognostic factors for a binary outcome. We consider both binary and continuous factors and provide a three-step approach to estimating the power in advance of collecting IPD, under an assumption of the true prognostic effect of each factor of interest. The first step uses routinely available (published) aggregate data for each study to approximate Fisher's information matrix and thereby estimate the anticipated variance of the unadjusted prognostic factor effect in each study. These variances are then used in step 2 to estimate the anticipated variance of the summary prognostic effect from the IPDMA. Finally, step 3 uses this variance to estimate the corresponding IPDMA power, based on a two-sided Wald test and the assumed true effect. Extensions are provided to adjust the power calculation for the presence of additional covariates correlated with the prognostic factor of interest (by using a variance inflation factor) and to allow for between-study heterogeneity in prognostic effects. An example is provided for illustration, and Stata code is supplied to enable researchers to implement the method.

Nonlinear effects and effect modification at the participant-level in IPD meta-analysis part 2: methodological guidance is available

OBJECTIVES

To review methodological guidance for nonlinear covariate-outcome associations (NL), and linear effect modification and nonlinear effect modification (LEM and NLEM) at the participant level in individual participant data meta-analyses (IPDMAs) and their power requirements.

STUDY DESIGN AND SETTING

We searched Medline, Embase, Web of Science, Scopus, PsycINFO and the Cochrane Library to identify methodology publications on IPDMA of LEM, NL or NLEM (PROSPERO CRD42019126768).

RESULTS

Through screening 6,466 records we identified 54 potential articles of which 23 full texts were relevant. Nine further relevant publications were published before or after the literature search and were added. Of these 32 references, 21 articles considered LEM, 6 articles NL or NLEM and 6 articles described sample size calculations. A book described all four. Sample size may be calculated through simulation or closed form. Assessments of LEM or NLEM at the participant level need to be based on within-trial information alone. Nonlinearity (NL or NLEM) can be modeled using polynomials or splines to avoid categorization.

CONCLUSION

Detailed methodological guidance on IPDMA of effect modification at participant-level is available. However, methodology papers for sample size and nonlinearity are rarer and may not cover all scenarios. On these aspects, further guidance is needed.

Nonlinear effects and effect modification at the participant-level in IPD meta-analysis part 1: analysis methods are often substandard

OBJECTIVES

To review analysis methods used for linear effect modification (LEM), nonlinear covariate-outcome associations (NL) and nonlinear effect modification (NLEM) at the participant-level in individual participant data meta-analysis (IPDMA).

STUDY DESIGN AND SETTING

We searched Medline, Embase, Web of Science, Scopus, PsycINFO and the Cochrane Library to identify IPDMA of randomized controlled trials (PROSPERO CRD42019126768). We investigated if and how IPDMA examined LEM, NL and NLEM, including whether aggregation bias was addressed and if power was considered.

RESULTS

We screened 6,466 records, randomly sampled 207 and identified 100 IPDMA of LEM, NL or NLEM. Power for LEM was calculated a priori in 3 IPDMA. Of 100 IPDMA, 94 analyzed LEM, 4 NLEM and 8 NL. One-stage models were favoured for all three (56%, 100%, 50%, respectively). Two-stage models were used in 15%, 0% and 25% of IPDMA with unclear descriptions in 30%, 0% and 25%, respectively. Only 12% of one-stage LEM and NLEM IPDMA provided sufficient detail to confirm they had addressed aggregation bias.

CONCLUSION

Investigation of effect modification at the participant-level is common in IPDMA projects, but methods are often open to bias or lack detailed descriptions. Nonlinearity of continuous covariates and power of IPDMA are rarely assessed.

Calcium supplementation to prevent pre-eclampsia: protocol for an individual participant data meta-analysis, network meta-analysis and health economic evaluation

INTRODUCTION

Low dietary calcium intake is a risk factor for pre-eclampsia, a major contributor to maternal and perinatal mortality and morbidity worldwide. Calcium supplementation can prevent pre-eclampsia in women with low dietary calcium. However, the optimal dose and timing of calcium supplementation are not known. We plan to undertake an individual participant data (IPD) meta-analysis of randomised trials to determine the effects of various calcium supplementation regimens in preventing pre-eclampsia and its complications and rank these by effectiveness. We also aim to evaluate the cost-effectiveness of calcium supplementation to prevent pre-eclampsia.

METHODS AND ANALYSIS

We will identify randomised trials on calcium supplementation before and during pregnancy by searching major electronic databases including Embase, CINAHL, MEDLINE, CENTRAL, PubMed, Scopus, AMED, LILACS, POPLINE, AIM, IMSEAR, ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform, without language restrictions, from inception to February 2022. Primary researchers of the identified trials will be invited to join the International Calcium in Pregnancy Collaborative Network and share their IPD. We will check each study's IPD for consistency with the original authors before standardising and harmonising the data. We will perform a series of one-stage and two-stage IPD random-effect meta-analyses to obtain the summary intervention effects on pre-eclampsia with 95% CIs and summary treatment-covariate interactions (maternal risk status, dietary intake, timing of intervention, daily dose of calcium prescribed and total intake of calcium). Heterogeneity will be summarised using tau2, I2 and 95% prediction intervals for effect in a new study. Sensitivity analysis to explore robustness of statistical and clinical assumptions will be carried out. Minor study effects (potential publication bias) will be investigated using funnel plots. A decision analytical model for use in low-income and middle-income countries will assess the cost-effectiveness of calcium supplementation to prevent pre-eclampsia.

ETHICS AND DISSEMINATION

No ethical approvals are required. We will store the data in a secure repository in an anonymised format. The results will be published in peer-reviewed journals.

PROSPERO REGISTRATION NUMBER

CRD42021231276.

Do better nights lead to better days? Guided internet-based cognitive behavioral therapy for insomnia in people suffering from a range of mental health problems: Protocol of a pragmatic randomized clinical trial

BACKGROUND

Insomnia is the transdiagnostically shared most common complaint in disorders of anxiety, stress and emotion regulation. Current cognitive behavioral therapies (CBT) for these disorders do not address sleep, while good sleep is essential for regulating emotions and learning new cognitions and behaviours: the core fundaments of CBT. This transdiagnostic randomized control trial (RCT) evaluates whether guided internet-delivered cognitive behavioral therapy for insomnia (iCBT-I) (1) improves sleep, (2) affects the progression of emotional distress and (3) enhances the effectiveness of regular treatment of people with clinically relevant symptoms of emotional disorders across all mental health care (MHC) echelons.

METHODS

We aim for 576 completers with clinically relevant symptoms of insomnia as well as at least one of the dimensions of generalized anxiety disorder (GAD), social anxiety disorder (SAD), panic disorder (PD), posttraumatic stress disorder (PTSD) or borderline personality disorder (BPD). Participants are either pre-clinical, unattended, or referred to general- or specialized MHC. Using covariate-adaptive randomization, participants will be assigned to a 5 to 8-week iCBT-I (i-Sleep) or a control condition (sleep diary only) and assessed at baseline, and after two and eight months. The primary outcome is insomnia severity. Secondary outcomes address sleep, severity of mental health symptoms, daytime functioning, mental health protective lifestyles, well-being, and process evaluation measures. Analyses use linear mixed-effect regression models.

DISCUSSION

This study can reveal for whom, and at which stage of disease progression, better nights could mean substantially better days.

TRIAL REGISTRATION

International Clinical Trial Registry Platform (NL9776). Registered on 2021-10-07.

Calculating the power to examine treatment-covariate interactions when planning an individual participant data meta-analysis of randomized trials with a binary outcome

Before embarking on an individual participant data meta-analysis (IPDMA) project, researchers and funders need assurance it is worth their time and cost. This should include consideration of how many studies are promising their IPD and, given the characteristics of these studies, the power of an IPDMA including them. Here, we show how to estimate the power of a planned IPDMA of randomized trials to examine treatment-covariate interactions at the participant level (ie, treatment effect modifiers). We focus on a binary outcome with binary or continuous covariates, and propose a three-step approach, which assumes the true interaction size is common to all trials. In step one, the user must specify a minimally important interaction size and, for each trial separately (eg, as obtained from trial publications), the following aggregate data: the number of participants and events in control and treatment groups, the mean and SD for each continuous covariate, and the proportion of participants in each category for each binary covariate. This allows the variance of the interaction estimate to be calculated for each trial, using an analytic solution for Fisher's information matrix from a logistic regression model. Step 2 calculates the variance of the summary interaction estimate from the planned IPDMA (equal to the inverse of the sum of the inverse trial variances from step 1), and step 3 calculates the corresponding power based on a two-sided Wald test. Stata and R code are provided, and two examples given for illustration. Extension to allow for between-study heterogeneity is also considered.

Cut points and contexts

In research, policy, and practice, continuous variables are often categorized. Statisticians have generally advised against categorization for many reasons, such as loss of information and precision as well as distortion of estimated statistics. Here, a different kind of problem with categorization is considered: the idea that, for a given continuous variable, there is a unique set of cut points that is the objectively correct or best categorization. It is shown that this is unlikely to be the case because categorized variables typically exist in webs of statistical relationships with other variables. The choice of cut points for a categorized variable can influence the values of many statistics relating that variable to others. This essay explores the substantive trade‐offs that can arise between different possible cut points to categorize a continuous variable, making it difficult to say that any particular categorization is objectively best. Limitations of different approaches to selecting cut points are discussed. Contextual trade‐offs may often be an argument against categorization. At the very least, such trade‐offs mean that research inferences, or decisions about policy or practice, that involve categorized variables should be framed and acted upon with flexibility and humility.

In practical settings, the choice of cut points for categorizing a continuous variable is likely to entail trade‐offs across multiple statistical relationships between the categorized variable and other variables. These trade‐offs mean that no single categorization is objectively best or correct.

Psychosocial factors and cancer incidence (PSY-CA): Protocol for individual participant data meta-analyses

OBJECTIVES

Psychosocial factors have been hypothesized to increase the risk of cancer. This study aims (1) to test whether psychosocial factors (depression, anxiety, recent loss events, subjective social support, relationship status, general distress, and neuroticism) are associated with the incidence of any cancer (any, breast, lung, prostate, colorectal, smoking-related, and alcohol-related); (2) to test the interaction between psychosocial factors and factors related to cancer risk (smoking, alcohol use, weight, physical activity, sedentary behavior, sleep, age, sex, education, hormone replacement therapy, and menopausal status) with regard to the incidence of cancer; and (3) to test the mediating role of health behaviors (smoking, alcohol use, weight, physical activity, sedentary behavior, and sleep) in the relationship between psychosocial factors and the incidence of cancer.

METHODS

The psychosocial factors and cancer incidence (PSY-CA) consortium was established involving experts in the field of (psycho-)oncology, methodology, and epidemiology. Using data collected in 18 cohorts (N = 617,355), a preplanned two-stage individual participant data (IPD) meta-analysis is proposed. Standardized analyses will be conducted on harmonized datasets for each cohort (stage 1), and meta-analyses will be performed on the risk estimates (stage 2).

CONCLUSION

PSY-CA aims to elucidate the relationship between psychosocial factors and cancer risk by addressing several shortcomings of prior meta-analyses.

Diet and physical activity in pregnancy to prevent gestational diabetes: a protocol for an individual participant data (IPD) meta-analysis on the differential effects of interventions with economic evaluation

INTRODUCTION

Mothers with gestational diabetes mellitus (GDM) are at increased risk of pregnancy-related complications and developing type 2 diabetes after delivery. Diet and physical activity-based interventions may prevent GDM, but variations in populations, interventions and outcomes in primary trials have limited the translation of available evidence into practice. We plan to undertake an individual participant data (IPD) meta-analysis of randomised trials to assess the differential effects and cost-effectiveness of diet and physical activity-based interventions in preventing GDM and its complications.

METHODS

The International Weight Management in Pregnancy Collaborative Network database is a living repository of IPD from randomised trials on diet and physical activity in pregnancy identified through a systematic literature search. We shall update our existing search on MEDLINE, Embase, BIOSIS, LILACS, Pascal, Science Citation Index, Cochrane Database of Systematic Reviews, Cochrane Central Register of Controlled Trials, Database of Abstracts of Reviews of Effects and Health Technology Assessment Database without language restriction to identify relevant trials until March 2021. Primary researchers will be invited to join the Network and share their IPD. Trials including women with GDM at baseline will be excluded. We shall perform a one and two stage random-effect meta-analysis for each intervention type (all interventions, diet-based, physical activity-based and mixed approach) to obtain summary intervention effects on GDM with 95% CIs and summary treatment-covariate interactions. Heterogeneity will be summarised using I2 and tau2 statistics with 95% prediction intervals. Publication and availability bias will be assessed by examining small study effects. Study quality of included trials will be assessed by the Cochrane Risk of Bias tool, and the Grading of Recommendations, Assessment, Development and Evaluations approach will be used to grade the evidence in the results. A model-based economic analysis will be carried out to assess the cost-effectiveness of interventions to prevent GDM and its complications compared with usual care.

ETHICS AND DISSEMINATION

Ethics approval is not required. The study is registered on the International Prospective Register of Systematic Reviews (CRD42020212884). Results will be submitted for publication in peer-reviewed journals.

sparrpowR: a flexible R package to estimate statistical power to identify spatial clustering of two groups and its application

Background

Cancer epidemiology studies require sufficient power to assess spatial relationships between exposures and cancer incidence accurately. However, methods for power calculations of spatial statistics are complicated and underdeveloped, and therefore underutilized by investigators. The spatial relative risk function, a cluster detection technique that detects spatial clusters of point-level data for two groups (e.g., cancer cases and controls, two exposure groups), is a commonly used spatial statistic but does not have a readily available power calculation for study design.

Results

We developed sparrpowR as an open-source R package to estimate the statistical power of the spatial relative risk function. sparrpowR generates simulated data applying user-defined parameters (e.g., sample size, locations) to detect spatial clusters with high statistical power. We present applications of sparrpowR that perform a power calculation for a study designed to detect a spatial cluster of incident cancer in relation to a point source of numerous environmental emissions. The conducted power calculations demonstrate the functionality and utility of sparrpowR to calculate the local power for spatial cluster detection.

Conclusions

sparrpowR improves the current capacity of investigators to calculate the statistical power of spatial clusters, which assists in designing more efficient studies. This newly developed R package addresses a critically underdeveloped gap in cancer epidemiology by estimating statistical power for a common spatial cluster detection technique.

Trials number, funding support, and intervention type associated with IPDMA data retrieval: a cross-sectional study

OBJECTIVES

This study aimed to investigate the predictors for high data retrieval and the reporting of individual participant data meta-analyses (IPDMAs).

STUDY DESIGN AND SETTING

We searched EMBASE, MEDLINE, and the Cochrane Library for articles pertaining to IPDMA from 2011 to 2019. Only IPDMA assessing treatment effects, including randomized controlled trials (RCTs), were included. Adherence to the PRISMA-IPD guideline was checked.

RESULTS

A total of 210 IPDMA covering 18 diseases were sampled; 80 (38.1%) and 123 (58.6%) of the IPDMA retrieved IPD from all and ≥80% RCTs, respectively. Non-Cochrane reviews, IPDMA on nonpharmacological interventions, analyses of smaller numbers of RCTs, and having funding supports were predictors of complete IPD retrieval. Owners of RCTs had an increased probability of obtaining IPD. Only 4.3% described the eligibility criteria covering all the PICO components, 11.0% reported the methods for assessing risk of bias across studies, 11.4% mentioned the IPD integrity, and 9.0% presented detailed results of syntheses.

CONCLUSION

IPD retrieval and reporting was not satisfactory among the published IPDMA. Encouraging RCT owners to conduct or join in the IPDMA is a potential strategy to maximize the IPD retrieval. IPDMA are suggested to adhere to the PRISMA-IPD guideline during reporting.

Individual participant data meta-analysis to examine interactions between treatment effect and participant-level covariates: Statistical recommendations for conduct and planning

Precision medicine research often searches for treatment‐covariate interactions, which refers to when a treatment effect (eg, measured as a mean difference, odds ratio, hazard ratio) changes across values of a participant‐level covariate (eg, age, gender, biomarker). Single trials do not usually have sufficient power to detect genuine treatment‐covariate interactions, which motivate the sharing of individual participant data (IPD) from multiple trials for meta‐analysis. Here, we provide statistical recommendations for conducting and planning an IPD meta‐analysis of randomized trials to examine treatment‐covariate interactions. For conduct, two‐stage and one‐stage statistical models are described, and we recommend: (i) interactions should be estimated directly, and not by calculating differences in meta‐analysis results for subgroups; (ii) interaction estimates should be based solely on within‐study information; (iii) continuous covariates and outcomes should be analyzed on their continuous scale; (iv) nonlinear relationships should be examined for continuous covariates, using a multivariate meta‐analysis of the trend (eg, using restricted cubic spline functions); and (v) translation of interactions into clinical practice is nontrivial, requiring individualized treatment effect prediction. For planning, we describe first why the decision to initiate an IPD meta‐analysis project should not be based on between‐study heterogeneity in the overall treatment effect; and second, how to calculate the power of a potential IPD meta‐analysis project in advance of IPD collection, conditional on characteristics (eg, number of participants, standard deviation of covariates) of the trials (potentially) promising their IPD. Real IPD meta‐analysis projects are used for illustration throughout.