Meta-analysis of a binary outcome using individual participant data and aggregate data

Combining randomized and non-randomized data to predict heterogeneous effects of competing treatments

Some patients benefit from a treatment while others may do so less or do not benefit at all. We have previously developed a two-stage network meta-regression prediction model that synthesized randomized trials and evaluates how treatment effects vary across patient characteristics. In this article, we extended this model to combine different sources of types in different formats: aggregate data (AD) and individual participant data (IPD) from randomized and non-randomized evidence. In the first stage, a prognostic model is developed to predict the baseline risk of the outcome using a large cohort study. In the second stage, we recalibrated this prognostic model to improve our predictions for patients enrolled in randomized trials. In the third stage, we used the baseline risk as effect modifier in a network meta-regression model combining AD, IPD randomized clinical trial to estimate heterogeneous treatment effects. We illustrated the approach in the re-analysis of a network of studies comparing three drugs for relapsing-remitting multiple sclerosis. Several patient characteristics influence the baseline risk of relapse, which in turn modifies the effect of the drugs. The proposed model makes personalized predictions for health outcomes under several treatment options and encompasses all relevant randomized and non-randomized evidence.

Pharmacokinetic analysis across studies to drive knowledge-integration: A tutorial on individual patient data meta-analysis (IPDMA)

Answering challenging questions in drug development sometimes requires pharmacokinetic (PK) data analysis across different studies, for example, to characterize PKs across diverse regions or populations, or to increase statistical power for subpopulations by combining smaller size trials. Given the growing interest in data sharing and advanced computational methods, knowledge integration based on multiple data sources is increasingly applied in the context of model-informed drug discovery and development. A powerful analysis method is the individual patient data meta-analysis (IPDMA), leveraging systematic review of databases and literature, with the most detailed data type of the individual patient, and quantitative modeling of the PK processes, including capturing heterogeneity of variance between studies. The methodology that should be used in IPDMA in the context of population PK analysis is summarized in this tutorial, highlighting areas of special attention compared to standard PK modeling, including hierarchical nested variability terms for interstudy variability, and handling between-assay differences in limits of quantification within a single analysis. This tutorial is intended for any pharmacological modeler who is interested in performing an integrated analysis of PK data across different studies in a systematic and thorough manner, to answer questions that transcend individual primary studies.

Multi-domain prognostic models used in middle-aged adults without known cognitive impairment for predicting subsequent dementia

BACKGROUND

Dementia, a global health priority, has no current cure. Around 50 million people worldwide currently live with dementia, and this number is expected to treble by 2050. Some health conditions and lifestyle behaviours can increase or decrease the risk of dementia and are known as 'predictors'. Prognostic models combine such predictors to measure the risk of future dementia. Models that can accurately predict future dementia would help clinicians select high-risk adults in middle age and implement targeted risk reduction.

OBJECTIVES

Our primary objective was to identify multi-domain prognostic models used in middle-aged adults (aged 45 to 65 years) for predicting dementia or cognitive impairment. Eligible multi-domain prognostic models involved two or more of the modifiable dementia predictors identified in a 2020 Lancet Commission report and a 2019 World Health Organization (WHO) report (less education, hearing loss, traumatic brain injury, hypertension, excessive alcohol intake, obesity, smoking, depression, social isolation, physical inactivity, diabetes mellitus, air pollution, poor diet, and cognitive inactivity). Our secondary objectives were to summarise the prognostic models, to appraise their predictive accuracy (discrimination and calibration) as reported in the development and validation studies, and to identify the implications of using dementia prognostic models for the management of people at a higher risk for future dementia.

SEARCH METHODS

We searched MEDLINE, Embase, PsycINFO, CINAHL, and ISI Web of Science Core Collection from inception until 6 June 2022. We performed forwards and backwards citation tracking of included studies using the Web of Science platform.  SELECTION CRITERIA: We included development and validation studies of multi-domain prognostic models. The minimum eligible follow-up was five years. Our primary outcome was an incident clinical diagnosis of dementia based on validated diagnostic criteria, and our secondary outcome was dementia or cognitive impairment determined by any other method.

DATA COLLECTION AND ANALYSIS

Two review authors independently screened the references, extracted data using a template based on the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modelling Studies (CHARMS), and assessed risk of bias and applicability of included studies using the Prediction model Risk Of Bias ASsessment Tool (PROBAST). We synthesised the C-statistics of models that had been externally validated in at least three comparable studies.  MAIN RESULTS: We identified 20 eligible studies; eight were development studies and 12 were validation studies. There were 14 unique prognostic models: seven models with validation studies and seven models with development-only studies. The models included a median of nine predictors (range 6 to 34); the median number of modifiable predictors was five (range 2 to 11). The most common modifiable predictors in externally validated models were diabetes, hypertension, smoking, physical activity, and obesity. In development-only models, the most common modifiable predictors were obesity, diabetes, hypertension, and smoking. No models included hearing loss or air pollution as predictors. Nineteen studies had a high risk of bias according to the PROBAST assessment, mainly because of inappropriate analysis methods, particularly lack of reported calibration measures. Applicability concerns were low for 12 studies, as their population, predictors, and outcomes were consistent with those of interest for this review. Applicability concerns were high for nine studies, as they lacked baseline cognitive screening or excluded an age group within the range of 45 to 65 years. Only one model, Cardiovascular Risk Factors, Ageing, and Dementia (CAIDE), had been externally validated in multiple studies, allowing for meta-analysis. The CAIDE model included eight predictors (four modifiable predictors): age, education, sex, systolic blood pressure, body mass index (BMI), total cholesterol, physical activity and APOEƐ4 status. Overall, our confidence in the prediction accuracy of CAIDE was very low; our main reasons for downgrading the certainty of the evidence were high risk of bias across all the studies, high concern of applicability, non-overlapping confidence intervals (CIs), and a high degree of heterogeneity. The summary C-statistic was 0.71 (95% CI 0.66 to 0.76; 3 studies; very low-certainty evidence) for the incident clinical diagnosis of dementia, and 0.67 (95% CI 0.61 to 0.73; 3 studies; very low-certainty evidence) for dementia or cognitive impairment based on cognitive scores. Meta-analysis of calibration measures was not possible, as few studies provided these data.

AUTHORS' CONCLUSIONS

We identified 14 unique multi-domain prognostic models used in middle-aged adults for predicting subsequent dementia. Diabetes, hypertension, obesity, and smoking were the most common modifiable risk factors used as predictors in the models. We performed meta-analyses of C-statistics for one model (CAIDE), but the summary values were unreliable. Owing to lack of data, we were unable to meta-analyse the calibration measures of CAIDE. This review highlights the need for further robust external validations of multi-domain prognostic models for predicting future risk of dementia in middle-aged adults.

Linear mixed models for investigating effect modification in subgroup meta-analysis

Subgroup meta-analysis can be used for comparing treatment effects between subgroups using information from multiple trials. If the effect of treatment is differential depending on subgroup, the results could enable personalization of the treatment. We propose using linear mixed models for estimating treatment effect modification in aggregate data meta-analysis. The linear mixed models capture existing subgroup meta-analysis methods while allowing for additional features such as flexibility in modeling heterogeneity, handling studies with missing subgroups and more. Reviews and simulation studies of the best suited models for estimating possible differential effect of treatment depending on subgroups have been studied mostly within individual participant data meta-analysis. While individual participant data meta-analysis in general is recommended over aggregate data meta-analysis, conducting an aggregate data subgroup meta-analysis could be valuable for exploring treatment effect modifiers before committing to an individual participant data subgroup meta-analysis. Additionally, using solely individual participant data for subgroup meta-analysis requires collecting sufficient individual participant data which may not always be possible. In this article, we compared existing methods with linear mixed models for aggregate data subgroup meta-analysis under a broad selection of scenarios using simulation and two case studies. Both the case studies and simulation studies presented here demonstrate the advantages of the linear mixed model approach in aggregate data subgroup meta-analysis.

Systolic Blood Pressure and Cardiovascular Risk in Patients With Diabetes: A Prospective Cohort Study

BACKGROUND

Whether the association between systolic blood pressure (SBP) and risk of cardiovascular disease is monotonic or whether there is a nadir of optimal blood pressure remains controversial. We investigated the association between SBP and cardiovascular events in patients with diabetes across the full spectrum of SBP.

METHODS

A cohort of 49 000 individuals with diabetes aged 50 to 90 years between 1990 and 2005 was identified from linked electronic health records in the United Kingdom. Associations between SBP and cardiovascular outcomes (ischemic heart disease, heart failure, stroke, and cardiovascular death) were analyzed using a deep learning approach.

RESULTS

Over a median follow-up of 7.3 years, 16 378 cardiovascular events were observed. The relationship between SBP and cardiovascular events followed a monotonic pattern, with the group with the lowest baseline SBP of <120 mm Hg exhibiting the lowest risk of cardiovascular events. In comparison to the reference group with the lowest SBP (<120 mm Hg), the adjusted risk ratio for cardiovascular disease was 1.03 (95% CI, 0.97-1.10) for SBP between 120 and 129 mm Hg, 1.05 (0.99-1.11) for SBP between 130 and 139 mm Hg, 1.08 (1.01-1.15) for SBP between 140 and 149 mm Hg, 1.12 (1.03-1.20) for SBP between 150 and 159 mm Hg, and 1.19 (1.09-1.28) for SBP ≥160 mm Hg.

CONCLUSIONS

Using deep learning modeling, we found a monotonic relationship between SBP and risk of cardiovascular outcomes in patients with diabetes, without evidence of a J-shaped relationship.

A hierarchical meta-analysis for settings involving multiple outcomes across multiple cohorts

Evidence from animal models and epidemiological studies has linked prenatal alcohol exposure (PAE) to a broad range of long-term cognitive and behavioural deficits. However, there is a paucity of evidence regarding the nature and levels of PAE associated with increased risk of clinically significant cognitive deficits. To derive robust and efficient estimates of the effects of PAE on cognitive function, we have developed a hierarchical meta-analysis approach to synthesize information regarding the effects of PAE on cognition, integrating data on multiple outcomes from six U.S. Iongitudinal cohort studies. A key assumption of standard methods of meta-analysis, effect sizes are independent, is violated when multiple intercorrelated outcomes are synthesized across studies. Our approach involves estimating the dose-response coefficients for each outcome and then pooling these correlated dose-response coefficients to obtain an estimated "global" effect of exposure on cognition. In the first stage, we use individual participant data to derive estimates of the effects of PAE by fitting regression models that adjust for potential confounding variables using propensity scores. The correlation matrix characterizing the dependence between the outcome-specific dose-response coefficients estimated within each cohort is then run, while accommodating incomplete information on some outcome. We also compare inferences based on the proposed approach to inferences based on a full multivariate analysis.

Blood pressure-lowering treatment for prevention of major cardiovascular diseases in people with and without type 2 diabetes: an individual participant-level data meta-analysis

BACKGROUND

Controversy exists as to whether the threshold for blood pressure-lowering treatment should differ between people with and without type 2 diabetes. We aimed to investigate the effects of blood pressure-lowering treatment on the risk of major cardiovascular events by type 2 diabetes status, as well as by baseline levels of systolic blood pressure.

METHODS

We conducted a one-stage individual participant-level data meta-analysis of major randomised controlled trials using the Blood Pressure Lowering Treatment Trialists' Collaboration dataset. Trials with information on type 2 diabetes status at baseline were eligible if they compared blood pressure-lowering medications versus placebo or other classes of blood pressure-lowering medications, or an intensive versus a standard blood pressure-lowering strategy, and reported at least 1000 persons-years of follow-up in each group. Trials exclusively on participants with heart failure or with short-term therapies and acute myocardial infarction or other acute settings were excluded. We expressed treatment effect per 5 mm Hg reduction in systolic blood pressure on the risk of developing a major cardiovascular event as the primary outcome, defined as the first occurrence of fatal or non-fatal stroke or cerebrovascular disease, fatal or non-fatal ischaemic heart disease, or heart failure causing death or requiring hospitalisation. Cox proportional hazard models, stratified by trial, were used to estimate hazard ratios (HRs) separately by type 2 diabetes status at baseline, with further stratification by baseline categories of systolic blood pressure (in 10 mm Hg increments from <120 mm Hg to ≥170 mm Hg). To estimate absolute risk reductions, we used a Poisson regression model over the follow-up duration. The effect of each of the five major blood pressure-lowering drug classes, including angiotensin-converting enzyme inhibitors, angiotensin II receptor blockers, β blockers, calcium channel blockers, and thiazide diuretics, was estimated using a network meta-analysis framework. This study is registered with PROSPERO, CRD42018099283.

FINDINGS

We included data from 51 randomised clinical trials published between 1981 and 2014 involving 358 533 participants (58% men), among whom 103 325 (29%) had known type 2 diabetes at baseline. The baseline mean systolic/diastolic blood pressure of those with and without type 2 diabetes was 149/84 mm Hg (SD 19/11) and 153/88 mm Hg (SD 21/12), respectively. Over 4·2 years median follow-up (IQR 3·0-5·0), a 5 mm Hg reduction in systolic blood pressure decreased the risk of major cardiovascular events in both groups, but with a weaker relative treatment effect in participants with type 2 diabetes (HR 0·94 [95% CI 0·91-0·98]) compared with those without type 2 diabetes (0·89 [0·87-0·92]; p_interaction=0·0013). However, absolute risk reductions did not differ substantially between people with and without type 2 diabetes because of the higher absolute cardiovascular risk among participants with type 2 diabetes. We found no reliable evidence for heterogeneity of treatment effects by baseline systolic blood pressure in either group. In keeping with the primary findings, analysis using stratified network meta-analysis showed no evidence that relative treatment effects differed substantially between participants with type 2 diabetes and those without for any of the drug classes investigated.

INTERPRETATION

Although the relative beneficial effects of blood pressure reduction on major cardiovascular events were weaker in participants with type 2 diabetes than in those without, absolute effects were similar. The difference in relative risk reduction was not related to the baseline blood pressure or allocation to different drug classes. Therefore, the adoption of differential blood pressure thresholds, intensities of blood pressure lowering, or drug classes used in people with and without type 2 diabetes is not warranted.

FUNDING

British Heart Foundation, UK National Institute for Health Research, and Oxford Martin School.

Bayesian network meta-analysis methods for combining individual participant data and aggregate data from single arm trials and randomised controlled trials

BACKGROUND

Increasingly in network meta-analysis (NMA), there is a need to incorporate non-randomised evidence to estimate relative treatment effects, and in particular in cases with limited randomised evidence, sometimes resulting in disconnected networks of treatments. When combining different sources of data, complex NMA methods are required to address issues associated with participant selection bias, incorporating single-arm trials (SATs), and synthesising a mixture of individual participant data (IPD) and aggregate data (AD). We develop NMA methods which synthesise data from SATs and randomised controlled trials (RCTs), using a mixture of IPD and AD, for a dichotomous outcome.

METHODS

We propose methods under both contrast-based (CB) and arm-based (AB) parametrisations, and extend the methods to allow for both within- and across-trial adjustments for covariate effects. To illustrate the methods, we use an applied example investigating the effectiveness of biologic disease-modifying anti-rheumatic drugs for rheumatoid arthritis (RA). We applied the methods to a dataset obtained from a literature review consisting of 14 RCTs and an artificial dataset consisting of IPD from two SATs and AD from 12 RCTs, where the artificial dataset was created by removing the control arms from the only two trials assessing tocilizumab in the original dataset.

RESULTS

Without adjustment for covariates, the CB method with independent baseline response parameters (CBunadjInd) underestimated the effectiveness of tocilizumab when applied to the artificial dataset compared to the original dataset, albeit with significant overlap in posterior distributions for treatment effect parameters. The CB method with exchangeable baseline response parameters produced effectiveness estimates in agreement with CBunadjInd, when the predicted baseline response estimates were similar to the observed baseline response. After adjustment for RA duration, there was a reduction in across-trial heterogeneity in baseline response but little change in treatment effect estimates.

CONCLUSIONS

Our findings suggest incorporating SATs in NMA may be useful in some situations where a treatment is disconnected from a network of comparator treatments, due to a lack of comparative evidence, to estimate relative treatment effects. The reliability of effect estimates based on data from SATs may depend on adjustment for covariate effects, although further research is required to understand this in more detail.

An evidence synthesis approach for combining different data sources illustrated using entomological efficacy of insecticides for indoor residual spraying

BACKGROUND

Prospective malaria public health interventions are initially tested for entomological impact using standardised experimental hut trials. In some cases, data are collated as aggregated counts of potential outcomes from mosquito feeding attempts given the presence of an insecticidal intervention. Comprehensive data i.e. full breakdowns of probable outcomes of mosquito feeding attempts, are more rarely available. Bayesian evidence synthesis is a framework that explicitly combines data sources to enable the joint estimation of parameters and their uncertainties. The aggregated and comprehensive data can be combined using an evidence synthesis approach to enhance our inference about the potential impact of vector control products across different settings over time.

METHODS

Aggregated and comprehensive data from a meta-analysis of the impact of Pirimiphos-methyl, an indoor residual spray (IRS) product active ingredient, used on wall surfaces to kill mosquitoes and reduce malaria transmission, were analysed using a series of statistical models to understand the benefits and limitations of each.

RESULTS

Many more data are available in aggregated format (N = 23 datasets, 4 studies) relative to comprehensive format (N = 2 datasets, 1 study). The evidence synthesis model had the smallest uncertainty at predicting the probability of mosquitoes dying or surviving and blood-feeding. Generating odds ratios from the correlated Bernoulli random sample indicates that when mortality and blood-feeding are positively correlated, as exhibited in our data, the number of successfully fed mosquitoes will be under-estimated. Analysis of either dataset alone is problematic because aggregated data require an assumption of independence and there are few and variable data in the comprehensive format.

CONCLUSIONS

We developed an approach to combine sources from trials to maximise the inference that can be made from such data and that is applicable to other systems. Bayesian evidence synthesis enables inference from multiple datasets simultaneously to give a more informative result and highlight conflicts between sources. Advantages and limitations of these models are discussed.

Protocol for individual participant data meta-analysis of interventions for post-traumatic stress

INTRODUCTION

Several evidence-based treatments are effective for post-traumatic stress disorder (PTSD), yet a substantial proportion of patients do not respond or dropout of treatment. We describe the protocol for a systematic review and individual participant data meta-analysis (IPD-MA) aimed at assessing the effectiveness and adverse effects of psychotherapy and pharmacotherapy interventions for treating PTSD. Additionally, we seek to examine moderators and predictors of treatment outcomes.

METHOD AND ANALYSIS

This IPD-MA includes randomised controlled trials comparing psychotherapy and pharmacotherapy interventions for PTSD. PubMed, Embase, PsycINFO, PTSDpubs and CENTRAL will be screened up till the 11th of January 2021. The target population is adults with above-threshold baseline PTSD symptoms on any standardised self-report measure. Trials will only be eligible if at least 70% of the study sample have been diagnosed with PTSD by means of a structured clinical interview. The primary outcomes of this IPD-MA are PTSD symptom severity, and response rate. Secondary outcomes include treatment dropout and adverse effects. Two independent reviewers will screen major bibliographic databases and past reviews. Authors will be contacted to contribute their participant-level datasets. Datasets will be merged into a master dataset. A one-stage IPD-MA will be conducted focusing on the effects of psychological and pharmacological interventions on PTSD symptom severity, response rate, treatment dropout and adverse effects. Subsequent analyses will focus on examining the effect of moderators and predictors of treatment outcomes. These will include sociodemographic, treatment-related, symptom-related, resilience, intervention, trauma and combat-related characteristics. By determining the individual factors that influence the effectiveness of specific PTSD treatments, we will gain insight into personalised treatment options for PTSD.

ETHICS AND DISSEMINATION

Specific ethics approval for an IPD-MA is not required as this study entails secondary analysis of existing anonymised data. The results of this study will be published in peer-reviewed scientific journals and presentations.

Transforming Obesity Prevention for CHILDren (TOPCHILD) Collaboration: protocol for a systematic review with individual participant data meta-analysis of behavioural interventions for the prevention of early childhood obesity

INTRODUCTION

Behavioural interventions in early life appear to show some effect in reducing childhood overweight and obesity. However, uncertainty remains regarding their overall effectiveness, and whether effectiveness differs among key subgroups. These evidence gaps have prompted an increase in very early childhood obesity prevention trials worldwide. Combining the individual participant data (IPD) from these trials will enhance statistical power to determine overall effectiveness and enable examination of individual and trial-level subgroups. We present a protocol for a systematic review with IPD meta-analysis to evaluate the effectiveness of obesity prevention interventions commencing antenatally or in the first year after birth, and to explore whether there are differential effects among key subgroups.

METHODS AND ANALYSIS

Systematic searches of Medline, Embase, Cochrane Central Register of Controlled Trials, Cumulative Index to Nursing and Allied Health Literature (CINAHL), PsycInfo and trial registries for all ongoing and completed randomised controlled trials evaluating behavioural interventions for the prevention of early childhood obesity have been completed up to March 2021 and will be updated annually to include additional trials. Eligible trialists will be asked to share their IPD; if unavailable, aggregate data will be used where possible. An IPD meta-analysis and a nested prospective meta-analysis will be performed using methodologies recommended by the Cochrane Collaboration. The primary outcome will be body mass index z-score at age 24±6 months using WHO Growth Standards, and effect differences will be explored among prespecified individual and trial-level subgroups. Secondary outcomes include other child weight-related measures, infant feeding, dietary intake, physical activity, sedentary behaviours, sleep, parenting measures and adverse events.

ETHICS AND DISSEMINATION

Approved by The University of Sydney Human Research Ethics Committee (2020/273) and Flinders University Social and Behavioural Research Ethics Committee (HREC CIA2133-1). Results will be relevant to clinicians, child health services, researchers, policy-makers and families, and will be disseminated via publications, presentations and media releases.

PROSPERO REGISTRATION NUMBER

CRD42020177408.

Efficient integration of aggregate data and individual participant data in one-way mixed models

Often both aggregate data (AD) studies and individual participant data (IPD) studies are available for specific treatments. Combining these two sources of data could improve the overall meta-analytic estimates of treatment effects. Moreover, often for some studies with AD, the associated IPD maybe available, albeit at some extra effort or cost to the analyst. We propose a method for combining treatment effects across trials when the response is from the exponential family of distribution and hence a generalized linear model structure can be used. We consider the case when treatment effects are fixed and common across studies. Using the proposed combination method, we study the relative efficiency of analyzing all IPD studies vs combining various percentages of AD and IPD studies. For many different models, design constraints under which the AD estimators are the IPD estimators, and hence fully efficient, are known. For such models, we advocate a selection procedure that chooses AD studies over IPD studies in a manner that force least departure from design constraints and hence ensures an efficient combined AD and IPD estimator.

Individual Patient Data Meta-Analysis and Network Meta-Analysis

Meta-analyses are often conducted using trial-level summary data. However, when individual patient data (IPD ) is available, there is greater flexibility in the analysis and a wider range of statistical models that can be fitted. There are two approaches to fitting IPD models. The traditional two-stage approach involves analyzing each trial individually in the first stage and then combining trial estimates of treatment effectiveness in the second stage using methods developed for aggregate data meta-analysis. Growing in popularity is the one-stage approach in which trials are analyzed and synthesized within one statistical model whilst the clustering of patients within trials is accounted for. This chapter outlines both fixed effect and random effects one- and two-stage meta-analysis models for continuous, binary, and time-to-event outcomes. The meta-analysis framework is then extended to the scenario where there are more than two treatments and network meta-analysis models are described.The availability of IPD provides greater statistical power for investigating interactions between treatments and covariates. Treatment-covariate interactions contain both within- and across-trial information where the across-trial information may be subject to ecological bias. This chapter presents network meta-analysis models separating out the within- and across-trial information and finishes by considering practical solutions for dealing with missing covariate data, assessing the consistency assumption, combining IPD and aggregate data and specific considerations for time-to-event outcomes.

Practical Considerations and Challenges When Conducting an Individual Participant Data (IPD) Meta-Analysis

This chapter provides a broad overview of the use of individual participant (sometimes referred to as patient) data (IPD ) within meta-analyses, the associated advantages of using IPD in meta-analysis compared to aggregate data, and when IPD should be used in meta-analysis.This chapter also outlines the steps of conducting an IPD meta-analysis, with practical guidance relating to requesting and obtaining IPD for meta-analysis. Challenges that can be associated with conducting an IPD meta-analysis are also discussed, including consideration of availability bias, when a subset of the relevant IPD is not available for meta-analysis.

Bayesian Methods for Meta-Analyses of Binary Outcomes: Implementations, Examples, and Impact of Priors

Bayesian methods are an important set of tools for performing meta-analyses. They avoid some potentially unrealistic assumptions that are required by conventional frequentist methods. More importantly, meta-analysts can incorporate prior information from many sources, including experts’ opinions and prior meta-analyses. Nevertheless, Bayesian methods are used less frequently than conventional frequentist methods, primarily because of the need for nontrivial statistical coding, while frequentist approaches can be implemented via many user-friendly software packages. This article aims at providing a practical review of implementations for Bayesian meta-analyses with various prior distributions. We present Bayesian methods for meta-analyses with the focus on odds ratio for binary outcomes. We summarize various commonly used prior distribution choices for the between-studies heterogeneity variance, a critical parameter in meta-analyses. They include the inverse-gamma, uniform, and half-normal distributions, as well as evidence-based informative log-normal priors. Five real-world examples are presented to illustrate their performance. We provide all of the statistical code for future use by practitioners. Under certain circumstances, Bayesian methods can produce markedly different results from those by frequentist methods, including a change in decision on statistical significance. When data information is limited, the choice of priors may have a large impact on meta-analytic results, in which case sensitivity analyses are recommended. Moreover, the algorithm for implementing Bayesian analyses may not converge for extremely sparse data; caution is needed in interpreting respective results. As such, convergence should be routinely examined. When select statistical assumptions that are made by conventional frequentist methods are violated, Bayesian methods provide a reliable alternative to perform a meta-analysis.

A review of the quantitative effectiveness evidence synthesis methods used in public health intervention guidelines

BACKGROUND

The complexity of public health interventions create challenges in evaluating their effectiveness. There have been huge advancements in quantitative evidence synthesis methods development (including meta-analysis) for dealing with heterogeneity of intervention effects, inappropriate 'lumping' of interventions, adjusting for different populations and outcomes and the inclusion of various study types. Growing awareness of the importance of using all available evidence has led to the publication of guidance documents for implementing methods to improve decision making by answering policy relevant questions.

METHODS

The first part of this paper reviews the methods used to synthesise quantitative effectiveness evidence in public health guidelines by the National Institute for Health and Care Excellence (NICE) that had been published or updated since the previous review in 2012 until the 19th August 2019.The second part of this paper provides an update of the statistical methods and explains how they address issues related to evaluating effectiveness evidence of public health interventions.

RESULTS

The proportion of NICE public health guidelines that used a meta-analysis as part of the synthesis of effectiveness evidence has increased since the previous review in 2012 from 23% (9 out of 39) to 31% (14 out of 45). The proportion of NICE guidelines that synthesised the evidence using only a narrative review decreased from 74% (29 out of 39) to 60% (27 out of 45).An application in the prevention of accidents in children at home illustrated how the choice of synthesis methods can enable more informed decision making by defining and estimating the effectiveness of more distinct interventions, including combinations of intervention components, and identifying subgroups in which interventions are most effective.

CONCLUSIONS

Despite methodology development and the publication of guidance documents to address issues in public health intervention evaluation since the original review, NICE public health guidelines are not making full use of meta-analysis and other tools that would provide decision makers with fuller information with which to develop policy. There is an evident need to facilitate the translation of the synthesis methods into a public health context and encourage the use of methods to improve decision making.

A brief introduction of meta-analyses in clinical practice and research

With the explosive growth of medical information, it is almost impossible for healthcare providers to review and evaluate all relevant evidence to make the best clinical decisions. Meta‐analyses, which summarize all existing evidence and quantitatively synthesize individual studies, have become the best available evidence for informing clinical practice. This article introduces the common methods, steps, principles, strengths and limitations of meta‐analyses and aims to help healthcare providers and researchers obtain a basic understanding of meta‐analyses in clinical practice and research.

Assessing the performance of population adjustment methods for anchored indirect comparisons: A simulation study

Standard network meta-analysis and indirect comparisons combine aggregate data from multiple studies on treatments of interest, assuming that any factors that interact with treatment effects (effect modifiers) are balanced across populations. Population adjustment methods such as multilevel network meta-regression (ML-NMR), matching-adjusted indirect comparison (MAIC), and simulated treatment comparison (STC) relax this assumption using individual patient data from one or more studies, and are becoming increasingly prevalent in health technology appraisals and the applied literature. Motivated by an applied example and two recent reviews of applications, we undertook an extensive simulation study to assess the performance of these methods in a range of scenarios under various failures of assumptions. We investigated the impact of varying sample size, missing effect modifiers, strength of effect modification and validity of the shared effect modifier assumption, validity of extrapolation and varying between-study overlap, and different covariate distributions and correlations. ML-NMR and STC performed similarly, eliminating bias when the requisite assumptions were met. Serious concerns are raised for MAIC, which performed poorly in nearly all simulation scenarios and may even increase bias compared with standard indirect comparisons. All methods incur bias when an effect modifier is missing, highlighting the necessity of careful selection of potential effect modifiers prior to analysis. When all effect modifiers are included, ML-NMR and STC are robust techniques for population adjustment. ML-NMR offers additional advantages over MAIC and STC, including extending to larger treatment networks and producing estimates in any target population, making this an attractive choice in a variety of scenarios.

Effectiveness of Internet- and Mobile-Based Cognitive Behavioral Therapy to Reduce Suicidal Ideation and Behaviors: Protocol for a Systematic Review and Meta-Analysis of Individual Participant Data

Internet- and mobile-based cognitive behavioral therapy (iCBT) might reduce suicidal ideation. However, recent meta-analyses found small effect sizes, and it remains unclear whether specific subgroups of participants experience beneficial or harmful effects. This is the study protocol for an individual participant meta-analysis (IPD-MA) aiming to determine the effectiveness of iCBT on suicidal ideation and identify moderators. We will systematically search CENTRAL, PsycINFO, Embase, and Pubmed for randomized controlled trials examining guided or self-guided iCBT for suicidality. All types of control conditions are eligible. Participants experiencing suicidal ideation will be included irrespective of age, diagnoses, or co-interventions. We will conduct a one-stage IPD-MA with suicidal ideation as the primary outcome, using a continuous measure, reliable improvement and deterioration, and response rate. Moderator analyses will be performed on participant-, study-, and intervention-level. Two independent reviewers will assess risk of bias and the quality of evidence using Cochrane's Risk of Bias Tool 2 and GRADE. This review was registered with OSF and is currently in progress. The IPD-MA will provide effect estimates while considering covariates and will offer novel insights into differential effects on a participant level. This will help to develop more effective, safe, and tailored digital treatment options for suicidal individuals.

The questionable efficacy of manualized psychological treatments for distressed breast cancer patients: An individual patient data meta-analysis

Previous meta-analyses conclude that psychological treatments are efficacious for emotional distress in breast cancer (BCa). However, the practical relevance of these meta-analyses is questionable; none focused specifically on clinically distressed patients or whether treatment effects were clinically significant. In a two-stage individual patient data (IPD) meta-analysis of 17 randomized controlled trials of manualized psychological treatments in BCa, we evaluated treatment efficacy in distressed BCa patients (n = 1591) using clinical significance and effect size analyses. Outcomes were anxiety, depression, and general distress, evaluated at post-treatment and follow-up. Moderators examined were treatment type, treatment format, therapists' profession, control condition, age, outcome measure, and trial quality. Treated patients were more likely than controls to recover from anxiety and general distress at post-treatment (14-15% more treated patients recovered), but not at mean 8-months follow-up. Overall recovery rates were low: across outcomes, at post-treatment, only 30-32% of treated patients and 15-25% of controls recovered; at follow-up, only 21-30% of treated patients and 18-35% of controls recovered. Small between-group effect sizes in favour of treatment were found across outcomes at post-treatment (g = 0.32-0.34) but not at follow-up. Across the different analysis methods, few moderator effects were found. More efficacious psychological treatments are needed for distressed BCa patients.

Individual participant data meta-analysis to examine interactions between treatment effect and participant-level covariates: Statistical recommendations for conduct and planning

Precision medicine research often searches for treatment‐covariate interactions, which refers to when a treatment effect (eg, measured as a mean difference, odds ratio, hazard ratio) changes across values of a participant‐level covariate (eg, age, gender, biomarker). Single trials do not usually have sufficient power to detect genuine treatment‐covariate interactions, which motivate the sharing of individual participant data (IPD) from multiple trials for meta‐analysis. Here, we provide statistical recommendations for conducting and planning an IPD meta‐analysis of randomized trials to examine treatment‐covariate interactions. For conduct, two‐stage and one‐stage statistical models are described, and we recommend: (i) interactions should be estimated directly, and not by calculating differences in meta‐analysis results for subgroups; (ii) interaction estimates should be based solely on within‐study information; (iii) continuous covariates and outcomes should be analyzed on their continuous scale; (iv) nonlinear relationships should be examined for continuous covariates, using a multivariate meta‐analysis of the trend (eg, using restricted cubic spline functions); and (v) translation of interactions into clinical practice is nontrivial, requiring individualized treatment effect prediction. For planning, we describe first why the decision to initiate an IPD meta‐analysis project should not be based on between‐study heterogeneity in the overall treatment effect; and second, how to calculate the power of a potential IPD meta‐analysis project in advance of IPD collection, conditional on characteristics (eg, number of participants, standard deviation of covariates) of the trials (potentially) promising their IPD. Real IPD meta‐analysis projects are used for illustration throughout.

Systematic review and network meta-analysis with individual participant data on cord management at preterm birth (iCOMP): study protocol

INTRODUCTION

Timing of cord clamping and other cord management strategies may improve outcomes at preterm birth. However, it is unclear whether benefits apply to all preterm subgroups. Previous and current trials compare various policies, including time-based or physiology-based deferred cord clamping, and cord milking. Individual participant data (IPD) enable exploration of different strategies within subgroups. Network meta-analysis (NMA) enables comparison and ranking of all available interventions using a combination of direct and indirect comparisons.

OBJECTIVES

(1) To evaluate the effectiveness of cord management strategies for preterm infants on neonatal mortality and morbidity overall and for different participant characteristics using IPD meta-analysis. (2) To evaluate and rank the effect of different cord management strategies for preterm births on mortality and other key outcomes using NMA.

METHODS AND ANALYSIS

Systematic searches of Medline, Embase, clinical trial registries, and other sources for all ongoing and completed randomised controlled trials comparing cord management strategies at preterm birth (before 37 weeks' gestation) have been completed up to 13 February 2019, but will be updated regularly to include additional trials. IPD will be sought for all trials; aggregate summary data will be included where IPD are unavailable. First, deferred clamping and cord milking will be compared with immediate clamping in pairwise IPD meta-analyses. The primary outcome will be death prior to hospital discharge. Effect differences will be explored for prespecified participant subgroups. Second, all identified cord management strategies will be compared and ranked in an IPD NMA for the primary outcome and the key secondary outcomes. Treatment effect differences by participant characteristics will be identified. Inconsistency and heterogeneity will be explored.

ETHICS AND DISSEMINATION

Ethics approval for this project has been granted by the University of Sydney Human Research Ethics Committee (2018/886). Results will be relevant to clinicians, guideline developers and policy-makers, and will be disseminated via publications, presentations and media releases.

REGISTRATION NUMBER

Australian New Zealand Clinical Trials Registry (ANZCTR) (ACTRN12619001305112) and International Prospective Register of Systematic Reviews (PROSPERO, CRD42019136640).

Individual participant data meta-analysis of intervention studies with time-to-event outcomes: A review of the methodology and an applied example

Many randomized trials evaluate an intervention effect on time-to-event outcomes. Individual participant data (IPD) from such trials can be obtained and combined in a so-called IPD meta-analysis (IPD-MA), to summarize the overall intervention effect. We performed a narrative literature review to provide an overview of methods for conducting an IPD-MA of randomized intervention studies with a time-to-event outcome. We focused on identifying good methodological practice for modeling frailty of trial participants across trials, modeling heterogeneity of intervention effects, choosing appropriate association measures, dealing with (trial differences in) censoring and follow-up times, and addressing time-varying intervention effects and effect modification (interactions).We discuss how to achieve this using parametric and semi-parametric methods, and describe how to implement these in a one-stage or two-stage IPD-MA framework. We recommend exploring heterogeneity of the effect(s) through interaction and non-linear effects. Random effects should be applied to account for residual heterogeneity of the intervention effect. We provide further recommendations, many of which specific to IPD-MA of time-to-event data from randomized trials examining an intervention effect.We illustrate several key methods in a real IPD-MA, where IPD of 1225 participants from 5 randomized clinical trials were combined to compare the effects of Carbamazepine and Valproate on the incidence of epileptic seizures.

Individual participant data (IPD) meta-analysis of psychological relapse prevention interventions versus control for patients in remission from depression: a protocol

INTRODUCTION

Psychological interventions and antidepressant medication can be effective interventions to prevent depressive relapse for patients currently in remission of depression. Less is known about overall factors that predict or moderate treatment response for patients receiving a psychological intervention for recurrent depression. This is a protocol for an individual participant data (IPD) meta-analysis which aims to assess predictors and moderators of relapse or recurrence for patients currently in remission from depression.

METHODS AND ANALYSIS

Searches of PubMed, PsycINFO, Embase and Cochrane Central Register of Controlled Trials were completed on 13 October 2019. Study extractions and risk of bias assessments have been completed. Study authors will be asked to contribute IPD. Standard aggregate meta-analysis and IPD analysis will be conducted, and the outcomes will be compared with assess whether results differ between studies supplying data and those that did not. IPD files of individual data will be merged and variables homogenised where possible for consistency. IPD will be analysed via Cox regression and one and two-stage analyses will be conducted.

ETHICS AND DISSEMINATION

The results will be published in peer review journals and shared in a policy briefing as well as accessible formats and shared with a range of stakeholders. The results will inform patients and clinicians and researchers about our current understanding of more personalised ways to prevent a depressive relapse. No local ethics approval was necessary following consultation with the legal department. Guidance on patient data storage and management will be adhered to.

PROSPERO REGISTRATION NUMBER

CRD42019127844.

Mass deworming for soil-transmitted helminths and schistosomiasis among pregnant women: A systematic review and individual participant data meta-analysis

The objective of the review is to use individual participant data (IPD) meta-analysis to explore the effect of mass deworming during pregnancy. We developed a search strategy and searched the databases till March 2018. We included individually randomised controlled trials; cluster randomised controlled trials and quasi randomised studies providing preventive or therapeutic deworming drugs for soil transmitted helminthiases and schistosomiasis during pregnancy. All IPD were assessed for completeness, compared to published reports and entered into a common data spreadsheet. Out of the seven trials elgible for IPD, we received data from three trials; out of 8,515 potential IPD participants; data were captured for 5,957 participants. Findings from this IPD suggest that mass deworming during pregnancy reduces maternal anaemia by 23% (Risk ratio [RR]: 0.77, 95% confidence intreval [CI]: 0.73-0.81; three trials; 5,216 participants; moderate quality evidence). We did not find any evidence of an effect of mass deworming during pregnancy on any of the other outcomes. There was no evidence of effect modification; however these findings should be interpreted with caution due to small sample sizes. The quality of evidence was rated as moderate for our findings. Our analyses suggest that mass deworming during pregnancy is associated with reducing anaemia with no evidence of impact on any other maternal or pregnancy outcomes. Our analyses were limited by the availability of data for the impact by subgroups and effect modification. There is also a need to support and promote open data for future IPDs.

Concussed athletes walk slower than non-concussed athletes during cognitive-motor dual-task assessments but not during single-task assessments 2 months after sports concussion: a systematic review and meta-analysis using individual participant data

Objectives

To determine whether individuals who sustained a sports concussion would exhibit persistent impairments in gait and quiet standing compared to non-injured controls during a dual-task assessment .

Design

Systematic review and meta-analysis using individual participant data (IPD).

Data sources

The search strategy was applied across seven electronic bibliographic and grey literature databases: MEDLINE, EMBASE, CINAHL, SportDISCUS, PsycINFO, PsycARTICLES and Web of Science, from database inception until June 2017.

Eligibility criteria for study selection

Studies were included if; individuals with a sports concussion and non-injured controls were included as participants; a steady-state walking or static postural balance task was used as the primary motor task; dual-task performance was assessed with the addition of a secondary cognitive task; spatiotemporal, kinematic or kinetic outcome variables were reported, and; included studies comprised an observational study design with case–control matching.

Data extraction and synthesis

Our review is reported in line with the Preferred Reporting Items for Systematic review and Meta-Analyses-IPD Statement. We implemented the Risk of Bias Assessment tool for Non-randomised Studies to undertake an outcome-level risk of bias assessment using a domain-based tool. Study-level data were synthesised in one of three tiers depending on the availability and quality of data: (1) homogeneous IPD; (2) heterogeneous IPD and (3) aggregate data for inclusion in a descriptive synthesis. IPD were aggregated using a ‘one-stage’, random-effects model.

Results

26 studies were included. IPD were available for 20 included studies. Consistently high and unclear risk of bias was identified for selection, detection, attrition, and reporting biases across studies. Individuals with a recent sports concussion walked with slower average walking speed (χ2=51.7; df=4; p<0.001; mean difference=0.06 m/s; 95% CI: 0.004 to 0.11) and greater frontal plane centre of mass displacement (χ2=10.3; df=4; p=0.036; mean difference −0.0039 m; 95% CI: −0.0075 to −0.0004) than controls when evaluated using a dual-task assessment up to 2 months following concussion.

Summary/conclusions

Our IPD evidence synthesis identifies that, when evaluated using a dual-task assessment, individuals who had incurred a sports concussion exhibited impairments in gait that persisted beyond reported standard clinical recovery timelines of 7–10 days. Dual-task assessment (with motion capture) may be a useful clinical assessment to evaluate recovery after sports concussion.

Protocol pre-registration

This systematic review was prospectively registered in PROSPERO CRD42017064861.

Evidence synthesis in prognosis research

Over the past few years, evidence synthesis has become essential to investigate and improve the generalizability of medical research findings. This strategy often involves a meta-analysis to formally summarize quantities of interest, such as relative treatment effect estimates. The use of meta-analysis methods is, however, less straightforward in prognosis research because substantial variation exists in research objectives, analysis methods and the level of reported evidence. We present a gentle overview of statistical methods that can be used to summarize data of prognostic factor and prognostic model studies. We discuss how aggregate data, individual participant data, or a combination thereof can be combined through meta-analysis methods. Recent examples are provided throughout to illustrate the various methods.

Adolescent Pregnancy Prevention: Meta-Analysis of Federally Funded Program Evaluations

BACKGROUND

Beginning in 2010, the US Department of Health and Human Services (HHS) funded more than 40 evaluations of adolescent pregnancy prevention interventions. The government's emphasis on rigor and transparency, along with a requirement that grantees collect standardized behavioral outcomes, ensured that findings could be meaningfully compared across evaluations.

OBJECTIVES

We used random and mixed-effects meta-analysis to analyze the findings generated by these evaluations to learn whether program elements, program implementation features, and participant demographics were associated with effects on adolescent sexual risk behavior.

SEARCH METHODS

We screened all 43 independent evaluation reports, some of which included multiple studies, funded by HHS and completed before October 1, 2016. HHS released, and our team considered, all such studies regardless of favorability or statistical significance.

SELECTION CRITERIA

Of these studies, we included those that used a randomized or high-quality quasi-experimental research design. We excluded studies that did not use statistical matching or provide pretest equivalence data on a measure of sexual behavior or a close proxy. We also excluded studies that compared 2 pregnancy prevention interventions without a control group. A total of 44 studies from 39 reports, comprising 51 150 youths, met the inclusion criteria.

DATA COLLECTION AND ANALYSIS

Two researchers extracted data from each study by using standard systematic reviewing and meta-analysis procedures. In addition, study authors provided individual participant data for a subset of 34 studies. We used mixed-effects meta-regressions with aggregate data to examine whether program or participant characteristics were associated with program effects on adolescent sexual risk behaviors and consequences. To examine whether individual-level participant characteristics such as age, gender, and race/ethnicity were associated with program effects, we used a 1-stage meta-regression approach combining participant-level data (48 635 youths) with aggregate data from the 10 studies for which participant-level data were not available.

MAIN RESULTS

Across all 44 studies, we found small but statistically insignificant mean effects favoring the programs and little variability around those means. Only 2 program characteristics showed statistically reliable relationships with program effects. First, gender-specific (girl-only) programs yielded a statistically significant average effect size (P < .05). Second, programs with individualized service delivery were more effective than programs delivering services to youths in small groups (P < .05). We found no other statistically significant associations between program effects and program or participant characteristics, or evaluation methods. Nor was there a statistically significant difference in the mean effect sizes for programs with previous evidence of effectiveness and previously untested programs.

CONCLUSIONS

Although several individual studies reported positive impacts, the average effects were small and there was minimal variation in effect sizes across studies on all of the outcomes assessed. Thus, we were unable to confidently identify which individual program characteristics were associated with effects. However, these studies examined relatively short-term effects and it is an open question whether some programs, perhaps with distinctive characteristics, will show longer-term effects as more of the adolescent participants become sexually active. Public Health Implications. The success of a small number of individualized interventions designed specifically for girls in changing behavioral outcomes suggests the need to reexamine the assumptions that underlie coed group approaches. However, given the almost total absence of similar programs targeting male adolescents, it is likely to be some time before evidence to support or reject such an approach for boys is available.

Conducting gene set tests in meta-analyses of transcriptome expression data

Research synthesis, eg, by meta-analysis, is more and more considered in the area of high-dimensional data from molecular research such as gene and protein expression data, especially because most studies and experiments are performed with very small sample sizes. In contrast to most clinical and epidemiological trials, raw data are often available for high-dimensional expression data. Therefore, direct data merging followed by a joint analysis of selected studies can be an alternative to meta-analysis by P value or effect-size merging or, more generally spoken, the merging of results. While several methods for meta-analysis of differential expression studies have been proposed, meta-analysis of gene set tests has very rarely been considered, although gene set tests are standard in the analysis of individual gene expression studies. We compare in this work the different strategies of research synthesis of gene set tests, in particularly the "early merging" of data cleaned from batch effects versus the "late merging" of individual results. In simulation studies and in examples of manipulated real-world data, we found that in most scenarios, the early merging has a higher sensitivity of detecting a gene set enrichment than the late merging. However, in scenarios with few studies, large batch effect, moderate and large sample sizes of late merging are more sensitive than early merging.

Assessing the consistency assumptions underlying network meta-regression using aggregate data

When numerous treatments exist for a disease (Treatments 1, 2, 3, etc), network meta‐regression (NMR) examines whether each relative treatment effect (eg, mean difference for 2 vs 1, 3 vs 1, and 3 vs 2) differs according to a covariate (eg, disease severity). Two consistency assumptions underlie NMR: consistency of the treatment effects at the covariate value 0 and consistency of the regression coefficients for the treatment by covariate interaction. The NMR results may be unreliable when the assumptions do not hold. Furthermore, interactions may exist but are not found because inconsistency of the coefficients is masking them, for example, when the treatment effect increases as the covariate increases using direct evidence but the effect decreases with the increasing covariate using indirect evidence.

We outline existing NMR models that incorporate different types of treatment by covariate interaction. We then introduce models that can be used to assess the consistency assumptions underlying NMR for aggregate data. We extend existing node‐splitting models, the unrelated mean effects inconsistency model, and the design by treatment inconsistency model to incorporate covariate interactions. We propose models for assessing both consistency assumptions simultaneously and models for assessing each of the assumptions in turn to gain a more thorough understanding of consistency.

We apply the methods in a Bayesian framework to trial‐level data comparing antimalarial treatments using the covariate average age and to four fabricated data sets to demonstrate key scenarios.

We discuss the pros and cons of the methods and important considerations when applying models to aggregated data.

Protease activity as a prognostic factor for wound healing in venous leg ulcers

BACKGROUND

Venous leg ulcers (VLUs) are a common type of complex wound that have a negative impact on people's lives and incur high costs for health services and society. It has been suggested that prolonged high levels of protease activity in the later stages of the healing of chronic wounds may be associated with delayed healing. Protease modulating treatments have been developed which seek to modulate protease activity and thereby promote healing in chronic wounds.

OBJECTIVES

To determine whether protease activity is an independent prognostic factor for the healing of venous leg ulcers.

SEARCH METHODS

In February 2018, we searched the following databases: Cochrane Central Register of Controlled Trials (CENTRAL), Ovid MEDLINE, Ovid Embase and CINAHL.

SELECTION CRITERIA

We included prospective and retrospective longitudinal studies with any follow-up period that recruited people with VLUs and investigated whether protease activity in wound fluid was associated with future healing of VLUs. We included randomised controlled trials (RCTs) analysed as cohort studies, provided interventions were taken into account in the analysis, and case-control studies if there were no available cohort studies. We also included prediction model studies provided they reported separately associations of individual prognostic factors (protease activity) with healing. Studies of any type of protease or combination of proteases were eligible, including proteases from bacteria, and the prognostic factor could be examined as a continuous or categorical variable; any cut-off point was permitted. The primary outcomes were time to healing (survival analysis) and the proportion of people with ulcers completely healed; the secondary outcome was change in ulcer size/rate of wound closure. We extracted unadjusted (simple) and adjusted (multivariable) associations between the prognostic factor and healing.

DATA COLLECTION AND ANALYSIS

Two review authors independently assessed studies for inclusion at each stage, and undertook data extraction, assessment of risk of bias and GRADE assessment. We collected association statistics where available. No study reported adjusted analyses: instead we collected unadjusted results or calculated association measures from raw data. We calculated risk ratios when both outcome and prognostic factor were dichotomous variables. When the prognostic factor was reported as continuous data and healing outcomes were dichotomous, we either performed regression analysis or analysed the impact of healing on protease levels, analysing as the standardised mean difference. When both prognostic factor and outcome were continuous data, we reported correlation coefficients or calculated them from individual participant data.We displayed all results on forest plots to give an overall visual representation. We planned to conduct meta-analyses where this was appropriate, otherwise we summarised narratively.

MAIN RESULTS

We included 19 studies comprising 21 cohorts involving 646 participants. Only 11 studies (13 cohorts, 522 participants) had data available for analysis. Of these, five were prospective cohort studies, four were RCTs and two had a type of case-control design. Follow-up time ranged from four to 36 weeks. Studies covered 10 different matrix metalloproteases (MMPs) and two serine proteases (human neutrophil elastase and urokinase-type plasminogen activators). Two studies recorded complete healing as an outcome; other studies recorded partial healing measures. There was clinical and methodological heterogeneity across studies; for example, in the definition of healing, the type of protease and its measurement, the distribution of active and bound protease species, the types of treatment and the reporting of results. Therefore, meta-analysis was not performed. No study had conducted multivariable analyses and all included evidence was of very low certainty because of the lack of adjustment for confounders, the high risk of bias for all studies except one, imprecision around the measures of association and inconsistency in the direction of association. Collectively the research indicated complete uncertainty as to the association between protease activity and VLU healing.

AUTHORS' CONCLUSIONS

This review identified very low validity evidence regarding any association between protease activity and VLU healing and there is complete uncertainty regarding the relationship. The review offers information for both future research and systematic review methodology.

A comparison of one-stage vs two-stage individual patient data meta-analysis methods: A simulation study

Background

Individual patient data (IPD) meta‐analysis allows for the exploration of heterogeneity and can identify subgroups that most benefit from an intervention (or exposure), much more successfully than meta‐analysis of aggregate data. One‐stage or two‐stage IPD meta‐analysis is possible, with the former using mixed‐effects regression models and the latter obtaining study estimates through simpler regression models before aggregating using standard meta‐analysis methodology. However, a comprehensive comparison of the two methods, in practice, is lacking.

Methods

We generated 1000 datasets for each of many simulation scenarios covering different IPD sizes and different between‐study variance (heterogeneity) assumptions at various levels (intercept and exposure). Numerous simulation settings of different assumptions were also used, while we evaluated performance both on main effects and interaction effects. Performance was assessed on mean bias, mean error, coverage, and power.

Results

Fully specified one‐stage models (random study intercept or fixed study‐specific intercept; random exposure effect; and fixed study‐specific effects for covariate) were the best performers overall, especially when investigating interactions. For main effects, performance was almost identical across models unless intercept heterogeneity was present, in which case the fully specified one‐stage and the two‐stage models performed better. For interaction effects, differences across models were greater with the two‐stage model consistently outperformed by the two fully specified one‐stage models.

Conclusions

A fully specified one‐stage model should be preferred (accounting for potential exposure, intercept, and, possibly, interaction heterogeneity), especially when investigating interactions. If non‐convergence is encountered with a random study intercept, the fixed study‐specific intercept one‐stage model should be used instead.

A framework for identifying treatment-covariate interactions in individual participant data network meta-analysis

Background: Stratified medicine seeks to identify patients most likely to respond to treatment. Individual participant data (IPD) network meta‐analysis (NMA) models have greater power than individual trials to identify treatment‐covariate interactions (TCIs). Treatment‐covariate interactions contain “within” and “across” trial interactions, where the across‐trial interaction is more susceptible to confounding and ecological bias.

Methods: We considered a network of IPD from 37 trials (5922 patients) for cervical cancer (2394 events), where previous research identified disease stage as a potential interaction covariate. We compare 2 models for NMA with TCIs: (1) 2 effects separating within‐ and across‐trial interactions and (2) a single effect combining within‐ and across‐trial interactions. We argue for a visual assessment of consistency of within‐ and across‐trial interactions and consider more detailed aspects of interaction modelling, eg, common vs trial‐specific effects of the covariate. This leads us to propose a practical framework for IPD NMA with TCIs.

Results: Following our framework, we found no evidence in the cervical cancer network for a treatment‐stage interaction on the basis of the within‐trial interaction. The NMA provided additional power for an across‐trial interaction over and above the pairwise evidence. Following our proposed framework, we found that the within‐ and across‐trial interactions should not be combined.

Conclusion: Across‐trial interactions are susceptible to confounding and ecological bias. It is important to separate the sources of evidence to check their consistency and identify which sources of evidence are driving the conclusion. Our framework provides practical guidance for researchers, reducing the risk of unduly optimistic interpretation of TCIs.

Effects of antenatal diet and physical activity on maternal and fetal outcomes: individual patient data meta-analysis and health economic evaluation

BACKGROUND

Diet- and physical activity-based interventions in pregnancy have the potential to alter maternal and child outcomes.

OBJECTIVES

To assess whether or not the effects of diet and lifestyle interventions vary in subgroups of women, based on maternal body mass index (BMI), age, parity, Caucasian ethnicity and underlying medical condition(s), by undertaking an individual patient data (IPD) meta-analysis. We also evaluated the association of gestational weight gain (GWG) with adverse pregnancy outcomes and assessed the cost-effectiveness of the interventions.

DATA SOURCES

MEDLINE, EMBASE, Cochrane Central Register of Controlled Trials, Database of Abstracts of Reviews of Effects and Health Technology Assessment database were searched from October 2013 to March 2015 (to update a previous search).

REVIEW METHODS

Researchers from the International Weight Management in Pregnancy Collaborative Network shared the primary data. For each intervention type and outcome, we performed a two-step IPD random-effects meta-analysis, for all women (except underweight) combined and for each subgroup of interest, to obtain summary estimates of effects and 95% confidence intervals (CIs), and synthesised the differences in effects between subgroups. In the first stage, we fitted a linear regression adjusted for baseline (for continuous outcomes) or a logistic regression model (for binary outcomes) in each study separately; estimates were combined across studies using random-effects meta-analysis models. We quantified the relationship between weight gain and complications, and undertook a decision-analytic model-based economic evaluation to assess the cost-effectiveness of the interventions.

RESULTS

Diet and lifestyle interventions reduced GWG by an average of 0.70 kg (95% CI -0.92 to -0.48 kg; 33 studies, 9320 women). The effects on composite maternal outcome [summary odds ratio (OR) 0.90, 95% CI 0.79 to 1.03; 24 studies, 8852 women] and composite fetal/neonatal outcome (summary OR 0.94, 95% CI 0.83 to 1.08; 18 studies, 7981 women) were not significant. The effect did not vary with baseline BMI, age, ethnicity, parity or underlying medical conditions for GWG, and composite maternal and fetal outcomes. Lifestyle interventions reduce Caesarean sections (OR 0.91, 95% CI 0.83 to 0.99), but not other individual maternal outcomes such as gestational diabetes mellitus (OR 0.89, 95% CI 0.72 to 1.10), pre-eclampsia or pregnancy-induced hypertension (OR 0.95, 95% CI 0.78 to 1.16) and preterm birth (OR 0.94, 95% CI 0.78 to 1.13). There was no significant effect on fetal outcomes. The interventions were not cost-effective. GWG, including adherence to the Institute of Medicine-recommended targets, was not associated with a reduction in complications. Predictors of GWG were maternal age (summary estimate -0.10 kg, 95% CI -0.14 to -0.06 kg) and multiparity (summary estimate -0.73 kg, 95% CI -1.24 to -0.23 kg).

LIMITATIONS

The findings were limited by the lack of standardisation in the components of intervention, residual heterogeneity in effects across studies for most analyses and the unavailability of IPD in some studies.

CONCLUSION

Diet and lifestyle interventions in pregnancy are clinically effective in reducing GWG irrespective of risk factors, with no effects on composite maternal and fetal outcomes.

FUTURE WORK

The differential effects of lifestyle interventions on individual pregnancy outcomes need evaluation.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42013003804.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

Synergistic effects of blood pressure-lowering drugs and statins: systematic review and meta-analysis

BACKGROUND

Synergistic effects of blood pressure-lowering drugs and statins are unknown, but are key to risk-based treatment decision strategies and fixed-combination polypills.

OBJECTIVE

We conducted a systematic literature review and meta-analysis to test the hypothesis that the combined relative effects of blood pressure-lowering drugs and statins on cardiovascular outcomes are multiplicative.

STUDY SELECTION

Two persons independently searched five data sources and hand-searched reference lists from earliest available to December 2017. We included factorial trials with at least two randomised interventions including one statin versus placebo factor and one blood pressure-lowering drug/more intense blood pressure-lowering regimen versus placebo/less intense regimen factor, and reported cardiovascular events or mortality as outcomes. We tested interactions as departures from additivity or multiplicativity using mixed-effects logistic regression models.

FINDINGS

Seven out of 1017 screened studies fulfilled the selection criteria, contributing a total of 27 020 patients with 857 major cardiovascular events and 725 deaths. The relative risk reduction of major cardiovascular events with active/more intense blood pressure-lowering regimen was not materially different in subgroups randomised to statins (risk ratio 0.81, 95% CI 0.66 to 1.00) or placebo (0.94, 0.79 to 1.11). Likewise, statin effects were not substantially different in subgroups randomised to active/more intense blood pressure-lowering regimen (0.69, 0.57 to 0.85) or placebo/less intense regimen (0.80, 0.67 to 0.96). No departures from either additivity or multiplicativity were observed. Heterogeneity was low.

CONCLUSIONS

The combined relative effects of blood pressure-lowering drugs and statins on cardiovascular events were multiplicative. This supports risk-based treatment decision strategies and fixed-combination polypills.

Graphs of study contributions and covariate distributions for network meta-regression

Background

Meta‐regression results must be interpreted taking into account the range of covariate values of the contributing studies. Results based on interpolation or extrapolation may be unreliable. In network meta‐regression (NMR) models, which include covariates in network meta‐analyses, results are estimated using direct and indirect evidence; therefore, it may be unclear which studies and covariate values contribute to which result. We propose graphs to help understand which trials and covariate values contribute to each NMR result and to highlight extrapolation or interpolation.

Methods

We introduce methods to calculate the contribution that each trial and covariate value makes to each result and compare them with existing methods. We show how to construct graphs including a network covariate distribution diagram, covariate‐contribution plot, heat plot, contribution‐NMR plot, and heat‐NMR plot. We demonstrate the methods using a dataset with treatments for malaria using the covariate average age and a dataset of topical fluoride interventions for preventing dental caries using the covariate randomisation year.

Results

For the malaria dataset, no contributing trials had an average age between 7–25 years and therefore results were interpolated within this range. For the fluoride dataset, there are no contributing trials randomised between 1954–1959 for most comparisons therefore, within this range, results would be extrapolated.

Conclusions

Even in a fully connected network, an NMR result may be estimated from trials with a narrower covariate range than the range of the whole dataset. Calculating contributions and graphically displaying them aids interpretation of NMR result by highlighting extrapolated or interpolated results.

Meta-Analysis of Complex Interventions

Meta-analysis is a prominent method for estimating the effects of public health interventions, yet these interventions are often complex in ways that pose challenges to using conventional meta-analytic methods. This article discusses meta-analytic techniques that can be used in research syntheses on the effects of complex public health interventions. We first introduce the use of complexity frameworks to conceptualize public health interventions. We then present a menu of meta-analytic procedures for addressing various sources of complexity when answering questions about the effects of public health interventions in research syntheses. We conclude with a review of important practices and key resources for conducting meta-analyses on complex interventions, as well as future directions for research synthesis more generally. Overall, we argue that it is possible to conduct meaningful quantitative syntheses of research on the effects of public health interventions, though these meta-analyses may require the use of advanced techniques to properly consider and attend to issues of complexity.

Stratifying risk in the prevention of recurrent variceal hemorrhage: Results of an individual patient meta-analysis

Endoscopic variceal ligation plus beta-blockers (EVL+BB) is currently recommended for variceal rebleeding prophylaxis, a recommendation that extends to all patients with cirrhosis with previous variceal bleeding irrespective of prognostic stage. Individualizing patient care is relevant, and in published studies on variceal rebleeding prophylaxis, there is a lack of information regarding response to therapy by prognostic stage. This study aimed at comparing EVL plus BB with monotherapy (EVL or BB) on all-source rebleeding and mortality in patients with cirrhosis and previous variceal bleeding stratified by cirrhosis severity (Child A versus B/C) by means of individual time-to-event patient data meta-analysis from randomized controlled trials. The study used individual data on 389 patients from three trials comparing EVL plus BB versus BB and 416 patients from four trials comparing EVL plus BB versus EVL. Compared with BB alone, EVL plus BB reduced overall rebleeding in Child A (incidence rate ratio 0.40; 95% confidence interval, 0.18-0.89; P = 0.025) but not in Child B/C, without differences in mortality. The effect of EVL on rebleeding was different according to Child (P for interaction <0.001). Conversely, compared with EVL, EVL plus BB reduced rebleeding in both Child A and B/C, with a significant reduction in mortality in Child B/C (incidence rate ratio 0.46; 95% confidence interval, 0.25-0.85; P = 0.013).

CONCLUSION

Outcomes of therapies to prevent variceal rebleeding differ depending on cirrhosis severity: in patients with preserved liver function (Child A), combination therapy is recommended because it is more effective in preventing rebleeding, without modifying survival, while in patients with advanced liver failure (Child B/C), EVL alone carries an increased risk of rebleeding and death compared with combination therapy, underlining that BB is the key element of combination therapy. (Hepatology 2017;66:1219-1231).

Glycemic Control During Continuous Subcutaneous Insulin Infusion Versus Multiple Daily Insulin Injections in Type 2 Diabetes: Individual Patient Data Meta-analysis and Meta-regression of Randomized Controlled Trials

OBJECTIVE

To compare glycemic control during continuous subcutaneous insulin infusion (CSII) and multiple daily insulin injections (MDI) in people with type 2 diabetes to identify patient characteristics that determine those best treated by CSII.

RESEARCH DESIGN AND METHODS

Randomized controlled trials were selected comparing HbA_1c during CSII versus MDI in people with type 2 diabetes. Data sources included Cochrane database and Ovid Medline. We explored patient-level determinants of final HbA_1c level and insulin dose using Bayesian meta-regression models of individual patient data and summary effects using two-step meta-analysis. Hypoglycemia data were unavailable.

RESULTS

Five trials were identified, with 287 patients randomized to receive MDI and 303 to receive CSII. Baseline HbA_1c was the best determinant of final HbA_1c: HbA_1c difference (%) = 1.575 - (0.216 [95% credible interval 0.371-0.043] × baseline HbA_1c) for all trials, but with largest effect in the trial with prerandomization optimization of control. Baseline insulin dose was best predictor of final insulin dose: insulin dose difference (units/kg) = 0.1245 - (0.382 [0.510-0.254] × baseline insulin dose). Overall HbA_1c difference was -0.40% (-0.86 to 0.05 [-4.4 mmol/mol (-9.4 to 0.6)]). Overall insulin dose was reduced by -0.25 units/kg (-0.31 to -0.19) (26% reduction on CSII), and by -24.0 units/day (-30.6 to -17.5). Mean weight did not differ between treatments (0.08 kg [-0.33 to 0.48]).

CONCLUSIONS

CSII achieves better glycemic control than MDI in people with poorly controlled type 2 diabetes, with ∼26% reduction in insulin requirements and no weight change. The best effect is in those worst controlled and with the highest insulin dose at baseline.

The effects of dietary and lifestyle interventions among pregnant women who are overweight or obese on longer-term maternal and early childhood outcomes: protocol for an individual participant data (IPD) meta-analysis

BACKGROUND

The aim of this individual participant data meta-analysis (IPDMA) is to evaluate the effects of dietary and lifestyle interventions among pregnant women who are overweight or obese on later maternal and early childhood outcomes at ages 3-5 years.

METHODS/DESIGN

We will build on the established International Weight Management in Pregnancy (i-WIP) IPD Collaborative Network, having identified researchers who have conducted randomised dietary and lifestyle interventions among pregnant women who are overweight or obese, and where ongoing childhood follow-up of participants has been or is being undertaken. The primary maternal outcome is a diagnosis of maternal metabolic syndrome. The primary childhood outcome is BMI above 90%. We have identified 7 relevant trials, involving 5425 women who were overweight or obese during pregnancy, with approximately 3544 women and children with follow-up assessments available for inclusion in the meta-analysis.

DISCUSSION

The proposed IPDMA provides an opportunity to evaluate the effect of dietary and lifestyle interventions among pregnant women who are overweight or obese on later maternal and early childhood health outcomes, including risk of obesity. This knowledge is essential to effectively translate research findings into clinical practice and public health policy.

SYSTEMATIC REVIEW REGISTRATION

This IPD has been prospectively registered (PROSPERO), ID number CRD42016047165 .

Deriving percentage study weights in multi-parameter meta-analysis models: with application to meta-regression, network meta-analysis and one-stage individual participant data models

Many meta-analysis models contain multiple parameters, for example due to multiple outcomes, multiple treatments or multiple regression coefficients. In particular, meta-regression models may contain multiple study-level covariates, and one-stage individual participant data meta-analysis models may contain multiple patient-level covariates and interactions. Here, we propose how to derive percentage study weights for such situations, in order to reveal the (otherwise hidden) contribution of each study toward the parameter estimates of interest. We assume that studies are independent, and utilise a decomposition of Fisher's information matrix to decompose the total variance matrix of parameter estimates into study-specific contributions, from which percentage weights are derived. This approach generalises how percentage weights are calculated in a traditional, single parameter meta-analysis model. Application is made to one- and two-stage individual participant data meta-analyses, meta-regression and network (multivariate) meta-analysis of multiple treatments. These reveal percentage study weights toward clinically important estimates, such as summary treatment effects and treatment-covariate interactions, and are especially useful when some studies are potential outliers or at high risk of bias. We also derive percentage study weights toward methodologically interesting measures, such as the magnitude of ecological bias (difference between within-study and across-study associations) and the amount of inconsistency (difference between direct and indirect evidence in a network meta-analysis).

One-stage individual participant data meta-analysis models: estimation of treatment-covariate interactions must avoid ecological bias by separating out within-trial and across-trial information

Stratified medicine utilizes individual‐level covariates that are associated with a differential treatment effect, also known as treatment‐covariate interactions. When multiple trials are available, meta‐analysis is used to help detect true treatment‐covariate interactions by combining their data. Meta‐regression of trial‐level information is prone to low power and ecological bias, and therefore, individual participant data (IPD) meta‐analyses are preferable to examine interactions utilizing individual‐level information. However, one‐stage IPD models are often wrongly specified, such that interactions are based on amalgamating within‐ and across‐trial information. We compare, through simulations and an applied example, fixed‐effect and random‐effects models for a one‐stage IPD meta‐analysis of time‐to‐event data where the goal is to estimate a treatment‐covariate interaction. We show that it is crucial to centre patient‐level covariates by their mean value in each trial, in order to separate out within‐trial and across‐trial information. Otherwise, bias and coverage of interaction estimates may be adversely affected, leading to potentially erroneous conclusions driven by ecological bias. We revisit an IPD meta‐analysis of five epilepsy trials and examine age as a treatment effect modifier. The interaction is −0.011 (95% CI: −0.019 to −0.003; p = 0.004), and thus highly significant, when amalgamating within‐trial and across‐trial information. However, when separating within‐trial from across‐trial information, the interaction is −0.007 (95% CI: −0.019 to 0.005; p = 0.22), and thus its magnitude and statistical significance are greatly reduced. We recommend that meta‐analysts should only use within‐trial information to examine individual predictors of treatment effect and that one‐stage IPD models should separate within‐trial from across‐trial information to avoid ecological bias. © 2016 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.

Meta-analysis using individual participant data: one-stage and two-stage approaches, and why they may differ

Meta‐analysis using individual participant data (IPD) obtains and synthesises the raw, participant‐level data from a set of relevant studies. The IPD approach is becoming an increasingly popular tool as an alternative to traditional aggregate data meta‐analysis, especially as it avoids reliance on published results and provides an opportunity to investigate individual‐level interactions, such as treatment‐effect modifiers. There are two statistical approaches for conducting an IPD meta‐analysis: one‐stage and two‐stage. The one‐stage approach analyses the IPD from all studies simultaneously, for example, in a hierarchical regression model with random effects. The two‐stage approach derives aggregate data (such as effect estimates) in each study separately and then combines these in a traditional meta‐analysis model. There have been numerous comparisons of the one‐stage and two‐stage approaches via theoretical consideration, simulation and empirical examples, yet there remains confusion regarding when each approach should be adopted, and indeed why they may differ.

In this tutorial paper, we outline the key statistical methods for one‐stage and two‐stage IPD meta‐analyses, and provide 10 key reasons why they may produce different summary results. We explain that most differences arise because of different modelling assumptions, rather than the choice of one‐stage or two‐stage itself. We illustrate the concepts with recently published IPD meta‐analyses, summarise key statistical software and provide recommendations for future IPD meta‐analyses. © 2016 The Authors. Statistics in Medicine published by John Wiley & Sons Ltd.

A systematic review and individual patient data meta-analysis of prognostic factors for foot ulceration in people with diabetes: the international research collaboration for the prediction of diabetic foot ulcerations (PODUS)

BACKGROUND

Annual foot risk assessment of people with diabetes is recommended in national and international clinical guidelines. At present, these are consensus based and use only a proportion of the available evidence.

OBJECTIVES

We undertook a systematic review of individual patient data (IPD) to identify the most highly prognostic factors for foot ulceration (i.e. symptoms, signs, diagnostic tests) in people with diabetes.

DATA SOURCES

Studies were identified from searches of MEDLINE and EMBASE.

REVIEW METHODS

The electronic search strategies for MEDLINE and EMBASE databases created during an aggregate systematic review of predictive factors for foot ulceration in diabetes were updated and rerun to January 2013. One reviewer applied the IPD review eligibility criteria to the full-text articles of the studies identified in our literature search and also to all studies excluded from our aggregate systematic review to ensure that we did not miss eligible IPD. A second reviewer applied the eligibility criteria to a 10% random sample of the abstract search yield to check that no relevant material was missed. This review includes exposure variables (risk factors) only from individuals who were free of foot ulceration at the time of study entry and who had a diagnosis of diabetes mellitus (either type 1 or type 2). The outcome variable was incident ulceration.

RESULTS

Our search identified 16 cohort studies and we obtained anonymised IPD for 10. These data were collected from more than 16,000 people with diabetes worldwide and reanalysed by us. One data set was kept for independent validation. The data sets contributing IPD covered a range of temporal, geographical and clinical settings. We therefore selected random-effects meta-analysis, which assumes not that all the estimates from each study are estimates of the same underlying true value, but rather that the estimates belong to the same distribution. We selected candidate variables for meta-analysis using specific criteria. After univariate meta-analyses, the most clinically important predictors were identified by an international steering committee for inclusion in the primary, multivariable meta-analysis. Age, sex, duration of diabetes, monofilaments and pulses were considered most prognostically important. Meta-analyses based on data from the entire IPD population found that an inability to feel a 10-g monofilament [odds ratio (OR) 3.184, 95% confidence interval (CI) 2.654 to 3.82], at least one absent pedal pulse (OR 1.968, 95% CI 1.624 to 2.386), a longer duration of a diagnosis of diabetes (OR 1.024, 95% CI 1.011 to 1.036) and a previous history of ulceration (OR 6.589, 95% CI 2.488 to 17.45) were all predictive of risk. Female sex was protective (OR 0.743, 95% CI 0.598 to 0.922).

LIMITATIONS

It was not possible to perform a meta-analysis using a one-step approach because we were unable to procure copies of one of the data sets and instead accessed data via Safe Haven.

CONCLUSIONS

The findings from this review identify risk assessment procedures that can reliably inform national and international diabetes clinical guideline foot risk assessment procedures. The evidence from a large sample of patients in worldwide settings show that the use of a 10-g monofilament or one absent pedal pulse will identify those at moderate or intermediate risk of foot ulceration, and a history of foot ulcers or lower-extremity amputation is sufficient to identify those at high risk. We propose the development of a clinical prediction rule (CPR) from our existing model using the following predictor variables: insensitivity to a 10-g monofilament, absent pedal pulses and a history of ulceration or lower-extremities amputations. This CPR could replace the many tests, signs and symptoms that patients currently have measured using equipment that is either costly or difficult to use.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42011001841.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.