Multivariate and network meta-analysis of multiple outcomes and multiple treatments: rationale, concepts, and examples

Organisations such as the National Institute for Health and Care Excellence require the synthesis of evidence from existing studies to inform their decisions—for example, about the best available treatments with respect to multiple efficacy and safety outcomes. However, relevant studies may not provide direct evidence about all the treatments or outcomes of interest. Multivariate and network meta-analysis methods provide a framework to address this, using correlated or indirect evidence from such studies alongside any direct evidence. In this article, the authors describe the key concepts and assumptions of these methods, outline how correlated and indirect evidence arises, and illustrate the contribution of such evidence in real clinical examples involving multiple outcomes and multiple treatments

Power analysis for random-effects meta-analysis

One of the reasons for the popularity of meta-analysis is the notion that these analyses will possess more power to detect effects than individual studies. This is inevitably the case under a fixed-effect model. However, the inclusion of the between-study variance in the random-effects model, and the need to estimate this parameter, can have unfortunate implications for this power. We develop methods for assessing the power of random-effects meta-analyses, and the average power of the individual studies that contribute to meta-analyses, so that these powers can be compared. In addition to deriving new analytical results and methods, we apply our methods to 1991 meta-analyses taken from the Cochrane Database of Systematic Reviews to retrospectively calculate their powers. We find that, in practice, 5 or more studies are needed to reasonably consistently achieve powers from random-effects meta-analyses that are greater than the studies that contribute to them. Not only is statistical inference under the random-effects model challenging when there are very few studies but also less worthwhile in such cases. The assumption that meta-analysis will result in an increase in power is challenged by our findings.

A matrix-based method of moments for fitting multivariate network meta-analysis models with multiple outcomes and random inconsistency effects

Random-effects meta-analyses are very commonly used in medical statistics. Recent methodological developments include multivariate (multiple outcomes) and network (multiple treatments) meta-analysis. Here, we provide a new model and corresponding estimation procedure for multivariate network meta-analysis, so that multiple outcomes and treatments can be included in a single analysis. Our new multivariate model is a direct extension of a univariate model for network meta-analysis that has recently been proposed. We allow two types of unknown variance parameters in our model, which represent between-study heterogeneity and inconsistency. Inconsistency arises when different forms of direct and indirect evidence are not in agreement, even having taken between-study heterogeneity into account. However, the consistency assumption is often assumed in practice and so we also explain how to fit a reduced model which makes this assumption. Our estimation method extends several other commonly used methods for meta-analysis, including the method proposed by DerSimonian and Laird (). We investigate the use of our proposed methods in the context of both a simulation study and a real example.

Single-cell studies of IFN-β promoter activation by wild-type and NS1-defective influenza A viruses

Deletion or truncation of NS1, the principal IFN antagonist of influenza viruses, leads to increased IFN induction during influenza virus infection. We have studied activation of the IFN induction cascade by both wild-type and NS1-defective viruses at the single-cell level using a cell line expressing GFP under the control of the IFN-β promoter and by examining MxA expression. The IFN-β promoter was not activated in all infected cells even during NS1-defective virus infections. Loss of NS1 expression is therefore insufficient per se to induce IFN in an infected cell, and factors besides NS1 expression status must dictate whether the IFN response is activated. The IFN response was efficiently stimulated in these cells following infection with other viruses; the differential IFN response we observe with influenza viruses is therefore not cell specific but is likely due to differences in the nature of the infecting virus particles and their subsequent replication.

Deriving percentage study weights in multi-parameter meta-analysis models: with application to meta-regression, network meta-analysis and one-stage individual participant data models

Many meta-analysis models contain multiple parameters, for example due to multiple outcomes, multiple treatments or multiple regression coefficients. In particular, meta-regression models may contain multiple study-level covariates, and one-stage individual participant data meta-analysis models may contain multiple patient-level covariates and interactions. Here, we propose how to derive percentage study weights for such situations, in order to reveal the (otherwise hidden) contribution of each study toward the parameter estimates of interest. We assume that studies are independent, and utilise a decomposition of Fisher's information matrix to decompose the total variance matrix of parameter estimates into study-specific contributions, from which percentage weights are derived. This approach generalises how percentage weights are calculated in a traditional, single parameter meta-analysis model. Application is made to one- and two-stage individual participant data meta-analyses, meta-regression and network (multivariate) meta-analysis of multiple treatments. These reveal percentage study weights toward clinically important estimates, such as summary treatment effects and treatment-covariate interactions, and are especially useful when some studies are potential outliers or at high risk of bias. We also derive percentage study weights toward methodologically interesting measures, such as the magnitude of ecological bias (difference between within-study and across-study associations) and the amount of inconsistency (difference between direct and indirect evidence in a network meta-analysis).

Large Scale Population Assessment of Physical Activity Using Wrist Worn Accelerometers: The UK Biobank Study

Background

Physical activity has not been objectively measured in prospective cohorts with sufficiently large numbers to reliably detect associations with multiple health outcomes. Technological advances now make this possible. We describe the methods used to collect and analyse accelerometer measured physical activity in over 100,000 participants of the UK Biobank study, and report variation by age, sex, day, time of day, and season.

Methods

Participants were approached by email to wear a wrist-worn accelerometer for seven days that was posted to them. Physical activity information was extracted from 100Hz raw triaxial acceleration data after calibration, removal of gravity and sensor noise, and identification of wear / non-wear episodes. We report age- and sex-specific wear-time compliance and accelerometer measured physical activity, overall and by hour-of-day, week-weekend day and season.

Results

103,712 datasets were received (44.8% response), with a median wear-time of 6.9 days (IQR:6.5–7.0). 96,600 participants (93.3%) provided valid data for physical activity analyses. Vector magnitude, a proxy for overall physical activity, was 7.5% (2.35mg) lower per decade of age (Cohen’s d = 0.9). Women had a higher vector magnitude than men, apart from those aged 45-54yrs. There were major differences in vector magnitude by time of day (d = 0.66). Vector magnitude differences between week and weekend days (d = 0.12 for men, d = 0.09 for women) and between seasons (d = 0.27 for men, d = 0.15 for women) were small.

Conclusions

It is feasible to collect and analyse objective physical activity data in large studies. The summary measure of overall physical activity is lower in older participants and age-related differences in activity are most prominent in the afternoon and evening. This work lays the foundation for studies of physical activity and its health consequences. Our summary variables are part of the UK Biobank dataset and can be used by researchers as exposures, confounding factors or outcome variables in future analyses.

A study of wrist-worn activity measurement as a potential real-world biomarker for late-life depression

BACKGROUND

Late-life depression (LLD) is associated with a decline in physical activity. Typically this is assessed by self-report questionnaires and, more recently, with actigraphy. We sought to explore the utility of a bespoke activity monitor to characterize activity profiles in LLD more precisely.

METHOD

The activity monitor was worn for 7 days by 29 adults with LLD and 30 healthy controls. Subjects underwent neuropsychological assessment and quality of life (QoL) (36-item Short-Form Health Survey) and activities of daily living (ADL) scales (Instrumental Activities of Daily Living Scale) were administered.

RESULTS

Physical activity was significantly reduced in LLD compared with controls (t = 3.63, p < 0.001), primarily in the morning. LLD subjects showed slower fine motor movements (t = 3.49, p < 0.001). In LLD patients, activity reductions were related to reduced ADL (r = 0.61, p < 0.001), lower QoL (r = 0.65, p < 0.001), associative learning (r = 0.40, p = 0.036), and higher Montgomery-Åsberg Depression Rating Scale score (r = -0.37, p < 0.05).

CONCLUSIONS

Patients with LLD had a significant reduction in general physical activity compared with healthy controls. Assessment of specific activity parameters further revealed the correlates of impairments associated with LLD. Our study suggests that novel wearable technology has the potential to provide an objective way of monitoring real-world function.

Residual plots for linear regression models with censored outcome data: A refined method for visualizing residual uncertainty

Residual plots are a standard tool for assessing model fit. When some outcome data are censored, standard residual plots become less appropriate. Here, we develop a new procedure for producing residual plots for linear regression models where some or all of the outcome data are censored. We implement two approaches for incorporating parameter uncertainty. We illustrate our methodology by examining the model fit for an analysis of bacterial load data from a trial for chronic obstructive pulmonary disease. Simulated datasets show that the method can be used when the outcome data consist of a variety of types of censoring.

Wristband Accelerometers to motiVate arm Exercise after Stroke (WAVES): study protocol for a pilot randomized controlled trial

Background

Loss of upper limb function affects up to 85 % of acute stroke patients. Recovery of upper limb function requires regular intensive practise of specific upper limb tasks. To enhance intensity of practice interventions are being developed to encourage patients to undertake self-directed exercise practice. Most interventions do not translate well into everyday activities and stroke patients continue to find it difficult remembering integration of upper limb movements into daily activities. A wrist-worn device has been developed that monitors and provides ‘live’ upper limb activity feedback to remind patients to use their stroke arm in daily activities (The CueS wristband). The aim of this trial is to assess the feasibility of a multi-centre, observer blind, pilot randomised controlled trial of the CueS wristband in clinical stroke services.

Methods/design

This pilot randomised controlled feasibility trial aims to recruit 60 participants over 15 months from North East England. Participants will be within 3 months of stroke which has caused new reduced upper limb function and will still be receiving therapy. Each participant will be randomised to an intervention or control group. Intervention participants will wear a CueS wristband (between 8 am and 8 pm) providing “live” feedback towards pre-set movement goals through a simple visual display and vibration prompts whilst undertaking a 4-week upper limb therapy programme (reviewed twice weekly by an occupational/physiotherapist). Control participants will also complete the 4-week upper limb therapy programme but will wear a ‘sham’ CueS wristband that monitors upper limb activity but provides no feedback. Outcomes will determine study feasibility in terms of recruitment, retention, adverse events, adherence and collection of descriptive clinical and accelerometer motor performance data at baseline, 4 weeks and 8 weeks.

Discussion

The WAVES study will address an important gap in the evidence base by reporting the feasibility of undertaking an evaluation of emerging and affordable technology to encourage impaired upper limb activity after stroke. The study will establish whether the study protocol can be supported by clinical stroke services, thereby informing the design of a future multi-centre randomised controlled trial of clinical and cost-effectiveness.

Trial registration

ISRCTN:82306027. Registered 12 July 2016.

Electronic supplementary material

The online version of this article (doi:10.1186/s13063-016-1628-2) contains supplementary material, which is available to authorized users.

Confidence intervals for the between-study variance in random-effects meta-analysis using generalised heterogeneity statistics: should we use unequal tails?

BACKGROUND

Confidence intervals for the between study variance are useful in random-effects meta-analyses because they quantify the uncertainty in the corresponding point estimates. Methods for calculating these confidence intervals have been developed that are based on inverting hypothesis tests using generalised heterogeneity statistics. Whilst, under the random effects model, these new methods furnish confidence intervals with the correct coverage, the resulting intervals are usually very wide, making them uninformative.

METHODS

We discuss a simple strategy for obtaining 95 % confidence intervals for the between-study variance with a markedly reduced width, whilst retaining the nominal coverage probability. Specifically, we consider the possibility of using methods based on generalised heterogeneity statistics with unequal tail probabilities, where the tail probability used to compute the upper bound is greater than 2.5 %. This idea is assessed using four real examples and a variety of simulation studies. Supporting analytical results are also obtained.

RESULTS

Our results provide evidence that using unequal tail probabilities can result in shorter 95 % confidence intervals for the between-study variance. We also show some further results for a real example that illustrates how shorter confidence intervals for the between-study variance can be useful when performing sensitivity analyses for the average effect, which is usually the parameter of primary interest.

CONCLUSIONS

We conclude that using unequal tail probabilities when computing 95 % confidence intervals for the between-study variance, when using methods based on generalised heterogeneity statistics, can result in shorter confidence intervals. We suggest that those who find the case for using unequal tail probabilities convincing should use the '1-4 % split', where greater tail probability is allocated to the upper confidence bound. The 'width-optimal' interval that we present deserves further investigation.

Implementing informative priors for heterogeneity in meta-analysis using meta-regression and pseudo data

Many meta‐analyses combine results from only a small number of studies, a situation in which the between‐study variance is imprecisely estimated when standard methods are applied. Bayesian meta‐analysis allows incorporation of external evidence on heterogeneity, providing the potential for more robust inference on the effect size of interest. We present a method for performing Bayesian meta‐analysis using data augmentation, in which we represent an informative conjugate prior for between‐study variance by pseudo data and use meta‐regression for estimation. To assist in this, we derive predictive inverse‐gamma distributions for the between‐study variance expected in future meta‐analyses. These may serve as priors for heterogeneity in new meta‐analyses. In a simulation study, we compare approximate Bayesian methods using meta‐regression and pseudo data against fully Bayesian approaches based on importance sampling techniques and Markov chain Monte Carlo (MCMC). We compare the frequentist properties of these Bayesian methods with those of the commonly used frequentist DerSimonian and Laird procedure. The method is implemented in standard statistical software and provides a less complex alternative to standard MCMC approaches. An importance sampling approach produces almost identical results to standard MCMC approaches, and results obtained through meta‐regression and pseudo data are very similar. On average, data augmentation provides closer results to MCMC, if implemented using restricted maximum likelihood estimation rather than DerSimonian and Laird or maximum likelihood estimation. The methods are applied to real datasets, and an extension to network meta‐analysis is described. The proposed method facilitates Bayesian meta‐analysis in a way that is accessible to applied researchers. © 2016 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.

Two new methods to fit models for network meta-analysis with random inconsistency effects

Background

Meta-analysis is a valuable tool for combining evidence from multiple studies. Network meta-analysis is becoming more widely used as a means to compare multiple treatments in the same analysis. However, a network meta-analysis may exhibit inconsistency, whereby the treatment effect estimates do not agree across all trial designs, even after taking between-study heterogeneity into account. We propose two new estimation methods for network meta-analysis models with random inconsistency effects.

Methods

The model we consider is an extension of the conventional random-effects model for meta-analysis to the network meta-analysis setting and allows for potential inconsistency using random inconsistency effects. Our first new estimation method uses a Bayesian framework with empirically-based prior distributions for both the heterogeneity and the inconsistency variances. We fit the model using importance sampling and thereby avoid some of the difficulties that might be associated with using Markov Chain Monte Carlo (MCMC). However, we confirm the accuracy of our importance sampling method by comparing the results to those obtained using MCMC as the gold standard. The second new estimation method we describe uses a likelihood-based approach, implemented in the metafor package, which can be used to obtain (restricted) maximum-likelihood estimates of the model parameters and profile likelihood confidence intervals of the variance components.

Results

We illustrate the application of the methods using two contrasting examples. The first uses all-cause mortality as an outcome, and shows little evidence of between-study heterogeneity or inconsistency. The second uses “ear discharge" as an outcome, and exhibits substantial between-study heterogeneity and inconsistency. Both new estimation methods give results similar to those obtained using MCMC.

Conclusions

The extent of heterogeneity and inconsistency should be assessed and reported in any network meta-analysis. Our two new methods can be used to fit models for network meta-analysis with random inconsistency effects. They are easily implemented using the accompanying R code in the Additional file 1. Using these estimation methods, the extent of inconsistency can be assessed and reported.

Electronic supplementary material

The online version of this article (doi:10.1186/s12874-016-0184-5) contains supplementary material, which is available to authorized users.

Sudden cardiac death in adults with congenitally corrected transposition of the great arteries

Background

Congenitally corrected transposition of the great arteries (ccTGA) is a rare congenital heart disease. There have been only few reports of sudden cardiac death (SCD) in patients with ccTGA and reasonable ventricular function.

Methods

A retrospective review of the medical records of all patients attending our adult congenital heart centre, with known ccTGA.

Results

From a database of over 3500 adult patients with congenital heart disease, we identified 39 (∼1%) with ccTGA and ‘two-ventricle’ circulations. 65% were male. The mean age at diagnosis was 12.4±11.4 years and the mean age at last time of review was 34.3±11.3 years. 24 patients (56%) had a history of surgical intervention. 8 (19%) had had pacemaker implantation and 2 had had a defibrillator implanted for non-sustained ventricular tachycardia (NSVT). In 544 years of patient follow-up, there had been five cases of SCD in our population; 1 death per 109 patient-years. Two of these patients had had previously documented supraventricular or NSVT. However, they were all classified as New York Heart Association (NYHA) class I or II, and systemic (right) ventricular function had been recorded as normal, mildly or mildly–moderately impaired, at most recent follow-up.

Conclusions

Our experience suggests the need for improved risk stratification and/or surveillance for malignant arrhythmia in adults with ccTGA, even in those with reasonable functional class on ventricular function.

The Clinical and Bacteriological Assessment of a Twice Daily Formulation of Erythromycin Stearate: A General Practice Study

This report presents the clinical and bacteriological assessment of a twice daily formulation of erythromycin stearate. The activity of this twice daily formulation was compared with erythromycin stearate 250 mg 6 hourly, and the assessment carried out in a number of general practices, both urban and rural, distributed widely throughout the United Kingdom. The twice daily dosage regime of ‘Erythrocin 500’ was seen to be an effective therapy, to compare favourably with the smaller dose given 6 hourly and with minimal side-effects. Antibiotic sensitivity tests included penicillin and tetracycline as well as erythromycin, and the negligible incidence of resistance demonstrated for erythromycin contrasted with that of the two other commonly used antibiotics.