A multivariate meta-analysis approach for reducing the impact of outcome reporting bias in systematic reviews

Effects of game-based digital interventions for mental disorders: A meta-analysis

With increasing research attention on game-based digital interventions for mental disorders, a number of studies have been conducted to explore the effectiveness of digital game-based interventions on mental disorders. However, findings from previous research were inconsistent. Thus, we conducted a comprehensive meta-analytic review of the effectiveness of game-based digital interventions for mental disorders. By searching the articles in databases, we identified 53 studies in which 2433 participants were involved, and 282 effect sizes were extracted. Among the 53 studies, 14 employed within-group (pre/post) designs, and the remaining 39 utilized controlled trial designs. Using a three-level random-effects meta-analytic model, a medium effect size of game-based digital interventions (g = 0.47, 95 % CI: 0.33, 0.61) was revealed in the controlled trial designs and a close-to-medium effect size (g = 0.45, 95 % CI: 0.32, 0.58) was found in the within-group (pre/post) designs, indicating close-to-medium-sized efficacy of game-based digital interventions for mental disorders. Moderator analyses showed that age in the controlled trial designs had contributed to the heterogeneity in previous studies, suggesting that interventions might be more effective for the elderly. However, given that only a limited number of studies were focused on the elderly, more studies with older participants should be conducted in the future to provide more robust evidence and explore the mechanisms of how digital gaming interventions can be more effective in improving mental disorders symptoms.

Is symptom outcome the whole story?-A multilevel meta-analysis of systemic therapy for adults including family system functioning

OBJECTIVE

Systemic Therapy conceives mental health symptoms in the context of social systems. Previous meta-analyses on Systemic Therapy focused on symptoms. This meta-analysis aims to focus on family system functioning while including all types of outcomes.

METHOD

We conducted a systematic literature research in multiple databases (PsycInfo, PubMed, Embase, Cochrane Central). We included RCT-studies on adults with psychiatric diagnoses, which compared Systemic Therapy with active psychosocial control. The literature research resulted in 171 coded effect sizes of 32 RCTs. We conducted a random-effects three-level meta-analysis. We categorized outcomes into symptoms of patients, family system functioning, further secondary outcomes of patients, and psychopathology of family members.

RESULTS

The results show a small significant overall effect size of g = .30 (CI: .15-.45, p < .001, k = 171, s = 32) for all outcomes. Systemic Therapy revealed small effect sizes with regard to family system functioning (g = .34, z = 3.51, p = .0004, k = 26, s = 12), symptoms (g = .30, z = 3.74, p = .0002, k = 73, s = 29), and further secondary outcomes (g = .32, z = 3.83, p = .0001, k = 63, s = 19). The effect sizes for psychopathology of family system members were reported rarely (k = 9, s = 6).

CONCLUSION

This meta-analysis shows the potential relevance of investigating family system functioning as a primary outcome for Systemic Therapy.

Exploiting multivariate network meta-analysis: A calibrated Bayesian composite likelihood inference

Multivariate network meta-analysis has emerged as a powerful tool in evidence synthesis by incorporating multiple outcomes and treatments. Despite its advantages, this method comes with methodological challenges, such as the issue of unreported within-study correlations among treatments and outcomes, which potentially lead to misleading conclusions. In this paper, we proposed a calibrated Bayesian composite likelihood approach to overcome this limitation. The proposed method eliminated the need to specify a full likelihood function while allowing for the unavailability of within-study correlations among treatments and outcomes. Additionally, we developed a hybrid Gibbs sampler algorithm along with the Open-Faced Sandwich post-sampling adjustment to enable robust posterior inference. Through comprehensive simulation studies, we demonstrated that the proposed approach yielded unbiased estimates while maintaining coverage probabilities close to the nominal level. Furthermore, we implemented the proposed method on two real-world network meta-analysis datasets; one comparing treatment procedures for the root coverage and another comparing treatments for anaemia in chronic kidney disease patients.

Network meta-analysis made simple: a composite likelihood approach

Network meta-analysis, also known as mixed treatments comparison meta-analysis or multiple treatments meta-analysis, extends conventional pairwise meta-analysis by simultaneously synthesizing multiple interventions in a single integrated analysis. Despite the growing popularity of network metaanalysis within comparative effectiveness research, it comes with potential challenges. For example, within-study correlations among treatment comparisons are rarely reported in the published literature. Yet, these correlations are pivotal for valid statistical inference. As demonstrated in earlier studies, ignoring these correlations can inflate mean squared errors of the resulting point estimates and lead to inaccurate standard error estimates. This paper introduces a composite likelihood-based approach that ensures accurate statistical inference without requiring knowledge of the within-study correlations. The proposed method is computationally robust and efficient, with substantially reduced computational time compared to the state-of-the-science methods implemented in R packages. The proposed method was evaluated through extensive simulations and applied to two important applications including a network meta-analysis comparing interventions for primary open-angle glaucoma, and another comparing treatments for chronic prostatitis and chronic pelvic pain syndrome.

Predicting Treatment Effects from Surrogate Endpoints in Historical Trials in First-Line Metastatic Castration-Resistant Prostate Cancer

Surrogate endpoints are becoming increasingly important in health technology assessment, where decisions are based on complex cost-effectiveness models (CEMs) that require numerous input parameters. Daniels and Hughes Surrogate Model was used to predict missing effect estimates in randomized controlled trials (RCTs) evaluating first-line treatments in metastatic castration-resistant prostate cancer (mCRPC) patients. Network meta-analyses (NMAs) were conducted to assess the comparative efficacy of these treatments. Databases were searched (inception to October 2022) using Ovid®. Several grey literature searches were also conducted (PROSPERO: CRD42021283512). Available trial data for radiographic progression-free survival (rPFS) and overall survival (OS) were used to predict the unreported effect of rPFS or OS for relevant comparator treatments. Bayesian NMAs were conducted using observed and predicted treatment effects. Effect estimates and 95% credible intervals were calculated for each comparison. Mean ranks and the probability of being best (p-best) were obtained. Twenty-five RCTs met the eligibility criteria and of these, 8 reported jointly rPFS and OS; while rPFS was predicted for 12 RCTs and 10 comparators, and OS was predicted for 5 RCTs and 6 comparators. A nonstandard dose of docetaxel (docetaxel 50 mg/m2 every 2 weeks) had the highest probability of being the most effective for rPFS (p-best: 59%) and OS (p-best: 48%), followed by talazoparib plus enzalutamide (13% and 19%, respectively). Advanced surrogate modelling techniques allowed obtaining relevant parameter and indirect estimates of previously unavailable data and may be used to populate future CEMs requiring rPFS and OS in first-line mCRPC.

Is cost of surgery and hospital length of stay increased for sickle cell disease patients undergoing total joint replacement surgeries? Systematic review and multivariate meta-analysis

Background

Sickle cell disease (SCD) patients undergo major joint replacement surgeries with significant improvement in quality of life. Previous literature have tried to explore differences in hospital charges and length of stay between patients with and without SCD. The aim of this meta-analysis is to find out if both outcomes are increased for patients with SCD patients compared to those without SCD.

Methods

Literature search was conducted and studies that compared hospital charges and length of stay between patients with and without sickle cell disease following major arthroplasties were retrieved. A multivariate meta-analysis was conducted using Random-Effect model with the Restricted Maximum Likelihood (REML) using the Metafor Package in R and Rstudio.

Results

Four observational studies were found eligible for the study. The estimated average mean difference based on the random effect model for hospital charges was 7548.50 (95 % CI: 3779.65 to 11317.65) and for length of stay was 2.28 (95 % CI: 1.32 to 3.24) while the prediction interval for the true mean difference for both outcomes were -1810.56 to 16907.56 and -0.01 to 4.57 respectively.

Conclusion

This present study showed that hospital charges and length of stay are increased for patients with SCD compared to patients without.

Twenty years of network meta-analysis: Continuing controversies and recent developments

Network meta-analysis (NMA) is an extension of pairwise meta-analysis (PMA) which combines evidence from trials on multiple treatments in connected networks. NMA delivers internally consistent estimates of relative treatment efficacy, needed for rational decision making. Over its first 20 years NMA's use has grown exponentially, with applications in both health technology assessment (HTA), primarily re-imbursement decisions and clinical guideline development, and clinical research publications. This has been a period of transition in meta-analysis, first from its roots in educational and social psychology, where large heterogeneous datasets could be explored to find effect modifiers, to smaller pairwise meta-analyses in clinical medicine on average with less than six studies. This has been followed by narrowly-focused estimation of the effects of specific treatments at specific doses in specific populations in sparse networks, where direct comparisons are unavailable or informed by only one or two studies. NMA is a powerful and well-established technique but, in spite of the exponential increase in applications, doubts about the reliability and validity of NMA persist. Here we outline the continuing controversies, and review some recent developments. We suggest that heterogeneity should be minimized, as it poses a threat to the reliability of NMA which has not been fully appreciated, perhaps because it has not been seen as a problem in PMA. More research is needed on the extent of heterogeneity and inconsistency in datasets used for decision making, on formal methods for making recommendations based on NMA, and on the further development of multi-level network meta-regression.

To seal or not to seal following an antimicrobial infusion at dry-off? A systematic review and multivariate meta-analysis of the incidence and prevalence of intramammary infections post-calving in dairy cows

Teat sealants (TSs) consist of sterile formulations with no antibacterial activity. Alone or in combination with antimicrobial (AM) or non-AM treatments, TSs have been commonly used in dairy cows at dry-off to prevent intra-mammary infections (IMIs) during the dry period. This study aimed to identify and synthesise the available evidence on the efficacy of combining TSs with AM treatments on the incidence and prevalence of IMIs. A comprehensive search of three electronic databases, two relevant conference proceedings, and reference lists of reviews and eligible articles was conducted to retrieve and identify studies that could answer the following question: in dairy cows, how does the efficacy of an AM-TS combination administered at dry-off compare with an AM alone for preventing new IMI? In addition to the general IMIs, bacterial species-specific data were extracted and combined into nine distinct pathogen groups: coagulase-positive and negative staphylococci; S. dysgalactiae; non-dysgalactiae Streptococci; E. coli; non-E. coli Enterobacteriaceae; Corynebacterium spp.; yeast and other frequent mastitis pathogens. The structural relationship between each study's prevalence and incidence, as the new (incidence) and persistent (uncured) infections make up the prevalence, was utilised to approximate a variance-covariance matrix for the within-study correlation between their study-specific log odds ratios (ORs). A bivariate random-effects meta-analysis was employed, utilising the within- and between-study correlations to synthesise both outcomes simultaneously. The risk of bias was assessed using the Cochrane ROBINS-I tool, and the quality of the body of evidence was rated using the GRADE approach. A total of 17 trials (16 studies), providing either IMIs incidence (n = 4), prevalence (n = 3) or both (n = 10), were identified. Overall, quarters infused with AM-TS combinations showed lower odds of new IMIs post-calving (OR=0.70; 95% CI=0.57-0.86; Wald test P < 0.001) than those which received only AMs. Across the pathogen groups, varying levels of reduction of new IMIs were found, where administration of TSs was most effective against S. dysgalactiae (OR=0.47; 95% CI=0.23-0.98), non-dysgalactiae streptococci (OR=0.60; 95% CI=0.49-0.74), E. coli (OR=0.62; 95% CI=0.50-0.77), Corynebacterium spp. (OR=0.68; 95% CI=0.52-0.90) and coagulase-negative staphylococci (OR=0.85; 95% CI=0.76-0.94). However, additional TS infusion did not significantly reduce new IMIs in the remaining pathogen groups. The current meta-analytic evidence supports the efficacy of using TS add-on infusions in dairy cows at dry-off for reducing the incidence and prevalence of IMIs post-calving; however, pathogen group differences should be considered.

PALM: PATIENT-CENTERED TREATMENT RANKING VIA LARGE-SCALE MULTIVARIATE NETWORK META-ANALYSIS

The growing number of available treatment options has led to urgent needs for reliable answers when choosing the best course of treatment for a patient. As it is often infeasible to compare a large number of treatments in a single randomized controlled trial, multivariate network meta-analyses (NMAs) are used to synthesize evidence from trials of a subset of the treatments, where both efficacy and safety related outcomes are considered simultaneously. However, these large-scale multiple-outcome NMAs have created challenges to existing methods due to the increasing complexity of the unknown correlations between outcomes and treatment comparisons. In this paper, we proposed a new framework for PAtient-centered treatment ranking via Large-scale Multivariate network meta-analysis, termed as PALM, which includes a parsimonious modeling approach, a fast algorithm for parameter estimation and inference, a novel visualization tool for presenting multivariate outcomes, termed as the origami plot, as well as personalized treatment ranking procedures taking into account the individual's considerations on multiple outcomes. In application to an NMA that compares 14 treatment options for labor induction, we provided a comprehensive illustration of the proposed framework and demonstrated its computational efficiency and practicality, and we obtained new insights and evidence to support patient-centered clinical decision making.

Comparing the Value of Data Visualization Methods for Communicating Harms in Clinical Trials

In clinical trials, harms (i.e., adverse events) are often reported by simply counting the number of people who experienced each event. Reporting only frequencies ignores other dimensions of the data that are important for stakeholders, including severity, seriousness, rate (recurrence), timing, and groups of related harms. Additionally, application of selection criteria to harms prevents most from being reported. Visualization of data could improve communication of multidimensional data. We replicated and compared the characteristics of 6 different approaches for visualizing harms: dot plot, stacked bar chart, volcano plot, heat map, treemap, and tendril plot. We considered binary events using individual participant data from a randomized trial of gabapentin for neuropathic pain. We assessed their value using a heuristic approach and a group of content experts. We produced all figures using R and share the open-source code on GitHub. Most original visualizations propose presenting individual harms (e.g., dizziness, somnolence) alone or alongside higher level (e.g., by body systems) summaries of harms, although they could be applied at either level. Visualizations can present different dimensions of all harms observed in trials. Except for the tendril plot, all other plots do not require individual participant data. The dot plot and volcano plot are favored as visualization approaches to present an overall summary of harms data. Our value assessment found the dot plot and volcano plot were favored by content experts. Using visualizations to report harms could improve communication. Trialists can use our provided code to easily implement these approaches.

Multivariate meta-analysis of multiple outcomes: characteristics and predictors of borrowing of strength from Cochrane reviews

OBJECTIVES

Multivariate meta-analysis allows the joint synthesis of multiple outcomes accounting for their correlation. This enables borrowing of strength (BoS) across outcomes, which may lead to greater efficiency and even different conclusions compared to separate univariate meta-analyses. However, multivariate meta-analysis is complex to apply, so guidance is needed to flag (in advance of analysis) when the approach is most useful.

STUDY DESIGN AND SETTING

We use 43 Cochrane intervention reviews to empirically investigate the characteristics of meta-analysis datasets that are associated with a larger BoS statistic (from 0 to 100%) when applying a bivariate meta-analysis of binary outcomes.

RESULTS

Four characteristics were identified as strongly associated with BoS: the total number of studies, the number of studies with the outcome of interest, the percentage of studies missing the outcome of interest, and the largest absolute within-study correlation. Using these characteristics, we then develop a model for predicting BoS in a new dataset, which is shown to have good performance (an adjusted R² of 50%). Applied examples are used to illustrate the use of the BoS prediction model.

CONCLUSIONS

Cochrane reviewers mainly use univariate meta-analysis methods, but the identified characteristics associated with BoS and our subsequent prediction model for BoS help to flag when a multivariate meta-analysis may also be beneficial in Cochrane reviews with multiple binary outcomes. Extension to non-Cochrane reviews and other outcome types is still required.

Approaches to Assessing and Adjusting for Selective Outcome Reporting in Meta-analysis

BACKGROUND

Selective or non-reporting of study outcomes results in outcome reporting bias.

OBJECTIVE

We sought to develop and assess tools for detecting and adjusting for outcome reporting bias.

DESIGN

Using data from a previously published systematic review, we abstracted whether outcomes were reported as collected, whether outcomes were statistically significant, and whether statistically significant outcomes were more likely to be reported. We proposed and tested a model to adjust for unreported outcomes and compared our model to three other methods (Copas, Frosi, trim and fill). Our approach assumes that unreported outcomes had a null intervention effect with variance imputed based on the published outcomes. We further compared our approach to these models using simulation, and by varying levels of missing data and study sizes.

RESULTS

There were 286 outcomes reported as collected from 47 included trials: 142 (48%) had the data provided and 144 (52%) did not. Reported outcomes were more likely to be statistically significant than those collected but for which data were unreported and for which non-significance was reported (RR, 2.4; 95% CI, 1.9 to 3.0). Our model and the Copas model provided similar decreases in the pooled effect sizes in both the meta-analytic data and simulation studies. The Frosi and trim and fill methods performed poorly.

LIMITATIONS

Single intervention of a single disease with only randomized controlled trials; approach may overestimate outcome reporting bias impact.

CONCLUSION

There was evidence of selective outcome reporting. Statistically significant outcomes were more likely to be published than non-significant ones. Our simple approach provided a quick estimate of the impact of unreported outcomes on the estimated effect. This approach could be used as a quick assessment of the potential impact of unreported outcomes.

Harms in Systematic Reviews Paper 1: An introduction to research on harms

OBJECTIVE

Most systematic reviews of interventions focus on potential benefits. Common methods and assumptions that are appropriate for assessing benefits can be inappropriate for harms. This paper provides a primer on researching harms, particularly in systematic reviews.

STUDY DESIGN AND SETTING

Commentary describing challenges with assessing harm.

RESULTS

Investigators should be familiar with various terminologies used to describe, classify, and group harms. Published reports of clinical trials include limited information about harms, so systematic reviewers should not depend on these studies and journal articles to reach conclusions about harms. Visualizations might improve communication of multiple dimensions of harms such as severity, relatedness, and timing.

CONCLUSION

The terminology, classification, detection, collection, and reporting of harms create unique challenges that take time, expertise, and resources to navigate in both primary studies and evidence syntheses. Systematic reviewers might reach incorrect conclusions if they focus on evidence about harms found in published reports of randomized trials of a particular health problem. Systematic reviews could be improved through better identification and reporting of harms in primary studies and through better training and uptake of appropriate methods for synthesizing evidence about harms.

Relative contributions of the nervous system, spinal tissue and psychosocial health to non-specific low back pain: Multivariate meta-analysis

BACKGROUND AND OBJECTIVES

Nervous system, psychosocial and spinal tissue biomarkers are associated with non-specific low back pain (nsLBP), though relative contributions are unclear.

DATABASES AND DATA TREATMENT

MEDLINE, EMBASE, CINAHL, PsycINFO and SPORTDiscus were searched up to 25 March 2020. Related reviews and reference lists were also screened. Observational studies examining structural and functional nervous system biomarkers (e.g. quantitative sensory tests, structural and functional brain measures), psychosocial factors (e.g. mental health, catastrophizing) and structural spinal imaging biomarkers (e.g. intervertebral disc degeneration, paraspinal muscle size) between nsLBP and pain-free controls were included. For multivariate meta-analysis, two of three domains were required in each study. Random-effects pairwise and multivariate meta-analyses were performed. GRADE approach assessed evidence certainty. Newcastle-Ottawa scale assessed risk of bias. Main outcomes were the effect size difference of domains between nsLBP and pain-free controls.

RESULTS

Of 4519 unique records identified, 33 studies (LBP = 1552, referents = 1322) were meta-analysed. Psychosocial state (Hedges' g [95%CI]: 0.90 [0.69-1.10], p < 0.001) in nsLBP showed larger effect sizes than nervous system (0.31 [0.13-0.49], p < 0.001; difference: 0.61 [0.36-0.86], p < 0.001) and spine imaging biomarkers (0.55 [0.37-0.73], p < 0.001; difference: 0.36 [0.04-0.67], p = 0.027). The relationship between domains changes depending on if pain duration is acute or chronic.

CONCLUSIONS

Psychosocial effect sizes in nsLBP are greater than that for spinal imaging and nervous system biomarkers. Limitations include cross-sectional design of studies included and inference of causality. Future research should investigate the clinical relevance of these effect size differences in relation to pain intensity and disability.

STUDY REGISTRATION

PROSPERO-CRD42020159188.

SIGNIFICANCE

Spinal imaging (e.g. intervertebral disc degeneration), psychosocial (e.g. depression) and nervous system (e.g. quantitative sensory tests, structural and functional brain measures) biomarkers contribute to non-specific low back pain. However, psychosocial factors may be more compromised than nervous system and spinal imaging biomarkers. This relationship depends on if the pain is acute or chronic. These findings underscore that the 'non-specific' label in back pain should be reconsidered, and more specific multidimensional categories evaluated to guide patient management.

Multivariate meta-analysis of critical care meta-analyses: a meta-epidemiological study

Background

Meta-analyses typically consider multiple outcomes and report univariate effect sizes considered as independent. Multivariate meta-analysis (MVMA) incorporates outcome correlation and synthesises direct evidence and related outcome estimates within a single analysis. In a series of meta-analyses from the critically ill literature, the current study contrasts multiple univariate effect estimates and their precision with those derived from MVMA.

Methods

A previous meta-epidemiological study was used to identify meta-analyses with either one or two secondary outcomes providing sufficient detail to structure bivariate or tri-variate MVMA, with mortality as primary outcome. Analysis was performed using a random effects model for both odds ratio (OR) and risk ratio (RR); borrowing of strength (BoS) between multivariate outcome estimates was reported. Estimate comparisons, β coefficients, standard errors (SE) and confidence interval (CI) width, univariate versus multivariate, were performed using Lin’s concordance correlation coefficient (CCC).

Results

In bivariate meta-analyses, for OR (n = 49) and RR (n = 48), there was substantial concordance (≥ 0.69) between estimates; but this was less so for tri-variate meta-analyses for both OR (n = 25; ≥ 0.38) and RR (≥ -0.10; n = 22). A variable change in the multivariate precision of primary mortality outcome estimates compared with univariate was present for both bivariate and tri-variate meta-analyses and for metrics. For second outcomes, precision tended to decrease and CI width increase for bivariate meta-analyses, but was variable in the tri-variate. For third outcomes, precision increased and CI width decreased. In bivariate meta-analyses, OR coefficient significance reversal, univariate versus MVMA, occurred once for mortality and 6 cases for second outcomes. RR coefficient significance reversal occurred in 4 cases; 2 were discordant with OR. For tri-variate OR meta-analyses reversal of coefficient estimate significance occurred in two cases for mortality, nine cases for second and 7 cases for third outcomes. In RR meta-analyses significance reversals occurred for mortality in 2 cases, 6 cases for second and 3 cases for third; there were 7 discordances with OR. BoS was greater in trivariate MVMAs compared with bivariate and for OR versus RR.

Conclusions

MVMA would appear to be the preferred solution to multiple univariate analyses; parameter significance changes may occur. Analytic metric appears to be a determinant.

Classifying information-sharing methods

BACKGROUND

Sparse relative effectiveness evidence is a frequent problem in Health Technology Assessment (HTA). Where evidence directly pertaining to the decision problem is sparse, it may be feasible to expand the evidence-base to include studies that relate to the decision problem only indirectly: for instance, when there is no evidence on a comparator, evidence on other treatments of the same molecular class could be used; similarly, a decision on children may borrow-strength from evidence on adults. Usually, in HTA, such indirect evidence is either included by ignoring any differences ('lumping') or not included at all ('splitting'). However, a range of more sophisticated methods exists, primarily in the biostatistics literature. The objective of this study is to identify and classify the breadth of the available information-sharing methods.

METHODS

Forwards and backwards citation-mining techniques were used on a set of seminal papers on the topic of information-sharing. Papers were included if they specified (network) meta-analytic methods for combining information from distinct populations, interventions, outcomes or study-designs.

RESULTS

Overall, 89 papers were included. A plethora of evidence synthesis methods have been used for information-sharing. Most papers (n=79) described methods that shared information on relative treatment effects. Amongst these, there was a strong emphasis on methods for information-sharing across multiple outcomes (n=42) and treatments (n=25), with fewer papers focusing on study-designs (n=23) or populations (n=8). We categorise and discuss the methods under four 'core' relationships of information-sharing: functional, exchangeability-based, prior-based and multivariate relationships, and explain the assumptions made within each of these core approaches.

CONCLUSIONS

This study highlights the range of information-sharing methods available. These methods often impose more moderate assumptions than lumping or splitting. Hence, the degree of information-sharing that they impose could potentially be considered more appropriate. Our identification of four 'core' methods of information-sharing allows for an improved understanding of the assumptions underpinning the different methods. Further research is required to understand how the methods differ in terms of the strength of sharing they impose and the implications of this for health care decisions.

A Comparison of Fixed-Effects and Random-Effects Models for Multivariate Meta-Analysis Using an SEM Approach

This study compared fixed-effects (FE) and random-effects (RE) models in meta-analysis for synthesizing multivariate effect sizes under the framework of structural equation modeling. Monte Carlo simulations were conducted to examine the performance characteristics of the two models under different data conditions. The results indicated that, for the homogeneous case, there was little difference between the FE model and the RE model applications. But the FE model had better performance in standard error estimation when number of studies is not large and the sample size of primary studies is small. Furthermore, under the heterogeneous case, FE model exhibited biased estimates of population parameters and extreme levels of inflated Type I error in testing the effect size estimates. However, RE model maintained unbiased estimates of the population parameters, and controlled Type I error well under various data conditions investigated. These findings provided empirical evidence that it is likely that RE model application in a meta-analysis would be preferred when the number of primary studies and the sample sizes in the primary studies are reasonably large, and FE model could be favored for situations with smaller numbers of studies and uniformly small sample sizes of primary studies.

A Bayesian multivariate meta-analysis of prevalence data

When conducting a meta-analysis involving prevalence data for an outcome with several subtypes, each of them is typically analyzed separately using a univariate meta-analysis model. Recently, multivariate meta-analysis models have been shown to correspond to a decrease in bias and variance for multiple correlated outcomes compared with univariate meta-analysis, when some studies only report a subset of the outcomes. In this article, we propose a novel Bayesian multivariate random effects model to account for the natural constraint that the prevalence of any given subtype cannot be larger than that of the overall prevalence. Extensive simulation studies show that this new model can reduce bias and variance when estimating subtype prevalences in the presence of missing data, compared with standard univariate and multivariate random effects models. The data from a rapid review on occupation and lower urinary tract symptoms by the Prevention of Lower Urinary Tract Symptoms Research Consortium are analyzed as a case study to estimate the prevalence of urinary incontinence and several incontinence subtypes among women in suspected high risk work environments.

A historical review of publication bias

Publication bias is a well-known threat to the validity of meta-analyses and, more broadly, the reproducibility of scientific findings. When policies and recommendations are predicated on an incomplete evidence base, it undermines the goals of evidence-based decision-making. Great strides have been made in the last 50 years to understand and address this problem, including calls for mandatory trial registration and the development of statistical methods to detect and correct for publication bias. We offer an historical account of seminal contributions by the evidence synthesis community, with an emphasis on the parallel development of graph-based and selection model approaches. We also draw attention to current innovations and opportunities for future methodological work.

Multivariate network meta-analysis incorporating class effects

BACKGROUND

Network meta-analysis synthesises data from a number of clinical trials in order to assess the comparative efficacy of multiple healthcare interventions in similar patient populations. In situations where clinical trial data are heterogeneously reported i.e. data are missing for one or more outcomes of interest, synthesising such data can lead to disconnected networks of evidence, increased uncertainty, and potentially biased estimates which can have severe implications for decision-making. To overcome this issue, strength can be borrowed between outcomes of interest in multivariate network meta-analyses. Furthermore, in situations where there are relatively few trials informing each treatment comparison, there is a potential issue with the sparsity of data in the treatment networks, which can lead to substantial parameter uncertainty. A multivariate network meta-analysis approach can be further extended to borrow strength between interventions of the same class using hierarchical models.

METHODS

We extend the trivariate network meta-analysis model to incorporate the exchangeability between treatment effects belonging to the same class of intervention to increase precision in treatment effect estimates. We further incorporate a missing data framework to estimate uncertainty in trials that did not report measures of variability in order to maximise the use of all available information for healthcare decision-making. The methods are applied to a motivating dataset in overactive bladder syndrome. The outcomes of interest were mean change from baseline in incontinence, voiding and urgency episodes. All models were fitted using Bayesian Markov Chain Monte Carlo (MCMC) methods in WinBUGS.

RESULTS

All models (univariate, multivariate, and multivariate models incorporating class effects) produced similar point estimates for all treatment effects. Incorporating class effects in multivariate models often increased precision in treatment effect estimates.

CONCLUSIONS

Multivariate network meta-analysis incorporating class effects allowed for the comparison of all interventions across all outcome measures to ameliorate the potential impact of outcome reporting bias, and further borrowed strength between interventions belonging to the same class of treatment to increase the precision in treatment effect estimates for healthcare policy and decision-making.

Association of Ketamine With Psychiatric Symptoms and Implications for Its Therapeutic Use and for Understanding Schizophrenia: A Systematic Review and Meta-analysis

Importance

Ketamine hydrochloride is increasingly used to treat depression and other psychiatric disorders but can induce schizophrenia-like or psychotomimetic symptoms. Despite this risk, the consistency and magnitude of symptoms induced by ketamine or what factors are associated with these symptoms remain unknown.

Objective

To conduct a meta-analysis of the psychopathological outcomes associated with ketamine in healthy volunteers and patients with schizophrenia and the experimental factors associated with these outcomes.

Data Sources

MEDLINE, Embase, and PsychINFO databases were searched for within-participant, placebo-controlled studies reporting symptoms using the Brief Psychiatric Rating Scale (BPRS) or the Positive and Negative Syndrome Scale (PANSS) in response to an acute ketamine challenge in healthy participants or patients with schizophrenia.

Study Selection

Of 8464 citations retrieved, 36 studies involving healthy participants were included. Inclusion criteria were studies (1) including healthy participants; (2) reporting symptoms occurring in response to acute administration of subanesthetic doses of ketamine (racemic ketamine, s-ketamine, r-ketamine) intravenously; (3) containing a placebo condition with a within-subject, crossover design; (4) measuring total positive or negative symptoms using BPRS or PANSS; and (5) providing data allowing the estimation of the mean difference and deviation between the ketamine and placebo condition.

Data Extraction and Synthesis

Two independent investigators extracted study-level data for a random-effects meta-analysis. Total, positive, and negative BPRS and PANSS scores were extracted. Subgroup analyses were conducted examining the effects of blinding status, ketamine preparation, infusion method, and time between ketamine and placebo conditions. The Meta-analysis of Observational Studies in Epidemiology (MOOSE) and Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) guidelines were followed.

Main Outcomes and Measures

Standardized mean differences (SMDs) were used as effect sizes for individual studies. Standardized mean differences between ketamine and placebo conditions were calculated for total, positive, and negative BPRS and PANSS scores.

Results

The overall sample included 725 healthy volunteers (mean [SD] age, 28.3 [3.6] years; 533 [73.6%] male) exposed to the ketamine and placebo conditions. Racemic ketamine or S-ketamine was associated with a statistically significant increase in transient psychopathology in healthy participants for total (SMD = 1.50 [95% CI, 1.23-1.77]; P < .001), positive (SMD = 1.55 [95% CI, 1.29-1.81]; P < .001), and negative (SMD = 1.16 [95% CI, 0.96-1.35]; P < .001) symptom ratings relative to the placebo condition. The effect size for this association was significantly greater for positive than negative symptoms of psychosis (estimate, 0.36 [95% CI, 0.12-0.61]; P = .004). There was significant inconsistency in outcomes between studies (I2 range, 77%-83%). Bolus followed by constant infusion increased ketamine's association with positive symptoms relative to infusion alone (effect size, 1.63 [95% CI, 1.36-1.90] vs 0.84 [95% CI, 0.35-1.33]; P = .006). Single-day study design increased ketamine's ability to generate total symptoms (effect size, 2.29 [95% CI, 1.69-2.89] vs 1.39 [95% CI, 1.12-1.66]; P = .007), but age and sex did not moderate outcomes. Insufficient studies were available for meta-analysis of studies in schizophrenia. Of these studies, 2 found a statistically significant increase in symptoms with ketamine administration in total and positive symptoms. Only 1 study found an increase in negative symptom severity with ketamine.

Conclusions and Relevance

This study found that acute ketamine administration was associated with schizophrenia-like or psychotomimetic symptoms with large effect sizes, but there was a greater increase in positive than negative symptoms and when a bolus was used. These findings suggest that bolus doses should be avoided in the therapeutic use of ketamine to minimize the risk of inducing transient positive (psychotic) symptoms.

A Bayesian approach to discrete multiple outcome network meta-analysis

In this paper we suggest a new Bayesian approach to network meta-analysis for the case of discrete multiple outcomes. The joint distribution of the discrete outcomes is modeled through a Gaussian copula with binomial marginals. The remaining elements of the hierarchial random effects model are specified in a standard way, with the logit of the success probabilities given by the sum of a baseline log-odds and random effects comparing the log-odds of each treatment against the reference and having a Gaussian distribution centered at the vector of pooled effects. An adaptive Markov Chain Monte Carlo algorithm is devised for running posterior inference. The model is applied to two datasets from Cochrane reviews, already analysed in two papers so to assess and compare its performance. We implemented the model in a freely available R package called netcopula.

Outcome choice and definition in systematic reviews leads to few eligible studies included in meta-analyses: a case study

BACKGROUND

There is broad recognition of the importance of evidence in informing clinical decisions. When information from all studies included in a systematic review ("review") does not contribute to a meta-analysis, decision-makers can be frustrated. Our objectives were to use the field of eyes and vision as a case study and examine the extent to which authors of Cochrane reviews conducted meta-analyses for their review's pre-specified main outcome domain and the reasons that some otherwise eligible studies were not incorporated into meta-analyses.

METHODS

We examined all completed systematic reviews published by Cochrane Eyes and Vision, as of August 11, 2017. We extracted information about each review's outcomes and, using an algorithm, categorized one outcome as its "main" outcome. We calculated the percentage of included studies incorporated into meta-analyses for any outcome and for the main outcome. We examined reasons for non-inclusion of studies into the meta-analysis for the main outcome.

RESULTS

We identified 175 completed reviews, of which 125 reviews included two or more studies. Across these 125 reviews, the median proportions of studies incorporated into at least one meta-analysis for any outcome and for the main outcome were 74% (interquartile range [IQR] 0-100%) and 28% (IQR 0-71%), respectively. Fifty-one reviews (41%) could not conduct a meta-analysis for the main outcome, mostly because fewer than two included studies measured the outcome (21/51 reviews) or the specific measurements for the outcome were inconsistent (16/51 reviews).

CONCLUSIONS

Outcome choice during systematic reviews can lead to few eligible studies included in meta-analyses. Core outcome sets and improved reporting of outcomes can help solve some of these problems.

Empirical comparison of univariate and multivariate meta-analyses in Cochrane Pregnancy and Childbirth reviews with multiple binary outcomes

BACKGROUND

Multivariate meta-analysis (MVMA) jointly synthesizes effects for multiple correlated outcomes. The MVMA model is potentially more difficult and time-consuming to apply than univariate models, so if its use makes little difference to parameter estimates, it could be argued that it is redundant.

METHODS

We assessed the applicability and impact of MVMA in Cochrane Pregnancy and Childbirth (CPCB) systematic reviews. We applied MVMA to CPCB reviews published between 2011 and 2013 with two or more binary outcomes with at least three studies and compared findings with results of univariate meta-analyses. Univariate random effects meta-analysis models were fitted using restricted maximum likelihood estimation (REML).

RESULTS

Eighty CPCB reviews were published. MVMA could not be applied in 70 of these reviews. MVMA was not feasible in three of the remaining 10 reviews because the appropriate models failed to converge. Estimates from MVMA agreed with those of univariate analyses in most of the other seven reviews. Statistical significance changed in two reviews: In one, this was due to a very small change in P value; in the other, the MVMA result for one outcome suggested that previous univariate results may be vulnerable to small-study effects and that the certainty of clinical conclusions needs consideration.

CONCLUSIONS

MVMA methods can be applied only in a minority of reviews of interventions in pregnancy and childbirth and can be difficult to apply because of missing correlations or lack of convergence. Nevertheless, clinical and/or statistical conclusions from MVMA may occasionally differ from those from univariate analyses.

Multivariate Methods for Meta-Analysis of Genetic Association Studies

Multivariate meta-analysis of genetic association studies and genome-wide association studies has received a remarkable attention as it improves the precision of the analysis. Here, we review, summarize and present in a unified framework methods for multivariate meta-analysis of genetic association studies and genome-wide association studies. Starting with the statistical methods used for robust analysis and genetic model selection, we present in brief univariate methods for meta-analysis and we then scrutinize multivariate methodologies. Multivariate models of meta-analysis for a single gene-disease association studies, including models for haplotype association studies, multiple linked polymorphisms and multiple outcomes are discussed. The popular Mendelian randomization approach and special cases of meta-analysis addressing issues such as the assumption of the mode of inheritance, deviation from Hardy-Weinberg Equilibrium and gene-environment interactions are also presented. All available methods are enriched with practical applications and methodologies that could be developed in the future are discussed. Links for all available software implementing multivariate meta-analysis methods are also provided.

Borrowing of strength from indirect evidence in 40 network meta-analyses

OBJECTIVES

Network meta-analysis (NMA) is increasingly being used to synthesize direct and indirect evidence and help decision makers simultaneously compare multiple treatments. We empirically evaluate the incremental gain in precision achieved by incorporating indirect evidence in NMAs.

STUDY DESIGN AND SETTING

We performed both network and pairwise meta-analyses on 40 published data sets of multiple-treatment comparisons. Their results were compared using the recently proposed borrowing of strength (BoS) statistic, which quantifies the percentage reduction in the uncertainty of the effect estimate when adding indirect evidence to an NMA.

RESULTS

We analyzed 915 possible treatment comparisons, from which 484 (53%) had no direct evidence (BoS = 100%). In 181 comparisons with only one study contributing direct evidence, NMAs resulted in reduced precision (BoS < 0) and no appreciable improvements in precision (0 < BoS < 30%) for 104 (57.5%) and 23 (12.7%) comparisons, respectively. In 250 comparisons with at least two studies contributing direct evidence, NMAs provided increased precision with BoS ≥ 30% for 166 (66.4%) comparisons.

CONCLUSION

Although NMAs have the potential to provide more precise results than those only based on direct evidence, the incremental gain may reliably occur only when at least two head-to-head studies are available and treatments are well connected. Researchers should routinely report and compare the results from both network and pairwise meta-analyses.

Multivariate network meta-analysis to mitigate the effects of outcome reporting bias

Outcome reporting bias (ORB) is recognized as a threat to the validity of both pairwise and network meta-analysis (NMA). In recent years, multivariate meta-analytic methods have been proposed to reduce the impact of ORB in the pairwise setting. These methods have shown that multivariate meta-analysis can reduce bias and increase efficiency of pooled effect sizes. However, it is unknown whether multivariate NMA (MNMA) can similarly reduce the impact of ORB. Additionally, it is quite challenging to implement MNMA due to the fact that correlation between treatments and outcomes must be modeled; thus, the dimension of the covariance matrix and number of components to estimate grows quickly with the number of treatments and number of outcomes. To determine whether MNMA can reduce the effects of ORB on pooled treatment effect sizes, we present an extensive simulation study of Bayesian MNMA. Via simulation studies, we show that MNMA reduces the bias of pooled effect sizes under a variety of outcome missingness scenarios, including missing at random and missing not at random. Further, MNMA improves the precision of estimates, producing narrower credible intervals. We demonstrate the applicability of the approach via application of MNMA to a multi-treatment systematic review of randomized controlled trials of anti-depressants for the treatment of depression in older adults.

Bayesian Multiparameter Evidence Synthesis to Inform Decision Making: A Case Study in Metastatic Hormone-Refractory Prostate Cancer

In health technology assessment, decisions are based on complex cost-effectiveness models that require numerous input parameters. When not all relevant estimates are available, the model may have to be simplified. Multiparameter evidence synthesis combines data from diverse sources of evidence, which results in obtaining estimates required in clinical decision making that otherwise may not be available. We demonstrate how bivariate meta-analysis can be used to predict an unreported estimate of a treatment effect enabling implementation of a multistate Markov model, which otherwise needs to be simplified. To illustrate this, we used an example of cost-effectiveness analysis for docetaxel in combination with prednisolone in metastatic hormone-refractory prostate cancer. Bivariate meta-analysis was used to model jointly available data on treatment effects on overall survival and progression-free survival (PFS) to predict the unreported effect on PFS in a study evaluating docetaxel with prednisolone. The predicted treatment effect on PFS enabled implementation of a 3-state Markov model comprising stable disease, progressive disease, and dead states, while lack of the estimate restricted the model to a 2-state model (with alive and dead states). The 2-state and 3-state models were compared by calculating the incremental cost-effectiveness ratio (which was much lower in the 3-state model: £22,148 per quality-adjusted life year gained compared to £30,026 obtained from the 2-state model) and the expected value of perfect information (which increased with the 3-state model). The 3-state model has the advantage of distinguishing surviving patients who progressed from those who did not progress. Hence, the use of advanced meta-analytic techniques allowed obtaining relevant parameter estimates to populate a model describing disease pathway in more detail while helping to prevent valuable clinical data from being discarded.

Dealing with effect size multiplicity in systematic reviews and meta-analyses

Systematic reviews often encounter primary studies that report multiple effect sizes based on data from the same participants. These have the potential to introduce statistical dependency into the meta-analytic data set. In this paper, we provide a tutorial on dealing with effect size multiplicity within studies in the context of meta-analyses of intervention and association studies, recommending a three-step approach. The first step is to define the research question and consider the extent to which it mainly reflects interest in mean effect sizes (which we term a convergent approach) or an interest in exploring heterogeneity (which we term a divergent approach). A second step is to identify the types of multiplicities that appear in the initial database of effect sizes relevant to the research question, and we propose a categorization scheme to differentiate them. The third step is to select a strategy for dealing with each type of multiplicity. The researcher can choose between a reductionist meta-analytic approach, which is characterized by inclusion of a single effect size per study, and an integrative approach, characterized by inclusion of multiple effect sizes per study. We present an overview of available analysis strategies for dealing with effect size multiplicity within studies and provide recommendations intended to help researchers decide which strategy might be preferable in particular situations. Last, we offer caveats and cautions about addressing the challenges multiplicity poses for systematic reviews and meta-analyses.

Bayesian multivariate meta-analysis of multiple factors

In medical sciences, a disease condition is typically associated with multiple risk and protective factors. Although many studies report results of multiple factors, nearly all meta-analyses separately synthesize the association between each factor and the disease condition of interest. The collected studies usually report different subsets of factors, and the results from separate analyses on multiple factors may not be comparable because each analysis may use different subpopulation. This may impact on selecting most important factors to design a multifactor intervention program. This article proposes a new concept, multivariate meta-analysis of multiple factors (MVMA-MF), to synthesize all available factors simultaneously. By borrowing information across factors, MVMA-MF can improve statistical efficiency and reduce biases compared with separate analyses when factors were missing not at random. As within-study correlations between factors are commonly unavailable from published articles, we use a Bayesian hybrid model to perform MVMA-MF, which effectively accounts for both within- and between-study correlations. The performance of MVMA-MF and the conventional methods are compared using simulations and an application to a pterygium dataset consisting of 29 studies on 8 risk factors.

The role of secondary outcomes in multivariate meta-analysis

Univariate meta‐analysis concerns a single outcome of interest measured across a number of independent studies. However, many research studies will have also measured secondary outcomes. Multivariate meta‐analysis allows us to take these secondary outcomes into account and can also include studies where the primary outcome is missing. We define the efficiency E as the variance of the overall estimate from a multivariate meta‐analysis relative to the variance of the overall estimate from a univariate meta‐analysis. The extra information gained from a multivariate meta‐analysis of n studies is then similar to the extra information gained if a univariate meta‐analysis of the primary effect had a further n(1−E)/E studies. The variance contribution of a study's secondary outcomes (its borrowing of strength) can be thought of as a contrast between the variance matrix of the outcomes in that study and the set of variance matrices of all the studies in the meta‐analysis. In the bivariate case this is given a simple graphical interpretation as the borrowing‐of‐strength plot. We discuss how these findings can also be used in the context of random‐effects meta‐analysis. Our discussion is motivated by a published meta‐analysis of 10 antihypertension clinical trials.

Comparative Efficacy of Drugs for Preventing Acute Kidney Injury after Cardiac Surgery: A Network Meta-Analysis

BACKGROUND

Acute kidney injury (AKI) occurs frequently after cardiac surgery and has been associated with increased hospital length of stay, mortality, and costs.

OBJECTIVE

We aimed to evaluate the efficacy of pharmacologic strategies for preventing AKI after cardiac surgery.

METHODS

We searched PubMed, Embase, the Cochrane Central Register of Controlled Trials (CENTRAL) up to 6 May 2017 and the reference lists of relevant articles about trials. The outcome was the occurrence of AKI. This is the first network meta-analysis of the different prevention strategies using Bayesian methodology.

RESULTS

The study included 63 articles with 19,520 participants and evaluated the effect of ten pharmacologic strategies to prevent AKI in patients undergoing cardiac surgery. Compared with placebo, the odds ratio (OR) for the occurrence of AKI was 0.24 [95% confidence interval (CI) 0.16-0.34] with natriuretic peptide, 0.33 (95% CI 0.14-0.70) with fenoldopam, 0.54 (95% CI 0.31-0.84) with dexmedetomidine, 0.56 (95% CI 0.29-0.95) with low-dose erythropoietin, 0.63 (95% CI 0.43-0.88) with levosimendan, 0.76 (95% CI 0.52-1.10) with steroids, 0.83 (95% CI 0.48-1.40) with high-dose erythropoietin, 0.85 (95% CI 0.64-1.14) with N-acetylcysteine, 0.96 (95% CI 0.69-1.29) with sodium bicarbonate, and 1.05 (95% CI 0.70-1.41) with statins. The surface under the cumulative ranking curve probabilities indicated that natriuretic peptide was the best treatment therapy and that fenoldopam ranked second.

CONCLUSIONS

Natriuretic peptide is probably the preferred pharmacologic strategy to prevent AKI in adult patients undergoing cardiac surgery, especially in those at high risk of AKI.

Borrowing of strength and study weights in multivariate and network meta-analysis

Multivariate and network meta-analysis have the potential for the estimated mean of one effect to borrow strength from the data on other effects of interest. The extent of this borrowing of strength is usually assessed informally. We present new mathematical definitions of 'borrowing of strength'. Our main proposal is based on a decomposition of the score statistic, which we show can be interpreted as comparing the precision of estimates from the multivariate and univariate models. Our definition of borrowing of strength therefore emulates the usual informal assessment. We also derive a method for calculating study weights, which we embed into the same framework as our borrowing of strength statistics, so that percentage study weights can accompany the results from multivariate and network meta-analyses as they do in conventional univariate meta-analyses. Our proposals are illustrated using three meta-analyses involving correlated effects for multiple outcomes, multiple risk factor associations and multiple treatments (network meta-analysis).

Model-based sensitivity analysis for outcome reporting bias in the meta analysis of benefit and harm outcomes

Outcome reporting bias occurs when outcomes in research studies are selectively reported, the selection being influenced by the study results. For benefit outcomes, we have shown how risk assessments using the Outcome Reporting Bias in Trials risk classification scale can be used to calculate bias-adjusted treatment effect estimates. This paper presents a new and simpler version of the benefits method, and shows how it can be extended to cover the partial reporting and non-reporting of harm outcomes. Our motivating example is a Cochrane systematic review of 12 studies of Topiramate add-on therapy for drug-resistant partial epilepsy. Bias adjustments for partially reported or unreported outcomes suggest that the review has overestimated the benefits and underestimated the harms of the test treatment.

Multivariate and network meta-analysis of multiple outcomes and multiple treatments: rationale, concepts, and examples

Organisations such as the National Institute for Health and Care Excellence require the synthesis of evidence from existing studies to inform their decisions—for example, about the best available treatments with respect to multiple efficacy and safety outcomes. However, relevant studies may not provide direct evidence about all the treatments or outcomes of interest. Multivariate and network meta-analysis methods provide a framework to address this, using correlated or indirect evidence from such studies alongside any direct evidence. In this article, the authors describe the key concepts and assumptions of these methods, outline how correlated and indirect evidence arises, and illustrate the contribution of such evidence in real clinical examples involving multiple outcomes and multiple treatments

A matrix-based method of moments for fitting multivariate network meta-analysis models with multiple outcomes and random inconsistency effects

Random-effects meta-analyses are very commonly used in medical statistics. Recent methodological developments include multivariate (multiple outcomes) and network (multiple treatments) meta-analysis. Here, we provide a new model and corresponding estimation procedure for multivariate network meta-analysis, so that multiple outcomes and treatments can be included in a single analysis. Our new multivariate model is a direct extension of a univariate model for network meta-analysis that has recently been proposed. We allow two types of unknown variance parameters in our model, which represent between-study heterogeneity and inconsistency. Inconsistency arises when different forms of direct and indirect evidence are not in agreement, even having taken between-study heterogeneity into account. However, the consistency assumption is often assumed in practice and so we also explain how to fit a reduced model which makes this assumption. Our estimation method extends several other commonly used methods for meta-analysis, including the method proposed by DerSimonian and Laird (). We investigate the use of our proposed methods in the context of both a simulation study and a real example.

Bayesian mixed treatment comparisons meta-analysis for correlated outcomes subject to reporting bias

Many randomized controlled trials (RCTs) report more than one primary outcome. As a result, multivariate meta-analytic methods for the assimilation of treatment effects in systematic reviews of RCTs have received increasing attention in the literature. These methods show promise with respect to bias reduction and efficiency gain compared to univariate meta-analysis. However, most methods for multivariate meta-analysis have focused on pairwise treatment comparisons (i.e., when the number of treatments is two). Current methods for mixed treatment comparisons (MTC) meta-analysis (i.e., when the number of treatments is more than two) have focused on univariate or very recently, bivariate outcomes. To broaden their application, we propose a framework for MTC meta-analysis of multivariate (≥ 2) outcomes where the correlations among multivariate outcomes within- and between-studies are accounted for through copulas, and the joint modeling of multivariate random effects, respectively. We consider a Bayesian hierarchical model using Markov Chain Monte Carlo methods for estimation. An important feature of the proposed framework is that it allows for borrowing of information across correlated outcomes. We show via simulation that our approach reduces the impact of outcome reporting bias (ORB) in a variety of missing outcome scenarios. We apply the method to a systematic review of RCTs of pharmacological treatments for alcohol dependence, which tends to report multiple outcomes potentially subject to ORB.

Multivariate meta-analysis with an increasing number of parameters

Meta-analysis can average estimates of multiple parameters, such as a treatment's effect on multiple outcomes, across studies. Univariate meta-analysis (UVMA) considers each parameter individually, while multivariate meta-analysis (MVMA) considers the parameters jointly and accounts for the correlation between their estimates. The performance of MVMA and UVMA has been extensively compared in scenarios with two parameters. Our objective is to compare the performance of MVMA and UVMA as the number of parameters, p, increases. Specifically, we show that (i) for fixed-effect (FE) meta-analysis, the benefit from using MVMA can substantially increase as p increases; (ii) for random effects (RE) meta-analysis, the benefit from MVMA can increase as p increases, but the potential improvement is modest in the presence of high between-study variability and the actual improvement is further reduced by the need to estimate an increasingly large between study covariance matrix; and (iii) when there is little to no between-study variability, the loss of efficiency due to choosing RE MVMA over FE MVMA increases as p increases. We demonstrate these three features through theory, simulation, and a meta-analysis of risk factors for non-Hodgkin lymphoma.

Comparing antibiotic treatment for leptospirosis using network meta-analysis: a tutorial

BACKGROUND

Network meta-analysis consists of simultaneous analysis of both direct comparisons of interventions within randomized controlled trials and indirect comparisons across trials based on a common comparator. In this paper, we aimed to characterise the conceptual understanding and the rationale for the use of network meta-analysis in assessing drug efficacy.

METHODS

We selected randomized controlled trials, assessing efficacy of antibiotics for the treatment of leptospirosis as a case study. A pairwise meta-analysis was conducted using a random effect model, assuming that different studies assessed different but related treatment effects. The analysis was then extended to a network meta-analysis, which consists of direct and indirect evidence in a network of antibiotics trials, using a suite of multivariate meta-analysis routines of STATA (mvmeta command). We also assessed an assumption of 'consistency' that estimates of treatment effects from direct and indirect evidence are in agreement.

RESULTS

Seven randomised controlled trials were identified for this analysis. These RCTs assessed the efficacy of antibiotics such as penicillin, doxycycline and cephalosporin for the treatment of human leptospirosis. These studies made comparisons between antibiotics (i.e. an antibiotic versus alternative antibiotic) in the primary study and a placebo, except for cephalosporin. These studies were sufficient to allow the creation of a network for the network meta-analysis; a closed loop in which three comparator antibiotics were connected to each other through a polygon. The comparison of penicillin versus the placebo has the largest contribution to the entire network (31.8%). The assessment of rank probabilities indicated that penicillin presented the greatest likelihood of improving efficacy among the evaluated antibiotics for treating leptospirosis.

CONCLUSIONS

Findings suggest that network meta-analysis, a meta-analysis comparing multiple treatments, is feasible and should be considered as better precision of effect estimates for decisions when several antibiotic options are available for the treatment of leptospirosis.

Meta-Analysis of Effect Sizes Reported at Multiple Time Points Using General Linear Mixed Model

Meta-analysis of longitudinal studies combines effect sizes measured at pre-determined time points. The most common approach involves performing separate univariate meta-analyses at individual time points. This simplistic approach ignores dependence between longitudinal effect sizes, which might result in less precise parameter estimates. In this paper, we show how to conduct a meta-analysis of longitudinal effect sizes where we contrast different covariance structures for dependence between effect sizes, both within and between studies. We propose new combinations of covariance structures for the dependence between effect size and utilize a practical example involving meta-analysis of 17 trials comparing postoperative treatments for a type of cancer, where survival is measured at 6, 12, 18 and 24 months post randomization. Although the results from this particular data set show the benefit of accounting for within-study serial correlation between effect sizes, simulations are required to confirm these results.

Rethinking the assessment of risk of bias due to selective reporting: a cross-sectional study

BACKGROUND

Selective reporting is included as a core domain of Cochrane's tool for assessing risk of bias in randomised trials. There has been no evaluation of review authors' use of this domain. We aimed to evaluate assessments of selective reporting in a cross-section of Cochrane reviews and to outline areas for improvement.

METHODS

We obtained data on selective reporting judgements for 8434 studies included in 586 Cochrane reviews published from issue 1-8, 2015. One author classified the reasons for judgements of high risk of selective reporting bias. We randomly selected 100 reviews with at least one trial rated at high risk of outcome non-reporting bias (non-/partial reporting of an outcome on the basis of its results). One author recorded whether the authors of these reviews incorporated the selective reporting assessment when interpreting results.

RESULTS

Of the 8434 studies, 1055 (13 %) were rated at high risk of bias on the selective reporting domain. The most common reason was concern about outcome non-reporting bias. Few studies were rated at high risk because of concerns about bias in selection of the reported result (e.g. reporting of only a subset of measurements, analysis methods or subsets of the data that were pre-specified). Review authors often specified in the risk of bias tables the study outcomes that were not reported (84 % of studies) but less frequently specified the outcomes that were partially reported (61 % of studies). At least one study was rated at high risk of outcome non-reporting bias in 31 % of reviews. In the random sample of these reviews, only 30 % incorporated this information when interpreting results, by acknowledging that the synthesis of an outcome was missing data that were not/partially reported.

CONCLUSIONS

Our audit of user practice in Cochrane reviews suggests that the assessment of selective reporting in the current risk of bias tool does not work well. It is not always clear which outcomes were selectively reported or what the corresponding risk of bias is in the synthesis with missing outcome data. New tools that will make it easier for reviewers to convey this information are being developed.

Bayesian bivariate meta-analysis of correlated effects: Impact of the prior distributions on the between-study correlation, borrowing of strength, and joint inferences

Multivariate random-effects meta-analysis allows the joint synthesis of correlated results from multiple studies, for example, for multiple outcomes or multiple treatment groups. In a Bayesian univariate meta-analysis of one endpoint, the importance of specifying a sensible prior distribution for the between-study variance is well understood. However, in multivariate meta-analysis, there is little guidance about the choice of prior distributions for the variances or, crucially, the between-study correlation, ρ_B; for the latter, researchers often use a Uniform(−1,1) distribution assuming it is vague. In this paper, an extensive simulation study and a real illustrative example is used to examine the impact of various (realistically) vague prior distributions for ρ_B and the between-study variances within a Bayesian bivariate random-effects meta-analysis of two correlated treatment effects. A range of diverse scenarios are considered, including complete and missing data, to examine the impact of the prior distributions on posterior results (for treatment effect and between-study correlation), amount of borrowing of strength, and joint predictive distributions of treatment effectiveness in new studies. Two key recommendations are identified to improve the robustness of multivariate meta-analysis results. First, the routine use of a Uniform(−1,1) prior distribution for ρ_B should be avoided, if possible, as it is not necessarily vague. Instead, researchers should identify a sensible prior distribution, for example, by restricting values to be positive or negative as indicated by prior knowledge. Second, it remains critical to use sensible (e.g. empirically based) prior distributions for the between-study variances, as an inappropriate choice can adversely impact the posterior distribution for ρ_B, which may then adversely affect inferences such as joint predictive probabilities. These recommendations are especially important with a small number of studies and missing data.

GetReal in network meta-analysis: a review of the methodology

Pairwise meta-analysis is an established statistical tool for synthesizing evidence from multiple trials, but it is informative only about the relative efficacy of two specific interventions. The usefulness of pairwise meta-analysis is thus limited in real-life medical practice, where many competing interventions may be available for a certain condition and studies informing some of the pairwise comparisons may be lacking. This commonly encountered scenario has led to the development of network meta-analysis (NMA). In the last decade, several applications, methodological developments, and empirical studies in NMA have been published, and the area is thriving as its relevance to public health is increasingly recognized. This article presents a review of the relevant literature on NMA methodology aiming to pinpoint the developments that have appeared in the field. Copyright © 2016 John Wiley & Sons, Ltd.

Multivariate Meta-Analysis of Genetic Association Studies: A Simulation Study

In a meta-analysis with multiple end points of interests that are correlated between or within studies, multivariate approach to meta-analysis has a potential to produce more precise estimates of effects by exploiting the correlation structure between end points. However, under random-effects assumption the multivariate estimation is more complex (as it involves estimation of more parameters simultaneously) than univariate estimation, and sometimes can produce unrealistic parameter estimates. Usefulness of multivariate approach to meta-analysis of the effects of a genetic variant on two or more correlated traits is not well understood in the area of genetic association studies. In such studies, genetic variants are expected to roughly maintain Hardy-Weinberg equilibrium within studies, and also their effects on complex traits are generally very small to modest and could be heterogeneous across studies for genuine reasons. We carried out extensive simulation to explore the comparative performance of multivariate approach with most commonly used univariate inverse-variance weighted approach under random-effects assumption in various realistic meta-analytic scenarios of genetic association studies of correlated end points. We evaluated the performance with respect to relative mean bias percentage, and root mean square error (RMSE) of the estimate and coverage probability of corresponding 95% confidence interval of the effect for each end point. Our simulation results suggest that multivariate approach performs similarly or better than univariate method when correlations between end points within or between studies are at least moderate and between-study variation is similar or larger than average within-study variation for meta-analyses of 10 or more genetic studies. Multivariate approach produces estimates with smaller bias and RMSE especially for the end point that has randomly or informatively missing summary data in some individual studies, when the missing data in the endpoint are imputed with null effects and quite large variance.

Multivariate meta-analysis of individual participant data helped externally validate the performance and implementation of a prediction model

OBJECTIVES

Our aim was to improve meta-analysis methods for summarizing a prediction model's performance when individual participant data are available from multiple studies for external validation.

STUDY DESIGN AND SETTING

We suggest multivariate meta-analysis for jointly synthesizing calibration and discrimination performance, while accounting for their correlation. The approach estimates a prediction model's average performance, the heterogeneity in performance across populations, and the probability of "good" performance in new populations. This allows different implementation strategies (e.g., recalibration) to be compared. Application is made to a diagnostic model for deep vein thrombosis (DVT) and a prognostic model for breast cancer mortality.

RESULTS

In both examples, multivariate meta-analysis reveals that calibration performance is excellent on average but highly heterogeneous across populations unless the model's intercept (baseline hazard) is recalibrated. For the cancer model, the probability of "good" performance (defined by C statistic ≥0.7 and calibration slope between 0.9 and 1.1) in a new population was 0.67 with recalibration but 0.22 without recalibration. For the DVT model, even with recalibration, there was only a 0.03 probability of "good" performance.

CONCLUSION

Multivariate meta-analysis can be used to externally validate a prediction model's calibration and discrimination performance across multiple populations and to evaluate different implementation strategies.