Bayesian approaches to random-effects meta-analysis: a comparative study

Heterogeneity in meta-analysis: a comprehensive overview

In recent years, meta-analysis has evolved to a critically important field of Statistics, and has significant applications in Medicine and Health Sciences. In this work we briefly present existing methodologies to conduct meta-analysis along with any discussion and recent developments accompanying them. Undoubtedly, studies brought together in a systematic review will differ in one way or another. This yields a considerable amount of variability, any kind of which may be termed heterogeneity. To this end, reports of meta-analyses commonly present a statistical test of heterogeneity when attempting to establish whether the included studies are indeed similar in terms of the reported output or not. We intend to provide an overview of the topic, discuss the potential sources of heterogeneity commonly met in the literature and provide useful guidelines on how to address this issue and to detect heterogeneity. Moreover, we review the recent developments in the Bayesian approach along with the various graphical tools and statistical software that are currently available to the analyst. In addition, we discuss sensitivity analysis issues and other approaches of understanding the causes of heterogeneity. Finally, we explore heterogeneity in meta-analysis for time to event data in a nutshell, pointing out its unique characteristics.

Writing a Scientific Review Article: Comprehensive Insights for Beginners

Review articles present comprehensive overview of relevant literature on specific themes and synthesise the studies related to these themes, with the aim of strengthening the foundation of knowledge and facilitating theory development. The significance of review articles in science is immeasurable as both students and researchers rely on these articles as the starting point for their research. Interestingly, many postgraduate students are expected to write review articles for journal publications as a way of demonstrating their ability to contribute to new knowledge in their respective fields. However, there is no comprehensive instructional framework to guide them on how to analyse and synthesise the literature in their niches into publishable review articles. The dearth of ample guidance or explicit training results in students having to learn all by themselves, usually by trial and error, which often leads to high rejection rates from publishing houses. Therefore, this article seeks to identify these challenges from a beginner's perspective and strives to plug the identified gaps and discrepancies. Thus, the purpose of this paper is to serve as a systematic guide for emerging scientists and to summarise the most important information on how to write and structure a publishable review article.

Rare events meta-analysis using the Bayesian beta-binomial model

In meta-analyses of rare events, it can be challenging to obtain a reliable estimate of the pooled effect, in particular when the meta-analysis is based on a small number of studies. Recent simulation studies have shown that the beta-binomial model is a promising candidate in this situation, but have thus far only investigated its performance in a frequentist framework. In this study, we aim to make the beta-binomial model for meta-analysis of rare events amenable to Bayesian inference by proposing prior distributions for the effect parameter and investigating the models' robustness to different specifications of priors for the scale parameter. To evaluate the performance of Bayesian beta-binomial models with different priors, we conducted a simulation study with two different data generating models in which we varied the size of the pooled effect, the degree of heterogeneity, the baseline probability, and the sample size. Our results show that while some caution must be exercised when using the Bayesian beta-binomial in meta-analyses with extremely sparse data, the use of a weakly informative prior for the effect parameter is beneficial in terms of mean bias, mean squared error, and coverage. For the scale parameter, half-normal and exponential distributions are identified as candidate priors in meta-analysis of rare events using the Bayesian beta-binomial model.

Goodness-of-fit testing for meta-analysis of rare binary events

Random-effects (RE) meta-analysis is a crucial approach for combining results from multiple independent studies that exhibit heterogeneity. Recently, two frequentist goodness-of-fit (GOF) tests were proposed to assess the fit of RE model. However, they tend to perform poorly when assessing rare binary events. Under a general binomial-normal framework, we propose a novel GOF test for the meta-analysis of rare events. Our method is based on pivotal quantities that play an important role in Bayesian model assessment. It further adopts the Cauchy combination idea proposed in a 2019 JASA paper, to combine dependent p-values computed using posterior samples from Markov Chain Monte Carlo. The advantages of our method include clear conception and interpretation, incorporation of all data including double zeros without the need for artificial correction, well-controlled Type I error, and generally improved ability in detecting model misfits compared to previous GOF methods. We illustrate the proposed method via simulation and three real data applications.

Design and Statistical Innovations in a Platform Trial for Amyotrophic Lateral Sclerosis

Platform trials allow efficient evaluation of multiple interventions for a specific disease. The HEALEY ALS Platform Trial is testing multiple investigational products in parallel and sequentially in persons with amyotrophic lateral sclerosis (ALS) with the goal of rapidly identifying novel treatments to slow disease progression. Platform trials have considerable operational and statistical efficiencies compared with typical randomized controlled trials due to their use of shared infrastructure and shared control data. We describe the statistical approaches required to achieve the objectives of a platform trial in the context of ALS. This includes following regulatory guidance for the disease area of interest and accounting for potential differences in outcomes of participants within the shared control (potentially due to differences in time of randomization, mode of administration, and eligibility criteria). Within the HEALEY ALS Platform Trial, the complex statistical objectives are met using a Bayesian shared parameter analysis of function and survival. This analysis serves to provide a common integrated estimate of treatment benefit, overall slowing in disease progression, as measured by function and survival while accounting for potential differences in the shared control group using Bayesian hierarchical modeling. Clinical trial simulation is used to provide a better understanding of this novel analysis method and complex design. ANN NEUROL 2023;94:547-560.

The effects of oral ferrous bisglycinate supplementation on hemoglobin and ferritin concentrations in adults and children: a systematic review and meta-analysis of randomized controlled trials

CONTEXT

Iron deficiency and anemia have serious consequences, especially for children and pregnant women. Iron salts are commonly provided as oral supplements to prevent and treat iron deficiency, despite poor bioavailability and frequently reported adverse side effects. Ferrous bisglycinate is a novel amino acid iron chelate that is thought to be more bioavailable and associated with fewer gastrointestinal (GI) adverse events as compared with iron salts.

OBJECTIVE

A systematic review and meta-analysis of randomized controlled trials (RCTs) was conducted to evaluate the effects of ferrous bisglycinate supplementation compared with other iron supplements on hemoglobin and ferritin concentrations and GI adverse events.

DATA SOURCES

A systematic search of electronic databases and grey literature was performed up to July 17, 2020, yielding 17 RCTs that reported hemoglobin or ferritin concentrations following at least 4 weeks' supplementation of ferrous bisglycinate compared with other iron supplements in any dose or frequency.

DATA EXTRACTION

Random-effects meta-analyses were conducted among trials of pregnant women (n = 9) and children (n = 4); pooled estimates were expressed as standardized mean differences (SMDs). Incidence rate ratios (IRRs) were estimated for GI adverse events, using Poisson generalized linear mixed-effects models. The remaining trials in other populations (n = 4; men and nonpregnant women) were qualitatively evaluated.

DATA ANALYSIS

Compared with other iron supplements, supplementation with ferrous bisglycinate for 4-20 weeks resulted in higher hemoglobin concentrations in pregnant women (SMD, 0.54 g/dL; 95% confidence interval [CI], 0.15-0.94; P < 0.01) and fewer reported GI adverse events (IRR, 0.36; 95%CI, 0.17-0.76; P < 0.01). We observed a non-significant trend for higher ferritin concentrations in pregnant women supplemented with ferrous bisglycinate. No significant differences in hemoglobin or ferritin concentrations were detected among children.

CONCLUSION

Ferrous bisglycinate shows some benefit over other iron supplements in increasing hemoglobin concentration and reducing GI adverse events among pregnant women. More trials are needed to assess the efficacy of ferrous bisglycinate against other iron supplements in other populations.

PROSPERO REGISTRATION NO

CRD42020196984.

Bayesian estimation and testing in random-effects meta-analysis of rare binary events allowing for flexible group variability

Rare binary events data arise frequently in medical research. Due to lack of statistical power in individual studies involving such data, meta-analysis has become an increasingly important tool for combining results from multiple independent studies. However, traditional meta-analysis methods often report severely biased estimates in such rare-event settings. Moreover, many rely on models assuming a pre-specified direction for variability between control and treatment groups for mathematical convenience, which may be violated in practice. Based on a flexible random-effects model that removes the assumption about the direction, we propose new Bayesian procedures for estimating and testing the overall treatment effect and inter-study heterogeneity. Our Markov chain Monte Carlo algorithm employs Pólya-Gamma augmentation so that all conditionals are known distributions, greatly facilitating computational efficiency. Our simulation shows that the proposed approach generally reports less biased and more stable estimates compared to existing methods. We further illustrate our approach using two real examples, one using rosiglitazone data from 56 studies and the other using stomach ulcers data from 41 studies.

The normality assumption on between-study random effects was questionable in a considerable number of Cochrane meta-analyses

BACKGROUND

Studies included in a meta-analysis are often heterogeneous. The traditional random-effects models assume their true effects to follow a normal distribution, while it is unclear if this critical assumption is practical. Violations of this between-study normality assumption could lead to problematic meta-analytical conclusions. We aimed to empirically examine if this assumption is valid in published meta-analyses.

METHODS

In this cross-sectional study, we collected meta-analyses available in the Cochrane Library with at least 10 studies and with between-study variance estimates > 0. For each extracted meta-analysis, we performed the Shapiro-Wilk (SW) test to quantitatively assess the between-study normality assumption. For binary outcomes, we assessed between-study normality for odds ratios (ORs), relative risks (RRs), and risk differences (RDs). Subgroup analyses based on sample sizes and event rates were used to rule out the potential confounders. In addition, we obtained the quantile-quantile (Q-Q) plot of study-specific standardized residuals for visually assessing between-study normality.

RESULTS

Based on 4234 eligible meta-analyses with binary outcomes and 3433 with non-binary outcomes, the proportion of meta-analyses that had statistically significant non-normality varied from 15.1 to 26.2%. RDs and non-binary outcomes led to more frequent non-normality issues than ORs and RRs. For binary outcomes, the between-study non-normality was more frequently found in meta-analyses with larger sample sizes and event rates away from 0 and 100%. The agreements of assessing the normality between two independent researchers based on Q-Q plots were fair or moderate.

CONCLUSIONS

The between-study normality assumption is commonly violated in Cochrane meta-analyses. This assumption should be routinely assessed when performing a meta-analysis. When it may not hold, alternative meta-analysis methods that do not make this assumption should be considered.

A Bayesian model for combining standardized mean differences and odds ratios in the same meta-analysis

In meta-analysis practice, researchers frequently face studies that report the same outcome differently, such as a continuous variable (e.g., scores for rating depression) or a binary variable (e.g., counts of patients with depression dichotomized by certain latent and unreported depression scores). For combining these two types of studies in the same analysis, a simple conversion method has been widely used to handle standardized mean differences (SMDs) and odds ratios (ORs). This conventional method uses a linear function connecting the SMD and log OR; it assumes logistic distributions for (latent) continuous measures. However, the normality assumption is more commonly used for continuous measures, and the conventional method may be inaccurate when effect sizes are large or cutoff values for dichotomizing binary events are extreme (leading to rare events). This article proposes a Bayesian hierarchical model to synthesize SMDs and ORs without using the conventional conversion method. This model assumes exact likelihoods for continuous and binary outcome measures, which account for full uncertainties in the synthesized results. We performed simulation studies to compare the performance of the conventional and Bayesian methods in various settings. The Bayesian method generally produced less biased results with smaller mean squared errors and higher coverage probabilities than the conventional method in most cases. Nevertheless, this superior performance depended on the normality assumption for continuous measures; the Bayesian method could lead to nonignorable biases for non-normal data. In addition, we used two case studies to illustrate the proposed Bayesian method in real-world settings.

Using the bayesmeta R package for Bayesian random-effects meta-regression

BACKGROUND

Random-effects meta-analysis within a hierarchical normal modeling framework is commonly implemented in a wide range of evidence synthesis applications. More general problems may even be tackled when considering meta-regression approaches that in addition allow for the inclusion of study-level covariables.

METHODS

We describe the Bayesian meta-regression implementation provided in the bayesmetaR package including the choice of priors, and we illustrate its practical use.

RESULTS

A wide range of example applications are given, such as binary and continuous covariables, subgroup analysis, indirect comparisons, and model selection. Example R code is provided.

CONCLUSIONS

The bayesmeta package provides a flexible implementation. Due to the avoidance of MCMC methods, computations are fast and reproducible, facilitating quick sensitivity checks or large-scale simulation studies.

Bayesian fusion: scalable unification of distributed statistical analyses

There has been considerable interest in addressing the problem of unifying distributed analyses into a single coherent inference, which arises in big-data settings, when working under privacy constraints, and in Bayesian model choice. Most existing approaches relied upon approximations of the distributed analyses, which have significant shortcomings—the quality of the inference can degrade rapidly with the number of analyses being unified, and can be substantially biased when unifying analyses that do not concur. In contrast, recent Monte Carlo fusion approach is exact and based on rejection sampling. In this paper, we introduce a practical Bayesian fusion approach by embedding the Monte Carlo fusion framework within a sequential Monte Carlo algorithm. We demonstrate theoretically and empirically that Bayesian fusion is more robust than existing methods.

Heterogeneity estimation in meta-analysis of standardized mean differences when the distribution of random effects departs from normal: A Monte Carlo simulation study

BACKGROUND

Advantages of meta-analysis depend on the assumptions underlying the statistical procedures used being met. One of the main assumptions that is usually taken for granted is the normality underlying the population of true effects in a random-effects model, even though the available evidence suggests that this assumption is often not met. This paper examines how 21 frequentist and 24 Bayesian methods, including several novel procedures, for computing a point estimate of the heterogeneity parameter ([Formula: see text]) perform when the distribution of random effects departs from normality compared to normal scenarios in meta-analysis of standardized mean differences.

METHODS

A Monte Carlo simulation was carried out using the R software, generating data for meta-analyses using the standardized mean difference. The simulation factors were the number and average sample size of primary studies, the amount of heterogeneity, as well as the shape of the random-effects distribution. The point estimators were compared in terms of absolute bias and variance, although results regarding mean squared error were also discussed.

RESULTS

Although not all the estimators were affected to the same extent, there was a general tendency to obtain lower and more variable [Formula: see text] estimates as the random-effects distribution departed from normality. However, the estimators ranking in terms of their absolute bias and variance did not change: Those estimators that obtained lower bias also showed greater variance. Finally, a large number and sample size of primary studies acted as a bias-protective factor against a lack of normality for several procedures, whereas only a high number of studies was a variance-protective factor for most of the estimators analyzed.

CONCLUSIONS

Although the estimation and inference of the combined effect have proven to be sufficiently robust, our work highlights the role that the deviation from normality may be playing in the meta-analytic conclusions from the simulation results and the numerical examples included in this work. With the aim to exercise caution in the interpretation of the results obtained from random-effects models, the tau2() R function is made available for obtaining the range of [Formula: see text] values computed from the 45 estimators analyzed in this work, as well as to assess how the pooled effect, its confidence and prediction intervals vary according to the estimator chosen.

Global monkeypox case hospitalisation rates: A rapid systematic review and meta-analysis

Background

Estimates of the case hospitalization rate and case fatality rate when hospital care is available for monkeypox (MPX) infections have not been well defined. This rapid systematic review and meta-analysis aimed to estimate the case hospitalisation rate and case fatality rate where hospital care is available.

Methods

We systematically searched PubMed, Embase, the Lancet Preprints, and MedRxiv for studies published between Jan 1, 1950 and Aug 2, 2022. We included documents which contained both the number of cases and associated hospitalisations of MPX infections. From eligible studies we extracted the country, the year of the study, the study design type, the clade of MPX, the participant characteristics, transmission type, any treatments used, number of cases (including suspected, probable, or laboratory confirmed diagnosis), number of hospitalizations, hospitalized patient outcomes, and case definition. Case hospitalization rate (CHR) was defined as the proportion of cases that were admitted to hospital care while case fatality rate (CFR) was defined as the proportion of cases that died. CHR and CFR were analysed in a fully Bayesian meta-analytic framework using random effects models, including sub-group analysis with heterogeneity assessed using I2.

Findings

Of the 259 unique documents identified, 19 studies were eligible for inclusion. Included studies represented 7553 reported cases among which there were 555 hospitalizations. Of the 7540 cases for which outcomes were available, there were 15 recorded deaths. The median age of cases was 35 years (interquartile range 28-38, n = 2010) and primarily male (7339/7489, 98%) in studies where age or sex were available. Combined CHR was estimated to be 14.1% (95% credible interval, 7.5-25.0, I2 97.4%), with a high degree of heterogeneity. Further analysis by outbreak period indicates CHRs of 49.8% (28.2-74.0, I2 81.4%), 21.7% (7.2-52.1, I2 57.7%), and 5.8% (3.2-9.4, I2 92.4%) during the pre-2017, 2017-2021, and 2022 outbreaks, respectively, again with high levels of heterogeneity. CFR was estimated to be 0.03% (0.0-0.44, I2 99.9%), with evidence of large heterogeneity between the studies.

Interpretation

There is limited data for MPX hospitalization rates in countries where MPX has been traditionally non-endemic until the current outbreak. Due to substantial heterogeneity, caution is needed when interpreting these findings. Health care organizations should be cognizant of the potential increase in healthcare utilization. Rapid identification of infection and use of appropriate therapies such as antivirals play a role reducing the CHR and associated CFR.

Funding

None.

Bayesian sparse modeling to identify high-risk subgroups in meta-analysis of safety data

Meta-analysis allows researchers to combine evidence from multiple studies, making it a powerful tool for synthesizing information on the safety profiles of new medical interventions. There is a critical need to identify subgroups at high risk of experiencing treatment-related toxicities. However, this remains quite challenging from a statistical perspective as there are a variety of clinical risk factors that may be relevant for different types of adverse events, and adverse events of interest may be rare or incompletely reported. We frame this challenge as a variable selection problem and propose a Bayesian hierarchical model which incorporates a horseshoe prior on the interaction terms to identify high-risk groups. Our proposed model is motivated by a meta-analysis of adverse events in cancer immunotherapy, and our results uncover key factors driving the risk of specific types of treatment-related adverse events.

Theory and practice of Bayesian and frequentist frameworks for network meta-analysis

Network meta-analysis (NMA) is an increasingly popular statistical method of synthesising evidence to assess the comparative benefits and harms of multiple treatments in a single analysis. Several automated software packages facilitate conducting NMA using either of two alternative approaches, Bayesian or frequentist frameworks. Researchers must choose a framework for conducting NMA (Bayesian or frequentist) and select appropriate model(s), and those conducting NMA need to understand the assumptions and limitations of different approaches. Bayesian models are more frequently used and can be more flexible but require checking additional assumptions and greater statistical expertise that are often ignored. The present paper describes the important theoretical aspects of Bayesian and frequentist models for NMA and the applications and considerations of contrast-synthesis and arm-synthesis NMAs. In addition, we present evidence from a limited number of simulation and empirical studies that compared different frequentist and Bayesian models and provide an overview of available automated software packages to perform NMA. We will conclude that when analysts choose appropriate models, there are seldom important differences in the results of Bayesian and frequentist approaches and that network meta-analysts should therefore focus on model features rather than the statistical framework.

Assessing and visualizing fragility of clinical results with binary outcomes in R using the fragility package

With the growing concerns about research reproducibility and replicability, the assessment of scientific results' fragility (or robustness) has been of increasing interest. The fragility index was proposed to quantify the robustness of statistical significance of clinical studies with binary outcomes. It is defined as the minimal event status modifications that can alter statistical significance. It helps clinicians evaluate the reliability of the conclusions. Many factors may affect the fragility index, including the treatment groups in which event status is modified, the statistical methods used for testing for the association between treatments and outcomes, and the pre-specified significance level. In addition to assessing the fragility of individual studies, the fragility index was recently extended to both conventional pairwise meta-analyses and network meta-analyses of multiple treatment comparisons. It is not straightforward for clinicians to calculate these measures and visualize the results. We have developed an R package called "fragility" to offer user-friendly functions for such purposes. This article provides an overview of methods for assessing and visualizing the fragility of individual studies as well as pairwise and network meta-analyses, introduces the usage of the "fragility" package, and illustrates the implementations with several worked examples.

A group-sequential randomized trial design utilizing supplemental trial data

Definitive clinical trials are resource intensive, often requiring a large number of participants over several years. One approach to improve the efficiency of clinical trials is to incorporate historical information into the primary trial analysis. This approach has tremendous potential in the areas of pediatric or rare disease trials, where achieving reasonable power is difficult. In this article, we introduce a novel Bayesian group-sequential trial design based on Multisource Exchangeability Models, which allows for dynamic borrowing of historical information at the interim analyses. Our approach achieves synergy between group sequential and adaptive borrowing methodology to attain improved power and reduced sample size. We explore the frequentist operating characteristics of our design through simulation and compare our method to a traditional group-sequential design. Our method achieves earlier stopping of the primary study while increasing power under the alternative hypothesis but has a potential for type I error inflation under some null scenarios. We discuss the issues of decision boundary determination, power and sample size calculations, and the issue of information accrual. We present our method for a continuous and binary outcome, as well as in a linear regression setting.

Physical therapy and orthopaedic equipment-induced reduction in the biomechanical risk factors related to knee osteoarthritis: a systematic review and Bayesian network meta-analysis of randomised controlled trials

OBJECTIVE

Are physical therapy or orthopaedic equipment efficacious in reducing the biomechanical risk factors in people with tibiofemoral osteoarthritis (OA)? Is there a better therapeutic intervention than others to improve these outcomes?

DESIGN

Systematic review with network meta-analysis (NMA) of randomised trials.

DATA SOURCES

PubMed, Web of Science, Cochrane Library, Embase and MEDLINE were searched through January 2021.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

We included randomised controlled trials exploring the benefits of using physical therapy or orthopaedic equipment in reducing the biomechanical risk factors which included knee adduction moment (KAM) and knee adduction angular impulse (KAAI) in individuals with tibiofemoral OA.

DATA EXTRACTION AND SYNTHESIS

Two authors extracted data independently and assessed risk of bias. We conducted an NMA to compare multiple interventions, including both direct and indirect evidences. Heterogeneity was assessed (sensitivity analysis) and quantified (I2 statistic). Grading of Recommendations Assessment, Development and Evaluation assessed the certainty of the evidence.

RESULTS

Eighteen randomised controlled trials, including 944 participants, met the inclusion criteria, of which 14 trials could be included in the NMA. Based on the collective probability of being the overall best therapy for reducing the first peak KAM, lateral wedge insoles (LWI) plus knee brace was closely followed by gait retraining, and knee brace only. Although no significant difference was observed among the eight interventions, variable-stiffness shoes and neuromuscular exercise exhibited an increase in the first peak KAM compared with the control condition group. And based on the collective probability of being the overall best therapy for reducing KAAI, gait retraining was followed by LWI only, and lower limb exercise.

CONCLUSION

The results of our study support the use of LWI plus knee brace for reducing the first peak KAM. Gait retraining did not rank highest but it influenced both KAM and KAAI and therefore it was the most recommended therapy for reducing the biomechanical risk factors.

Comparison of commonly used methods in random effects meta-analysis: application to preclinical data in drug discovery research

Background

Meta-analysis of preclinical data is used to evaluate the consistency of findings and to inform the design and conduct of future studies. Unlike clinical meta-analysis, preclinical data often involve many heterogeneous studies reporting outcomes from a small number of animals. Here, we review the methodological challenges in preclinical meta-analysis in estimating and explaining heterogeneity in treatment effects.

Methods

Assuming aggregate-level data, we focus on two topics: (1) estimation of heterogeneity using commonly used methods in preclinical meta-analysis: method of moments (DerSimonian and Laird; DL), maximum likelihood (restricted maximum likelihood; REML) and Bayesian approach; (2) comparison of univariate versus multivariable meta-regression for adjusting estimated treatment effects for heterogeneity. Using data from a systematic review on the efficacy of interleukin-1 receptor antagonist in animals with stroke, we compare these methods, and explore the impact of multiple covariates on the treatment effects.

Results

We observed that the three methods for estimating heterogeneity yielded similar estimates for the overall effect, but different estimates for between-study variability. The proportion of heterogeneity explained by a covariate is estimated larger using REML and the Bayesian method as compared with DL. Multivariable meta-regression explains more heterogeneity than univariate meta-regression.

Conclusions

Our findings highlight the importance of careful selection of the estimation method and the use of multivariable meta-regression to explain heterogeneity. There was no difference between REML and the Bayesian method and both methods are recommended over DL. Multiple meta-regression is worthwhile to explain heterogeneity by more than one variable, reducing more variability than any univariate models and increasing the explained proportion of heterogeneity.

A flexible Bayesian framework for individualized inference via adaptive borrowing

The explosion in high-resolution data capture technologies in health has increased interest in making inferences about individual-level parameters. While technology may provide substantial data on a single individual, how best to use multisource population data to improve individualized inference remains an open research question. One possible approach, the multisource exchangeability model (MEM), is a Bayesian method for integrating data from supplementary sources into the analysis of a primary source. MEM was originally developed to improve inference for a single study by asymmetrically borrowing information from a set of similar previous studies and was further developed to apply a more computationally intensive symmetric borrowing in the context of basket trial; however, even for asymmetric borrowing, its computational burden grows exponentially with the number of supplementary sources, making it unsuitable for applications where hundreds or thousands of supplementary sources (i.e., individuals) could contribute to inference on a given individual. In this article, we propose the data-driven MEM (dMEM), a two-stage approach that includes both source selection and clustering to enable the inclusion of an arbitrary number of sources to contribute to individualized inference in a computationally tractable and data-efficient way. We illustrate the application of dMEM to individual-level human behavior and mental well-being data collected via smartphones, where our approach increases individual-level estimation precision by 84% compared with a standard no-borrowing method and outperforms recently proposed competing methods in 80% of individuals.

Adaptive Platform Trials to Transform ALS Therapy Development

Current therapeutic development in ALS relies on individual randomized clinical trials to test a specific investigational product in a single patient population. This approach has intrinsic limitations including cost, time, and lack of flexibility. Adaptive platform trials represent a novel approach to investigate several interventions for a single disease in a continuous manner. Already in use in oncology, this approach is now being employed more often in neurology. Here, we describe a newly launched, platform trial for amyotrophic lateral sclerosis (ALS). The HEALEY ALS Platform Trial is testing multiple investigational products concurrently in people with ALS, with the goal of rapidly identifying novel treatments, biomarkers, and trial endpoints. This article is protected by copyright. All rights reserved.

Sensitivity and identification quantification by a relative latent model complexity perturbation in Bayesian meta-analysis

In recent years, Bayesian meta-analysis expressed by a normal-normal hierarchical model (NNHM) has been widely used for combining evidence from multiple studies. Data provided for the NNHM are frequently based on a small number of studies and on uncertain within-study standard deviation values. Despite the widespread use of Bayesian NNHM, it has always been unclear to what extent the posterior inference is impacted by the heterogeneity prior (sensitivity S ) and by the uncertainty in the within-study standard deviation values (identification I ). Thus, to answer this question, we developed a unified method to simultaneously quantify both sensitivity and identification ( S - I ) for all model parameters in a Bayesian NNHM, based on derivatives of the Bhattacharyya coefficient with respect to relative latent model complexity (RLMC) perturbations. Three case studies exemplify the applicability of the method proposed: historical data for a conventional therapy, data from which one large study is first included and then excluded, and two subgroup meta-analyses specified by their randomization status. We analyzed six scenarios, crossing three RLMC targets with two heterogeneity priors (half-normal, half-Cauchy). The results show that S - I explicitly reveals which parameters are affected by the heterogeneity prior and by the uncertainty in the within-study standard deviation values. In addition, we compare the impact of both heterogeneity priors and quantify how S - I values are affected by omitting one large study and by the randomization status. Finally, the range of applicability of S - I is extended to Bayesian NtHM. A dedicated R package facilitates automatic S - I quantification in applied Bayesian meta-analyses.

Effectiveness and safety of non-steroidal anti-inflammatory drugs and opioid treatment for knee and hip osteoarthritis: network meta-analysis

OBJECTIVE

To assess the effectiveness and safety of different preparations and doses of non-steroidal anti-inflammatory drugs (NSAIDs), opioids, and paracetamol for knee and hip osteoarthritis pain and physical function to enable effective and safe use of these drugs at their lowest possible dose.

DESIGN

Systematic review and network meta-analysis of randomised trials.

DATA SOURCES

Cochrane Central Register of Controlled Trials (CENTRAL), Medline, Embase, regulatory agency websites, and ClinicalTrials.gov from inception to 28 June 2021.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

Randomised trials published in English with ≥100 patients per group that evaluated NSAIDs, opioids, or paracetamol (acetaminophen) to treat osteoarthritis.

OUTCOMES AND MEASURES

The prespecified primary outcome was pain. Physical function and safety outcomes were also assessed.

REVIEW METHODS

Two reviewers independently extracted outcomes data and evaluated the risk of bias of included trials. Bayesian random effects models were used for network meta-analysis of all analyses. Effect estimates are comparisons between active treatments and oral placebo.

RESULTS

192 trials comprising 102 829 participants examined 90 different active preparations or doses (68 for NSAIDs, 19 for opioids, and three for paracetamol). Five oral preparations (diclofenac 150 mg/day, etoricoxib 60 and 90 mg/day, and rofecoxib 25 and 50 mg/day) had ≥99% probability of more pronounced treatment effects than the minimal clinically relevant reduction in pain. Topical diclofenac (70-81 and 140-160 mg/day) had ≥92.3% probability, and all opioids had ≤53% probability of more pronounced treatment effects than the minimal clinically relevant reduction in pain. 18.5%, 0%, and 83.3% of the oral NSAIDs, topical NSAIDs, and opioids, respectively, had an increased risk of dropouts due to adverse events. 29.8%, 0%, and 89.5% of oral NSAIDs, topical NSAIDs, and opioids, respectively, had an increased risk of any adverse event. Oxymorphone 80 mg/day had the highest risk of dropouts due to adverse events (51%) and any adverse event (88%).

CONCLUSIONS

Etoricoxib 60 mg/day and diclofenac 150 mg/day seem to be the most effective oral NSAIDs for pain and function in patients with osteoarthritis. However, these treatments are probably not appropriate for patients with comorbidities or for long term use because of the slight increase in the risk of adverse events. Additionally, an increased risk of dropping out due to adverse events was found for diclofenac 150 mg/day. Topical diclofenac 70-81 mg/day seems to be effective and generally safer because of reduced systemic exposure and lower dose, and should be considered as first line pharmacological treatment for knee osteoarthritis. The clinical benefit of opioid treatment, regardless of preparation or dose, does not outweigh the harm it might cause in patients with osteoarthritis.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO number CRD42020213656.

A Primer on Bayesian Model-Averaged Meta-Analysis

Meta-analysis is the predominant approach for quantitatively synthesizing a set of studies. If the studies themselves are of high quality, meta-analysis can provide valuable insights into the current scientific state of knowledge about a particular phenomenon. In psychological science, the most common approach is to conduct frequentist meta-analysis. In this primer, we discuss an alternative method, Bayesian model-averaged meta-analysis. This procedure combines the results of four Bayesian meta-analysis models: (a) fixed-effect null hypothesis, (b) fixed-effect alternative hypothesis, (c) random-effects null hypothesis, and (d) random-effects alternative hypothesis. These models are combined according to their plausibilities given the observed data to address the two key questions “Is the overall effect nonzero?” and “Is there between-study variability in effect size?” Bayesian model-averaged meta-analysis therefore avoids the need to select either a fixed-effect or random-effects model and instead takes into account model uncertainty in a principled manner.

Sequential Bayesian Data Synthesis for Mediation and Regression Analysis

Science is an inherently cumulative process, and knowledge on a specific topic is organized through synthesis of findings from related studies. Meta-analysis has been the most common statistical method for synthesizing findings from multiple studies in prevention science and other fields. In recent years, Bayesian statistics have been put forth as another way to synthesize findings and have been praised for providing a natural framework for update existing knowledge with new data. This article presents a Bayesian method for cumulative science and describes a SAS macro %SBDS for synthesizing findings from multiple studies or multiple data sets from a single study using three different methods: meta-analysis using raw data, sequential Bayesian data synthesis, and a single-level analysis on pooled data. Sequential Bayesian data synthesis and Bayesian statistics in general are discussed in an accessible manner, and guidelines are provided on how researchers can use the accompanying SAS macro for synthesizing data from their own studies. Four alcohol use studies were used to demonstrate how to apply the three data synthesis methods using the SAS macro.

Bayesian hypothesis testing and estimation under the marginalized random-effects meta-analysis model

Meta-analysis methods are used to synthesize results of multiple studies on the same topic. The most frequently used statistical model in meta-analysis is the random-effects model containing parameters for the overall effect, between-study variance in primary study’s true effect size, and random effects for the study-specific effects. We propose Bayesian hypothesis testing and estimation methods using the marginalized random-effects meta-analysis (MAREMA) model where the study-specific true effects are regarded as nuisance parameters which are integrated out of the model. We propose using a flat prior distribution on the overall effect size in case of estimation and a proper unit information prior for the overall effect size in case of hypothesis testing. For the between-study variance (which can attain negative values under the MAREMA model), a proper uniform prior is placed on the proportion of total variance that can be attributed to between-study variability. Bayes factors are used for hypothesis testing that allow testing point and one-sided hypotheses. The proposed methodology has several attractive properties. First, the proposed MAREMA model encompasses models with a zero, negative, and positive between-study variance, which enables testing a zero between-study variance as it is not a boundary problem. Second, the methodology is suitable for default Bayesian meta-analyses as it requires no prior information about the unknown parameters. Third, the proposed Bayes factors can even be used in the extreme case when only two studies are available because Bayes factors are not based on large sample theory. We illustrate the developed methods by applying it to two meta-analyses and introduce easy-to-use software in the R package BFpack to compute the proposed Bayes factors.

Incorporating single-arm studies in meta-analysis of randomised controlled trials: a simulation study

Background

Use of real world data (RWD) from non-randomised studies (e.g. single-arm studies) is increasingly being explored to overcome issues associated with data from randomised controlled trials (RCTs). We aimed to compare methods for pairwise meta-analysis of RCTs and single-arm studies using aggregate data, via a simulation study and application to an illustrative example.

Methods

We considered contrast-based methods proposed by Begg & Pilote (1991) and arm-based methods by Zhang et al (2019). We performed a simulation study with scenarios varying (i) the proportion of RCTs and single-arm studies in the synthesis (ii) the magnitude of bias, and (iii) between-study heterogeneity. We also applied methods to data from a published health technology assessment (HTA), including three RCTs and 11 single-arm studies.

Results

Our simulation study showed that the hierarchical power and commensurate prior methods by Zhang et al provided a consistent reduction in uncertainty, whilst maintaining over-coverage and small error in scenarios where there was limited RCT data, bias and differences in between-study heterogeneity between the two sets of data. The contrast-based methods provided a reduction in uncertainty, but performed worse in terms of coverage and error, unless there was no marked difference in heterogeneity between the two sets of data.

Conclusions

The hierarchical power and commensurate prior methods provide the most robust approach to synthesising aggregate data from RCTs and single-arm studies, balancing the need to account for bias and differences in between-study heterogeneity, whilst reducing uncertainty in estimates. This work was restricted to considering a pairwise meta-analysis using aggregate data.

Supplementary Information

The online version contains supplementary material available at (10.1186/s12874-021-01301-1).

Risks of vaptans in hypernatremia and serum sodium overcorrection: A systematic review and meta-analysis of randomised controlled trials

OBJECTIVE

Serum sodium overcorrection and hypernatremia are significant health risks. We conducted a systematic review and meta-analysis to evaluate the risks of vaptans in hypernatremia and serum sodium overcorrection.

METHODS

We searched PubMed, Embase, and CENTRAL for randomised controlled trials. We included studies comparing vaptans and placebo with data on hypernatremia and serum sodium overcorrection. The study quality was assessed using the Cochrane Collaboration's risk-of-bias assessment tool. Fixed-effect model meta-analysis was used to pool the data. Different analyses were performed to ensure the accuracy of the results.

RESULTS

Twenty-eight studies were included in the meta-analysis of hypernatremia incidence. Treatment with vaptans resulted in a higher risk of hypernatremia than placebo (3.8% vs 1.0%, odds ratio [OR] 2.69; 95% confidence interval [CI] 1.97-3.68). The subgroup with baseline hyponatremia had a lower risk of hypernatremia incidence; however, the use of loop diuretics increased the risk. Fourteen studies were included in the analysis of the incidence of serum sodium overcorrection. A higher risk of serum sodium overcorrection was found in using vaptans vs placebo (4.4% vs 1.4%; OR 2.26; 95% CI 1.32-3.86).

CONCLUSION

Vaptans showed higher risks in the incidence of hypernatremia and serum sodium overcorrection than placebo. In addition, combination with loop diuretics increased the risk of hypernatremia. The risk of serum sodium overcorrection should be concerned in patients with hyponatremia and normal serum sodium equally. Using a low dose of vaptans can reduce both risks.

Jackknife empirical likelihood confidence intervals for assessing heterogeneity in meta-analysis of rare binary event data

In meta-analysis, the heterogeneity of effect sizes across component studies is typically described by a variance parameter in a random-effects (Re) model. In the literature, methods for constructing confidence intervals (CIs) for the parameter often assume that study-level effect sizes are normally distributed. However, this assumption might be violated in practice, especially in meta-analysis of rare binary events. We propose to use jackknife empirical likelihood (JEL), a nonparametric approach that uses jackknife pseudo-values, to construct CIs for the heterogeneity parameter. To compute jackknife pseudo-values, we employ a moment-based estimator and consider two commonly used weighing schemes (i.e., equal and inverse variance weights). We prove that with each scheme, the resulting log empirical likelihood ratio follows a chi-square distribution asymptotically. We further examine the performance of the proposed JEL methods and compare them with existing CIs through simulation studies and data examples that focus on data of rare binary events. Our numerical results suggest that the JEL method with equal weights compares favorably to alternatives, especially when (observed) effect sizes are non-normal and the number of component studies is large. Thus, it is worth serious consideration in statistical inference.

Bayesian adaptive design for clinical trials in Duchenne muscular dystrophy

A Bayesian adaptive design is proposed for a clinical trial in Duchenne muscular dystrophy. The trial was designed to demonstrate treatment efficacy on an ambulatory-based clinical endpoint and to identify early success on a biomarker (dystrophin protein levels) that can serve as a basis for accelerated approval in the United States. The trial incorporates placebo augmentation using placebo data from past clinical trials. A thorough simulation study was conducted to understand the operating characteristics of the trial. This trial design was selected for the US FDA Complex Innovative Trial Design Pilot Meeting Program and the experience in that program is summarized.

A Bayesian dose-response meta-analysis model: A simulations study and application

Dose-response models express the effect of different dose or exposure levels on a specific outcome. In meta-analysis, where aggregated-level data is available, dose-response evidence is synthesized using either one-stage or two-stage models in a frequentist setting. We propose a hierarchical dose-response model implemented in a Bayesian framework. We develop our model assuming normal or binomial likelihood and accounting for exposures grouped in clusters. To allow maximum flexibility, the dose-response association is modelled using restricted cubic splines. We implement these models in R using JAGS and we compare our approach to the one-stage dose-response meta-analysis model in a simulation study. We found that the Bayesian dose-response model with binomial likelihood has lower bias than the Bayesian model with normal likelihood and the frequentist one-stage model when studies have small sample size. When the true underlying shape is log-log or half-sigmoid, the performance of all models depends on choosing an appropriate location for the knots. In all other examined situations, all models perform very well and give practically identical results. We also re-analyze the data from 60 randomized controlled trials (15,984 participants) examining the efficacy (response) of various doses of serotonin-specific reuptake inhibitor (SSRI) antidepressant drugs. All models suggest that the dose-response curve increases between zero dose and 30-40 mg of fluoxetine-equivalent dose, and thereafter shows small decline. We draw the same conclusion when we take into account the fact that five different antidepressants have been studied in the included trials. We show that implementation of the hierarchical model in Bayesian framework has similar performance to, but overcomes some of the limitations of the frequentist approach and offers maximum flexibility to accommodate features of the data.

Life stage-specific inbreeding depression in long-lived Pinaceae species depends on population connectivity

Inbreeding depression (ID) is a fundamental selective pressure that shapes mating systems and population genetic structures in plants. Although it has been shown that ID varies over the life stages of shorter-lived plants, less is known about how the fitness effects of inbreeding vary across life stages in long-lived species. We conducted a literature survey in the Pinaceae, a tree family known to harbour some of the highest mutational loads ever reported. Using a meta-regression model, we investigated distributions of inbreeding depression over life stages, adjusting for effects of inbreeding levels and the genetic differentiation of populations within species. The final dataset contained 147 estimates of ID across life stages from 41 studies. 44 Fst estimates were collected from 40 peer-reviewed studies for the 18 species to aid genetic differentiation modelling. Partitioning species into fragmented and well-connected groups using Fst resulted in the best way (i.e. trade-off between high goodness-of-fit of the model to the data and reduced model complexity) to incorporate genetic connectivity in the meta-regression analysis. Inclusion of a life stage term and its interaction with the inbreeding coefficient (F) dramatically increased model precision. We observed that the correlation between ID and F was significant at the earliest life stage. Although partitioning of species populations into fragmented and well-connected groups explained little of the between-study heterogeneity, the inclusion of an interaction between life stage and population differentiation revealed that populations with fragmented distributions suffered lower inbreeding depression at early embryonic stages than species with well-connected populations. There was no evidence for increased ID in late life stages in well-connected populations, although ID tended to increase across life stages in the fragmented group. These findings suggest that life stage data should be included in inbreeding depression studies and that inbreeding needs to be managed over life stages in commercial populations of long-lived plants.

Slamming the sham: A Bayesian model for adaptive adjustment with noisy control data

It is not always clear how to adjust for control data in causal inference, balancing the goals of reducing bias and variance. We show how, in a setting with repeated experiments, Bayesian hierarchical modeling yields an adaptive procedure that uses the data to determine how much adjustment to perform. The result is a novel analysis with increased statistical efficiency compared with the default analysis based on difference estimates. We demonstrate this procedure on two real examples, as well as on a series of simulated datasets. We show that the increased efficiency can have real-world consequences in terms of the conclusions that can be drawn from the experiments. We also discuss the relevance of this work to causal inference and statistical design and analysis more generally.

Bayesian Methods for Meta-Analyses of Binary Outcomes: Implementations, Examples, and Impact of Priors

Bayesian methods are an important set of tools for performing meta-analyses. They avoid some potentially unrealistic assumptions that are required by conventional frequentist methods. More importantly, meta-analysts can incorporate prior information from many sources, including experts’ opinions and prior meta-analyses. Nevertheless, Bayesian methods are used less frequently than conventional frequentist methods, primarily because of the need for nontrivial statistical coding, while frequentist approaches can be implemented via many user-friendly software packages. This article aims at providing a practical review of implementations for Bayesian meta-analyses with various prior distributions. We present Bayesian methods for meta-analyses with the focus on odds ratio for binary outcomes. We summarize various commonly used prior distribution choices for the between-studies heterogeneity variance, a critical parameter in meta-analyses. They include the inverse-gamma, uniform, and half-normal distributions, as well as evidence-based informative log-normal priors. Five real-world examples are presented to illustrate their performance. We provide all of the statistical code for future use by practitioners. Under certain circumstances, Bayesian methods can produce markedly different results from those by frequentist methods, including a change in decision on statistical significance. When data information is limited, the choice of priors may have a large impact on meta-analytic results, in which case sensitivity analyses are recommended. Moreover, the algorithm for implementing Bayesian analyses may not converge for extremely sparse data; caution is needed in interpreting respective results. As such, convergence should be routinely examined. When select statistical assumptions that are made by conventional frequentist methods are violated, Bayesian methods provide a reliable alternative to perform a meta-analysis.

A comparison of hypothesis tests for homogeneity in meta-analysis with focus on rare binary events

Analysis of rare binary events is an important problem for biomedical researchers. Due to the sparsity of events in such problems, meta-analysis that integrates information across multiple studies can be applied to increase the efficiency of statistical inference. Although it is critical to examine whether the effect sizes are homogeneous across all studies, a comprehensive review of homogeneity tests has been lacking, and in particular, no attention has been paid to infrequent dichotomous outcomes. We systematically review statistical methods for homogeneity testing. By conducting an extensive simulation analysis and two case studies, we examine the performance of 30 tests in meta-analysis of rare binary outcomes. When using log-odds ratio as the association measure, our simulation results suggest that there is no uniform winner. However, we recommend the test proposed by Kulinskaya and Dollinger (BMC Med Res Methodol, 2015, 15), which uses a gamma distribution to approximate the null distribution, for its generally good performance; for very rare events coupled with small within-study sample sizes, in addition to the Kulinskaya-Dollinger test, we further recommend the conditional score test based on the random-effects hypergeometric model proposed by Liang and Self (Biometrika, 1985, 72:353-358). One should be cautious about the use of the Wald tests, the Lipsitz tests (Biometrics, 1998, 54:148-160), and tests proposed by Bhaumik et al (J Am Stat Assoc, 2012, 107:555-567).

On weakly informative prior distributions for the heterogeneity parameter in Bayesian random-effects meta-analysis

The normal-normal hierarchical model (NNHM) constitutes a simple and widely used framework for meta-analysis. In the common case of only few studies contributing to the meta-analysis, standard approaches to inference tend to perform poorly, and Bayesian meta-analysis has been suggested as a potential solution. The Bayesian approach, however, requires the sensible specification of prior distributions. While noninformative priors are commonly used for the overall mean effect, the use of weakly informative priors has been suggested for the heterogeneity parameter, in particular in the setting of (very) few studies. To date, however, a consensus on how to generally specify a weakly informative heterogeneity prior is lacking. Here we investigate the problem more closely and provide some guidance on prior specification.

Interval estimation of the overall treatment effect in random-effects meta-analyses: Recommendations from a simulation study comparing frequentist, Bayesian, and bootstrap methods

There exists a variety of interval estimators for the overall treatment effect in a random-effects meta-analysis. A recent literature review summarizing existing methods suggested that in most situations, the Hartung-Knapp/Sidik-Jonkman (HKSJ) method was preferable. However, a quantitative comparison of those methods in a common simulation study is still lacking. Thus, we conduct such a simulation study for continuous and binary outcomes, focusing on the medical field for application. Based on the literature review and some new theoretical considerations, a practicable number of interval estimators is selected for this comparison: the classical normal-approximation interval using the DerSimonian-Laird heterogeneity estimator, the HKSJ interval using either the Paule-Mandel or the Sidik-Jonkman heterogeneity estimator, the Skovgaard higher-order profile likelihood interval, a parametric bootstrap interval, and a Bayesian interval using different priors. We evaluate the performance measures (coverage and interval length) at specific points in the parameter space, that is, not averaging over a prior distribution. In this sense, our study is conducted from a frequentist point of view. We confirm the main finding of the literature review, the general recommendation of the HKSJ method (here with the Sidik-Jonkman heterogeneity estimator). For meta-analyses including only two studies, the high length of the HKSJ interval limits its practical usage. In this case, the Bayesian interval using a weakly informative prior for the heterogeneity may help. Our recommendations are illustrated using a real-world meta-analysis dealing with the efficacy of an intramyocardial bone marrow stem cell transplantation during coronary artery bypass grafting.

Screening rate and influential factors of lung cancer with low-dose computed tomography in Asian population: a systematic review and meta-analysis

BACKGROUND

There are currently no studies synthesizing the screening rate and influential factors of low-dose computed tomography (LDCT)-screened lung cancer in Asian population.

METHODS

A systematic review was conducted, using both English and Chinese language databases on March, 2019. The pooled screening rate and estimated odds ratios (ORs) of influential factors were analyzed using random effects models. Subgroup and meta-regression analyses were also employed to explore the heterogeneity.

RESULTS

The pooled LDCT lung cancer screening rate was 1.12% (95% confidence interval (CI): 0.94%, 1.32%), and increased with age. Adenocarcinoma and stage I lung cancer had higher screening rates. Analysis of influential factors in the general population showed that female and elder age (≥50 years) were significantly influencing LDCT lung cancer screening rate (for female, OR = 1.32, 95% CI: 1.15-1.52; for adults ≥ 50 years, OR = 1.94, 95% CI: 1.52-2.49). Meta-regression analysis indicated that the heterogeneity maybe significantly correlated with the sample size, risk population and source of population.

CONCLUSIONS

Unlike European and American populations, female and adults > 50 years rather than smoking adults were positively associated with screening rate in Asian populations. It is important to further study the benefits of lung cancer screening with LDCT in Asian populations.

Meta-analysis of rare adverse events in randomized clinical trials: Bayesian and frequentist methods

BACKGROUND/AIMS

Regulatory approval of a drug or device involves an assessment of not only the benefits but also the risks of adverse events associated with the therapeutic agent. Although randomized controlled trials (RCTs) are the gold standard for evaluating effectiveness, the number of treated patients in a single RCT may not be enough to detect a rare but serious side effect of the treatment. Meta-analysis plays an important role in the evaluation of the safety of medical products and has advantage over analyzing a single RCT when estimating the rate of adverse events.

METHODS

In this article, we compare 15 widely used meta-analysis models under both Bayesian and frequentist frameworks when outcomes are extremely infrequent or rare. We present extensive simulation study results and then apply these methods to a real meta-analysis that considers RCTs investigating the effect of rosiglitazone on the risks of myocardial infarction and of death from cardiovascular causes.

RESULTS

Our simulation studies suggest that the beta hyperprior method modeling treatment group-specific parameters and accounting for heterogeneity performs the best. Most models ignoring between-study heterogeneity give poor coverage probability when such heterogeneity exists. In the data analysis, different methods provide a wide range of log odds ratio estimates between rosiglitazone and control treatments with a mixed conclusion on their statistical significance based on 95% confidence (or credible) intervals.

CONCLUSION

In the rare event setting, treatment effect estimates obtained from traditional meta-analytic methods may be biased and provide poor coverage probability. This trend worsens when the data have large between-study heterogeneity. In general, we recommend methods that first estimate the summaries of treatment-specific risks across studies and then relative treatment effects based on the summaries when appropriate. Furthermore, we recommend fitting various methods, comparing the results and model performance, and investigating any significant discrepancies among them.

Frequentist performances of Bayesian prediction intervals for random-effects meta-analysis

The prediction interval has been increasingly used in meta-analyses as a useful measure for assessing the magnitude of treatment effect and between-studies heterogeneity. In calculations of the prediction interval, although the Higgins-Thompson-Spiegelhalter method is used most often in practice, it might not have adequate coverage probability for the true treatment effect of a future study under realistic situations. An effective alternative candidate is the Bayesian prediction interval, which has also been widely used in general prediction problems. However, these prediction intervals are constructed based on the Bayesian philosophy, and their frequentist validities are only justified by large-sample approximations even if noninformative priors are adopted. There has been no certain evidence that evaluated their frequentist performances under realistic situations of meta-analyses. In this study, we conducted extensive simulation studies to assess the frequentist coverage performances of Bayesian prediction intervals with 11 noninformative prior distributions under general meta-analysis settings. Through these simulation studies, we found that frequentist coverage performances strongly depended on what prior distributions were adopted. In addition, when the number of studies was smaller than 10, there were no prior distributions that retained accurate frequentist coverage properties. We also illustrated these methods via applications to two real meta-analysis datasets. The resultant prediction intervals also differed according to the adopted prior distributions. Inaccurate prediction intervals may provide invalid evidence and misleading conclusions. Thus, if frequentist accuracy is required, Bayesian prediction intervals should be used cautiously in practice.

Zero-cell corrections in random-effects meta-analyses

The standard estimator for the log odds ratio (the unconditional maximum likelihood estimator) and the delta-method estimator for its standard error are not defined if the corresponding 2 × 2 table contains at least one "zero cell". This is also an issue when estimating the overall log odds ratio in a meta-analysis. It is well known that correcting for zero cells by adding a small increment should be avoided. Nevertheless, these zero-cell corrections continue to be used. With this Brief Method Note, we want to warn of a particularly bad zero-cell correction. For this, we conduct a simulation study comparing the following two zero-cell corrections under the ordinary random-effects model: (a) adding 1 2 to all cells of all the individual studies' 2 × 2 tables independently of any zero-cell occurrences and (b) adding 1 2 to all cells of only those 2 × 2 tables containing at least one zero cell. The main finding is that correction (a) performs worse than correction (b). Thus, we strongly discourage the use of correction (a).

A Bayesian multivariate meta-analysis of prevalence data

When conducting a meta-analysis involving prevalence data for an outcome with several subtypes, each of them is typically analyzed separately using a univariate meta-analysis model. Recently, multivariate meta-analysis models have been shown to correspond to a decrease in bias and variance for multiple correlated outcomes compared with univariate meta-analysis, when some studies only report a subset of the outcomes. In this article, we propose a novel Bayesian multivariate random effects model to account for the natural constraint that the prevalence of any given subtype cannot be larger than that of the overall prevalence. Extensive simulation studies show that this new model can reduce bias and variance when estimating subtype prevalences in the presence of missing data, compared with standard univariate and multivariate random effects models. The data from a rapid review on occupation and lower urinary tract symptoms by the Prevention of Lower Urinary Tract Symptoms Research Consortium are analyzed as a case study to estimate the prevalence of urinary incontinence and several incontinence subtypes among women in suspected high risk work environments.

Statistical approaches for meta-analysis of genetic mutation prevalence

Estimating the prevalence of rare germline genetic mutations in the general population is of interest as it can inform genetic counseling and risk management. Most studies that estimate the prevalence of mutations are performed in high-risk populations, and each study is designed with differing inclusion criteria, resulting in ascertained populations. Quantifying the effects of ascertainment is necessary to estimate the prevalence in the general population. This quantification is difficult as the inclusion criteria is often based on disease status and/or family history. Combining estimates from multiple studies through a meta-analysis is challenging due to the variety of study designs and ascertainment mechanisms as well as the complexity of quantifying the effect of these mechanisms. We provide guidelines on how to quantify the ascertainment mechanism for a wide range of settings and propose a general approach for conducting a meta-analysis in these complex settings by incorporating study-specific ascertainment mechanisms into a joint likelihood function. We implement the proposed likelihood-based approach using both frequentist and Bayesian methodologies. We evaluate these approaches in simulations and show that the methods are robust and produce unbiased estimates of the prevalence. An advantage of the Bayesian approach is that it can easily incorporate uncertainty in ascertainment probability values. We apply our methods to estimate the prevalence of PALB2 mutations in the United States by combining data from multiple studies and obtain a prevalence estimate of around 0.02%.

Degree irregularity and rank probability bias in network meta-analysis

Network meta-analysis (NMA) is a statistical technique for the comparison of treatment options. Outcomes of Bayesian NMA include estimates of treatment effects, and the probabilities that each treatment is ranked best, second best and so on. How exactly network topology affects the accuracy and precision of these outcomes is not fully understood. Here we carry out a simulation study and find that disparity in the number of trials involving different treatments leads to a systematic bias in estimated rank probabilities. This bias is associated with an increased variation in the precision of treatment effect estimates. Using ideas from the theory of complex networks, we define a measure of "degree irregularity" to quantify asymmetry in the number of studies involving each treatment. Our simulations indicate that more regular networks have more precise treatment effect estimates and smaller bias of rank probabilities. Conversely, these topological effects are not observed for the accuracy of treatment effect estimates. This reinforces the importance of taking into account multiple measures, rather than making decisions based on a single metric. We also find that degree regularity is a better indicator for the accuracy and precision of parameter estimates in NMA than both the total number of studies in a network and the disparity in the number of trials per comparison. These results have implications for planning future trials. We demonstrate that choosing trials which reduce the network's irregularity can improve the precision and accuracy of parameter estimates from NMA.

Estimating the reference range from a meta-analysis

Often clinicians are interested in determining whether a subject's measurement falls within a normal range, defined as a range of values of a continuous outcome which contains some proportion (eg, 95%) of measurements from a healthy population. Several studies in the biomedical field have estimated reference ranges based on a meta-analysis of multiple studies with healthy individuals. However, the literature currently gives no guidance about how to estimate the reference range of a new subject in such settings. Instead, meta-analyses of such normative range studies typically report the pooled mean as a reference value, which does not incorporate natural variation across healthy individuals in different studies. We present three approaches to calculating the normal reference range of a subject from a meta-analysis of normally or lognormally distributed outcomes: a frequentist random effects model, a Bayesian random effects model, and an empirical approach. We present the results of a simulation study demonstrating that the methods perform well under a variety of scenarios, though users should be cautious when the number of studies is small and between-study heterogeneity is large. Finally, we apply these methods to two examples: pediatric time spent awake after sleep onset and frontal subjective postural vertical measurements.

Cabozantinib and vandetanib for unresectable locally advanced or metastatic medullary thyroid cancer: a systematic review and economic model

BACKGROUND

Medullary thyroid cancer (MTC) is a rare form of cancer that affects patients' health-related quality of life (HRQoL) and survival. Cabozantinib (Cometriq^®; Ipsen, Paris, France) and vandetanib (Caprelsa^®; Sanofi Genzyme, Cambridge, MA, USA) are currently the treatment modality of choice for treating unresectable progressive and symptomatic MTC.

OBJECTIVES

(1) To evaluate the clinical effectiveness and safety of cabozantinib and vandetanib. (2) To estimate the incremental cost-effectiveness of cabozantinib and vandetanib versus each other and best supportive care. (3) To identify key areas for primary research. (4) To estimate the overall cost of these treatments in England.

DATA SOURCES

Peer-reviewed publications (searched from inception to November 2016), European Public Assessment Reports and manufacturers' submissions.

REVIEW METHODS

A systematic review [including a network meta-analysis (NMA)] was conducted to evaluate the clinical effectiveness and safety of cabozantinib and vandetanib. The economic analysis included a review of existing analyses and the development of a de novo model.

RESULTS

The systematic review identified two placebo-controlled trials. The Efficacy of XL184 (Cabozantinib) in Advanced Medullary Thyroid Cancer (EXAM) trial evaluated the efficacy and safety of cabozantinib in patients with unresectable locally advanced, metastatic and progressive MTC. The ZETA trial evaluated the efficacy and safety of vandetanib in patients with unresectable locally advanced or metastatic MTC. Both drugs significantly improved progression-free survival (PFS) more than the placebo (p < 0.001). The NMA suggested that, within the symptomatic and progressive MTC population, the effects on PFS were similar (vandetanib vs. cabozantinib: hazard ratio 1.14, 95% credible interval 0.41 to 3.09). Neither trial demonstrated a significant overall survival benefit for cabozantinib or vandetanib versus placebo, although data from ZETA were subject to potential confounding. Both cabozantinib and vandetanib demonstrated significantly better objective response rates and calcitonin (CTN) and carcinoembryonic antigen (CEA) response rates than placebo. Both cabozantinib and vandetanib produced frequent adverse events, often leading to dose interruption or reduction. The assessment group model indicates that, within the EU-label population (symptomatic and progressive MTC), the incremental cost-effectiveness ratios (ICERs) for cabozantinib and vandetanib are > £138,000 per quality-adjusted life-year (QALY) gained. Within the restricted EU-label population (symptomatic and progressive MTC with CEA/CTN doubling times of ≤ 24 months), the ICER for vandetanib is expected to be > £66,000 per QALY gained. The maximum annual budget impact within the symptomatic and progressive population is estimated to be ≈£2.35M for cabozantinib and ≈£5.53M for vandetanib. The costs of vandetanib in the restricted EU-label population are expected to be lower.

LIMITATIONS

The intention-to-treat populations of the EXAM and ZETA trials are notably different. The analyses of ZETA subgroups may be subject to confounding as a result of differences in baseline characteristics and open-label vandetanib use. Attempts to statistically adjust for treatment switching were unsuccessful. No HRQoL evidence was identified for the MTC population.

CONCLUSIONS

The identified trials suggest that cabozantinib and vandetanib improve PFS more than the placebo; however, significant OS benefits were not demonstrated. The economic analyses indicate that within the EU-label population, the ICERs for cabozantinib and vandetanib are > £138,000 per QALY gained. Within the restricted EU-label population, the ICER for vandetanib is expected to be > £66,000 per QALY gained.

FUTURE RESEARCH PRIORITIES

(1) Primary research assessing the long-term effectiveness of cabozantinib and vandetanib within relevant subgroups. (2) Reanalyses of the ZETA trial to investigate the impact of adjusting for open-label vandetanib use using appropriate statistical methods. (3) Studies assessing the impact of MTC on HRQoL.

STUDY REGISTRATION

This study is registered as PROSPERO CRD42016050403.

FUNDING

The National Institute for Health Research Health Technology Assessment programme.

Arcsine-based transformations for meta-analysis of proportions: Pros, cons, and alternatives

Meta-analyses have been increasingly used to synthesize proportions (eg, disease prevalence) from multiple studies in recent years. Arcsine-based transformations, especially the Freeman-Tukey double-arcsine transformation, are popular tools for stabilizing the variance of each study's proportion in two-step meta-analysis methods. Although they offer some benefits over the conventional logit transformation, they also suffer from several important limitations (eg, lack of interpretability) and may lead to misleading conclusions. Generalized linear mixed models and Bayesian models are intuitive one-step alternative approaches, and can be readily implemented via many software programs. This article explains various pros and cons of the arcsine-based transformations, and discusses the alternatives that may be generally superior to the currently popular practice.