When should meta-analysis avoid making hidden normality assumptions?

Influence of habitat features on the colonisation of native and non-indigenous species

Marine artificial structures provide substrates on which organisms can settle and grow. These structures facilitate establishment and spread of non-indigenous species, in part due to their distinct physical features (substrate material, movement, orientation) compared to natural habitat analogues such as rocky shores, and because following construction, they have abundant resources (space) for species to colonise. Despite the perceived importance of these habitat features, few studies have directly compared distributions of native and non-indigenous species or considered how functional identity and associated environmental preferences drive associations. We undertook a meta-analysis to investigate whether colonisation of native and non-indigenous species varies between artificial structures with features most closely resembling natural habitats (natural substrates, fixed structures, surfaces oriented upwards) and those least resembling natural habitats (artificial materials, floating structures, downfacing or vertical surfaces), or whether functional identity is the primary driver of differences. Analyses were done at global and more local (SE Australia) scales to investigate if patterns held regardless of scale. Our results suggest that functional group (i.e., algae, ascidians. barnacles, bryozoans, polychaetes) rather than species classification (i.e., native or non-indigenous) are the main drivers of differences in communities between different types of artificial structures. Specifically, there were differences in the abundance of ascidians, barnacles, and polychaetes between (1) upfacing and downfacing/vertical surfaces, and (2) floating and fixed substrates. When differences were detected, taxa were most abundant on features least resembling natural habitats. Results varied between global and SE Australian analyses, potentially due to reduced variability across studies in the SE Australian dataset. Thus, the functional group and associated preferences of the highest threat NIS in the area should be considered in design strategies (e.g., ecological engineering) to limit their establishment on newly built infrastructure.

Longitudinal Changes in Human Milk Minerals and Vitamins in the Chinese Population: A Scoping Review

This scoping review aims to investigate longitudinal changes in minerals and vitamins concentrations in human milk among the Chinese population. Following the PRISMA-ScR guidelines, a comprehensive and systematic literature search was conducted using both English and Chinese databases. Data were extracted and categorized into six defined lactation stages. We found that the concentration of most minerals decreased throughout the lactation period, although calcium (Ca) and magnesium (Mg) fluctuated slightly across lactation periods. Fat-soluble vitamins also showed a decline throughout the lactation period, while water-soluble vitamins exhibited an increasing trend. However, folic acid, biotin, and pantothenic acid demonstrated a downward trend. Overall, this review has identified the longitudinal changes in minerals and vitamins concentrations in human milk among the Chinese population. In order to conduct a more in-depth examination of maternal characteristics and nutritional factors of the composition of human milk, it is recommended to utilize standardized protocols for the collection and analysis of human milk samples.

A Bayesian nonparametric meta-analysis model for estimating the reference interval

A reference interval represents the normative range for measurements from a healthy population. It plays an important role in laboratory testing, as well as in differentiating healthy from diseased patients. The reference interval based on a single study might not be applicable to a broader population. Meta-analysis can provide a more generalizable reference interval based on the combined population by synthesizing results from multiple studies. However, the assumptions of normally distributed underlying study-specific means and equal within-study variances, which are commonly used in existing methods, are strong and may not hold in practice. We propose a Bayesian nonparametric model with more flexible assumptions to extend random effects meta-analysis for estimating reference intervals. We illustrate through simulation studies and two real data examples the performance of our proposed approach when the assumptions of normally distributed study means and equal within-study variances do not hold.

Comparisons of various estimates of the I2 statistic for quantifying between-study heterogeneity in meta-analysis

Assessing heterogeneity between studies is a critical step in determining whether studies can be combined and whether the synthesized results are reliable. The I 2 statistic has been a popular measure for quantifying heterogeneity, but its usage has been challenged from various perspectives in recent years. In particular, it should not be considered an absolute measure of heterogeneity, and it could be subject to large uncertainties. As such, when using I 2 to interpret the extent of heterogeneity, it is essential to account for its interval estimate. Various point and interval estimators exist for I 2 . This article summarizes these estimators. In addition, we performed a simulation study under different scenarios to investigate preferable point and interval estimates of I 2 . We found that the Sidik-Jonkman method gave precise point estimates for I 2 when the between-study variance was large, while in other cases, the DerSimonian-Laird method was suggested to estimate I 2 . When the effect measure was the mean difference or the standardized mean difference, the Q -profile method, the Biggerstaff-Jackson method, or the Jackson method was suggested to calculate the interval estimate for I 2 due to reasonable interval length and more reliable coverage probabilities than various alternatives. For the same reason, the Kulinskaya-Dollinger method was recommended to calculate the interval estimate for I 2 when the effect measure was the log odds ratio.

How trace plots help interpret meta-analysis results

The trace plot is seldom used in meta-analysis, yet it is a very informative plot. In this article, we define and illustrate what the trace plot is, and discuss why it is important. The Bayesian version of the plot combines the posterior density of τ , the between-study standard deviation, and the shrunken estimates of the study effects as a function of τ . With a small or moderate number of studies, τ is not estimated with much precision, and parameter estimates and shrunken study effect estimates can vary widely depending on the correct value of τ . The trace plot allows visualization of the sensitivity to τ along with a plot that shows which values of τ are plausible and which are implausible. A comparable frequentist or empirical Bayes version provides similar results. The concepts are illustrated using examples in meta-analysis and meta-regression; implementation in R is facilitated in a Bayesian or frequentist framework using the bayesmeta and metafor packages, respectively.

Problematic meta-analyses: Bayesian and frequentist perspectives on combining randomized controlled trials and non-randomized studies

PURPOSE

In the literature, the propriety of the meta-analytic treatment-effect produced by combining randomized controlled trials (RCT) and non-randomized studies (NRS) is questioned, given the inherent confounding in NRS that may bias the meta-analysis. The current study compared an implicitly principled pooled Bayesian meta-analytic treatment-effect with that of frequentist pooling of RCT and NRS to determine how well each approach handled the NRS bias.

MATERIALS & METHODS

Binary outcome Critical-Care meta-analyses, reflecting the importance of such outcomes in Critical-Care practice, combining RCT and NRS were identified electronically. Bayesian pooled treatment-effect and 95% credible-intervals (BCrI), posterior model probabilities indicating model plausibility and Bayes-factors (BF) were estimated using an informative heavy-tailed heterogeneity prior (half-Cauchy). Preference for pooling of RCT and NRS was indicated for Bayes-factors > 3 or < 0.333 for the converse. All pooled frequentist treatment-effects and 95% confidence intervals (FCI) were re-estimated using the popular DerSimonian-Laird (DSL) random effects model.

RESULTS

Fifty meta-analyses were identified (2009-2021), reporting pooled estimates in 44; 29 were pharmaceutical-therapeutic and 21 were non-pharmaceutical therapeutic. Re-computed pooled DSL FCI excluded the null (OR or RR = 1) in 86% (43/50). In 18 meta-analyses there was an agreement between FCI and BCrI in excluding the null. In 23 meta-analyses where FCI excluded the null, BCrI embraced the null. BF supported a pooled model in 27 meta-analyses and separate models in 4. The highest density of the posterior model probabilities for 0.333 < Bayes factor < 1 was 0.8.

CONCLUSIONS

In the current meta-analytic cohort, an integrated and multifaceted Bayesian approach gave support to including NRS in a pooled-estimate model. Conversely, caution should attend the reporting of naïve frequentist pooled, RCT and NRS, meta-analytic treatment effects.

Prevalence and Risk Factors Associated with Feline Infectious Peritonitis (FIP) in Mainland China between 2008 and 2023: A Systematic Review and Meta-Analysis

To evaluate the overall prevalence of FIP infection in cats in mainland China and associated risk factors, studies on the prevalence of FIP conducted from 1 January 2008 to 20 December 2023 were retrieved from five databases-CNKI, Wanfang, PubMed, Web of Science, and ScienceDirect-and comprehensively reviewed. The 21 studies selected, with a total of 181,014 samples, underwent a rigorous meta-analysis after quality assessment. The results revealed a 2% prevalence of FIP (95% CI: 1-2%) through the random-effects model, showing considerable heterogeneity (I2 = 95.2%). The subsequent subgroup analysis revealed that the age and gender of cats are significant risk factors for FIP infection in mainland China. In order to effectively reduce and control the prevalence of FIP on the Chinese mainland, we suggest improving the immunity of cats, with special attention given to health management in kittens and intact cats, and continuously monitoring FIPV.

A meta-analysis on the potency of foot-and-mouth disease vaccines in different animal models

Whether mice can be used as a foot-and-mouth disease (FMD) model has been debated for a long time. However, the major histocompatibility complex between pigs and mice is very different. In this study, the protective effects of FMD vaccines in different animal models were analyzed by a meta-analysis. The databases PubMed, China Knowledge Infrastructure, EMBASE, and Baidu Academic were searched. For this purpose, we evaluated evidence from 14 studies that included 869 animals with FMD vaccines. A random effects model was used to combine effects using Review Manager 5.4 software. A forest plot showed that the protective effects in pigs were statistically non-significant from those in mice [MH = 0.56, 90% CI (0.20, 1.53), P = 0.26]. The protective effects in pigs were also statistically non-significant from those in guinea pigs [MH = 0.67, 95% CI (0.37, 1.21), P = 0.18] and suckling mice [MH = 1.70, 95% CI (0.10, 28.08), P = 0.71]. Non-inferiority test could provide a hypothesis that the models (mice, suckling mice and guinea pigs) could replace pigs as FMDV vaccine models to test the protective effect of the vaccine. Strict standard procedures should be established to promote the assumption that mice and guinea pigs should replace pigs in vaccine evaluation.

Sensitivity analysis with iterative outlier detection for systematic reviews and meta-analyses

Meta-analysis is a widely used tool for synthesizing results from multiple studies. The collected studies are deemed heterogeneous when they do not share a common underlying effect size; thus, the factors attributable to the heterogeneity need to be carefully considered. A critical problem in meta-analyses and systematic reviews is that outlying studies are frequently included, which can lead to invalid conclusions and affect the robustness of decision-making. Outliers may be caused by several factors such as study selection criteria, low study quality, small-study effects, and so on. Although outlier detection is well-studied in the statistical community, limited attention has been paid to meta-analysis. The conventional outlier detection method in meta-analysis is based on a leave-one-study-out procedure. However, when calculating a potentially outlying study's deviation, other outliers could substantially impact its result. This article proposes an iterative method to detect potential outliers, which reduces such an impact that could confound the detection. Furthermore, we adopt bagging to provide valid inference for sensitivity analyses of excluding outliers. Based on simulation studies, the proposed iterative method yields smaller bias and heterogeneity after performing a sensitivity analysis to remove the identified outliers. It also provides higher accuracy on outlier detection. Two case studies are used to illustrate the proposed method's real-world performance.

A meta-analysis for prevalence of infectious laryngotracheitis in chickens in mainland China in 1981-2022

BACKGROUND

Infectious laryngotracheitis (ILT) is a highly infectious upper respiratory tract disease of chickens caused by infectious laryngotracheitis virus or Gallid herpesvirus 1 (GaHV-1). ILT is an important respiratory disease of chickens and annually causes significant economic losses in the chicken industry. Although numerous relevant studies have been published, the overall prevalence of ILT infection among chicken in mainland China is still unknown, and associated risk factors need to be evaluated to establish preventive measures.

RESULTS

The present study reviewed the literature on the prevalence of ILT in chickens in China as of December 20, 2022, retrieved from six databases-CNKI, Wanfang, VIP, PubMed, Web of Science, and ScienceDirect-were used to retrieve relevant studies published between January 1, 1981 and December 20, 2022. The literature quality of studies was assessed, and 20 studies with a total of 108,587 samples were included in the meta-analysis. Results of the meta-analysis showed that the overall prevalence of ILT was 10% (95% confidence interval: 8 -12%) through the random-effects model, which showed high heterogeneity, I2 = 99.4%. Further subgroup analyses showed that the prevalence of ILT decreased over time; furthermore, the prevalence in Northwest China was slightly lower than that in North China and South China, and the prevalence estimated using the diagnostic technique AGP was higher than that reported using other diagnostic techniques.

CONCLUSIONS

ILT is prevalent to some extent in mainland China. Given that the ILT attenuated live vaccine has a certain level of virulence and the prevalence differences between regions, we recommend controlling breeding density, improving immunization programs and continuously monitoring viruses and to prevent ILT prevailing in mainland China.

Comparisons of the mean differences and standardized mean differences for continuous outcome measures on the same scale

When conducting systematic reviews and meta-analyses of continuous outcomes, the mean differences (MDs) and standardized mean differences (SMDs) are 2 commonly used choices for effect measures. The SMDs are motivated by scenarios where studies collected in a systematic review do not report the continuous measures on the same scale. The standardization process transfers the MDs to be unit-free measures that can be synthesized across studies. As such, some evidence synthesis researchers tend to prefer the SMD over the MD. However, other researchers have concerns about the interpretability of the SMD. The standardization process could also yield additional heterogeneity between studies. In this paper, we use simulation studies to illustrate that, in a scenario where the continuous measures are on the same scale, the SMD could have considerably poorer performance compared with the MD in some cases. The simulations compare the MD and SMD in various settings, including cases where the normality assumption of continuous measures does not hold. We conclude that although the SMD remains useful for evidence synthesis of continuous measures on different scales, the SMD could have substantially greater biases, greater mean squared errors, and lower coverage probabilities of CIs than the MD. The MD is generally more robust to the violation of the normality assumption for continuous measures. In scenarios where continuous measures are inherently comparable or can be transformed to a common scale, the MD is the preferred choice for an effect measure.

Simulation models for aggregated data meta-analysis: Evaluation of pooling effect sizes and publication biases

Simulation studies are commonly used to evaluate the performance of newly developed meta-analysis methods. For methodology that is developed for an aggregated data meta-analysis, researchers often resort to simulation of the aggregated data directly, instead of simulating individual participant data from which the aggregated data would be calculated in reality. Clearly, distributional characteristics of the aggregated data statistics may be derived from distributional assumptions of the underlying individual data, but they are often not made explicit in publications. This article provides the distribution of the aggregated data statistics that were derived from a heteroscedastic mixed effects model for continuous individual data and a procedure for directly simulating the aggregated data statistics. We also compare our simulation approach with other simulation approaches used in literature. We describe their theoretical differences and conduct a simulation study for three meta-analysis methods: DerSimonian and Laird method for pooling aggregated study effect sizes and the Trim & Fill and precision-effect test and precision-effect estimate with standard errors method for adjustment of publication bias. We demonstrate that the choice of simulation model for aggregated data may have an impact on (the conclusions of) the performance of the meta-analysis method. We recommend the use of multiple aggregated data simulation models to investigate the sensitivity in the performance of the meta-analysis method. Additionally, we recommend that researchers try to make the individual participant data model explicit and derive from this model the distributional consequences of the aggregated statistics to help select appropriate aggregated data simulation models.

Correcting for outcome reporting bias in a meta-analysis: A meta-regression approach

Outcome reporting bias (ORB) refers to the biasing effect caused by researchers selectively reporting outcomes within a study based on their statistical significance. ORB leads to inflated effect size estimates in meta-analysis if only the outcome with the largest effect size is reported due to ORB. We propose a new method (CORB) to correct for ORB that includes an estimate of the variability of the outcomes' effect size as a moderator in a meta-regression model. An estimate of the variability of the outcomes' effect size can be computed by assuming a correlation among the outcomes. Results of a Monte-Carlo simulation study showed that the effect size in meta-analyses may be severely overestimated without correcting for ORB. Estimates of CORB are close to the true effect size when overestimation caused by ORB is the largest. Applying the method to a meta-analysis on the effect of playing violent video games on aggression showed that the effect size estimate decreased when correcting for ORB. We recommend to routinely apply methods to correct for ORB in any meta-analysis. We provide annotated R code and functions to help researchers apply the CORB method.

Multistep estimators of the between-study covariance matrix under the multivariate random-effects model for meta-analysis

A wide variety of methods are available to estimate the between-study variance under the univariate random-effects model for meta-analysis. Some, but not all, of these estimators have been extended so that they can be used in the multivariate setting. We begin by extending the univariate generalised method of moments, which immediately provides a wider class of multivariate methods than was previously available. However, our main proposal is to use this new type of estimator to derive multivariate multistep estimators of the between-study covariance matrix. We then use the connection between the univariate multistep and Paule-Mandel estimators to motivate taking the limit, where the number of steps tends toward infinity. We illustrate our methodology using two contrasting examples and investigate its properties in a simulation study. We conclude that the proposed methodology is a fully viable alternative to existing estimation methods, is well suited to sensitivity analyses that explore the use of alternative estimators, and should be used instead of the existing DerSimonian and Laird-type moments based estimator in application areas where data are expected to be heterogeneous. However, multistep estimators do not seem to outperform the existing estimators when the data are more homogeneous. Advantages of the new multivariate multistep estimator include its semi-parametric nature and that it is computationally feasible in high dimensions. Our proposed estimation methods are also applicable for multivariate random-effects meta-regression, where study-level covariates are included in the model.

Comparative efficacy of materials used in patients undergoing pulpotomy or direct pulp capping in carious teeth: A systematic review and meta-analysis

OBJECTIVES

Different materials have been used for capping the pulp after exposure during caries removal in permanent teeth. The purpose of this study was to collate and analyze all pertinent evidence from randomized controlled trials (RCTs) on different materials used in patients undergoing pulpotomy or direct pulp capping in carious teeth.

MATERIALS AND METHODS

Trials comparing two or more capping agents used for direct pulp capping (DPC) or pulpotomy were considered eligible. An electronic search of four databases and two clinical trial registries was carried out up to February 28, 2021 using a search strategy properly adapted to the PICO framework. Screening, data extraction, and risk of bias (RoB) assessment of primary studies were performed in duplicate and independently. The primary outcome was clinical and radiological success; secondary outcomes included continued root formation, tooth discoloration, and dentin bridge formation.

RESULTS

21 RCTs were included in the study. The RoB assessment indicated a moderate risk among the studies. Due to significant clinical and statistical heterogeneity among the studies, performing network meta-analysis (NMA) was not possible. An ad hoc subgroup analysis revealed strong evidence of a higher success of DPC with Mineral Trioxide Aggregate (MTA) compared to calcium hydroxide (CH) (odds ratio [OR] = 3.10, 95% confidence interval [CI]: 1.66-5.79). MTA performed better than CH in pulp capping (both DPC and pulpotomy) of mature compared to immature teeth (OR = 3.34, 95% CI: 1.81-6.17). The GRADE assessment revealed moderate strength of evidence for DPC and mature teeth, and low to very low strength of evidence for the remaining subgroups.

CONCLUSIONS

Considerable clinical and statistical heterogeneity among the trials did not allow NMA. The ad hoc subgroup analysis indicated that the clinical and radiographic success of MTA was higher than that of CH but only in mature teeth and DPC cases where the strength of evidence was moderate. PROSPERO Registration: number CRD42020127239.

Two-stage or not two-stage? That is the question for IPD meta-analysis projects

Individual participant data meta-analysis (IPDMA) projects obtain, check, harmonise and synthesise raw data from multiple studies. When undertaking the meta-analysis, researchers must decide between a two-stage or a one-stage approach. In a two-stage approach, the IPD are first analysed separately within each study to obtain aggregate data (e.g., treatment effect estimates and standard errors); then, in the second stage, these aggregate data are combined in a standard meta-analysis model (e.g., common-effect or random-effects). In a one-stage approach, the IPD from all studies are analysed in a single step using an appropriate model that accounts for clustering of participants within studies and, potentially, between-study heterogeneity (e.g., a general or generalised linear mixed model). The best approach to take is debated in the literature, and so here we provide clearer guidance for a broad audience. Both approaches are important tools for IPDMA researchers and neither are a panacea. If most studies in the IPDMA are small (few participants or events), a one-stage approach is recommended due to using a more exact likelihood. However, in other situations, researchers can choose either approach, carefully following best practice. Some previous claims recommending to always use a one-stage approach are misleading, and the two-stage approach will often suffice for most researchers. When differences do arise between the two approaches, often it is caused by researchers using different modelling assumptions or estimation methods, rather than using one or two stages per se.

Meta-analyzing partial correlation coefficients using Fisher's z transformation

The partial correlation coefficient (PCC) is used to quantify the linear relationship between two variables while taking into account/controlling for other variables. Researchers frequently synthesize PCCs in a meta-analysis, but two of the assumptions of the common equal-effect and random-effects meta-analysis model are by definition violated. First, the sampling variance of the PCC cannot assumed to be known, because the sampling variance is a function of the PCC. Second, the sampling distribution of each primary study's PCC is not normal since PCCs are bounded between -1 and 1. I advocate applying the Fisher's z transformation analogous to applying Fisher's z transformation for Pearson correlation coefficients, because the Fisher's z transformed PCC is independent of the sampling variance and its sampling distribution more closely follows a normal distribution. Reproducing a simulation study by Stanley and Doucouliagos and adding meta-analyses based on Fisher's z transformed PCCs shows that the meta-analysis based on Fisher's z transformed PCCs had lower bias and root mean square error than meta-analyzing PCCs. Hence, meta-analyzing Fisher's z transformed PCCs is a viable alternative to meta-analyzing PCCs, and I recommend to accompany any meta-analysis based on PCCs with one using Fisher's z transformed PCCs to assess the robustness of the results.

Comparison of statistical methods for integrating real-world evidence in a rare events meta-analysis of randomized controlled trials

Rare events meta-analyses of randomized controlled trials (RCTs) are often underpowered because the outcomes are infrequent. Real-world evidence (RWE) from non-randomized studies may provide valuable complementary evidence about the effects of rare events, and there is growing interest in including such evidence in the decision-making process. Several methods for combining RCTs and RWE studies have been proposed, but the comparative performance of these methods is not well understood. We describe a simulation study that aims to evaluate an array of alternative Bayesian methods for including RWE in rare events meta-analysis of RCTs: the naïve data synthesis, the design-adjusted synthesis, the use of RWE as prior information, the three-level hierarchical models, and the bias-corrected meta-analysis model. The percentage bias, root-mean-square-error, mean 95% credible interval width, coverage probability, and power are used to measure performance. The various methods are illustrated using a systematic review to evaluate the risk of diabetic ketoacidosis among patients using sodium/glucose co-transporter 2 inhibitors as compared with active-comparators. Our simulations show that the bias-corrected meta-analysis model is comparable to or better than the other methods in terms of all evaluated performance measures and simulation scenarios. Our results also demonstrate that data solely from RCTs may not be sufficiently reliable for assessing the effects of rare events. In summary, the inclusion of RWE could increase the certainty and comprehensiveness of the body of evidence of rare events from RCTs, and the bias-corrected meta-analysis model may be preferable.

The prevalence of vancomycin-resistant Staphylococcus aureus in Ethiopia: a systematic review and meta-analysis

INTRODUCTION

Vancomycin-resistant Staphylococcus aureus, identified as a "high priority antibiotic-resistant pathogen" by the World Health Organization, poses a significant threat to human health. This systematic review and meta-analysis aimed to estimate the pooled prevalence of vancomycin-resistant Staphylococcus aureus in Ethiopia.

METHODS

This systematic review and meta-analysis was reported in accordance with the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. Studies that reported VRSA prevalence due to infection or carriage from human clinical specimens were extensively searched in bibliographic databases and grey literatures using entry terms and combination key words. Electronic databases like PubMed, Google Scholar, Wiley Online Library, African Journal Online, Scopus, Science Direct, Embase, and ResearchGate were used to find relevant articles. In addition, the Joanna Briggs Institute quality appraisal tool was used to assess the quality of the included studies. Stata version 14 software was used for statistical analysis. Forest plots using the random-effect model were used to compute the overall pooled prevalence of VRSA and for the subgroup analysis. Heterogeneity was assessed using Cochrane chi-square (I2) statistics. After publication bias was assessed using a funnel plot and Egger's test, trim & fill analysis was carried out. Furthermore, sensitivity analysis was done to assess the impact of a single study on pooled effect size.

RESULTS

Of the 735 studies identified, 31 studies that fulfilled the eligibility criteria were included for meta-analysis consisted of 14,966 study participants and 2,348 S. aureus isolates. The overall pooled prevalence of VRSA was 14.52% (95% CI: 11.59, 17.44). Significantly high level of heterogeneity was observed among studies (I2 = 93.0%, p < 0.001). The region-based subgroup analysis depicted highest pooled prevalence of 47.74% (95% CI: 17.79, 77.69) in Sidama region, followed by 14.82% (95% CI: 8.68, 19.88) in Amhara region, while Oromia region had the least pooled prevalence 8.07% (95% CI: 4.09, 12.06). The subgroup analysis based on AST methods depicted a significant variation in pooled prevalence of VRSA (6.3% (95% CI: 3.14, 9.43) for MIC-based methods, and 18.4% (95% CI: 14.03, 22.79) for disk diffusion AST method) which clearly showed that disk diffusion AST method overestimates the pooled VRSA prevalence. The total number of S. aureus isolates was found to be the responsible variable for the existence of heterogeneity among studies (p = 0.033).

CONCLUSION

This study showed an alarmingly high pooled prevalence of VRSA necessitating routine screening, appropriate antibiotic usage, and robust infection prevention measures to manage MRSA infections and control the emergence of drug resistance. Furthermore, mainly attributable to the overestimation of VRSA burden while using disk diffusion method, there is an urgent need to improve the methods to determine vancomycin resistance in Ethiopia and incorporate MIC-based VRSA detection methods in routine clinical laboratory tests, and efforts should be directed at improving it nationally.

TRIAL REGISTRATION

PROSPERO registration identification number: CRD42023422043.

Metal Exposure and Risk of Parkinson Disease: A Systematic Review and Meta-Analysis

Metal exposure has been suggested as a possible environmental risk factor for Parkinson disease (PD). We searched the PubMed, EMBASE, and Cochrane databases to systematically review the literature on the relationship between metal exposure and PD risk and to examine the overall quality of each study and the exposure assessment method. A total of 83 case-control studies and 5 cohort studies published during the period 1963-July 2021 were included, of which 73 were graded as being of low or moderate overall quality. Investigators in 69 studies adopted self-reported exposure and biomonitoring after disease diagnosis for exposure assessment approaches. The meta-analyses showed that concentrations of copper and iron in serum and concentrations of zinc in either serum or plasma were lower, while concentrations of magnesium in CSF and zinc in hair were higher, among PD cases as compared with controls. Cumulative lead levels in bone were found to be associated with increased risk of PD. We did not find associations between other metals and PD. The current level of evidence for associations between metals and PD risk is limited, as biases from methodological limitations cannot be ruled out. High-quality studies assessing metal levels before disease onset are needed to improve our understanding of the role of metals in the etiology of PD.

The normality assumption on between-study random effects was questionable in a considerable number of Cochrane meta-analyses

BACKGROUND

Studies included in a meta-analysis are often heterogeneous. The traditional random-effects models assume their true effects to follow a normal distribution, while it is unclear if this critical assumption is practical. Violations of this between-study normality assumption could lead to problematic meta-analytical conclusions. We aimed to empirically examine if this assumption is valid in published meta-analyses.

METHODS

In this cross-sectional study, we collected meta-analyses available in the Cochrane Library with at least 10 studies and with between-study variance estimates > 0. For each extracted meta-analysis, we performed the Shapiro-Wilk (SW) test to quantitatively assess the between-study normality assumption. For binary outcomes, we assessed between-study normality for odds ratios (ORs), relative risks (RRs), and risk differences (RDs). Subgroup analyses based on sample sizes and event rates were used to rule out the potential confounders. In addition, we obtained the quantile-quantile (Q-Q) plot of study-specific standardized residuals for visually assessing between-study normality.

RESULTS

Based on 4234 eligible meta-analyses with binary outcomes and 3433 with non-binary outcomes, the proportion of meta-analyses that had statistically significant non-normality varied from 15.1 to 26.2%. RDs and non-binary outcomes led to more frequent non-normality issues than ORs and RRs. For binary outcomes, the between-study non-normality was more frequently found in meta-analyses with larger sample sizes and event rates away from 0 and 100%. The agreements of assessing the normality between two independent researchers based on Q-Q plots were fair or moderate.

CONCLUSIONS

The between-study normality assumption is commonly violated in Cochrane meta-analyses. This assumption should be routinely assessed when performing a meta-analysis. When it may not hold, alternative meta-analysis methods that do not make this assumption should be considered.

A Bayesian model for combining standardized mean differences and odds ratios in the same meta-analysis

In meta-analysis practice, researchers frequently face studies that report the same outcome differently, such as a continuous variable (e.g., scores for rating depression) or a binary variable (e.g., counts of patients with depression dichotomized by certain latent and unreported depression scores). For combining these two types of studies in the same analysis, a simple conversion method has been widely used to handle standardized mean differences (SMDs) and odds ratios (ORs). This conventional method uses a linear function connecting the SMD and log OR; it assumes logistic distributions for (latent) continuous measures. However, the normality assumption is more commonly used for continuous measures, and the conventional method may be inaccurate when effect sizes are large or cutoff values for dichotomizing binary events are extreme (leading to rare events). This article proposes a Bayesian hierarchical model to synthesize SMDs and ORs without using the conventional conversion method. This model assumes exact likelihoods for continuous and binary outcome measures, which account for full uncertainties in the synthesized results. We performed simulation studies to compare the performance of the conventional and Bayesian methods in various settings. The Bayesian method generally produced less biased results with smaller mean squared errors and higher coverage probabilities than the conventional method in most cases. Nevertheless, this superior performance depended on the normality assumption for continuous measures; the Bayesian method could lead to nonignorable biases for non-normal data. In addition, we used two case studies to illustrate the proposed Bayesian method in real-world settings.

Random-effects meta-analysis models for the odds ratio in the case of rare events under different data-generating models: A simulation study

Meta-analysis of binary data is challenging when the event under investigation is rare, and standard models for random-effects meta-analysis perform poorly in such settings. In this simulation study, we investigate the performance of different random-effects meta-analysis models in terms of point and interval estimation of the pooled log odds ratio in rare events meta-analysis. First and foremost, we evaluate the performance of a hypergeometric-normal model from the family of generalized linear mixed models (GLMMs), which has been recommended, but has not yet been thoroughly investigated for rare events meta-analysis. Performance of this model is compared to performance of the beta-binomial model, which yielded favorable results in previous simulation studies, and to the performance of models that are frequently used in rare events meta-analysis, such as the inverse variance model and the Mantel-Haenszel method. In addition to considering a large number of simulation parameters inspired by real-world data settings, we study the comparative performance of the meta-analytic models under two different data-generating models (DGMs) that have been used in past simulation studies. The results of this study show that the hypergeometric-normal GLMM is useful for meta-analysis of rare events when moderate to large heterogeneity is present. In addition, our study reveals important insights with regard to the performance of the beta-binomial model under different DGMs from the binomial-normal family. In particular, we demonstrate that although misalignment of the beta-binomial model with the DGM affects its performance, it shows more robustness to the DGM than its competitors.

A Practical Approach in Refining Binary Outcome for Treatment Effect of COVID-19 According to Geographical Diversity

The recent COVID-19 pandemic has drawn attention to health and economics worldwide. Initially, diseases only ravage local populations, while a pandemic could aggravate global economic burdens. Lopinavir/Ritonavir is an anti-HIV drug that was used on small scale patients during SARS, but its effectiveness for COVID-19 treatment is still unclear. Previous studies or meta-analysis have retrieved clinical data of subgroup analysis to evaluate the efficacy and safety of Lopinavir/Ritonavir for the treatment of COVID-19 in a few affected regions. However, geographical diversity and small number of studies bias correction were not achieved in such subgroup analysis of published meta-analysis. The present study demonstrates a practical approach in refining the binary outcome for COVID-19 treatment of Lopinavir/Ritonavir according to geographical location diversity and small number of studies (less than or equal to five) for subgroup analysis. After performing practical approach, the risk of adverse event with LPV/RTV for treatment of COVID-19 becomes nonsignificant compared to previous meta-analysis. Furthermore, we also notice heterogeneity of random effect of meta-analysis may be declined after proposed adjustment. In conclusion, proposed practical approach is recommend for performing a subgroup analysis to avoid concentration in a single geographical location and small number of studies bias.

Heterogeneity estimation in meta-analysis of standardized mean differences when the distribution of random effects departs from normal: A Monte Carlo simulation study

BACKGROUND

Advantages of meta-analysis depend on the assumptions underlying the statistical procedures used being met. One of the main assumptions that is usually taken for granted is the normality underlying the population of true effects in a random-effects model, even though the available evidence suggests that this assumption is often not met. This paper examines how 21 frequentist and 24 Bayesian methods, including several novel procedures, for computing a point estimate of the heterogeneity parameter ([Formula: see text]) perform when the distribution of random effects departs from normality compared to normal scenarios in meta-analysis of standardized mean differences.

METHODS

A Monte Carlo simulation was carried out using the R software, generating data for meta-analyses using the standardized mean difference. The simulation factors were the number and average sample size of primary studies, the amount of heterogeneity, as well as the shape of the random-effects distribution. The point estimators were compared in terms of absolute bias and variance, although results regarding mean squared error were also discussed.

RESULTS

Although not all the estimators were affected to the same extent, there was a general tendency to obtain lower and more variable [Formula: see text] estimates as the random-effects distribution departed from normality. However, the estimators ranking in terms of their absolute bias and variance did not change: Those estimators that obtained lower bias also showed greater variance. Finally, a large number and sample size of primary studies acted as a bias-protective factor against a lack of normality for several procedures, whereas only a high number of studies was a variance-protective factor for most of the estimators analyzed.

CONCLUSIONS

Although the estimation and inference of the combined effect have proven to be sufficiently robust, our work highlights the role that the deviation from normality may be playing in the meta-analytic conclusions from the simulation results and the numerical examples included in this work. With the aim to exercise caution in the interpretation of the results obtained from random-effects models, the tau2() R function is made available for obtaining the range of [Formula: see text] values computed from the 45 estimators analyzed in this work, as well as to assess how the pooled effect, its confidence and prediction intervals vary according to the estimator chosen.

Meta-analysis methods for risk difference: A comparison of different models

Risk difference is a frequently-used effect measure for binary outcomes. In a meta-analysis, commonly-used methods to synthesize risk differences include: (1) the two-step methods that estimate study-specific risk differences first, then followed by the univariate common-effect model, fixed-effects model, or random-effects models; and (2) the one-step methods using bivariate random-effects models to estimate the summary risk difference from study-specific risks. These methods are expected to have similar performance when the number of studies is large and the event rate is not rare. However, studies with zero events are common in meta-analyses, and bias may occur with the conventional two-step methods from excluding zero-event studies or using an artificial continuity correction to zero events. In contrast, zero-event studies can be included and modeled by bivariate random-effects models in a single step. This article compares various methods to estimate risk differences in meta-analyses. Specifically, we present two case studies and three simulation studies to compare the performance of conventional two-step methods and bivariate random-effects models in the presence or absence of zero-event studies. In conclusion, we recommend researchers using bivariate random-effects models to estimate risk differences in meta-analyses, particularly in the presence of zero events.

A meta-analysis: the efficacy and effectiveness of polypeptide vaccines protect pigs from foot and mouth disease

The protective effects of peptides on pigs are controversial. In this study, meta-analysis was used to analyze the protective immune response of peptides. The China National Knowledge Infrastructure, PubMed, Wanfang Data, Cochrane Library, Embase, and gray literature sources were searched for FMDV articles published from the inception of the databases to March 2022. Of the 1403 articles obtained, 14 were selected using inclusion criteria. The experimental data on polypeptide vaccines were analyzed using Microsoft Office Home and Student 2019 Software. From the results, polypeptide vaccine doses (PPVDs) ≤ 1 mg offered protection against FMDV in 69.41% pigs lower than World Organization for Animal Health (OIE) standard (75%, 12/16). PPVDs ≥ 2 mg provided protection against FMDV in 97.22% pigs. When the two groups were compared directly, PPVDs ≥ 2 mg (93.75%) was higher than PPVDs ≤ 1 mg (63.16%). PPVDs ≤ 1 mg provided protection 56% pigs and the inactivated vaccine was 93.33% in direct comparison. In conclusion, PPVDs has a dose-dependent protective effect on pigs and PPVDs ≤ 1 mg group was lower than the inactivated vaccines group.

Estimating risk and rate ratio in rare events meta-analysis with the Mantel–Haenszel estimator and assessing heterogeneity

Meta-analysis of binary outcome data faces often a situation where studies with a rare event are part of the set of studies to be considered. These studies have low occurrence of event counts to the extreme that no events occur in one or both groups to be compared. This raises issues how to estimate validly the summary risk or rate ratio across studies. A preferred choice is the Mantel–Haenszel estimator, which is still defined in the situation of zero studies unless all studies have zeros in one of the groups to be compared. For this situation, a modified Mantel–Haenszel estimator is suggested and shown to perform well by means of simulation work. Also, confidence interval estimation is discussed and evaluated in a simulation study. In a second part, heterogeneity of relative risk across studies is investigated with a new chi-square type statistic which is based on a conditional binomial distribution where the conditioning is on the event margin for each study. This is necessary as the conventional Q-statistic is undefined in the occurrence of zero studies. The null-distribution of the proposed Q-statistic is obtained by means of a parametric bootstrap as a chi-square approximation is not valid for rare events meta-analysis, as bootstrapping of the null-distribution shows. In addition, for the effect heterogeneity situation, confidence interval estimation is considered using a nonparametric bootstrap procedure. The proposed techniques are illustrated at hand of three meta-analytic data sets.

Jointly pooling aggregated effect sizes and their standard errors from studies with continuous clinical outcomes

The DerSimonian-Laird (DL) weighted average method for aggregated data meta-analysis has been widely used for the estimation of overall effect sizes. It is criticized for its underestimation of the standard error of the overall effect size in the presence of heterogeneous effect sizes. Due to this negative property, many alternative estimation approaches have been proposed in the literature. One of the earliest alternative approaches was developed by Hardy and Thompson (HT), who implemented a profile likelihood instead of the moment-based approach of DL. Others have further extended this likelihood approach and proposed higher-order likelihood inferences (e.g., Bartlett-type corrections). In addition, corrections factors for the estimated DL standard error, like the Hartung-Knapp-Sidik-Jonkman (HKSJ) adjustment, and the restricted maximum likelihood (REML) estimation have been suggested too. Although these improvements address the uncertainty in estimating the between-study variance better than the DL method, they all assume that the true within-study standard errors are known and equal to the observed standard errors of the effect sizes. Here, we will treat the observed standard errors as estimators for the within-study variability and we propose a bivariate likelihood approach that jointly estimates the overall effect size, the between-study variance, and the potentially heteroskedastic within-study variances. We study the performance of the proposed method by means of simulation, and compare it to DL (with and without HKSJ), HT, their higher-order likelihood methods, and REML. Our proposed approach seems to have better or similar coverages compared to the other approaches and it appears to be less biased in the case of heteroskedastic within-study variances when this heteroskedasticty is correlated with the effect size.

Meta-analysis of the severe acute respiratory syndrome coronavirus 2 serial intervals and the impact of parameter uncertainty on the coronavirus disease 2019 reproduction number

The serial interval of an infectious disease, commonly interpreted as the time between the onset of symptoms in sequentially infected individuals within a chain of transmission, is a key epidemiological quantity involved in estimating the reproduction number. The serial interval is closely related to other key quantities, including the incubation period, the generation interval (the time between sequential infections), and time delays between infection and the observations associated with monitoring an outbreak such as confirmed cases, hospital admissions, and deaths. Estimates of these quantities are often based on small data sets from early contact tracing and are subject to considerable uncertainty, which is especially true for early coronavirus disease 2019 data. In this paper, we estimate these key quantities in the context of coronavirus disease 2019 for the UK, including a meta-analysis of early estimates of the serial interval. We estimate distributions for the serial interval with a mean of 5.9 (95% CI 5.2; 6.7) and SD 4.1 (95% CI 3.8; 4.7) days (empirical distribution), the generation interval with a mean of 4.9 (95% CI 4.2; 5.5) and SD 2.0 (95% CI 0.5; 3.2) days (fitted gamma distribution), and the incubation period with a mean 5.2 (95% CI 4.9; 5.5) and SD 5.5 (95% CI 5.1; 5.9) days (fitted log-normal distribution). We quantify the impact of the uncertainty surrounding the serial interval, generation interval, incubation period, and time delays, on the subsequent estimation of the reproduction number, when pragmatic and more formal approaches are taken. These estimates place empirical bounds on the estimates of most relevant model parameters and are expected to contribute to modeling coronavirus disease 2019 transmission.

A meta-analysis approach for evaluating the effectiveness of complex multisite programs

The National Institutes of Health (NIH) created the Building Infrastructure Leading to Diversity (BUILD) initiative to incentivize undergraduate institutions to create innovative approaches to increasing diversity in biomedical research, with the ultimate goal of diversifying the NIH-funded research enterprise. Initiatives such as BUILD involve designing and implementing programs at multiple sites that share common objectives. Evaluation of initiatives like this often includes statistical analyses that combine data across sites to estimate the program's impact on particular outcomes. Meta-analysis is a statistical technique for combining effect estimates from different studies to obtain a single overall effect estimate and to estimate heterogeneity across studies. However, it has not been commonly applied to evaluate the impact of a program across multiple different sites. In this chapter, we use the BUILD Scholar program-one component of the broader initiative-to demonstrate the application of meta-analysis to combine effect estimates from different sites of a multisite initiative. We analyze three student outcomes using a typical "single-stage" modeling approach and a meta-analysis approach. We show how a meta-analysis approach can provide more nuanced information about program impacts on student outcomes and thus can help support a robust evaluation.

Estimation of the COVID-19 Mean Incubation Time: Systematic Review, Meta-analysis, and Sensitivity Analysis

Providing sensible estimates of the mean incubation time for COVID‐19 is important yet complex. This study aims to provide synthetic estimates of the mean incubation time of COVID‐19 by capitalizing on available estimates reported in the literature and exploring different ways to accommodate heterogeneity involved in the reported studies. Online databases between January 1, 2020 and May 20, 2021 are first searched to obtain estimates of the mean incubation time of COVID‐19, and meta‐analyses are then conducted to generate synthetic estimates. Heterogeneity of the studies is examined via the use of Cochran's Q statistic and Higgin's & Thompson's I2 statistic, and subgroup analyses are conducted using mixed effects models. The publication bias issue is assessed using the funnel plot and Egger's test. Using all those reported mean incubation estimates for COVID‐19, the synthetic mean incubation time is estimated to be 6.43 days with a 95% confidence interval (CI) [5.90, 6.96], and using all those reported mean incubation estimates together with those transformed median incubation estimates, the estimated mean incubation time is 6.07 days with a 95% CI [5.70, 6.45]. The reported estimates of the mean incubation time of COVID‐19 vary considerably due to multiple reasons, including heterogeneity and publication bias. To alleviate these issues, we take different angles to provide a sensible estimate of the mean incubation time of COVID‐19. Our analyses show that the mean incubation time of COVID‐19 between January 1, 2020 and May 20, 2021 ranges from 5.68 to 8.30 days.

Tranexamic acid for the prevention of postpartum hemorrhage in women undergoing cesarean delivery: an updated meta-analysis

OBJECTIVE

This study aimed to assess the efficacy and safety of prophylactic tranexamic acid administration vs standard uterotonic agents alone among women undergoing cesarean delivery.

DATA SOURCES

MEDLINE, Scopus, Web of Science, Cochrane Central Register of Controlled Trials, ClinicalTrials.gov, and Google Scholar were systematically searched from inception to June 30, 2021.

STUDY ELIGIBILITY CRITERIA

Randomized controlled trials comparing intravenous tranexamic acid administration with placebo in women undergoing cesarean delivery and receiving standard prophylactic uterotonic agents were held eligible.

STUDY APPRAISAL AND SYNTHESIS METHODS

The risk of bias of individual studies was appraised with the Risk of Bias 2 tool. Meta-analysis was conducted by fitting random-effects models using restricted maximum likelihood. Subgroup analysis was performed on the basis of country, protocol availability, double-blinding, risk of bias, sample size, and tranexamic acid dose. A 1-stage meta-analysis was performed as a sensitivity analysis. The credibility of outcomes was appraised with the Grading of Recommendations Assessment, Development and Evaluation approach.

RESULTS

Overall, 36 studies with 10,659 women were included. Tranexamic acid administration was associated with significantly lower total blood loss (mean difference, -189.44 mL; 95% confidence intervals, -218.63 to -160.25), lower hemoglobin drop (mean difference, 8.22%; 95% confidence interval, 5.54-10.90), decreased risk of blood loss of >1000 mL (odds ratio, 0.37; 95% confidence interval, 0.22-0.60), transfusion requirement (odds ratio, 0.41; 95% confidence interval, 0.26-0.65), and need of additional uterotonics (odds ratio, 0.36; 95% confidence interval, 0.25-0.52). Subgroup analysis indicated a greater effect of tranexamic acid on total blood loss reduction in low-middle income countries. The outcomes remained stable by separately evaluating women at low bleeding risk. The 1-stage meta-analysis demonstrated similar outcomes with the primary analysis. The quality of evidence was judged to be moderate regarding total blood loss and hemoglobin percentage change and low for the other outcomes.

CONCLUSION

This meta-analysis suggested that prophylactic tranexamic acid administration is effective among women undergoing cesarean delivery in lowering postpartum blood loss and limiting hemoglobin drop. Further research is needed to test its efficacy in high-risk populations and verify its safety profile.

Shapley variable importance cloud for interpretable machine learning

Interpretable machine learning has been focusing on explaining final models that optimize performance. The state-of-the-art Shapley additive explanations (SHAP) locally explains the variable impact on individual predictions and has recently been extended to provide global assessments across the dataset. Our work further extends “global” assessments to a set of models that are “good enough” and are practically as relevant as the final model to a prediction task. The resulting Shapley variable importance cloud consists of Shapley-based importance measures from each good model and pools information across models to provide an overall importance measure, with uncertainty explicitly quantified to support formal statistical inference. We developed visualizations to highlight the uncertainty and to illustrate its implications to practical inference. Building on a common theoretical basis, our method seamlessly complements the widely adopted SHAP assessments of a single final model to avoid biased inference, which we demonstrate in two experiments using recidivism prediction data and clinical data.

•

Comprehensive global variable importance assessments beyond final (optimal) models

•

Integrates with SHAP to complement current interpretable machine learning research

•

ShapleyVIC quantifies uncertainty in variable importance for rigorous assessments

•

ShapleyVIC visualizes uncertainty to explore good models with specific properties

With the wide use of machine learning models in decision making, various explanation methods have been developed to help researchers understand how each variable contributes to predictions. However, the current explanation approach focuses on explaining the final (often best performing) models, ignoring the fact that in practice, researchers are willing to consider models that are “good enough” and are easier to understand and/or implement. We propose the Shapley variable importance cloud to address this practical need by extending the current explanation approach to a set of “good models,” which pools information across models to derive a more reliable measure for overall variable importance. Moreover, we analyze and visualize the uncertainty of variable importance across models, which enables rigorous statistical assessments and helps discover alternative models with preferrable properties.

Association between SARS-Cov-2 infection during pregnancy and adverse pregnancy outcomes: a re-analysis of the data reported by Wei et al. (2021)

Objectives and background

Wei et al. have published a meta-analysis (MA), which aimed to evaluate the association between SARS-CoV-2 infection during pregnancy and adverse pregnancy outcomes. Using classical random-effects model, they found that SARS-CoV-2 infection was associated with preeclampsia, preterm birth and stillbirth. Performing MA with low event rates or with few studies may be challenging insofar as MA relies on several within and between-study distributional assumptions. The objective was to assess the robustness of the results provided by Wei et al.

Methods

We performed a sensitivity analysis using frequentist and Bayesian meta-analysis methods. We also estimated fragility indexes.

Results

For eclampsia, the confidence intervals of most frequentist models contain 1. All beta-binomial models (Bayesian) lead to credible intervals containing 1. The prediction interval, based on DL method, ranges from 0.75 to 2.38. The fragility index is 2 for the DL method. For preterm, the confidence (credible) intervals exclude 1. The prediction interval is broad, ranging from 0.84 to 20.61. The fragility index ranges from 27 to 10. For stillbirth, the confidence intervals of most frequentist models contain 1. Six Bayesian MA models lead to credible intervals containing 1. The prediction interval ranges from 0.52 to 8.49. The fragility index is 3.

Conclusion

Given the available data and the results of our broad sensitivity analysis, we can suggest that SARS-CoV-2 infection during pregnancy is associated with preterm, and that it may be associated with preeclampsia. For stillbirth, more data are needed as none of the Bayesian analyses are conclusive.

Empirical Comparisons of 12 Meta-analysis Methods for Synthesizing Proportions of Binary Outcomes

BACKGROUND

Meta-analysis is increasingly used to synthesize proportions (e.g., disease prevalence). It can be implemented with widely used two-step methods or one-step methods, such as generalized linear mixed models (GLMMs). Existing simulation studies have shown that GLMMs outperform the two-step methods in some settings. It is, however, unclear whether these simulation settings are common in the real world. We aim to compare the real-world performance of various meta-analysis methods for synthesizing proportions.

METHODS

We extracted datasets of proportions from the Cochrane Library and applied 12 two-step and one-step methods to each dataset. We used Spearman's ρ and the Bland-Altman plot to assess their results' correlation and agreement. The GLMM with the logit link was chosen as the reference method. We calculated the absolute difference and fold change (ratio of estimates) of the overall proportion estimates produced by each method vs. the reference method.

RESULTS

We obtained a total of 43,644 datasets. The various methods generally had high correlations (ρ > 0.9) and agreements. GLMMs had computational issues more frequently than the two-step methods. However, the two-step methods generally produced large absolute differences from the GLMM with the logit link for small total sample sizes (< 50) and crude event rates within 10-20% and 90-95%, and large fold changes for small total event counts (< 10) and low crude event rates (< 20%).

CONCLUSIONS

Although different methods produced similar overall proportion estimates in most datasets, one-step methods should be considered in the presence of small total event counts or sample sizes and very low or high event rates.

Software to Conduct a Meta-Analysis and Network Meta-Analysis

Statistical software for meta-analysis (MA) and network meta-analysis (NMA) have become indispensable for researchers. The aim of this chapter is to introduce key features of MA and NMA software to compare the effectiveness of interventions. Commonly used or routinely maintained statistical software are reviewed, including commercial and open-sourced programs such as Stata, R and Excel plug-ins. It does not provide a comprehensive overview of all features available in the software covered. Rather, it focuses on the essential features required to carry out an MA or NMA . This chapter begins with a review of key considerations when implementing an MA or NMA , then presents a summary of the software. Key features of each software option are discussed.

Combining chains of Bayesian models with Markov melding

A challenge for practitioners of Bayesian inference is specifying a model that incorporates multiple relevant, heterogeneous data sets. It may be easier to instead specify distinct submodels for each source of data, then join the submodels together. We consider chains of submodels, where submodels directly relate to their neighbours via common quantities which may be parameters or deterministic functions thereof. We propose chained Markov melding, an extension of Markov melding, a generic method to combine chains of submodels into a joint model. One challenge we address is appropriately capturing the prior dependence between common quantities within a submodel, whilst also reconciling differences in priors for the same common quantity between two adjacent submodels. Estimating the posterior of the resulting overall joint model is also challenging, so we describe a sampler that uses the chain structure to incorporate information contained in the submodels in multiple stages, possibly in parallel. We demonstrate our methodology using two examples. The first example considers an ecological integrated population model, where multiple data sets are required to accurately estimate population immigration and reproduction rates. We also consider a joint longitudinal and time-to-event model with uncertain, submodel-derived event times. Chained Markov melding is a conceptually appealing approach to integrating submodels in these settings.

How robust are findings of pairwise and network meta-analysis in the presence of missing participant outcome data?

BACKGROUND

To investigate the prevalence of robust conclusions in systematic reviews addressing missing (participant) outcome data via a novel framework of sensitivity analyses and examine the agreement with the current sensitivity analysis standards.

METHODS

We performed an empirical study on systematic reviews with two or more interventions. Pairwise meta-analyses (PMA) and network meta-analyses (NMA) were identified from empirical studies on the reporting and handling of missing outcome data in systematic reviews. PMAs with at least three studies and NMAs with at least three interventions on one primary outcome were considered eligible. We applied Bayesian methods to obtain the summary effect estimates whilst modelling missing outcome data under the missing-at-random assumption and different assumptions about the missingness mechanism in the compared interventions. The odds ratio in the logarithmic scale was considered for the binary outcomes and the standardised mean difference for the continuous outcomes. We calculated the proportion of primary analyses with robust and frail conclusions, quantified by our proposed metric, the robustness index (RI), and current sensitivity analysis standards. Cohen's kappa statistic was used to measure the agreement between the conclusions derived by the RI and the current sensitivity analysis standards.

RESULTS

One hundred eight PMAs and 34 NMAs were considered. When studies with a substantial number of missing outcome data dominated the analyses, the number of frail conclusions increased. The RI indicated that 59% of the analyses failed to demonstrate robustness compared to 39% when the current sensitivity analysis standards were employed. Comparing the RI with the current sensitivity analysis standards revealed that two in five analyses yielded contradictory conclusions concerning the robustness of the primary analysis results.

CONCLUSIONS

Compared with the current sensitivity analysis standards, the RI offers an explicit definition of similar results and does not unduly rely on statistical significance. Hence, it may safeguard against possible spurious conclusions regarding the robustness of the primary analysis results.

Study specific prediction intervals for random-effects meta-analysis: A tutorial: Prediction intervals in meta-analysis

The pooled estimate of the average effect is of primary interest when fitting the random-effects model for meta-analysis. However estimates of study specific effects, for example those displayed on forest plots, are also often of interest. In this tutorial, we present the case, with the accompanying statistical theory, for estimating the study specific true effects using so called 'empirical Bayes estimates' or 'Best Unbiased Linear Predictions' under the random-effects model. These estimates can be accompanied by prediction intervals that indicate a plausible range of study specific true effects. We coalesce and elucidate the available literature and illustrate the methodology using two published meta-analyses as examples. We also perform a simulation study that reveals that coverage probability of study specific prediction intervals are substantially too low if the between-study variance is small but not negligible. Researchers need to be aware of this defect when interpreting prediction intervals. We also show how empirical Bayes estimates, accompanied with study specific prediction intervals, can embellish forest plots. We hope that this tutorial will serve to provide a clear theoretical underpinning for this methodology and encourage its widespread adoption.

Jackknife empirical likelihood confidence intervals for assessing heterogeneity in meta-analysis of rare binary event data

In meta-analysis, the heterogeneity of effect sizes across component studies is typically described by a variance parameter in a random-effects (Re) model. In the literature, methods for constructing confidence intervals (CIs) for the parameter often assume that study-level effect sizes are normally distributed. However, this assumption might be violated in practice, especially in meta-analysis of rare binary events. We propose to use jackknife empirical likelihood (JEL), a nonparametric approach that uses jackknife pseudo-values, to construct CIs for the heterogeneity parameter. To compute jackknife pseudo-values, we employ a moment-based estimator and consider two commonly used weighing schemes (i.e., equal and inverse variance weights). We prove that with each scheme, the resulting log empirical likelihood ratio follows a chi-square distribution asymptotically. We further examine the performance of the proposed JEL methods and compare them with existing CIs through simulation studies and data examples that focus on data of rare binary events. Our numerical results suggest that the JEL method with equal weights compares favorably to alternatives, especially when (observed) effect sizes are non-normal and the number of component studies is large. Thus, it is worth serious consideration in statistical inference.

Estimating the reference interval from a fixed effects meta-analysis

A reference interval provides a basis for physicians to determine whether a measurement is typical of a healthy individual. It can be interpreted as a prediction interval for a new individual from the overall population. However, a reference interval based on a single study may not be representative of the broader population. Meta-analysis can provide a general reference interval based on the overall population by combining results from multiple studies. Methods for estimating the reference interval from a random effects meta-analysis have been recently proposed to incorporate the within and between-study variation, but a random effects model may give imprecise estimates of the between-study variation with only few studies. In addition, the normal distribution of underlying study-specific means, and equal within-study variance assumption in these methods may be inappropriate in some settings. In this article, we aim to estimate the reference interval based on the fixed effects model assuming study effects are unrelated, which is useful for a meta-analysis with only a few studies (e.g., ≤5). We propose a mixture distribution method only assuming parametric distributions (e.g., normal) for individuals within each study and integrating them to form the overall population distribution. This method is compared to an empirical method only assuming a parametric overall population distribution. Simulation studies have shown that both methods can estimate a reference interval with coverage close to the targeted value (i.e., 95%). Meta-analyses of women daytime urination frequency and frontal subjective postural vertical measurements are reanalyzed to demonstrate the application of our methods.

A metaresearch study revealed susceptibility of Covid-19 treatment research to white hat bias: first, do no harm

Objective

To investigate the presence of white hat bias in Covid-19 treatment research by evaluating the effects of citation and reporting bias.

Study design and setting

Citation bias was investigated by assessing the degree of agreement between evidence provided by a remdesivir randomized controlled trial and its citing articles. The dissimilarity of outcomes derived from nonrandomized and randomized studies was tested by a meta-analysis of hydroxychloroquine effects on mortality. The differential influence of studies with beneficial over those with neutral results was evaluated by a bibliometric analysis.

Results

The articles citing the ACTT-1 remdesivir trial preferentially presented its positive outcomes in 55.83% and its negative outcomes in 6.43% of cases. The hydroxychloroquine indicated no significant effect by randomized studies, but a significant survival benefit by nonrandomized ones. Citation mapping revealed that the study reporting survival benefit from the hydroxychloroquine-azithromycin combination was the most influential, despite subsequent studies reporting potential harmful effects.

Conclusion

The present study raises concerns about citation bias and a predilection of reporting beneficial over harmful effects in the Covid-19 treatment research, potentially in the context of white hat bias. Preregistration, data sharing and avoidance of selective reporting are crucial to ensure the credibility of future research.

Bounds for the weight of external data in shrinkage estimation

Shrinkage estimation in a meta-analysis framework may be used to facilitate dynamical borrowing of information. This framework might be used to analyze a new study in the light of previous data, which might differ in their design (e.g., a randomized controlled trial and a clinical registry). We show how the common study weights arise in effect and shrinkage estimation, and how these may be generalized to the case of Bayesian meta-analysis. Next we develop simple ways to compute bounds on the weights, so that the contribution of the external evidence may be assessed a priori. These considerations are illustrated and discussed using numerical examples, including applications in the treatment of Creutzfeldt-Jakob disease and in fetal monitoring to prevent the occurrence of metabolic acidosis. The target study's contribution to the resulting estimate is shown to be bounded below. Therefore, concerns of evidence being easily overwhelmed by external data are largely unwarranted.

On weakly informative prior distributions for the heterogeneity parameter in Bayesian random-effects meta-analysis

The normal-normal hierarchical model (NNHM) constitutes a simple and widely used framework for meta-analysis. In the common case of only few studies contributing to the meta-analysis, standard approaches to inference tend to perform poorly, and Bayesian meta-analysis has been suggested as a potential solution. The Bayesian approach, however, requires the sensible specification of prior distributions. While noninformative priors are commonly used for the overall mean effect, the use of weakly informative priors has been suggested for the heterogeneity parameter, in particular in the setting of (very) few studies. To date, however, a consensus on how to generally specify a weakly informative heterogeneity prior is lacking. Here we investigate the problem more closely and provide some guidance on prior specification.

Meta-analysis using flexible random-effects distribution models

BACKGROUND

In meta-analysis, the normal distribution assumption has been adopted in most systematic reviews of random-effects distribution models due to its computational and conceptual simplicity. However, this restrictive model assumption is possibly unsuitable and might have serious influences in practices.

METHODS

We provide two examples of real-world evidence that clearly show that the normal distribution assumption is explicitly unsuitable. We propose new random-effects meta-analysis methods using five flexible random-effects distribution models that can flexibly regulate skewness, kurtosis and tailweight: skew normal distribution, skew t-distribution, asymmetric Subbotin distribution, Jones-Faddy distribution, and sinh-arcsinh distribution. We also developed a statistical package, flexmeta, that can easily perform these methods.

RESULTS

Using the flexible random-effects distribution models, the results of the two meta-analyses were markedly altered, potentially influencing the overall conclusions of these systematic reviews.

CONCLUSIONS

The restrictive normal distribution assumption in the random-effects model can yield misleading conclusions. The proposed flexible methods can provide more precise conclusions in systematic reviews.

Evaluation of various estimators for standardized mean difference in meta-analysis

Meta-analyses of a treatment's effect compared with a control frequently calculate the meta-effect from standardized mean differences (SMDs). SMDs are usually estimated by Cohen's d or Hedges' g. Cohen's d divides the difference between sample means of a continuous response by the pooled standard deviation, but is subject to nonnegligible bias for small sample sizes. Hedges' g removes this bias with a correction factor. The current literature (including meta-analysis books and software packages) is confusingly inconsistent about methods for synthesizing SMDs, potentially making reproducibility a problem. Using conventional methods, the variance estimate of SMD is associated with the point estimate of SMD, so Hedges' g is not guaranteed to be unbiased in meta-analyses. This article comprehensively reviews and evaluates available methods for synthesizing SMDs. Their performance is compared using extensive simulation studies and analyses of actual datasets. We find that because of the intrinsic association between point estimates and standard errors, the usual version of Hedges' g can result in more biased meta-estimation than Cohen's d. We recommend using average-adjusted variance estimators to obtain an unbiased meta-estimate, and the Hartung-Knapp-Sidik-Jonkman method for accurate estimation of its confidence interval.

Pattern-mixture model in network meta-analysis of binary missing outcome data: one-stage or two-stage approach?

Background

Trials with binary outcomes can be synthesised using within-trial exact likelihood or approximate normal likelihood in one-stage or two-stage approaches, respectively. The performance of the one-stage and the two-stage approaches has been documented extensively in the literature. However, little is known about how these approaches behave in the presence of missing outcome data (MOD), which are ubiquitous in clinical trials. In this work, we compare the one-stage versus two-stage approach via a pattern-mixture model in the network meta-analysis using Bayesian methods to handle MOD appropriately.

Methods

We used 29 published networks to empirically compare the two approaches concerning the relative treatment effects of several competing interventions and the between-trial variance (τ²), while considering the extent and level of balance of MOD in the included trials. We additionally conducted a simulation study to compare the competing approaches regarding the bias and width of the 95% credible interval of the (summary) log odds ratios (OR) and τ² in the presence of moderate and large MOD.

Results

The empirical study did not reveal any systematic bias between the compared approaches regarding the log OR, but showed systematically larger uncertainty around the log OR under the one-stage approach for networks with at least one small trial or low event risk and moderate MOD. For these networks, the simulation study revealed that the bias in log OR for comparisons with the reference intervention in the network was relatively higher in the two-stage approach. Contrariwise, the bias in log OR for the remaining comparisons was relatively higher in the one-stage approach. Overall, bias increased for large MOD. For these networks, the empirical results revealed slightly higher τ² estimates under the one-stage approach irrespective of the extent of MOD. The one-stage approach also led to less precise log OR and τ² when compared with the two-stage approach for large MOD.

Conclusions

Due to considerable bias in the log ORs overall, especially for large MOD, none of the competing approaches was superior. Until a more competent model is developed, the researchers may prefer the one-stage approach to handle MOD, while acknowledging its limitations.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-020-01205-6.

Continuous(ly) missing outcome data in network meta-analysis: A one-stage pattern-mixture model approach

Appropriate handling of aggregate missing outcome data is necessary to minimise bias in the conclusions of systematic reviews. The two-stage pattern-mixture model has been already proposed to address aggregate missing continuous outcome data. While this approach is more proper compared with the exclusion of missing continuous outcome data and simple imputation methods, it does not offer flexible modelling of missing continuous outcome data to investigate their implications on the conclusions thoroughly. Therefore, we propose a one-stage pattern-mixture model approach under the Bayesian framework to address missing continuous outcome data in a network of interventions and gain knowledge about the missingness process in different trials and interventions. We extend the hierarchical network meta-analysis model for one aggregate continuous outcome to incorporate a missingness parameter that measures the departure from the missing at random assumption. We consider various effect size estimates for continuous data, and two informative missingness parameters, the informative missingness difference of means and the informative missingness ratio of means. We incorporate our prior belief about the missingness parameters while allowing for several possibilities of prior structures to account for the fact that the missingness process may differ in the network. The method is exemplified in two networks from published reviews comprising a different amount of missing continuous outcome data.