Multivariate meta-analysis of critical care meta-analyses: a meta-epidemiological study

Problematic meta-analyses: Bayesian and frequentist perspectives on combining randomized controlled trials and non-randomized studies

PURPOSE

In the literature, the propriety of the meta-analytic treatment-effect produced by combining randomized controlled trials (RCT) and non-randomized studies (NRS) is questioned, given the inherent confounding in NRS that may bias the meta-analysis. The current study compared an implicitly principled pooled Bayesian meta-analytic treatment-effect with that of frequentist pooling of RCT and NRS to determine how well each approach handled the NRS bias.

MATERIALS & METHODS

Binary outcome Critical-Care meta-analyses, reflecting the importance of such outcomes in Critical-Care practice, combining RCT and NRS were identified electronically. Bayesian pooled treatment-effect and 95% credible-intervals (BCrI), posterior model probabilities indicating model plausibility and Bayes-factors (BF) were estimated using an informative heavy-tailed heterogeneity prior (half-Cauchy). Preference for pooling of RCT and NRS was indicated for Bayes-factors > 3 or < 0.333 for the converse. All pooled frequentist treatment-effects and 95% confidence intervals (FCI) were re-estimated using the popular DerSimonian-Laird (DSL) random effects model.

RESULTS

Fifty meta-analyses were identified (2009-2021), reporting pooled estimates in 44; 29 were pharmaceutical-therapeutic and 21 were non-pharmaceutical therapeutic. Re-computed pooled DSL FCI excluded the null (OR or RR = 1) in 86% (43/50). In 18 meta-analyses there was an agreement between FCI and BCrI in excluding the null. In 23 meta-analyses where FCI excluded the null, BCrI embraced the null. BF supported a pooled model in 27 meta-analyses and separate models in 4. The highest density of the posterior model probabilities for 0.333 < Bayes factor < 1 was 0.8.

CONCLUSIONS

In the current meta-analytic cohort, an integrated and multifaceted Bayesian approach gave support to including NRS in a pooled-estimate model. Conversely, caution should attend the reporting of naïve frequentist pooled, RCT and NRS, meta-analytic treatment effects.

To seal or not to seal following an antimicrobial infusion at dry-off? A systematic review and multivariate meta-analysis of the incidence and prevalence of intramammary infections post-calving in dairy cows

Teat sealants (TSs) consist of sterile formulations with no antibacterial activity. Alone or in combination with antimicrobial (AM) or non-AM treatments, TSs have been commonly used in dairy cows at dry-off to prevent intra-mammary infections (IMIs) during the dry period. This study aimed to identify and synthesise the available evidence on the efficacy of combining TSs with AM treatments on the incidence and prevalence of IMIs. A comprehensive search of three electronic databases, two relevant conference proceedings, and reference lists of reviews and eligible articles was conducted to retrieve and identify studies that could answer the following question: in dairy cows, how does the efficacy of an AM-TS combination administered at dry-off compare with an AM alone for preventing new IMI? In addition to the general IMIs, bacterial species-specific data were extracted and combined into nine distinct pathogen groups: coagulase-positive and negative staphylococci; S. dysgalactiae; non-dysgalactiae Streptococci; E. coli; non-E. coli Enterobacteriaceae; Corynebacterium spp.; yeast and other frequent mastitis pathogens. The structural relationship between each study's prevalence and incidence, as the new (incidence) and persistent (uncured) infections make up the prevalence, was utilised to approximate a variance-covariance matrix for the within-study correlation between their study-specific log odds ratios (ORs). A bivariate random-effects meta-analysis was employed, utilising the within- and between-study correlations to synthesise both outcomes simultaneously. The risk of bias was assessed using the Cochrane ROBINS-I tool, and the quality of the body of evidence was rated using the GRADE approach. A total of 17 trials (16 studies), providing either IMIs incidence (n = 4), prevalence (n = 3) or both (n = 10), were identified. Overall, quarters infused with AM-TS combinations showed lower odds of new IMIs post-calving (OR=0.70; 95% CI=0.57-0.86; Wald test P < 0.001) than those which received only AMs. Across the pathogen groups, varying levels of reduction of new IMIs were found, where administration of TSs was most effective against S. dysgalactiae (OR=0.47; 95% CI=0.23-0.98), non-dysgalactiae streptococci (OR=0.60; 95% CI=0.49-0.74), E. coli (OR=0.62; 95% CI=0.50-0.77), Corynebacterium spp. (OR=0.68; 95% CI=0.52-0.90) and coagulase-negative staphylococci (OR=0.85; 95% CI=0.76-0.94). However, additional TS infusion did not significantly reduce new IMIs in the remaining pathogen groups. The current meta-analytic evidence supports the efficacy of using TS add-on infusions in dairy cows at dry-off for reducing the incidence and prevalence of IMIs post-calving; however, pathogen group differences should be considered.

Multivariate meta-analysis of multiple outcomes: characteristics and predictors of borrowing of strength from Cochrane reviews

OBJECTIVES

Multivariate meta-analysis allows the joint synthesis of multiple outcomes accounting for their correlation. This enables borrowing of strength (BoS) across outcomes, which may lead to greater efficiency and even different conclusions compared to separate univariate meta-analyses. However, multivariate meta-analysis is complex to apply, so guidance is needed to flag (in advance of analysis) when the approach is most useful.

STUDY DESIGN AND SETTING

We use 43 Cochrane intervention reviews to empirically investigate the characteristics of meta-analysis datasets that are associated with a larger BoS statistic (from 0 to 100%) when applying a bivariate meta-analysis of binary outcomes.

RESULTS

Four characteristics were identified as strongly associated with BoS: the total number of studies, the number of studies with the outcome of interest, the percentage of studies missing the outcome of interest, and the largest absolute within-study correlation. Using these characteristics, we then develop a model for predicting BoS in a new dataset, which is shown to have good performance (an adjusted R² of 50%). Applied examples are used to illustrate the use of the BoS prediction model.

CONCLUSIONS

Cochrane reviewers mainly use univariate meta-analysis methods, but the identified characteristics associated with BoS and our subsequent prediction model for BoS help to flag when a multivariate meta-analysis may also be beneficial in Cochrane reviews with multiple binary outcomes. Extension to non-Cochrane reviews and other outcome types is still required.