Commentary on ‘Misunderstandings aboutQand “Cochran'sQtest” in meta analysis’

Does Remdesivir Lower COVID-19 Mortality? A Subgroup Analysis of Hospitalized Adults Receiving Supplemental Oxygen

The first Adaptive COVID-19 Treatment Trial (ACTT-1) showed that remdesivir improved COVID-19 recovery time compared with placebo in hospitalized adults. The secondary outcome of mortality was almost significant overall (p = 0.07) and highly significant for people receiving supplemental oxygen at enrollment (p = 0.002), suggesting a mortality benefit concentrated in this group. We explore analysis methods that are helpful when a single subgroup benefits from treatment and apply them to ACTT-1, using baseline oxygen use to define subgroups. We consider two questions: (1) is the remdesivir effect for people receiving supplemental oxygen real, and (2) does this effect differ from the overall effect? For Question 1, we apply a Bonferroni adjustment to subgroup-specific hypothesis tests and the Westfall and Young permutation test, which is valid when small cell counts preclude normally distributed test statistics (a frequently unexamined condition in subgroup analyses). For Question 2, we introduce Qmax, the largest standardized difference between subgroup-specific effects and the overall effect. Qmax simultaneously tests whether any subgroup effect differs from the overall effect and identifies the subgroup benefitting most. We demonstrate that Qmax strongly controls the familywise error rate (FWER) when test statistics are normally distributed with no mean-variance relationship. We compare Qmax to a related permutation test, SEAMOS, which was previously proposed but not extensively applied or tested. We show that SEAMOS can have inflated Type 1 error under the global null when control arm event rates differ between subgroups. Our results support a mortality benefit from remdesivir in people receiving supplemental oxygen.

On ratio measures of heterogeneity for meta-analyses

As a measure of heterogeneity in meta-analysis, the coefficient of variation (CV) has been recently considered, providing researchers with a complement to the very popular I 2 measure. While I 2 measures the proportion of total variance that is due to variance of the random effects, the CV is the ratio of the standard deviation of the random effects to the effect of interest. Consequently, the CV provides a different measure of the extent of heterogeneity in a meta-analysis relative to the effect being measured. However, very large CV values can occur when the effect is small making interpretation difficult. The purpose of this article is two-fold. First, we consider variants of the CV that exist in the interval 0 , 1 which may be preferable for some researchers. Second, we provide interval estimators for the CV and its variants with excellent coverage properties. We perform simulation studies based on simulated and real data sets and draw comparisons between the methods. For both the CV and its transformations, we recommend confidence intervals using the propagating imprecision method or, as a simpler alternative but at the expense of slightly worse performance in terms of coverage, combining reduced-coverage confidence intervals for the two parameters. These interval estimators typically have better coverage properties for the CV measure than those previously considered.

Beta-binomial model for meta-analysis of odds ratios

In meta-analysis of odds ratios (ORs), heterogeneity between the studies is usually modelled via the additive random effects model (REM). An alternative, multiplicative REM for ORs uses overdispersion. The multiplicative factor in this overdispersion model (ODM) can be interpreted as an intra-class correlation (ICC) parameter. This model naturally arises when the probabilities of an event in one or both arms of a comparative study are themselves beta-distributed, resulting in beta-binomial distributions. We propose two new estimators of the ICC for meta-analysis in this setting. One is based on the inverted Breslow-Day test, and the other on the improved gamma approximation by Kulinskaya and Dollinger (2015, p. 26) to the distribution of Cochran's Q. The performance of these and several other estimators of ICC on bias and coverage is studied by simulation. Additionally, the Mantel-Haenszel approach to estimation of ORs is extended to the beta-binomial model, and we study performance of various ICC estimators when used in the Mantel-Haenszel or the inverse-variance method to combine ORs in meta-analysis. The results of the simulations show that the improved gamma-based estimator of ICC is superior for small sample sizes, and the Breslow-Day-based estimator is the best for n⩾100. The Mantel-Haenszel-based estimator of OR is very biased and is not recommended. The inverse-variance approach is also somewhat biased for ORs≠1, but this bias is not very large in practical settings. Developed methods and R programs, provided in the Web Appendix, make the beta-binomial model a feasible alternative to the standard REM for meta-analysis of ORs. © 2017 The Authors. Statistics in Medicine Published by John Wiley & Sons Ltd.