Extremely large outlier treatment effects may be a footprint of bias in trials from less developed countries: randomized trials of gabapentinoids

Sensitivity analysis with iterative outlier detection for systematic reviews and meta-analyses

Meta-analysis is a widely used tool for synthesizing results from multiple studies. The collected studies are deemed heterogeneous when they do not share a common underlying effect size; thus, the factors attributable to the heterogeneity need to be carefully considered. A critical problem in meta-analyses and systematic reviews is that outlying studies are frequently included, which can lead to invalid conclusions and affect the robustness of decision-making. Outliers may be caused by several factors such as study selection criteria, low study quality, small-study effects, and so on. Although outlier detection is well-studied in the statistical community, limited attention has been paid to meta-analysis. The conventional outlier detection method in meta-analysis is based on a leave-one-study-out procedure. However, when calculating a potentially outlying study's deviation, other outliers could substantially impact its result. This article proposes an iterative method to detect potential outliers, which reduces such an impact that could confound the detection. Furthermore, we adopt bagging to provide valid inference for sensitivity analyses of excluding outliers. Based on simulation studies, the proposed iterative method yields smaller bias and heterogeneity after performing a sensitivity analysis to remove the identified outliers. It also provides higher accuracy on outlier detection. Two case studies are used to illustrate the proposed method's real-world performance.

GRADE guidelines 33: Addressing imprecision in a network meta-analysis

OBJECTIVE

This article describes GRADE guidance for assessing imprecision when rating the certainty of the evidence from network meta-analysis.

STUDY DESIGN AND SETTING

A project group within the GRADE working group conducted iterative discussions, computer simulations, and presentations at GRADE working group meetings to produce and obtain approval for this guidance.

RESULTS

When addressing imprecision of a network estimate, reviewers should consider the 95% confidence or credible interval, and the optimal information size. If the 95% confidence or credible interval crosses a pre-specified threshold, reviewers should rate down the certainty of the evidence. If the 95% confidence interval does not cross any pre-specfied threshold, reviewers should consider the optimal information size. Because addressing the optimal information size may be challenging, reviewers can use the effect size to decide if any calculations are necessary. When the size of the effect is modest or the optimal information size is met, reviewers should not rate down for imprecision.

CONCLUSION

Reviewers should use this guidance when to appropriately address imprecision in the context of the assessment of certainty of evidence from network meta-analysis.

Efficacy and acceptability of pharmacological and non-pharmacological interventions for non-specific chronic low back pain: a protocol for a systematic review and network meta-analysis

BACKGROUND

Despite the enormous financial and humanistic burden of chronic low back pain (CLBP), there is little consensus on what constitutes the best treatment options from a multitude of competing interventions. The objective of this network meta-analysis (NMA) is to determine the relative efficacy and acceptability of primary care treatments for non-specific CLBP, with the overarching aim of providing a comprehensive evidence base for informing treatment decisions.

METHODS

We will perform a systematic search to identify randomised controlled trials of interventions endorsed in primary care guidelines for the treatment of non-specific CLBP in adults. Information sources searched will include major bibliographic databases (MEDLINE, Embase, CENTRAL, CINAHL, PsycINFO and LILACS) and clinical trial registries. Our primary outcomes will be patient-reported pain ratings and treatment acceptability (all-cause discontinuation), and secondary outcomes will be functional ability, quality of life and patient/physician ratings of overall improvement. A hierarchical Bayesian class-based NMA will be performed to determine the relative effects of different classes of pharmacological (NSAIDs, opioids, paracetamol, anti-depressants, muscle relaxants) and non-pharmacological (exercise, patient education, manual therapies, psychological therapy, multidisciplinary approaches, massage, acupuncture, mindfulness) interventions and individual treatments within a class (e.g. NSAIDs: diclofenac, ibuprofen, naproxen). We will conduct risk of bias assessments and threshold analysis to assess the robustness of the findings to potential bias. We will compute the effect of different interventions relative to placebo/no treatment for both short- and long-term efficacy and acceptability.

DISCUSSION

While many factors are important in selecting an appropriate intervention for an individual patient, evidence for the analgesic effects and acceptability of a treatment are key factors in guiding this selection. Thus, this NMA will provide an important source of evidence to inform treatment decisions and future clinical guidelines.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO registry number: CRD42019138115.

Graphical augmentations to sample-size-based funnel plot in meta-analysis

Assessing publication bias is a critical procedure in meta-analyses for rating the synthesized overall evidence. Because statistical tests for publication bias are usually not powerful and only give P values that inform either the presence or absence of the bias, examining the asymmetry of funnel plots has been popular to investigate potentially missing studies and the direction of the bias. Most funnel plots present treatment effects against their standard errors, and the contours depicting studies' significance levels have been used in the plots to distinguish publication bias from other factors (such as heterogeneity and subgroup effects) that may cause the plots' asymmetry. However, treatment effects and their standard errors are frequently associated even if no publication bias exists (eg, both variables depend on the four data cells in a 2 × 2 table for the odds ratio), so standard-error-based funnel plots may lead to false positive conclusions when such association may not be negligible. In addition, the missingness of studies may relate to their sample sizes besides P values (which are partly determined by standard errors); studies with more samples are more likely published. Therefore, funnel plots based on sample sizes can be an alternative tool. However, the contours for standard-error-based funnel plots cannot be directly applied to sample-size-based ones. This article introduces contours for sample-size-based funnel plots of various effect sizes, which may help meta-analysts properly interpret such plots' asymmetry. We provide five examples to illustrate the use of the proposed contours.