Protocols for meta-analysis of intervention safety seldom specified methods to deal with rare events

Effect of important modifiers on harmful effects in evidence synthesis practice of adverse events were insufficiently investigated: an empirical investigation

BACKGROUND

Safety is important in the assessment of health interventions, while the results of adverse events are often susceptive to potential effect modifiers since the event risk tends to be rare. In this study, we investigated whether the potential impact of the important effect modifiers on harmful effects was analyzed in meta-analyses of adverse events.

METHODS

Systematic reviews of healthcare interventions, had adverse events as the exclusive outcomes, had at least one meta-analysis, and published between 1st January 2015, and 1st January 2020 were collected. An adverse event was defined as any untoward medical occurrence in a patient or subject in healthcare practice. Six effect modifiers that are the most important for harmful effects were identified by a group discussion. The proportions of eligible systematic reviews that investigated the potential impact of the six effect modifiers on harmful effects were summarized.

RESULTS

We identified 279 systematic reviews eligible for this study. Except for the modifier of interventions/controls (70.61%, 197/279), most of the systematic reviews failed to investigate the potential impact of treatment duration (21.15%, 59/279), dosage (24.73%, 69/279), age (11.47%, 32/279), risk of bias (6.45%, 18/279), and source of funding (1.08%, 3/279) on harmful effects. Systematic reviews with meta-analyses containing more studies were more likely to investigate the potential impacts of these modifiers on the effects, but the proportion was still low (2.3% to 33.3%). Systematic reviews that developed a protocol were significantly more likely to investigate the potential impact of all these effect modifiers (e.g. treatment duration: odds ratio = 5.08, 95% CI: 2.76 to 9.35) on the results.

CONCLUSIONS

Current systematic reviews rarely investigated the potential impact of the important effect modifiers on harmful effects. Methodological guidelines for meta-analysis of adverse events should consider "effect modifier" as one of the domains to help systematic review authors better investigate harmful effects.

Conducting separate reviews of benefits and harms could improve systematic reviews and meta-analyses

Guidance for systematic reviews of interventions recommends both benefits and harms be included. Systematic reviews may reach conclusions about harms (or lack of harms) that are not true when reviews include only some relevant studies, rely on incomplete data from eligible studies, use inappropriate methods for synthesizing data, and report results selectively. Separate reviews about harms could address some of these problems, and we argue that conducting separate reviews of harms is a feasible alternative to current standards and practices. Systematic reviews of potential benefits could be organized around the use of interventions for specific health problems. Systematic reviews of potential harms could be broader, including more diverse study designs and including all people at risk of harms (who might use the same intervention to treat different health problems). Multiple reviews about benefits could refer to a single review of harms. This approach could improve the reliability, completeness, and efficiency of systematic reviews.

Random-effects meta-analysis models for the odds ratio in the case of rare events under different data-generating models: A simulation study

Meta-analysis of binary data is challenging when the event under investigation is rare, and standard models for random-effects meta-analysis perform poorly in such settings. In this simulation study, we investigate the performance of different random-effects meta-analysis models in terms of point and interval estimation of the pooled log odds ratio in rare events meta-analysis. First and foremost, we evaluate the performance of a hypergeometric-normal model from the family of generalized linear mixed models (GLMMs), which has been recommended, but has not yet been thoroughly investigated for rare events meta-analysis. Performance of this model is compared to performance of the beta-binomial model, which yielded favorable results in previous simulation studies, and to the performance of models that are frequently used in rare events meta-analysis, such as the inverse variance model and the Mantel-Haenszel method. In addition to considering a large number of simulation parameters inspired by real-world data settings, we study the comparative performance of the meta-analytic models under two different data-generating models (DGMs) that have been used in past simulation studies. The results of this study show that the hypergeometric-normal GLMM is useful for meta-analysis of rare events when moderate to large heterogeneity is present. In addition, our study reveals important insights with regard to the performance of the beta-binomial model under different DGMs from the binomial-normal family. In particular, we demonstrate that although misalignment of the beta-binomial model with the DGM affects its performance, it shows more robustness to the DGM than its competitors.

Meta-computational techniques' for managing spare data: An application in off-pump heart surgery

BACKGROUND AND OBJECTIVES

This research looked at the key considerations to remember when selecting a model for working with sparse data. In the presence of sparse evidence, it proposes ideal conditions for conducting meta-analysis.

METHODS

Monte Carlo simulations were used to produce study results, and three forms of continuity correction were used in the research. Besides, meta-analytical approaches were used to measure the cumulative effect of treatment and estimate each method's efficiency. A clinical trial in off-pump surgery met the main objectives of this research. Meta-analysis methods were used to determine the outcome of postoperative risk results. After that, with a total population of 3030, Monte Carlo simulations were used to produce research data to run fixed and random-effect models with three continuity correction forms. The type of consistency adjustment used, group imbalances, statistical analysis used, and variance values between studies all affect meta-analytical methods' results.

RESULTS

MSE values for balanced groups are normally zero. While the Arc-sine variation approach does a decent job of coping with inconsistent results on the effect of treatment, it has concerns with boundary estimates of variance between tests. Furthermore, using continuity correction methods introduces bias and imprecise medication outcome calculations. The spectrum of statistical analysis, such as fixed effects and random effects, can be inferred as completely based on data in samples. The sensitivity analysis of correction decisions could increase the reliability of meta-analysis approaches by enabling researchers to analyze various effect estimation findings.

CONCLUSION

This research study can be expanded upon by identifying alternative approaches to continuity correction methods and resolving boundary estimate problems. The range of statistical analysis, such as fixed effects and random effects, can be entirely dependent on the samples' type of data. The sensitivity analysis of correction decisions could improve the efficiency of meta-analysis methods by allowing researchers to investigate a wide range of effect estimation results.

Utilization of the evidence from studies with no events in meta-analyses of adverse events: an empirical investigation

BACKGROUNDS

Zero-events studies frequently occur in systematic reviews of adverse events, which consist of an important source of evidence. We aimed to examine how evidence of zero-events studies was utilized in the meta-analyses of systematic reviews of adverse events.

METHODS

We conducted a survey of systematic reviews published in two periods: January 1, 2015, to January 1, 2020, and January 1, 2008, to April 25, 2011. Databases were searched for systematic reviews that conducted at least one meta-analysis of any healthcare intervention and used adverse events as the exclusive outcome. An adverse event was defined as any untoward medical occurrence in a patient or subject in healthcare practice. We summarized the frequency of occurrence of zero-events studies in eligible systematic reviews and how these studies were dealt with in the meta-analyses of these systematic reviews.

RESULTS

We included 640 eligible systematic reviews. There were 406 (63.45%) systematic reviews involving zero-events studies in their meta-analyses, among which 389 (95.11%) involved single-arm-zero-events studies and 223 (54.93%) involved double-arm-zero-events studies. The majority (98.71%) of these systematic reviews incorporated single-arm-zero-events studies into the meta-analyses. On the other hand, the majority (76.23%) of them excluded double-arm-zero-events studies from the meta-analyses, of which the majority (87.06%) did not discuss the potential impact of excluding such studies. Systematic reviews published at present (2015-2020) tended to incorporate zero-events studies in meta-analyses than those published in the past (2008-2011), but the difference was not significant (proportion difference=-0.09, 95% CI -0.21 to 0.03, p = 0.12).

CONCLUSION

Systematic review authors routinely treated studies with zero-events in both arms as "non-informative" carriers and excluded them from their reviews. Whether studies with no events are "informative" or not largely depends on the methods and assumptions applied, thus sensitivity analyses using different methods should be considered in future meta-analyses.

A proposed framework to guide evidence synthesis practice for meta-analysis with zero-events studies

OBJECTIVE

In evidence synthesis practice, researchers often face the problem of how to deal with zero-events. Inappropriately dealing with zero-events studies may lead to research waste and mislead healthcare practice. We propose a framework to guide researchers to better deal with zero-events in meta-analysis.

STUDY DESIGN AND SETTING

We used two dimensions, one with respect to the total events count across all studies in the comparative arms in a meta-analysis, and a second with respect to whether included studies have single or both arms with zero-events, to establish the framework for the classification of meta-analysis with zero-events studies. A dataset from Cochrane systematic reviews was used to evaluate the classification.

RESULTS

The proposed framework classifies meta-analysis with zero-events studies into six subtypes. The classification matched well to the large real-world dataset. The applicability of existing methods for zero-events were then presented under each meta-analysis subtype based on this framework, with a 5-step principle to help researchers in evidence synthesis practice.

CONCLUSIONS

The proposed framework should be considered by researchers when making decisions on the selection of the synthesis methods in a meta-analysis. It also provides a reasonable basis for the development of methodological guidelines to deal with zero-events in meta-analysis.