1
|
Zhou X, Yang X, Cai F, Wang L, Xu C, Jia P. Effect of important modifiers on harmful effects in evidence synthesis practice of adverse events were insufficiently investigated: an empirical investigation. BMC Med Res Methodol 2023; 23:106. [PMID: 37118664 PMCID: PMC10142201 DOI: 10.1186/s12874-023-01928-2] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/19/2022] [Accepted: 04/20/2023] [Indexed: 04/30/2023] Open
Abstract
BACKGROUND Safety is important in the assessment of health interventions, while the results of adverse events are often susceptive to potential effect modifiers since the event risk tends to be rare. In this study, we investigated whether the potential impact of the important effect modifiers on harmful effects was analyzed in meta-analyses of adverse events. METHODS Systematic reviews of healthcare interventions, had adverse events as the exclusive outcomes, had at least one meta-analysis, and published between 1st January 2015, and 1st January 2020 were collected. An adverse event was defined as any untoward medical occurrence in a patient or subject in healthcare practice. Six effect modifiers that are the most important for harmful effects were identified by a group discussion. The proportions of eligible systematic reviews that investigated the potential impact of the six effect modifiers on harmful effects were summarized. RESULTS We identified 279 systematic reviews eligible for this study. Except for the modifier of interventions/controls (70.61%, 197/279), most of the systematic reviews failed to investigate the potential impact of treatment duration (21.15%, 59/279), dosage (24.73%, 69/279), age (11.47%, 32/279), risk of bias (6.45%, 18/279), and source of funding (1.08%, 3/279) on harmful effects. Systematic reviews with meta-analyses containing more studies were more likely to investigate the potential impacts of these modifiers on the effects, but the proportion was still low (2.3% to 33.3%). Systematic reviews that developed a protocol were significantly more likely to investigate the potential impact of all these effect modifiers (e.g. treatment duration: odds ratio = 5.08, 95% CI: 2.76 to 9.35) on the results. CONCLUSIONS Current systematic reviews rarely investigated the potential impact of the important effect modifiers on harmful effects. Methodological guidelines for meta-analysis of adverse events should consider "effect modifier" as one of the domains to help systematic review authors better investigate harmful effects.
Collapse
Affiliation(s)
- Xiaoqin Zhou
- Center of Biostatistics, Design, Measurement and Evaluation (CBDME), Department of Clinical Research Management, West China Hospital, Sichuan University, Chengdu, Sichuan, China
- Research Center of Clinical Epidemiology and Evidence-Based Medicine, West China Hospital, Sichuan University, Chengdu, Sichuan, China
| | - Xi Yang
- Key Laboratory for Population Health Across-Life Cycle, Ministry of Education, Anhui, China
- School of Public Health, Anhui Medical University, Anhui, China
| | - Fei Cai
- School of Public Health, Anhui Medical University, Anhui, China
| | - Li Wang
- School of Public Health, Anhui Medical University, Anhui, China
| | - Chang Xu
- Key Laboratory for Population Health Across-Life Cycle, Ministry of Education, Anhui, China
- School of Public Health, Anhui Medical University, Anhui, China
| | - Pengli Jia
- School of Management, Shanxi Medical University, No.56, Xinjian South Road, Yingze District, Taiyuan, 030001, Shanxi, China.
| |
Collapse
|
2
|
Mayo-Wilson E, Qureshi R, Li T. Conducting separate reviews of benefits and harms could improve systematic reviews and meta-analyses. Syst Rev 2023; 12:67. [PMID: 37061724 PMCID: PMC10105415 DOI: 10.1186/s13643-023-02234-0] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 11/02/2022] [Accepted: 04/10/2023] [Indexed: 04/17/2023] Open
Abstract
Guidance for systematic reviews of interventions recommends both benefits and harms be included. Systematic reviews may reach conclusions about harms (or lack of harms) that are not true when reviews include only some relevant studies, rely on incomplete data from eligible studies, use inappropriate methods for synthesizing data, and report results selectively. Separate reviews about harms could address some of these problems, and we argue that conducting separate reviews of harms is a feasible alternative to current standards and practices. Systematic reviews of potential benefits could be organized around the use of interventions for specific health problems. Systematic reviews of potential harms could be broader, including more diverse study designs and including all people at risk of harms (who might use the same intervention to treat different health problems). Multiple reviews about benefits could refer to a single review of harms. This approach could improve the reliability, completeness, and efficiency of systematic reviews.
Collapse
Affiliation(s)
- Evan Mayo-Wilson
- Department of Epidemiology, UNC Gillings School of Global Public Health, Chapel Hill, NC, 27599, USA.
| | - Riaz Qureshi
- Department of Ophthalmology, School of Medicine, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| | - Tianjing Li
- Department of Ophthalmology, School of Medicine, University of Colorado Anschutz Medical Campus, Aurora, CO, USA
| |
Collapse
|
3
|
Jansen K, Holling H. Random-effects meta-analysis models for the odds ratio in the case of rare events under different data-generating models: A simulation study. Biom J 2023; 65:e2200132. [PMID: 36216590 DOI: 10.1002/bimj.202200132] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/02/2022] [Revised: 07/19/2022] [Accepted: 08/25/2022] [Indexed: 11/06/2022]
Abstract
Meta-analysis of binary data is challenging when the event under investigation is rare, and standard models for random-effects meta-analysis perform poorly in such settings. In this simulation study, we investigate the performance of different random-effects meta-analysis models in terms of point and interval estimation of the pooled log odds ratio in rare events meta-analysis. First and foremost, we evaluate the performance of a hypergeometric-normal model from the family of generalized linear mixed models (GLMMs), which has been recommended, but has not yet been thoroughly investigated for rare events meta-analysis. Performance of this model is compared to performance of the beta-binomial model, which yielded favorable results in previous simulation studies, and to the performance of models that are frequently used in rare events meta-analysis, such as the inverse variance model and the Mantel-Haenszel method. In addition to considering a large number of simulation parameters inspired by real-world data settings, we study the comparative performance of the meta-analytic models under two different data-generating models (DGMs) that have been used in past simulation studies. The results of this study show that the hypergeometric-normal GLMM is useful for meta-analysis of rare events when moderate to large heterogeneity is present. In addition, our study reveals important insights with regard to the performance of the beta-binomial model under different DGMs from the binomial-normal family. In particular, we demonstrate that although misalignment of the beta-binomial model with the DGM affects its performance, it shows more robustness to the DGM than its competitors.
Collapse
Affiliation(s)
- Katrin Jansen
- University of Münster, Department of Psychology, Münster, Germany
| | - Heinz Holling
- University of Münster, Department of Psychology, Münster, Germany
| |
Collapse
|
4
|
Lai H, Khan YA, Abbas SZ, Chammam W. Meta-computational techniques' for managing spare data: An application in off-pump heart surgery. COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE 2021; 208:106267. [PMID: 34293493 DOI: 10.1016/j.cmpb.2021.106267] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/07/2021] [Accepted: 06/29/2021] [Indexed: 06/13/2023]
Abstract
BACKGROUND AND OBJECTIVES This research looked at the key considerations to remember when selecting a model for working with sparse data. In the presence of sparse evidence, it proposes ideal conditions for conducting meta-analysis. METHODS Monte Carlo simulations were used to produce study results, and three forms of continuity correction were used in the research. Besides, meta-analytical approaches were used to measure the cumulative effect of treatment and estimate each method's efficiency. A clinical trial in off-pump surgery met the main objectives of this research. Meta-analysis methods were used to determine the outcome of postoperative risk results. After that, with a total population of 3030, Monte Carlo simulations were used to produce research data to run fixed and random-effect models with three continuity correction forms. The type of consistency adjustment used, group imbalances, statistical analysis used, and variance values between studies all affect meta-analytical methods' results. RESULTS MSE values for balanced groups are normally zero. While the Arc-sine variation approach does a decent job of coping with inconsistent results on the effect of treatment, it has concerns with boundary estimates of variance between tests. Furthermore, using continuity correction methods introduces bias and imprecise medication outcome calculations. The spectrum of statistical analysis, such as fixed effects and random effects, can be inferred as completely based on data in samples. The sensitivity analysis of correction decisions could increase the reliability of meta-analysis approaches by enabling researchers to analyze various effect estimation findings. CONCLUSION This research study can be expanded upon by identifying alternative approaches to continuity correction methods and resolving boundary estimate problems. The range of statistical analysis, such as fixed effects and random effects, can be entirely dependent on the samples' type of data. The sensitivity analysis of correction decisions could improve the efficiency of meta-analysis methods by allowing researchers to investigate a wide range of effect estimation results.
Collapse
Affiliation(s)
- Han Lai
- School of Information Engineering, Huanghuai University. China.
| | - Yousaf Ali Khan
- Department of Mathematics and Statistics, Hazara University Mansehra, Pakistan
| | - Syed Zaheer Abbas
- Department of Mathematics and Statistics, Hazara University Mansehra, Pakistan.
| | - Wathek Chammam
- Department of Mathematics, College of Science Al-Zulfi, Majmaah University, PO Box 66, Al-Majmaah 11952, Saudi Arabia.
| |
Collapse
|
5
|
Xu C, Zhou X, Zorzela L, Ju K, Furuya-Kanamori L, Lin L, Lu C, Musa OAH, Vohra S. Utilization of the evidence from studies with no events in meta-analyses of adverse events: an empirical investigation. BMC Med 2021; 19:141. [PMID: 34126999 PMCID: PMC8204528 DOI: 10.1186/s12916-021-02008-2] [Citation(s) in RCA: 9] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/19/2021] [Accepted: 05/17/2021] [Indexed: 02/08/2023] Open
Abstract
BACKGROUNDS Zero-events studies frequently occur in systematic reviews of adverse events, which consist of an important source of evidence. We aimed to examine how evidence of zero-events studies was utilized in the meta-analyses of systematic reviews of adverse events. METHODS We conducted a survey of systematic reviews published in two periods: January 1, 2015, to January 1, 2020, and January 1, 2008, to April 25, 2011. Databases were searched for systematic reviews that conducted at least one meta-analysis of any healthcare intervention and used adverse events as the exclusive outcome. An adverse event was defined as any untoward medical occurrence in a patient or subject in healthcare practice. We summarized the frequency of occurrence of zero-events studies in eligible systematic reviews and how these studies were dealt with in the meta-analyses of these systematic reviews. RESULTS We included 640 eligible systematic reviews. There were 406 (63.45%) systematic reviews involving zero-events studies in their meta-analyses, among which 389 (95.11%) involved single-arm-zero-events studies and 223 (54.93%) involved double-arm-zero-events studies. The majority (98.71%) of these systematic reviews incorporated single-arm-zero-events studies into the meta-analyses. On the other hand, the majority (76.23%) of them excluded double-arm-zero-events studies from the meta-analyses, of which the majority (87.06%) did not discuss the potential impact of excluding such studies. Systematic reviews published at present (2015-2020) tended to incorporate zero-events studies in meta-analyses than those published in the past (2008-2011), but the difference was not significant (proportion difference=-0.09, 95% CI -0.21 to 0.03, p = 0.12). CONCLUSION Systematic review authors routinely treated studies with zero-events in both arms as "non-informative" carriers and excluded them from their reviews. Whether studies with no events are "informative" or not largely depends on the methods and assumptions applied, thus sensitivity analyses using different methods should be considered in future meta-analyses.
Collapse
Affiliation(s)
- Chang Xu
- Department of Population Medicine, College of Medicine, Qatar University, Al Jamiaa Street, P. O. Box, 2713, Doha, Qatar.
| | - Xiaoqin Zhou
- Department of Clinical Research Management, West China Hospital, Sichuan University, Chengdu, China
| | - Liliane Zorzela
- Department of Pediatrics, Faculty of Medicine & Dentistry, University of Alberta, Edmonton, Alberta, Canada
| | - Ke Ju
- West China School of Public Health and West China Fourth Hospital, Sichuan University, Chengdu, China
| | - Luis Furuya-Kanamori
- UQ Centre for Clinical Research, Faculty of Medicine, University of Queensland, Brisbane, Australia
| | - Lifeng Lin
- Department of Statistics, Florida State University, Tallahassee, FL, USA
| | - Cuncun Lu
- Evidence-Based Medicine Center, School of Basic Medical Sciences, Lanzhou University, Lanzhou, China
- Institute of Basic Research in Clinical Medicine, China Academy of Chinese Medical Sciences, Beijing, China
| | - Omran A H Musa
- Department of Population Medicine, College of Medicine, Qatar University, Al Jamiaa Street, P. O. Box, 2713, Doha, Qatar
| | - Sunita Vohra
- Department of Pediatrics, Faculty of Medicine & Dentistry, University of Alberta, Edmonton, Alberta, Canada
- Department of Psychiatry, Faculty of Medicine & Dentistry, University of Alberta, Edmonton, Alberta, Canada
| |
Collapse
|
6
|
Xu C, Furuya-Kanamori L, Zorzela L, Lin L, Vohra S. A proposed framework to guide evidence synthesis practice for meta-analysis with zero-events studies. J Clin Epidemiol 2021; 135:70-78. [PMID: 33592277 DOI: 10.1016/j.jclinepi.2021.02.012] [Citation(s) in RCA: 43] [Impact Index Per Article: 14.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/03/2020] [Revised: 01/28/2021] [Accepted: 02/09/2021] [Indexed: 11/29/2022]
Abstract
OBJECTIVE In evidence synthesis practice, researchers often face the problem of how to deal with zero-events. Inappropriately dealing with zero-events studies may lead to research waste and mislead healthcare practice. We propose a framework to guide researchers to better deal with zero-events in meta-analysis. STUDY DESIGN AND SETTING We used two dimensions, one with respect to the total events count across all studies in the comparative arms in a meta-analysis, and a second with respect to whether included studies have single or both arms with zero-events, to establish the framework for the classification of meta-analysis with zero-events studies. A dataset from Cochrane systematic reviews was used to evaluate the classification. RESULTS The proposed framework classifies meta-analysis with zero-events studies into six subtypes. The classification matched well to the large real-world dataset. The applicability of existing methods for zero-events were then presented under each meta-analysis subtype based on this framework, with a 5-step principle to help researchers in evidence synthesis practice. CONCLUSIONS The proposed framework should be considered by researchers when making decisions on the selection of the synthesis methods in a meta-analysis. It also provides a reasonable basis for the development of methodological guidelines to deal with zero-events in meta-analysis.
Collapse
Affiliation(s)
- Chang Xu
- Department of Population Medicine, College of Medicine, Qatar University, Doha, Qatar.
| | - Luis Furuya-Kanamori
- Research School of Population Health, College of Health and Medicine, Australian National University, Canberra, Australia
| | - Liliane Zorzela
- Department of Pediatrics, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada
| | - Lifeng Lin
- Department of Statistics, Florida State University, Tallahassee, FL, USA
| | - Sunita Vohra
- Department of Pediatrics, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada; Department of Psychiatry, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Alberta, Canada
| |
Collapse
|