1
|
Chapelle C, Ollier E, Bonjean P, Locher C, Zufferey PJ, Cucherat M, Laporte S. Replication of systematic reviews: is it to the benefit or detriment of methodological quality? J Clin Epidemiol 2023; 162:98-106. [PMID: 37648071 DOI: 10.1016/j.jclinepi.2023.08.012] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/28/2023] [Revised: 07/09/2023] [Accepted: 08/22/2023] [Indexed: 09/01/2023]
Abstract
OBJECTIVES To perform an overview of the overlap of systematic reviews (SRs) assessing direct oral anticoagulants and characterize these reviews in terms of bias and methodological quality (PROSPERO: CRD42022316273). STUDY DESIGN AND SETTING A PubMed-indexed search was performed from inception to January 31, 2022 to identify SRs evaluating direct oral anticoagulants in patients treated for an acute venous thromboembolism. The risk of bias of these SRs was assessed according to the Risk Of Bias In Systematic reviews tool. Redundancy was defined as overlap in terms of the type of population considered, the interventions compared, and the studies included. RESULTS A total of 144 SRs were evaluated, of which 26 (18.1%) were classified as original, 87 (60.4%) as conceptual replications, and 31 (21.5%) as excessive replications. The risk of bias was high in 19 (73.1%) of the original SRs, 65 (74.7%) of the conceptual replications, and 21 (67.7%) of the excessive replications. Compared to the original SRs, the overall methodological quality was not improved in either conceptual or excessive replications. CONCLUSION A large number of SRs was classified as replications; a fifth constituted excessive replications. The replications showed no improvement in overall methodological quality compared to the original SRs.
Collapse
Affiliation(s)
- Céline Chapelle
- Univ. Jean Monnet, Mines Saint- Étienne, INSERM, U1059, SAINBIOSE, CHU Saint-Étienne, Service de Pharmacologie Clinique, F-42023 Saint-Étienne, France.
| | - Edouard Ollier
- Univ. Jean Monnet, Mines Saint- Étienne, INSERM, U1059, SAINBIOSE, CHU Saint-Étienne, Service de Pharmacologie Clinique, F-42023 Saint-Étienne, France
| | - Paul Bonjean
- Département d'Information Médical, CH Roanne, F-42328 Roanne, France
| | - Clara Locher
- Univ. Rennes, CHU Rennes, INSERM, Centre d'Investigation Clinique de Rennes (CIC1414), Service de Pharmacologie Clinique, Institut de Recherche en Santé, Environnement et Travail (Irset), UMR S 1085, EHESP, 35000 Rennes, France
| | - Paul Jacques Zufferey
- CHU Saint-Étienne, Département d'Anesthésie et Réanimation, Service de Pharmacologie Clinique, Univ. Jean Monnet, Mines Saint- Étienne, INSERM, U1059, SAINBIOSE, F-42055 Saint-Étienne, France
| | - Michel Cucherat
- UMR CNRS 5558, Laboratoire de Biométrie et Biologie Évolutive-Evaluation et Modélisation des Effets Thérapeutiques, Université Claude Bernard, Lyon 1, F-69376 Lyon, France
| | - Silvy Laporte
- Univ. Jean Monnet, Mines Saint- Étienne, INSERM, U1059, SAINBIOSE, CHU Saint-Étienne, Service de Pharmacologie Clinique, F-42023 Saint-Étienne, France
| |
Collapse
|
2
|
Li L, Asemota I, Liu B, Gomez-Valencia J, Lin L, Arif AW, Siddiqi TJ, Usman MS. AMSTAR 2 appraisal of systematic reviews and meta-analyses in the field of heart failure from high-impact journals. Syst Rev 2022; 11:147. [PMID: 35871099 PMCID: PMC9308914 DOI: 10.1186/s13643-022-02029-9] [Citation(s) in RCA: 16] [Impact Index Per Article: 8.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 10/20/2021] [Accepted: 07/14/2022] [Indexed: 11/19/2022] Open
Abstract
BACKGROUND The Measurement Tool to Assess systematic Reviews (AMSTAR) 2 is a critical appraisal tool for systematic reviews (SRs) and meta-analyses (MAs) of interventions. We aimed to perform the first AMSTAR 2-based quality assessment of heart failure-related studies. METHODS Eleven high-impact journals were searched from 2009 to 2019. The included studies were assessed on the basis of 16 domains. Seven domains were deemed critical for high-quality studies. On the basis of the performance in these 16 domains with different weights, overall ratings were generated, and the quality was determined to be "high," "moderate," "low," or "critically low." RESULTS Eighty-one heart failure-related SRs with MAs were included. Overall, 79 studies were of "critically low quality" and two were of "low quality." These findings were attributed to insufficiency in the following critical domains: a priori protocols (compliance rate, 5%), complete list of exclusions with justification (5%), risk of bias assessment (69%), meta-analysis methodology (78%), and investigation of publication bias (60%). CONCLUSIONS The low ratings for these potential high-quality heart failure-related SRs and MAs challenge the discrimination capacity of AMSTAR 2. In addition to identifying certain areas of insufficiency, these findings indicate the need to justify or modify AMSTAR 2's rating rules.
Collapse
Affiliation(s)
- Lin Li
- Department of Medicine, Cook County Health, Chicago, IL, USA.
| | | | - Bolun Liu
- Department of Medicine, Cook County Health, Chicago, IL, USA
| | - Javier Gomez-Valencia
- Division of Cardiology, Cook County Health, Rush University Medical Center, Chicago, IL, USA
| | - Lifeng Lin
- Department of Statistics, Florida State University, Tallahassee, Florida, USA
| | | | - Tariq Jamal Siddiqi
- Department of Medicine, Dow University of Health Sciences, Karachi, Pakistan
| | | |
Collapse
|
3
|
Riera R, Pacheco RL, Bagattini ÂM, Martimbianco ALC. Efficacy and safety of therapeutic use of cannabis derivatives and their synthetic analogs: Overview of systematic reviews. Phytother Res 2021; 36:5-21. [PMID: 34841610 DOI: 10.1002/ptr.7263] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/24/2021] [Revised: 08/05/2021] [Accepted: 08/14/2021] [Indexed: 01/08/2023]
Abstract
The debate on the use of cannabinoids for therapeutic purposes is constantly on the rise. This overview aimed to map the evidence on the therapeutic effects of cannabis derivatives and their synthetic analogs. Systematic reviews (SRs) of randomized trials were identified through a comprehensive search in several databases, and their methodological quality were evaluated with AMSTAR-2. The results for main outcomes are presented, prioritizing those from updated and better quality SRs. Finally, 68 SRs, addressing 37 different health conditions, were included. The methodological quality was high for eight SRs. The evidence certainty (GRADE) for the effects of cannabinoids is not high for any of the outcomes identified. Evidence certainty was moderate for the following: (a) cannabidiol appears to be beneficial for quality of life but increases the risk of adverse events in ulcerative colitis; (b) cannabinoids in general appear to have no clinically important benefit for chronic non-oncologic pain, spasticity-related pain in multiple sclerosis, or for acute post-operative pain; (c) cannabinoids in general appear to have a benefit in reducing chemotherapy-related nausea and vomiting. For all other outcomes from remaining comparisons, the evidence certainty was low, very low, or not evaluated, which prevents recommendations for or against their routine use.
Collapse
Affiliation(s)
- Rachel Riera
- Health Technology Assessment Centre, Hospital Sírio-Libanês, São Paulo - SP, Brazil.,Escola Paulista de Medicina (EPM), Universidade Federal de São Paulo (Unifesp), São Paulo - SP, Brazil
| | - Rafael Leite Pacheco
- Health Technology Assessment Centre, Hospital Sírio-Libanês, São Paulo - SP, Brazil.,Escola Paulista de Medicina (EPM), Universidade Federal de São Paulo (Unifesp), São Paulo - SP, Brazil.,Centro de Pesquisa Médica, Centro Universitário São Camilo (CUSC), São Paulo - SP, Brazil
| | - Ângela Maria Bagattini
- Health Technology Assessment Centre, Hospital Sírio-Libanês, São Paulo - SP, Brazil.,Instituto de Medicina Tropical e Saúde Publica, Universidade Federal de Goiás (UFG), Goiânia - GO, Brazil
| | - Ana Luiza Cabrera Martimbianco
- Health Technology Assessment Centre, Hospital Sírio-Libanês, São Paulo - SP, Brazil.,Mestrado Profissional em Saúde e Meio Ambiente, Universidade Metropolitana de Santos (Unimes), Santos - SP, Brazil
| |
Collapse
|
4
|
Sigurdson MK, Khoury MJ, Ioannidis JPA. Redundant meta-analyses are common in genetic epidemiology. J Clin Epidemiol 2020; 127:40-48. [PMID: 32540390 DOI: 10.1016/j.jclinepi.2020.05.035] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2019] [Revised: 05/02/2020] [Accepted: 05/13/2020] [Indexed: 11/18/2022]
Abstract
OBJECTIVES The massive growth in the publication of meta-analyses may cause redundancy and wasted efforts. We performed a metaepidemiologic study to evaluate the extent of potential redundancy in published meta-analyses in genetic epidemiology. STUDY DESIGN AND SETTING Using a sample of 38 index meta-analyses of genetic associations published in 2010, we retrieved additional meta-analyses that evaluated identical associations (same genetic variant and phenotype) using the Human Genome Epidemiology (HuGE) Navigator and PubMed databases. We analyzed the frequency of potential duplication and examined whether subsequent meta-analyses cited previous meta-analyses on the exact same association. RESULTS Based on 38 index meta-analyses, we retrieved a total of 99 duplicate meta-analyses. Only 12 (32%) of the index meta-analyses were unambiguously unique. We found a mean of 2.6 duplicates and a median of 2 duplicates per meta-analysis. In case studies, only 29-54% of previously published meta-analyses were cited by subsequent ones. CONCLUSION These results suggest that duplication is common in meta-analyses of genetic associations.
Collapse
Affiliation(s)
- Matthew K Sigurdson
- Department of Health Research and Policy, Stanford University School of Medicine, Stanford University, Stanford, CA, USA; Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA.
| | - Muin J Khoury
- Office of Public Health Genomics, Centers for Disease Control and Prevention (CDC), Atlanta, GA, USA
| | - John P A Ioannidis
- Department of Health Research and Policy, Stanford University School of Medicine, Stanford University, Stanford, CA, USA; Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA; Department of Statistics, Stanford University School of Humanities and Science, Stanford University, Stanford, CA, USA; Department of Biomedical Data Science, Stanford University School of Medicine, Stanford University, Stanford, CA, USA; Stanford Prevention Research Center, Department of Medicine, Stanford University, Stanford, CA, USA
| |
Collapse
|
5
|
Chen L, Jiang CY. Outcome differences between carotid artery stenting (CAS) and carotid endarterectomy (CEA) in postoperative ventricular arrhythmia, neurological complications, and in-hospital mortality. Postgrad Med 2020; 132:756-763. [PMID: 32396028 DOI: 10.1080/00325481.2020.1768765] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/10/2023]
Abstract
Objective: We study whether the carotid artery stenting (CAS) and carotid endarterectomy (CEA) differ from each other in postoperative ventricular arrhythmia, along with neurological complications (perioperative stroke and transient ischemic attack), in-hospital mortality, and estimated medical cost. Methods: This study used data of patients with carotid artery stenosis from the National Inpatient Sample (NIS) database (2011-2014) from the United States of America. Based on the procedure that patients received, individuals were categorized into groups of CAS and CEA. Multilevel analyses were conducted to examine the difference in the following outcomes: postoperative ventricular arrhythmia, neurological complications, in-hospital mortality, and medical costs between CAS and CEA. The patient age, gender, race, Charlson Comorbidity Index, primary payer, emergency department service record, bed size of hospital, region of the hospital, and location of the hospital were adjusted in each model. In addition, preexisting cardiovascular diseases (CVDs) were adjusted for when predicting postoperative ventricular arrhythmia; postoperative CVDs were adjusted for in the model of in-hospital mortality. Results: A total of 127,321 carotid artery stenosis hospitalizations were included in our analyses (n = 17,074 in CAS, n = 110,247 in CEA). Multivariate logistic regressions showed that compared with patients underwent CAS, those with CEA had a lower odds of postoperative ventricular arrhythmia (odds ratio [OR] = 0.81, 95% confidence interval [CI]: [0.66-0.98]), less neurological complications (OR = 0.55, 95% CI: [0.51-0.59] in general; OR = 0.63, 95% CI: [0.57-0.69] in ischemic stroke; OR = 0.26, 95% CI: [0.20-0.32] in hemorrhagic stroke; and OR = 0.58, 95% CI: [0.47-0.71] in transient ischemic attack), and in-hospital mortality (OR = 0.52, 95% CI: [0.42-0.64]). Generalized linear model indicated patients undergoing CEA had lower medical cost (β = -4329.99, 95% CI: [-4552.61, -4107.38]) than patients undergoing CAS. Conclusions: In short-term outcomes, CEA was associated with a lower risk of postoperative ventricular arrhythmia, neurological complications, in-hospital mortality, and lower cost as compared with CAS.
Collapse
Affiliation(s)
- LaiTe Chen
- School of Medicine, Zhejiang University , Hangzhou, Zhejiang Province, China
| | - Chen-Yang Jiang
- Sir Run Run Shaw Hospital, School of Medicine, Zhejiang University , Hangzhou, China
| |
Collapse
|
6
|
Storman M, Storman D, Jasinska KW, Swierz MJ, Bala MM. The quality of systematic reviews/meta-analyses published in the field of bariatrics: A cross-sectional systematic survey using AMSTAR 2 and ROBIS. Obes Rev 2020; 21:e12994. [PMID: 31997545 DOI: 10.1111/obr.12994] [Citation(s) in RCA: 34] [Impact Index Per Article: 8.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 09/27/2019] [Revised: 11/24/2019] [Accepted: 12/09/2019] [Indexed: 12/17/2022]
Abstract
High-quality systematic reviews (SR) and meta-analyses (MA) are considered to be reliable sources of information. This study aims to assess the quality of studies published as SR or MA in the field of bariatrics in 2016 and 2017. We identified SR and MA in the field of bariatrics by searching electronic databases (MEDLINE, Embase, and Cochrane Database of Systematic Reviews). Eligible studies were those identified as SR/MA in the title/abstract, which aimed to assess any outcome in patients with morbid obesity undergoing or scheduled to undergo bariatric surgery. Two authors independently reviewed all titles and abstracts, assessed full texts of potentially eligible studies, and assessed the quality of included studies. Any discrepancies were resolved by the third reviewer. We evaluated the quality and risk of bias of each SR/MA using AMSTAR 2 checklist and ROBIS tool, respectively. Seventy-eight of 4236 references met inclusion criteria and were assessed for their quality/risk of bias. The methodological quality of 99% of all papers was classified as "critically low." A total of 6% of the studies were at low risk of bias, and 78% were assessed as being at high risk of bias. The methodological quality of studies published in 2016 and 2017 as SR/MA is highly unsatisfactory.
Collapse
Affiliation(s)
- Monika Storman
- Systematic Reviews Unit, Jagiellonian University Medical College, Krakow, Poland
| | - Dawid Storman
- Department of Hygiene and Dietetics, Jagiellonian University Medical College, Krakow, Poland
| | - Katarzyna W Jasinska
- Students' Research Group, Systematic Reviews Unit, Jagiellonian University Medical College, Krakow, Poland
| | - Mateusz J Swierz
- Department of Hygiene and Dietetics, Jagiellonian University Medical College, Krakow, Poland
| | - Malgorzata M Bala
- Systematic Reviews Unit, Jagiellonian University Medical College, Krakow, Poland.,Department of Hygiene and Dietetics, Jagiellonian University Medical College, Krakow, Poland.,Chair of Epidemiology and Preventive Medicine, Jagiellonian University Medical College, Krakow, Poland
| |
Collapse
|