Redundant meta-analyses are common in genetic epidemiology

A framework to characterise the reproducibility of meta-analysis results with its application to direct oral anticoagulants in the acute treatment of venous thromboembolism

The number of meta-analyses of aggregate data has dramatically increased due to the facility of obtaining data from publications and the development of free, easy-to-use, and specialised statistical software. Even when meta-analyses include the same studies, their results may vary owing to different methodological choices. Assessment of the replication of meta-analysis provides an example of the variation of effect 'naturally' observed between multiple research projects. Reproducibility of results has mostly been reported using graphical descriptive representations. A quantitative analysis of such results would enable (i) breakdown of the total observed variability with quantification of the variability generated by the replication process and (ii) identification of which variables account for this variability, such as methodological quality or the statistical analysis procedures used. These variables might explain systematic mean differences between results and dispersion of the results. To quantitatively characterise the reproducibility of meta-analysis results, a bivariate linear mixed-effects model was developed to simulate both mean results and their corresponding uncertainty. Results were assigned to several replication groups, those assessing the same studies, outcomes, treatment indication and comparisons classified in the same replication group. A nested random effect structure was used to break down the total variability within each replication group and between these groups to enable calculation of an intragroup correlation coefficient and quantification of reproducibility. Determinants of variability were investigated by modelling both mean and variance parameters using covariates. The proposed model was applied to the example of meta-analyses evaluating direct oral anticoagulants in the acute treatment of venous thromboembolism.

Replication of systematic reviews: is it to the benefit or detriment of methodological quality?

OBJECTIVES

To perform an overview of the overlap of systematic reviews (SRs) assessing direct oral anticoagulants and characterize these reviews in terms of bias and methodological quality (PROSPERO: CRD42022316273).

STUDY DESIGN AND SETTING

A PubMed-indexed search was performed from inception to January 31, 2022 to identify SRs evaluating direct oral anticoagulants in patients treated for an acute venous thromboembolism. The risk of bias of these SRs was assessed according to the Risk Of Bias In Systematic reviews tool. Redundancy was defined as overlap in terms of the type of population considered, the interventions compared, and the studies included.

RESULTS

A total of 144 SRs were evaluated, of which 26 (18.1%) were classified as original, 87 (60.4%) as conceptual replications, and 31 (21.5%) as excessive replications. The risk of bias was high in 19 (73.1%) of the original SRs, 65 (74.7%) of the conceptual replications, and 21 (67.7%) of the excessive replications. Compared to the original SRs, the overall methodological quality was not improved in either conceptual or excessive replications.

CONCLUSION

A large number of SRs was classified as replications; a fifth constituted excessive replications. The replications showed no improvement in overall methodological quality compared to the original SRs.

Justification of research using systematic reviews continues to be inconsistent in clinical health science-A systematic review and meta-analysis of meta-research studies

BACKGROUND

Redundancy is an unethical, unscientific, and costly challenge in clinical health research. There is a high risk of redundancy when existing evidence is not used to justify the research question when a new study is initiated. Therefore, the aim of this study was to synthesize meta-research studies evaluating if and how authors of clinical health research studies use systematic reviews when initiating a new study.

METHODS

Seven electronic bibliographic databases were searched (final search June 2021). Meta-research studies assessing the use of systematic reviews when justifying new clinical health studies were included. Screening and data extraction were performed by two reviewers independently. The primary outcome was defined as the percentage of original studies within the included meta-research studies using systematic reviews of previous studies to justify a new study. Results were synthesized narratively and quantitatively using a random-effects meta-analysis. The protocol has been registered in Open Science Framework (https://osf.io/nw7ch/).

RESULTS

Twenty-one meta-research studies were included, representing 3,621 original studies or protocols. Nineteen of the 21 studies were included in the meta-analysis. The included studies represented different disciplines and exhibited wide variability both in how the use of previous systematic reviews was assessed, and in how this was reported. The use of systematic reviews to justify new studies varied from 16% to 87%. The mean percentage of original studies using systematic reviews to justify their study was 42% (95% CI: 36% to 48%).

CONCLUSION

Justification of new studies in clinical health research using systematic reviews is highly variable, and fewer than half of new clinical studies in health science were justified using a systematic review. Research redundancy is a challenge for clinical health researchers, as well as for funders, ethics committees, and journals.

Conducting umbrella reviews

In this article, Lazaros Belbasis and colleagues explain the rationale for umbrella reviews and the key steps involved in conducting an umbrella review, using a working example.