Overlapping meta-analyses on the same topic: survey of published studies

Five aspects of research waste in biomedicine: A scoping review

BACKGROUND

The number of published journal articles has grown exponentially during the last 30 years, which may have led to some wasteful research. However, the terminology associated with research waste remains unclear. To address this, we aimed to identify, define, and categorize the aspects of research waste in published biomedical reports.

METHODS

In this scoping review, we systematically searched for biomedical literature reports from 1993 to 2023 in two databases, focusing on those addressing and defining research waste. Through data charting, we analyzed and categorized the aspects of research waste.

RESULTS

Based on 4285 initial records in the searches, a total of 832 reports were included in the analysis. The included reports were primarily narrative reviews (26%) and original reports (21%). We categorized research waste into five aspects: methodological, invisible, negligible, underreported, and structural (MINUS) research waste. More than half of the reports (56%) covered methodological research waste concerning flaws in study design, study conduct, or analysis. Invisible research waste covered nonpublication, discontinuation, and lack of data-sharing. Negligible research waste primarily concerned unnecessary repetition, for example, stemming from the absence of preceding a trial with a systematic review of the literature. Underreported research waste mainly included poor reporting, resulting in a lack of transparency. Structural research waste comprised inadequate management, collaboration, prioritization, implementation, and dissemination.

CONCLUSION

MINUS encapsulates the five main aspects of research waste. Recognizing these aspects of research waste is important for addressing and preventing further research waste and thereby ensuring efficient resource allocation and scientific integrity.

Causes and Risk Factors of Breast Cancer, What Do We Know for Sure? An Evidence Synthesis of Systematic Reviews and Meta-Analyses

Breast cancer affected more than 2.3 million women in 2022 and is the most diagnosed cancer among women worldwide. The incidence rates are greater in developed regions and are significantly higher among women with higher education and socioeconomic status. Therefore, it is reasonable to assume that the way women live their lives may impact their risk of being diagnosed with breast cancer. This systematic review aimed to identify what is known about the causes and risk factors of breast cancer, excluding genetic causes. A comprehensive systematic search identified 2387 systematic reviews, 122 were included and six overall themes identified. In our "top list" with the 36 most important findings, a study of breast density had the highest effect size for increasing the risk of breast cancer, and a high sex-hormone-binding globulin level was the most protective factor. Many of the included studies investigating the same topics had conflicting results. The conclusion from this evidence synthesis reveals a lack of consensus of factors associated with the causes and risk of breast cancer. These findings suggest that recommendations about lifestyle and breast cancer should be made with caution.

Prevalence of and factors associated with potentially redundant randomized controlled trials: a cross-sectional study

OBJECTIVES

To investigate the prevalence of meta-analyses containing potentially redundant randomized controlled trials (RCTs) and the factors associated with the presence of redundancy.

STUDY DESIGN AND SETTING

This is a cross-sectional study, based on a random sample of references (n = 4500) that were published during 2020 and 2021, indexed in PubMed, Embase, or the Cochrane Database of Systematic Reviews, and retrieved through comprehensive searches using terms about systematic reviews and meta-analysis. From each systematic review, one meta-analysis fulfilling all the following criteria, if available, was included in this study: (1) assessing the effect of the intervention on a primary outcome of the systematic review; (2) combining RCTs only. The primary outcome was prevalence of meta-analyses containing potentially redundant RCTs. Potentially redundant RCTs referred to the trials that started 1 year after the overall effect estimate from cumulative meta-analysis had been statistically robust, as determined by trial sequential analysis when appropriate. The number of potentially redundant trials (if any) in each eligible meta-analysis and the number of participants involved in those trials were documented and contrasted across groups. Logistic regression analysis was conducted to explore the factors associated with presence of potential redundancy.

RESULTS

Of the 448 eligible meta-analyses, 57 (12.7%, 95% confidence interval (CI) 9.8-16.2%) contained potentially redundant RCTs. When limited to the 333 low-heterogeneity meta-analyses, the prevalence was 17.1% (95% CI 13.5-21.5%). The total number of potentially redundant RCTs was 295 (involving 85,385 participants), accounting for 38.5% of the RCTs (and 30.3% of the participants) included in the 57 meta-analyses. In these meta-analyses, the median number of potentially redundant RCTs and the participants involved were 2 (range: 1-50) and 352 (range: 17-26997), respectively. Potentially redundant RCTs were more likely to be present in the meta-analyses evaluating pharmaceutical intervention (odds ratio [OR] 2.31, 95% CI 1.16-4.49), assessing efficacy outcomes (OR 7.25, 95% CI 0.85-61.87), containing more than 5 RCTs (OR 6.47, 95% CI 3.22-12.99), or with the earliest RCT reporting statistically significant effect estimate (OR 5.30, 95% CI 2.64-10.64).

CONCLUSION

This study found that 12.7% to 17.1% of recently published meta-analyses contained potentially redundant RCTs, highlighting the importance of conducting or examining systematic reviews of existing evidence to justify new RCTs. More importantly, the study identified some scenarios in which redundancy was more likely to occur and thus has implications for trialists, funding agencies, ethics committees, and journal editors.

Determining the conclusiveness of a meta-analysis

The pursuit of conclusive evidence related to an unanswered foreground (decision-making) question has been the driving factor behind multiple ongoing and planned randomized controlled trials as well as meta-analyses. However, a fundamental challenge lies in establishing robust methods for ascertaining whether a collection of synthesized trials has yielded a definitive answer to that foreground question through the process of meta-analysis. This article explores the evolution of methods that attempt to address this challenge. These methods have primarily focused on defining and measuring the sufficiency and stability of evidence within a meta-analytic context. Cumulative meta-analysis and trial sequential analysis are the tools currently used, but they both come with limitations and challenges. We further discuss methods aimed at evaluating the evolution of effects over time more directly, such as the recursive cumulative meta-analysis. The latter method can be considered a better alternative, as it serves to demonstrate whether there is a true underlying treatment effect to which the meta-analysis is converging. However, recursive cumulative meta-analysis falls short of a specific indicator that establishes whether convergence has been reached. We coin the term exit for a meta-analysis where convergence can be demonstrated. Developing methods to determine the exit status of a meta-analysis is the next priority in research synthesis methods, as it will indicate that the research journey has concluded on a particular foreground question with no expectation of a different result with the addition of future trials.

Biomarkers for Psychosis: Are We There Yet? Umbrella Review of 1478 Biomarkers

Background and Hypothesis

This umbrella review aims to comprehensively synthesize the evidence of association between peripheral, electrophysiological, neuroimaging, neuropathological, and other biomarkers and diagnosis of psychotic disorders.

Study Design

We selected systematic reviews and meta-analyses of observational studies on diagnostic biomarkers for psychotic disorders, published until February 1, 2018. Data extraction was conducted according to the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines. Evidence of association between biomarkers and psychotic disorders was classified as convincing, highly suggestive, suggestive, weak, or non-significant, using a standardized classification. Quality analyses used the Assessment of Multiple Systematic Reviews (AMSTAR) tool.

Study Results

The umbrella review included 110 meta-analyses or systematic reviews corresponding to 3892 individual studies, 1478 biomarkers, and 392 210 participants. No factor showed a convincing level of evidence. Highly suggestive evidence was observed for transglutaminase autoantibodies levels (odds ratio [OR] = 7.32; 95% CI: 3.36, 15.94), mismatch negativity in auditory event-related potentials (standardized mean difference [SMD] = 0.73; 95% CI: 0.5, 0.96), P300 component latency (SMD = -0.6; 95% CI: -0.83, -0.38), ventricle-brain ratio (SMD = 0.61; 95% CI: 0.5, 0.71), and minor physical anomalies (SMD = 0.99; 95% CI: 0.64, 1.34). Suggestive evidence was observed for folate, malondialdehyde, brain-derived neurotrophic factor, homocysteine, P50 sensory gating (P50 S2/S1 ratio), frontal N-acetyl-aspartate, and high-frequency heart rate variability. Among the remaining biomarkers, weak evidence was found for 626 and a non-significant association for 833 factors.

Conclusions

While several biomarkers present highly suggestive or suggestive evidence of association with psychotic disorders, methodological biases, and underpowered studies call for future higher-quality research.

An umbrella review on the association between factors of oral health and cognitive dysfunction

An increasing number of systematic reviews and meta-analyses have been published on the association between oral health and cognitive dysfunction, also known as oral-cognitive links. However, there is great diversity in the oral and cognitive factors included in these studies, with different opinions for clinical practice drawn from the evidence. To understand which oral and cognitive factors are involved in those associations, we conducted an umbrella review of 28 systematic reviews, including 12 meta-analyses, on oral-cognitive links. We found that (a) periodontal diseases, oral microbiome, and dementia were frequently studied, while other factors, such as mastication and mild cognitive impairment, were less commonly investigated, and (b) severe deterioration of oral health, such as severe periodontitis or extensive tooth loss, rather than the presence of oral diseases alone, was strongly associated with cognitive dysfunction. In conclusion, the diversity of oral and cognitive factors included in the review studies reflects the complexity of oral-cognitive links. Clarifying the factors helps to form evidence-based clinical advice for healthcare.

Perspectives on systematic review protocol registration: a survey amongst stakeholders in the clinical research publication process

BACKGROUND

As systematic reviews (SRs) inform healthcare decisions, it is key that they address relevant questions and use rigorous methodology. Registration of SR protocols helps researchers identify relevant topics for future reviews and aims to prevent bias and duplication of effort. However, most SRs protocols are currently not registered, despite its significance. To guide future recommendations to enhance preregistration of SRs, it is important to gain a comprehensive understanding of the perspectives within the research community. Therefore, this study aims to examine the experiences with and factors of influence (barriers and facilitators) on prospective SR registration amongst researchers, peer reviewers and journal editors.

METHODS

Two different surveys were distributed to two groups: researchers and journal editors both identified from an existing sample of SRs. Researchers who indicated to have peer reviewed a SR were surveyed on their perspectives as peer reviewers as well. Survey design and analysis were informed by the Consolidated Framework for Implementation Research (CFIR). Shared and unique subthemes from the perspectives of researchers, peer reviewers and journal editors were identified and linked to the SR registration process (Innovation), to team, organisation (Inner setting) and (inter)national research community (Outer setting), and to characteristics of researchers, peer reviewers or journal editors (Individuals).

RESULTS

The survey's response rates were 65/727 (9%) for researchers, of which 37 were peer reviewers, and 22/308 (7%) for journal editors. Most respondents (n = 76, 94%) were familiar with SR protocol registration and 81% of researchers had registered minimally one SR protocol. Shared SR registration process subthemes were the importance and advantages of SR protocol registration, as well as barriers such as a high administrative burden. Shared subthemes regarding the inner and outer setting centred on journal processes, external standards and time. Shared individual factors were knowledge, skills and awareness.

CONCLUSIONS

The majority of the respondents were familiar with SR protocol registration and had a positive attitude towards it. This study identified suboptimal registration process, administrative burden and lack of mandatory SR protocol registration as barriers. By overcoming these barriers, SR protocol registration could contribute more effectively to the goals of open science.

SYSTEMATIC REVIEW REGISTRATION

osf.io/gmv6z.

Effect of PRISMA 2009 on reporting quality in systematic reviews and meta-analyses in high-impact dental medicine journals between 1993-2018

INTRODUCTION

The PRISMA guidelines were published in 2009 to address inadequate reporting of key methodological details in systematic reviews and meta-analyses (SRs/MAs). This study sought to assess the impact of PRISMA on the quality of reporting in the full text of dental medicine journals.

METHODS

This study assessed the impact of PRISMA (2009) on thirteen methodological details in SRs/MAs published in the highest-impact dental medicine journals between 1993-2009 (n = 211) and 2012-2018 (n = 618). The study further examined the rate of described use of PRISMA in the abstract or full text of included studies published post- PRISMA and the impact of described use of PRISMA on level of reporting. This study also examined potential effects of inclusion of PRISMA in Instructions for Authors, along with study team characteristics.

RESULTS

The number of items reported in SRs/MAs increased following the publication of PRISMA (pre-PRISMA: M = 7.83, SD = 3.267; post-PRISMA: M = 10.55, SD = 1.4). Post-PRISMA, authors rarely mention PRISMA in abstracts (8.9%) and describe the use of PRISMA in the full text in 59.87% of SRs/MAs. The described use of PRISMA within the full text indicates that its intent (guidance for reporting) is not well understood, with over a third of SRs/MAs (35.6%) describing PRISMA as guiding the conduct of the review. However, any described use of PRISMA was associated with improved reporting. Among author team characteristics examined, only author team size had a positive relationship with improved reporting.

CONCLUSION

Following the 2009 publication of PRISMA, the level of reporting of key methodological details improved for systematic reviews/meta-analyses published in the highest-impact dental medicine journals. The positive relationship between reference to PRISMA in the full text and level of reporting provides further evidence of the impact of PRISMA on improving transparent reporting in dental medicine SRs/MAs.

Placebo effects in mental health disorders: protocol for an umbrella review

INTRODUCTION

Given the high prevalence of mental health disorders and their significant socioeconomic burden, there is a need to develop improved treatments, and to evaluate them through placebo-controlled trials. However, the magnitude of the placebo response in randomised controlled trials to test medications may be substantial, affecting their interpretation. Therefore, improved understanding of the patient, trial and mental disorder factors that influence placebo responses would inform clinical trial design to better detect active treatment effects. There is a growing literature exploring the placebo response within specific mental health disorders, but no overarching synthesis of this research has been produced to date. We present a protocol for an umbrella review of systematic reviews and/or meta-analyses in which we aim to understand the effect size and potential predictors of placebo response within, and across, mental health disorders.

METHODS AND ANALYSIS

We will systematically search databases (Medline, PsycINFO, EMBASE+EMBASE Classic, Web of Knowledge) for systematic reviews and/or meta-analyses that report placebo effect size in clinical trials in patients with mental health disorders (initial search date 23 October 2022). Screening of abstracts and full texts will be done in pairs. We will extract data to qualitatively examine how placebo effect size varies across mental health disorders. We also plan to qualitatively summarise predictors of increased placebo response identified either quantitatively (eg, through meta-regression) or qualitatively. Risk of bias will be assessed using the AMSTAR-2 tool. We aim to not only summarise the current literature but also to identify gaps in knowledge and generate further hypotheses.

ETHICS AND DISSEMINATION

We do not believe there are any specific ethical considerations relevant to this study. We will publish the results in a peer-reviewed journal.

Diversity when interpreting evidence in network meta-analyses (NMAs) on similar topics: an example case of NMAs on diabetic macular oedema

BACKGROUND

Different network meta-analyses (NMAs) on the same topic result in differences in findings. In this review, we investigated NMAs comparing aflibercept with ranibizumab for diabetic macular oedema (DME) in the hope of illuminating why the differences in findings occurred.

METHODS

Studies were searched for in English and Chinese electronic databases (PubMed, Embase, Cochrane Library, Web of Science, CNKI, Wanfang, VIP; see detailed search strategy in the main body). Two independent reviewers systematically screened to identify target NMAs that included a comparison of aflibercept and ranibizumab in patients with DME. The key outcome of interest in this review is the change in best-corrected visual acuity (BCVA), including various ways of reporting (such as the proportion of participants who gain ≥ 10 ETDRS letters at 12 months; average change in BCVA at 12 months).

RESULTS

For the binary outcome of BCVA, different NMAs all agreed that there is no clear difference between the two treatments, while continuous outcomes all favour aflibercept over ranibizumab. We discussed four points of particular concern that are illustrated by five similar NMAs, including network differences, PICO (participants, interventions, comparators, outcomes) differences, different data from the same measures of effect, and differences in what is truly significant.

CONCLUSIONS

A closer inspection of each of these trials shows how the methods, including the searches and analyses, all differ, but the findings, although presented differently and sometimes interpreted differently, were similar.

Replication of systematic reviews: is it to the benefit or detriment of methodological quality?

OBJECTIVES

To perform an overview of the overlap of systematic reviews (SRs) assessing direct oral anticoagulants and characterize these reviews in terms of bias and methodological quality (PROSPERO: CRD42022316273).

STUDY DESIGN AND SETTING

A PubMed-indexed search was performed from inception to January 31, 2022 to identify SRs evaluating direct oral anticoagulants in patients treated for an acute venous thromboembolism. The risk of bias of these SRs was assessed according to the Risk Of Bias In Systematic reviews tool. Redundancy was defined as overlap in terms of the type of population considered, the interventions compared, and the studies included.

RESULTS

A total of 144 SRs were evaluated, of which 26 (18.1%) were classified as original, 87 (60.4%) as conceptual replications, and 31 (21.5%) as excessive replications. The risk of bias was high in 19 (73.1%) of the original SRs, 65 (74.7%) of the conceptual replications, and 21 (67.7%) of the excessive replications. Compared to the original SRs, the overall methodological quality was not improved in either conceptual or excessive replications.

CONCLUSION

A large number of SRs was classified as replications; a fifth constituted excessive replications. The replications showed no improvement in overall methodological quality compared to the original SRs.

Factors associated with successful publication for systematic review protocol registration: an analysis of 397 registered protocols

BACKGROUND

Meta-analyses are on top of the evidence-based medicine pyramid, yet many of them are not completed after they are begun. Many factors impacting the publication of meta-analysis works have been discussed, and their association with publication likelihood has been investigated. These factors include the type of systematic review, journal metrics, h-index of the corresponding author, country of the corresponding author, funding sources, and duration of publication. In our current review, we aim to investigate these various factors and their impact on the likelihood of publication. A comprehensive review of 397 registered protocols retrieved from five databases was performed to investigate the different factors that might affect the likelihood of publication. These factors include the type of systematic review, journal metrics, h-index of the corresponding author, country of the corresponding author, funding sources, and duration of publication.

RESULTS

We found that corresponding authors in developed countries and English-speaking countries had higher likelihoods of publication: 206/320 (p = 0.018) and 158/236 (p = 0.006), respectively. Factors affecting publications are the countries of corresponding author (p = 0.033), whether they are from developed countries (OR: 1.9, 95% CI: 1.2-3.1, p = 0.016), from English-speaking countries (OR: 1.8, 95% CI: 1.2-2.7, p = 0.005), update status of the protocol (OR: 1.6, 95% CI: 1.0-2.6, p = 0.033), and external funding (OR: 1.7, 95% CI: 1.1-2.7, p = 0.025). Multivariable regression retains three variables as significant predictors for the publication of a systematic review: whether it is the corresponding author from developed countries (p = 0.013), update status of the protocol (p = 0.014), and external funding (p = 0.047).

CONCLUSION

Being on top of the evidence hierarchy, systematic review and meta-analysis are the keys to informed clinical decision-making. Updating protocol status and external funding are significant influences on their publications. More attentions should be paid to the methodological quality of this type of publication.

Systematic online living evidence summaries: emerging tools to accelerate evidence synthesis

Systematic reviews and meta-analysis are the cornerstones of evidence-based decision making and priority setting. However, traditional systematic reviews are time and labour intensive, limiting their feasibility to comprehensively evaluate the latest evidence in research-intensive areas. Recent developments in automation, machine learning and systematic review technologies have enabled efficiency gains. Building upon these advances, we developed Systematic Online Living Evidence Summaries (SOLES) to accelerate evidence synthesis. In this approach, we integrate automated processes to continuously gather, synthesise and summarise all existing evidence from a research domain, and report the resulting current curated content as interrogatable databases via interactive web applications. SOLES can benefit various stakeholders by (i) providing a systematic overview of current evidence to identify knowledge gaps, (ii) providing an accelerated starting point for a more detailed systematic review, and (iii) facilitating collaboration and coordination in evidence synthesis.

Evidence-based health policy in Germany: lack of communication and coordination between academia and health authorities?

Health-care decision making should consider the best available evidence, often in the form of systematic reviews (SRs). The number of existing SRs and their overlap make their identification and use difficult. Decision makers often rely on de novo SRs instead of using existing SRs. We describe two cases of duplicate reviews (minimum volume threshold of total knee arthroplasties and lung cancer screening) and one case of duplicate primary data analysis (transcatheter aortic valve implantation). All cases have in common that unintended duplication of research occurred between health authorities and academia, demonstrating a lack of communication and coordination between them.It is important to note that academia and health authorities have different incentives. Academics are often measured by the number of peer-reviewed publications and grants awarded. In contrast, health authorities must comply with laws and are commissioned to deliver a specific report within a defined period of time. Most replication is currently unintended. A solution may be the collaboration of stakeholders commonly referred to as integrated knowledge translation (IKT). The IKT approach means that research is conducted in collaboration with the end users of the research. It requires active collaborations between researchers and decision-makers or knowledge users (clinicians, managers, policy makers) throughout the research process. Wherever cooperation is possible in spite of requirements for independence or confidentiality, legal regulations should facilitate and support collaborative approaches between academia and health authorities.

Cemented vs Uncemented hemiarthroplasties for femoral neck fractures: An overlapping systematic review and evidence appraisal

BACKGROUND

The purpose of our study is to assess the methodology of overlapping systematic reviews related to cemented vs uncemented hip hemiarthroplasties for the treatment of femoral neck fractures to find the study with the best evidence. Also, we assess the gaps in methodology and information to help with direction of future studies.

METHODS

A systematic search was conducted in September 2022 using Pubmed, Embase, and Cochrane Library. Clinical outcome data and characteristics of each study were extracted to see which treatment had better favorability. The outcomes and characteristics extracted from each study includes, first author, search date, publication journal and date, number of studies included, databases, level of evidence, software used, subgroup analyses that were conducted, and heterogeneity with the use of I2 statistics Methodological quality information was extracted from each study using four different methodologic scores (Oxford Levels of Evidence; Assessment of Multiple Systematic Reviews (AMSTAR); Quality of reporting of meta-analyses (QUROM); Oxman and Guyatt. After that, the Jadad decision algorithm was used to identify which studies in our sample contained the best available evidence. Finally, overlap of each systematic review was assessed using Corrected Covered Area (CCA) to look at redundancy and research waste among the systematic reviews published on the topic.

RESULTS

After screening, 12 studies were included in our sample. For the Oxford Levels of Evidence, we found that all the studies were Level I evidence. For the QUORUM assessment, we had 1 study with the highest score of 18. Additionally, we did the Oxman and Guyatt assessment, where we found 4 studies with a maximum score of 6. Finally, we did an AMSTAR assessment and found 2 studies with a score of 9. After conducting the methodological scores; the authors determined that Li. L et al 2021 had the highest quality. In addition, it was found that the CCA found among the primary studies in each systematic review calculated to .22. Any CCA above .15 is considered "very high overlap".

CONCLUSIONS

The best available evidence suggests that Cemented HAs are better at preventing Prosthesis-related complications. Conversely, the best evidence also suggests that Cemented HA also results in longer operative time and increased intraoperative blood loss. When conducting future systematic reviews related to the topic, we ask that authors restrict conducting another systematic review until new evidence emerges so as not to confuse the clinical decision-making of physicians.

From standard systematic reviews to living systematic reviews

Systematic reviews (SRs) have become a central tool for evidence-based health care over the last 30 years. The number of SRs being published has increased steadily. However, concerns have been raised regarding the duplication of work, methodological flaws and the currency of many systematic reviews, also in the context of the COVID-19 pandemic. Living systematic reviews (LSRs) offer a new approach to updating systematic reviews, particularly in high-priority research fields that face the challenge of dynamically evolving and sometimes uncertain evidence. Continual updates serve to ensure that LSRs remain current and methodologically rigorous. As a new element of the evidence ecosystem, LSRs can inform living guidelines and recommendations, user-adapted formats, decisions at the patient and system level as well as gaps in primary research.

Phosphodiesterase type 5 inhibitors and premature ejaculation: an overview of systematic reviews/meta-analyses using the AMSTAR 2, ROBIS, and GRADE tools

Introduction

The place of phosphodiesterase type 5 inhibitors (PDE5-Is) in treating premature ejaculation (PE) remains a therapeutic challenge.

Objectives

(1) To summarize the evidence of the efficacy and safety of PDE5-Is from published systematic reviews/meta-analyses (SRs/MAs). (2) To evaluate the reporting, methodological quality, and evidence quality of SRs/MAs concerning PE.

Methods

Nine databases were searched to retrieve SRs/MAs on using PDE5-Is for PE from inception to July 2022. Methodological quality and risk of bias were assessed with the AMSTAR 2 (A Measurement Tool to Assess Systematic Reviews) and ROBIS (Risk of Bias in Systematic Reviews). GRADE criteria (Grading of Recommendations, Assessment, Development, and Evaluations) were used to assess the evidence quality.

Results

The literature search revealed 15 relevant SRs/MAs covering 28 primary studies (9 pairwise MAs and 6 network MAs) rated as low or very low quality and high risk of bias except 1 review that was rated as moderate quality and low risk of bias. Among the 27 outcome measures related to efficacy and safety, the quality of evidence according to GRADE criteria was low in 4 and critically low in 23. Oral PDE5-Is have demonstrated a possible benefit over placebo in lifelong PE and mixed PE. The results of pairwise and network MAs advocated that the combined use of PDE5-Is and SSRIs is of possible benefit as compared with either SSRIs or PDE5-Is alone. The total adverse effects were more frequent with PDE5-Is than placebo.

Conclusion

PDE5-Is are of a possible benefit than placebo in lifelong PE and mixed PE. The results favor coadministration of PDE5-Is plus SSRIs over SSRIs alone or PDE5-I monotherapy. These conclusions should be interpreted cautiously due to the low methodological quality and low quality of evidence of most available reviews. Additional higher-quality randomized controlled trials, SRs, and MAs are warranted to provide a better estimate of any effect size.

Ocular Biomarkers for Alzheimer Disease Dementia: An Umbrella Review of Systematic Reviews and Meta-analyses

Importance

Several ocular biomarkers have been proposed for the early detection of Alzheimer disease (AD) and mild cognitive impairment (MCI), particularly fundus photography, optical coherence tomography (OCT), and OCT angiography (OCTA).

Objective

To perform an umbrella review of systematic reviews to assess the diagnostic accuracy of ocular biomarkers for early diagnosis of Alzheimer disease.

Data Sources

MEDLINE, Embase, and PsycINFO were searched from January 2000 to November 2021. The references of included reviews were also searched.

Study Selection

Systematic reviews investigating the diagnostic accuracy of ocular biomarkers to detect AD and MCI, in secondary care or memory clinics, against established clinical criteria or clinical judgment.

Data Extraction and Synthesis

The Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) reporting guideline checklist was followed and the Risk Of Bias in Systematic reviews tool was used to assess review quality.

Main Outcomes and Measures

The prespecified outcome was the accuracy of ocular biomarkers for diagnosing AD and MCI. The area under the curve (AUC) was derived from standardized mean difference.

Results

From the 591 titles, 14 systematic reviews were included (median [range] number of studies in each review, 14 [5-126]). Only 4 reviews were at low risk of bias on all Risk of Bias in Systematic Reviews domains. The imaging-derived parameters with the most evidence for detecting AD compared with healthy controls were OCT peripapillary retinal nerve fiber layer thickness (38 studies including 1883 patients with AD and 2510 controls; AUC = 0.70; 95% CI, 0.53-0.79); OCTA foveal avascular zone (5 studies including 177 patients with AD and 371 controls; AUC = 0.73; 95% CI, 0.50-0.89); and saccadic eye movements prosaccade latency (30 studies including 651 patients with AD/MCI and 771 controls; AUC = 0.64; 95% CI, 0.58-0.69). Antisaccade error was investigated in fewer studies (12 studies including 424 patients with AD/MCI and 382 controls) and yielded the best accuracy (AUC = 0.79; 95% CI, 0.70-0.88).

Conclusions and Relevance

This umbrella review has highlighted limitations in design and reporting of the existing research on ocular biomarkers for diagnosing AD. Parameters with the best evidence showed poor to moderate diagnostic accuracy in cross-sectional studies. Future longitudinal studies should investigate whether changes in OCT and OCTA measurements over time can yield accurate predictions of AD onset.

Duplicated network meta-analysis in advanced prostate cancer: a case study and recommendations for change

BACKGROUND

Research overlap and duplication is a recognised problem in the context of both pairwise and network systematic reviews and meta-analyses. As a case study, we carried out a scoping review to identify and examine duplicated network meta-analyses (NMAs) in a specific disease setting where several novel therapies have recently emerged: hormone-sensitive metastatic prostate cancer (mHSPC).

METHODS

MEDLINE and EMBASE were systematically searched, in January 2020, for indirect or mixed treatment comparisons or network meta-analyses of the systemic treatments docetaxel and abiraterone acetate in the mHSPC setting, with a time-to-event outcome reported on the hazard-ratio scale. Eligibility decisions were made, and data extraction performed, by two independent reviewers.

RESULTS

A total of 13 eligible reviews were identified, analysing between 3 and 8 randomised comparisons, and comprising between 1773 and 7844 individual patients. Although the included trials and treatments showed a high degree of overlap, we observed considerable variation between identified reviews in terms of review aims, eligibility criteria and included data, statistical methodology, reporting and inference. Furthermore, crucial methodological details and specific source data were often unclear.

CONCLUSIONS AND RECOMMENDATIONS

Variation across duplicated NMAs, together with reporting inadequacies, may compromise identification of best-performing treatments. Particularly in fast-moving fields, review authors should be aware of all relevant studies, and of other reviews with potential for overlap or duplication. We recommend that review protocols be published in advance, with greater clarity regarding the specific aims or scope of the project, and that reports include information on how the work builds upon existing knowledge. Source data and results should be clearly and completely presented to allow unbiased interpretation.

A multiverse analysis of meta-analyses assessing acupuncture efficacy for smoking cessation evidenced vibration of effects

OBJECTIVE

To explore the impact of methodological choices on the results of meta-analyses (MAs), with acupuncture for smoking cessation as a case study.

STUDY DESIGN AND SETTING

After performing an umbrella review (using MEDLINE, the COCHRANE Library, the Wan Fang database, and the Chinese Journal Full-text Database/March 2018) of MAs exploring the use of acupuncture for smoking cessation, we extracted all randomized controlled trials. Numerous MAs were performed as per every possible combination of various methodological choices (e.g., characteristics of the intervention and control procedures, outcome, publication status, language) to assess their vibration of effects or more precisely the existence of a Janus effect, that is, whether the 10th and 90th percentiles in the distribution of effect sizes were in opposite directions.

RESULTS

After including 7 MAs and 39 randomized controlled trials, we performed 496,528 MAs. The effect size was negative at the 10th percentile (-0.1, favoring controls) and positive at the 90th percentile (1.17, favoring acupuncture). In all, 104,491 MAs showed a statistically significant difference in favor of acupuncture, whereas 392,037 failed to demonstrate the efficacy of acupuncture (including 96 that showed a statistically significant difference in favor of the control).

CONCLUSION

The methodological choices made in performing pairwise MAs can result in substantial vibration of effects, occasionally leading to opposite results.

Commentary: collaborative systematic review may produce and share high-quality, comparative evidence more efficiently

Systematic reviews are necessary to synthesize available evidence and inform clinical practice and health policy decisions. There has been an explosion of evidence available in many fields; this makes it challenging to keep evidence syntheses up to date and useful. Comparative effectiveness systematic reviews are informative; however, producing these often-large reviews bring intense time and resource demands. This commentary describes the implementation of a systematic review using a collaborative model of evidence synthesis. We are implementing the collaborative review model to update a large Cochrane review investigating the efficacy and comparative effectiveness of the design, delivery, and type of exercise treatment for people with chronic low-back pain. Three key benefits of the collaborative review model for evidence synthesis are (1) team coordination and collaboration, (2) quality control measures, and (3) advanced comparative and other analyses. This new collaborative review model is developed and implemented to produce and share high-quality, comparative evidence more efficiently while building capacity and community within a research field.

Justification of research using systematic reviews continues to be inconsistent in clinical health science-A systematic review and meta-analysis of meta-research studies

BACKGROUND

Redundancy is an unethical, unscientific, and costly challenge in clinical health research. There is a high risk of redundancy when existing evidence is not used to justify the research question when a new study is initiated. Therefore, the aim of this study was to synthesize meta-research studies evaluating if and how authors of clinical health research studies use systematic reviews when initiating a new study.

METHODS

Seven electronic bibliographic databases were searched (final search June 2021). Meta-research studies assessing the use of systematic reviews when justifying new clinical health studies were included. Screening and data extraction were performed by two reviewers independently. The primary outcome was defined as the percentage of original studies within the included meta-research studies using systematic reviews of previous studies to justify a new study. Results were synthesized narratively and quantitatively using a random-effects meta-analysis. The protocol has been registered in Open Science Framework (https://osf.io/nw7ch/).

RESULTS

Twenty-one meta-research studies were included, representing 3,621 original studies or protocols. Nineteen of the 21 studies were included in the meta-analysis. The included studies represented different disciplines and exhibited wide variability both in how the use of previous systematic reviews was assessed, and in how this was reported. The use of systematic reviews to justify new studies varied from 16% to 87%. The mean percentage of original studies using systematic reviews to justify their study was 42% (95% CI: 36% to 48%).

CONCLUSION

Justification of new studies in clinical health research using systematic reviews is highly variable, and fewer than half of new clinical studies in health science were justified using a systematic review. Research redundancy is a challenge for clinical health researchers, as well as for funders, ethics committees, and journals.

Systematic evidence map of evidence addressing the top 10 priority research questions for hyperemesis gravidarum

OBJECTIVE

Knowledge gaps regarding hyperemesis gravidarum (HG) are substantial. We aimed to systematically identify and map recent evidence addressing the top 10 priority questions for HG, as published in 2021 in a James Lind Alliance Priority Setting Partnership.

DESIGN

Systematic evidence map.

METHODS

We searched MEDLINE and EMBASE on 12 January 2021 and CINAHL on 22 February 2021 with search terms hyperemesis gravidarum, pernicious vomiting in pregnancy and their synonyms. Results were limited to 2009 onwards. Two reviewers independently screened titles and abstracts to assess whether the studies addressed a top 10 priority questions for HG. Differences were discussed until consensus was reached. Publications were allocated to one or more top 10 research questions. Study design was noted, as was patient or public involvement. Two reviewers extracted data synchronously and both cross-checked 10%. Extracted data were imported into EPPI-Reviewer software to create an evidence map.

OUTCOME MEASURES

The number and design of studies in the search yield, displayed per the published 10 priority questions.

RESULTS

Searches returned 4338 results for screening; 406 publications were included in the evidence map. 136 publications addressed multiple questions. Numerous studies address the immediate and long-term outcomes or possible markers for HG (question 8 and 9, respectively 164 and 82 studies). Very few studies seek a possible cure for HG (question 1, 8 studies), preventative treatment (question 4, 2 studies) or how to achieve nutritional requirements of pregnancy (question 10, 17 studies). Case reports/series were most numerous with 125 (30.7%) included. Few qualitative studies (9, 2.2%) were identified. 25 (6.1%) systematic reviews addressed eight questions, or aspects of them. 31 (7.6%) studies included patient involvement.

CONCLUSIONS

There are significant gaps and overlap in the current HG literature addressing priority questions. Researchers and funders should direct their efforts at addressing the gaps in the top 10 questions.

The score after 10 years of registration of systematic review protocols

BACKGROUND

With the exponential growth of published systematic reviews (SR), there is a high potential for overlapping and redundant duplication of work. Prospective protocol registration gives the opportunity to assess the added value of a new study or review, thereby potentially reducing research waste and simultaneously increasing transparency and research quality. The PROSPERO database for SR protocol registration was launched 10 years ago. This study aims to assess the proportion SRs of intervention studies with a protocol registration (or publication) and explore associations of SR characteristics with protocol registration status.

METHODS

PubMed was searched for SRs of human intervention studies published in January 2020 and January 2021. After random-stratified sampling and eligibility screening, data extraction on publication and journal characteristics, and protocol registration status, was performed. Both descriptive and multivariable comparative statistical analyses were performed.

RESULTS

A total of 357 SRs (2020: n = 163; 2021: n = 194) were included from a random sample of 1267 publications. Of the published SRs, 38% had a protocol. SRs that reported using PRISMA as a reporting guideline had higher odds of having a protocol than publications that did not report PRISMA (OR 2.71; 95% CI: 1.21 to 6.09). SRs with a higher journal impact factor had higher odds of having a protocol (OR 1.12; 95% CI 1.04 to 1.25). Publications from Asia had a lower odds of having a protocol (OR 0.43; 95% CI 0.23 to 0.80, reference category = Europe). Of the 33 SRs published in journals that endorse PROSPERO, 45% did not have a protocol. Most SR protocols were registered in PROSPERO (n = 129; 96%).

CONCLUSIONS

We found that 38% of recently published SRs of interventions reported a registered or published protocol. Protocol registration was significantly associated with a higher impact factor of the journal publishing the SR and a more frequent self-reported use of the PRISMA guidelines. In some parts of the world, SR protocols are more often registered or published than others. To guide strategies to increase the uptake of SR protocol registration, further research is needed to gain understanding of the benefits and informativeness of SRs protocols among different stakeholders.

SYSTEMATIC REVIEW REGISTRATION

osf.io/9kj7r/.

Systematic reviews are rarely used to contextualise new results-a systematic review and meta-analysis of meta-research studies

BACKGROUND

Results of new studies should be interpreted in the context of what is already known to compare results and build the state of the science. This systematic review and meta-analysis aimed to identify and synthesise results from meta-research studies examining if original studies within health use systematic reviews to place their results in the context of earlier, similar studies.

METHODS

We searched MEDLINE (OVID), EMBASE (OVID), and the Cochrane Methodology Register for meta-research studies reporting the use of systematic reviews to place results of original clinical studies in the context of existing studies. The primary outcome was the percentage of original studies included in the meta-research studies using systematic reviews or meta-analyses placing new results in the context of existing studies. Two reviewers independently performed screening and data extraction. Data were synthesised using narrative synthesis and a random-effects meta-analysis was performed to estimate the mean proportion of original studies placing their results in the context of earlier studies. The protocol was registered in Open Science Framework.

RESULTS

We included 15 meta-research studies, representing 1724 original studies. The mean percentage of original studies within these meta-research studies placing their results in the context of existing studies was 30.7% (95% CI [23.8%, 37.6%], I2=87.4%). Only one of the meta-research studies integrated results in a meta-analysis, while four integrated their results within a systematic review; the remaining cited or referred to a systematic review. The results of this systematic review are characterised by a high degree of heterogeneity and should be interpreted cautiously.

CONCLUSION

Our systematic review demonstrates a low rate of and great variability in using systematic reviews to place new results in the context of existing studies. On average, one third of the original studies contextualised their results. Improvement is still needed in researchers' use of prior research systematically and transparently-also known as the use of an evidence-based research approach, to contribute to the accumulation of new evidence on which future studies should be based.

SYSTEMATIC REVIEW REGISTRATION

Open Science registration number https://osf.io/8gkzu/.

Critical Appraisal of a Systematic Review: A Concise Review

OBJECTIVES

Concise definitive review of how to read and critically appraise a systematic review.

DATA SOURCES

None.

STUDY SELECTION

Current literature describing the conduct, reporting, and appraisal of systematic reviews and meta-analyses.

DATA EXTRACTION

Best practices for conducting, reporting, and appraising systematic review were summarized.

DATA SYNTHESIS

A systematic review is a review of a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant original research, and to collect and analyze data from the studies that are included in the review. Critical appraisal methods address both the credibility (quality of conduct) and rate the confidence in the quality of summarized evidence from a systematic review. The A Measurement Tool to Assess Systematic Reviews-2 tool is a widely used practical tool to appraise the conduct of a systematic review. Confidence in estimates of effect is determined by assessing for risk of bias, inconsistency of results, imprecision, indirectness of evidence, and publication bias.

CONCLUSIONS

Systematic reviews are transparent and reproducible summaries of research and conclusions drawn from them are only as credible and reliable as their development process and the studies which form the systematic review. Applying evidence from a systematic review to patient care considers whether the results can be directly applied, whether all important outcomes have been considered, and if the benefits are worth potential harms and costs.

Retrieving Cochrane reviews is sometimes challenging and their reporting is not always optimal

Cochrane reviews are known to be a high quality source of evidence synthesis supporting health care decisions. In a recently conducted study, we analysed the trends in epidemiology and reporting of published systematic reviews over the last 20 years. This sample of 1,132 systematic reviews included 84 Cochrane reviews. We have learned several peculiarities of Cochrane reviews, that are worth being discussed in more detail due to their practical implications. Methodologists, clinicians and health care professionals should be aware of these limitations: 1) Cochrane reviews are not identified as systematic reviews in title, 2) Cochrane reviews do not always follow PRISMA reporting guidelines, 3) Some updates are only available via the Cochrane Library, and 4) Indexing of Cochrane Reviews in PubMed may be suboptimal. This article is protected by copyright. All rights reserved.

A new taxonomy was developed for overlap across 'overviews of systematic reviews': A meta-research study of research waste

Multiple 'overviews of reviews' conducted on the same topic ("overlapping overviews") represent a waste of research resources and can confuse clinicians making decisions amongst competing treatments. We aimed to assess the frequency and characteristics of overlapping overviews. MEDLINE, Epistemonikos and Cochrane Database of Systematic Reviews were searched for overviews that: synthesized reviews of health interventions and conducted systematic searches. Overlap was defined as: duplication of PICO eligibility criteria, and not reported as an update nor a replication. We categorized overview topics according to 22 WHO ICD-10 medical classifications, overviews as broad or narrow in scope, and overlap as identical, nearly identical, partial, or subsumed. Subsummation was defined as when broad overviews subsumed the populations, interventions and at least one outcome of another overview. Of 541 overviews included, 169 (31%) overlapped across similar PICO, fell within 13 WHO ICD-10 medical classifications, and 62 topics. 148/169 (88%) overlapping overviews were broad in scope. Fifteen overviews were classified as having nearly identical overlap (9%); 123 partial overlap (73%), and 31 subsumed (18%) others. One third of overviews overlapped in content and a majority covered broad topic areas. A multiplicity of overviews on the same topic adds to the ongoing waste of research resources, time, and effort across medical disciplines. Authors of overviews can use this study and the sample of overviews to identify gaps in the evidence for future analysis, and topics that are already studied, which do not need to be duplicated.

Efficacy and safety of Daoyin and massage for lumbar disc herniation: A protocol for overview of systematic reviews

BACKGROUND

Lumbar disc herniation (LDH) is a common disease, which can cause low back pain, sciatica, and even disability. The treatment of LDH is a global challenge. Conservative therapy with non-drugs is considered to be the first choice for patients with LDH. In recent years, an increasing number of systematic reviews and meta analyses on Daoyin and massage interventions in lumbar disc herniation have been implemented. However, the evidence quality and methodological quality of these systematic reviews/meta analyses are unknown and need to be systematically evaluated. This overview aims to systematically summarize and critically appraise the current evidence on Daoyin and massage for LDH.

METHODS

Eight electronic data will be retrieved, including China National Knowledge Infrastructure (CNKI), Wanfang database (WF), China Biomedical database (CBM), Chinese Scientific Journals Database (VIP), PubMed, Cochrane Library, Web of Science (WOS), and EMBASE from their inception to March 1, 2022. The reporting quality, methodological quality, risk of bias, quality of evidence will be assessed by using The Preferred Reporting Items for Systematic Reviews and Meta-analyses 2020 (PRISMA 2020), the Assessment of Multiple Systematic Reviews 2 (AMSTAR-2), the Risk of Bias in Systematic Review (ROBIS), and the Grading of Recommendations Assessment, Development, and Evaluation (GRADE). Two independent researchers conducted literature screening, data extraction, and quality evaluation process. In addition, we will establish an overlap matrix and calculate the corrected covered area to evaluate the impact of overlapping areas on conclusions.

RESULTS

The results will be published in a peer-reviewed journal.

CONCLUSION

This overview will provide comprehensive evidence of Daoyin and massage for treating lumbar disc herniation.

SYSTEMATIC REVIEW REGISTRATION

INPLASY202210019.

The role of diet in renal cell carcinoma incidence: an umbrella review of meta-analyses of observational studies

BACKGROUND

Evidence associating diet with the incidence of renal cell carcinoma (RCC) is inconclusive. We aimed to summarize evidence associating dietary factors with RCC incidence and assess the strength and validity of this evidence.

METHODS

We conducted an umbrella review of systematic reviews or meta-analyses (SRoMAs) that assessed the association between diet and RCC incidence. Through April 2021, PubMed, Web of Science, Embase, The Cochrane Library, Scopus, and WCRF were searched. Two independent reviewers selected studies, extracted data, and appraised the quality of SRoMAs. According to credibility assessment criteria, evidence can be divided into five categories: convincing (class I), highly suggestive (class II), suggestive (class III), weak (class IV), and nonsignificant (class V).

RESULTS

Twenty-nine meta-analyses were obtained after screening. After excluding 7 overlapping meta-analyses, 22 meta-analyses including 502 individual studies and 64 summary hazard ratios for RCC incidence were included: dietary patterns or dietary quality indices (n = 6), foods (n = 13), beverages (n = 4), alcohol (n = 7), macronutrients (n =15), and micronutrients (n =19). No meta-analyses had high methodological quality. Five meta-analyses exhibited small study effects; one meta-analysis showed evidence of excess significance bias. No dietary factors showed convincing or highly suggestive evidence of association with RCC in the overall analysis. Two protective factors had suggestive evidence (vegetables (0.74, 95% confidence interval 0.63 to 0.86) and vitamin C (0.77, 0.66 to 0.90)) in overall analysis. One protective factor had convincing evidence (moderate drinking (0.77, 0.70 to 0.84)) in Europe and North America and one protective factor had highly suggestive evidence (cruciferous vegetables (0.78, 0.70 to 0.86)) in North America.

CONCLUSIONS

Although many meta-analyses have assessed associations between dietary factors and RCC, no high-quality evidence exists (classes I and II) in the overall analysis. Increased intake of vegetables and vitamin C is negatively associated with RCC risk. Moderate drinking might be beneficial for Europeans and North Americans, and cruciferous vegetables might be beneficial to North Americans, but the results should be interpreted with caution. More researches are needed in the future.

TRIAL REGISTRATION

PROSPERO CRD42021246619.

Most published systematic reviews of remdesivir for COVID-19 were redundant and lacked currency

Objective

To investigate the completeness and currency of published systematic reviews of remdesivir for COVID-19 and to compare this with a living guidelines approach.

Study Design and Setting

In this cross-sectional study, we searched Europe PMC on May 20, 2021 for systematic reviews of remdesivir (including preprints, living review updates). Completeness and currency were based on the inclusion of four major randomized trials of remdesivir available at the time of publication of the review (including as preliminary results and preprints).

Results

We included 38 reviews (45 reports), equivalent to a new publication every 9 days. 23 (51%) reports were out of date at the time of publication. Eleven reviews that were current on publication had a median survival time of 10 days (range 4–57). A third of reviews cited other systematic reviews, but only four provided justifications for why another review was necessary. Eight (21%) of the reviews were registered in PROSPERO. The Australian COVID-19 Clinical Evidence Taskforce living guidelines were updated within 14 days for three of the remdesivir trials, and within 28 days for the fourth.

Conclusion

There was considerable duplication of systematic reviews of remdesivir, and half were already out of date at the time of publication.

Conducting umbrella reviews

In this article, Lazaros Belbasis and colleagues explain the rationale for umbrella reviews and the key steps involved in conducting an umbrella review, using a working example.

High variability in results and methodological quality among overlapping systematic reviews on the same topics in surgery: a meta-epidemiological study

BACKGROUND

Redundant publication of systematic reviews and meta-analyses (SRs/MAs) on the same topic presents an increasing burden for clinicians. The aim of this study was to describe variabilities in effect size and methodological quality of overlapping surgery-related SRs/MAs and to investigate factors associated with their postpublication citations.

METHODS

PubMed/MEDLINE was searched to identify SRs/MAs of RCTs on thoracoabdominal surgeries published in 2015. Previous SRs/MAs on the same topics published within the preceding 5 years (2011-2015) were identified and 5-year citation counts (through to 2020) were evaluated. Discrepancies in pooled effect sizes and their methodological quality using A Measurement Tool to Assess Systematic Reviews (AMSTAR) among overlapping SRs/MAs were assessed. The SR/MA-level factors associated with 5-year citation counts were explored, using a mixed-effects regression model with a random intercept for surgical topics.

RESULTS

A total of 57 surgery-related SRs/MAs (48 topics) published in 2015 were identified, and 146 SRs/MAs had overlapping publications on 29 topics (60.4 per cent of all topics) in the preceding 5 years. There was considerable variability in methodological quality of SRs/MAs and coverage probability for relevant RCTs, resulting in discrepant effect size estimates for the same topic. High quality (AMSTAR score 8-11) was independently associated with higher 5-year citation counts (coefficient = 32.82; 95 per cent c.i. 15.63 to 50.02; P < 0.001).

CONCLUSION

Overlapping SRs/MAs with high variability in results and methodological quality were common in surgery. A high-quality SR/MA score was an independent predictor of more frequent citations. Researchers and journal editors should concentrate their efforts on limiting publications to higher-quality reviews.

Umbrella reviews: A new tool to synthesize scientific evidence in surgery

Researchers and practitioners are faced with an exponential increase in the number of systematic reviews (SRs) (with or without meta-analysis), a so-called `secondary' research method that synthesizes data from primary research. This growing number, sometimes with discordant results on the same subject or with non-conclusions, has led to the introduction of the concept of reviews to synthesize SR in order to combine scientific knowledge useful to practitioners. These so-called ``umbrella reviews'' (UR) constitute a new tertiary research tool. Surgical research is no exception to this development but umbrella surgical reviews remain relatively rare. Any UR must be transparent and meet rigorous methodological criteria. The UR could thus provide answers to practical questions in the field of surgery, but only on condition that the bias of the included SRs is limited. Let us not forget that the base requirement of clinical surgical research remains the good methodological quality of clinical studies (primary research). Only thus can SRs or URs (secondary or tertiary research) be more useful and decisive.

Methodological Rigor and Temporal Trends of Cardiovascular Medicine Meta-Analyses in Highest-Impact Journals

Background Well-conducted meta-analyses are considered to be at the top of the evidence-based hierarchy pyramid, with an expansion of these publications within the cardiovascular research arena. There are limited data evaluating the trends and quality of such publications. The objective of this study was to evaluate the methodological rigor and temporal trends of cardiovascular medicine-related meta-analyses published in the highest impact journals. Methods and Results Using the Medline database, we retrieved cardiovascular medicine-related systematic reviews and meta-analyses published in The New England Journal of Medicine, The Lancet, Journal of the American Medical Association, The British Medical Journal, Annals of Internal Medicine, Circulation, European Heart Journal, and Journal of American College of Cardiology between January 1, 2012 and December 31, 2018. Among 6406 original investigations published during the study period, meta-analyses represented 422 (6.6%) articles, with an annual decline in the proportion of published meta-analyses (8.7% in 2012 versus 4.6% in 2018, Ptrend=0.002). A substantial number of studies failed to incorporate elements of Preferred Reporting Items for Systematic Reviews and Meta-Analyses or Meta-Analysis of Observational Studies in Epidemiology guidelines (51.9%) and only a minority of studies (10.4%) were registered in PROSPERO (International Prospective Register of Systematic Reviews). Fewer manuscripts failed to incorporate the Preferred Reporting Items for Systematic Reviews and Meta-Analyses or Meta-Analysis of Observational Studies in Epidemiology elements over time (60.2% in 2012 versus 40.0% in 2018, Ptrend<0.001) whereas the number of meta-analyses registered at PROSPERO has increased (2.4% in 2013 versus 17.5% in 2018, Ptrend<0.001). Conclusions The proportion of cardiovascular medicine-related meta-analyses published in the highest impact journals has declined over time. Although there is an increasing trend in compliance with quality-based guidelines, the overall compliance remains low.

Stepwise-Hierarchical Pooled Analysis for Synergistic Interpretation of Meta-analyses Involving Randomized and Observational Studies: Methodology Development

Background

The necessity of including observational studies in meta-analyses has been discussed in the literature, but a synergistic analysis method for combining randomized and observational studies has not been reported. Observational studies differ in validity depending on the degree of the confounders’ influence. Combining interpretations may be challenging, especially if the statistical directions are similar but the magnitude of the pooled results are different between randomized and observational studies (the ”gray zone”).

Objective

To overcome these hindrances, in this study, we aim to introduce a logical method for clinical interpretation of randomized and observational studies.

Methods

We designed a stepwise-hierarchical pooled analysis method to analyze both distribution trends and individual pooled results by dividing the included studies into at least three stages (eg, all studies, balanced studies, and randomized studies).

Results

According to the model, the validity of a hypothesis is mostly based on the pooled results of randomized studies (the highest stage). Ascending patterns in which effect size and statistical significance increase gradually with stage strengthen the validity of the hypothesis; in this case, the effect size of the observational studies is lower than that of the true effect (eg, because of the uncontrolled effect of negative confounders). Descending patterns in which decreasing effect size and statistical significance gradually weaken the validity of the hypothesis suggest that the effect size and statistical significance of the observational studies is larger than the true effect (eg, because of researchers’ bias).

Conclusions

We recommend using the stepwise-hierarchical pooled analysis approach for meta-analyses involving randomized and observational studies.

Association Between Prospective Registration and Quality of Systematic Reviews in Type 2 Diabetes Mellitus: A Meta-epidemiological Study

Background: We sought to investigate the methodological and reporting quality of published systematic reviews describing randomized controlled trials in type 2 diabetes mellitus and analyze their association with status of protocol registration.

Methods: We searched the PubMed database and identified non-Cochrane systematic reviews, with or without meta-analysis, reporting on type 2 diabetes mellitus and published between 2005 and 2018. We then randomly selected 20% of these reviews in each year, and performed methodological and reporting quality assessment using the Assessment of Multiple Systematic Review 2 (AMSTAR-2) checklist and Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) statement. We also conducted regression analyses to explore the association between characteristics of systematic reviews and AMSTAR-2 or PRISMA scores.

Results: A total of 238 systematic reviews, including 33 registered and 205 non-registered articles, met the inclusion criteria and were subsequently reviewed. Analysis indicated an increase in both registered rates and quality of systematic reviews in type 2 diabetes mellitus over the recent years. With regards to methodological and reporting quality, we found higher scores in registered, relative to non-registered reviews (AMSTAR-2 mean score: 18.0 vs. 14.5, P = 0.000; PRISMA mean score: 20.4 vs. 17.6, P = 0.000). AMSTAR-2 and PRISMA scores were associated with registration status, country of the first author, and statistical results, whereas the proportion of discussing publication bias and reporting funding sources were <40% for both registered and non-registered systematic reviews.

Conclusions: Methodological and reporting quality of systematic reviews in type 2 diabetes mellitus indicates an improvement in the recent years. However, the overall quality remains low, necessitating further improvement. Future studies are expected to pay more attention to prospective registration, description of publication bias and reporting of funding sources.

Nearly 80 systematic reviews were published each day: Observational study on trends in epidemiology and reporting over the years 2000-2019

BACKGROUND

Systematic reviews (SRs) are useful tools in synthesising the available evidence, but high numbers of overlapping SRs are also discussed in the context of research waste. Although it is often claimed that the number of SRs being published is increasing steadily, there are no precise data on that. We aimed to assess trends in the epidemiology and reporting of published SRs over the last 20 years.

METHODS

A retrospective observational study was conducted to identify potentially eligible SRs indexed in PubMed from 2000 to 2019. From all 572,871 records retrieved, we drew a simple random sample of 4,000. The PRISMA-P definition of SRs was applied to full texts and only SRs published in English were included. Characteristics were extracted by one reviewer, with a 20% sample verified by a second person.

RESULTS

A total of 1,132 SRs published in 710 different journals were included. The estimated number of SRs indexed in 2000 was 1,432 (95% CI: 547-2,317), 5,013 (95% CI: 3,375-6,650) in 2010 and 29,073 (95% CI: 25,445-32,702) in 2019. Transparent reporting of key items increased over the years. About 7 out of 10 named their article a SR (2000-2004: 41.9% and 2015-2019: 74.4%). In 2000-2004, 32.3% of SRs were based in the UK (0% in China), in 2015-2019 24.0% were from China and 10.8% from the UK. Nearly all articles from China (94.9%) conducted a meta-analysis (overall: 58.9%). Cochrane reviews (n = 84; 7.4%) less often imposed language restrictions, but often did not report the number of records and full texts screened and did not name their article a SR (22.6% vs. 73.4%).

CONCLUSIONS

We observed a more than 20-fold increase in the number of SRs indexed over the last 20 years. In 2019, this is equivalent to 80 SRs per day. Over time, SRs got more diverse in respect to journals, type of review, and country of corresponding authors. The high proportion of meta-analyses from China needs further investigation.

STUDY REGISTRATION

Open Science Framework (https://osf.io/pxjrv/).

Adverse events associated with sodium glucose co-transporter 2 inhibitors: an overview of quantitative systematic reviews

Background:

Multiple published quantitative systematic reviews have reported on adverse events associated with the use of sodium glucose co-transporter 2 (SGLT-2) inhibitors in patients with type 2 diabetes mellitus.

Aims:

To summarize and appraise the quality of evidence from quantitative systematic reviews assessing adverse events of SGLT-2 inhibitors.

Methods:

We searched PubMed, EMBASE and the Cochrane Library for quantitative systematic reviews assessing SGLT-2 inhibitor safety. Two reviewers extracted data and assessed methodological quality using the Assessment of Multiple Systematic Reviews 2 (AMSTAR 2) tool. Main outcomes included pooled and single study point estimaates (in the absence of pooled estimates) with corresponding 95% confidence intervals (CIs) of SGLT-2 inhibitors versus placebo or active comparators for genitourinary infections, volume depletion, acute kidney injury, bone fractures, diabetic ketoacidosis, lower limb amputations, cancers, and other notable adverse events.

Results:

Out of 1289 citations screened, 47 reviews assessed SGLT-2 inhibitor safety, of which 35 were of low quality. Canagliflozin, dapagliflozin and empagliflozin were consistently associated with an increased risk of genital tract infections versus placebo (point estimates ranged from 2.5 to 9.8) and other antihyperglycemic agents (point estimates ranged from 2.7 to 12.0). Canagliflozin and dapagliflozin were associated with an increased risk of diabetic ketoacidosis. Canagliflozin was the only agent associated with an increased amputation risk; however, this was driven by results from a single trial program. Dapagliflozin was the only agent that exhibited a statistically significant increased risk of urinary tract infections. Empagliflozin was associated with a statistically significant increased risk of bladder cancer; however, this finding was susceptible to detection bias. None of the agents were associated with a statistically significant increased risk of acute kidney injury, or bone fractures compared to placebo or mixed (active or placebo) comparators. Upper 95% CI limits do not rule out clinically meaningful outcomes.

Conclusion:

The majority of quantitative systematic reviews reporting on adverse events of SGLT-2 inhibitors were of low methodological quality. Despite almost 50 quantitative systematic reviews published on the safety of SGLT-2 inhibitors, clinicians are still left uncertain of the risks of important adverse effects.

Plain Language Summary

SGLT-2 iInhibitor side effects: overview of reviews

Many published systematic reviews have reported on side effects associated with the use of sodium glucose co-transporter 2 (SGLT-2) inhibitors in patients with type 2 diabetes. We aimed to summarize and appraise the quality of evidence from quantitative systematic reviews assessing side effects of SGLT-2 inhibitors. Using the Assessment of Multiple Systematic Reviews 2 (AMSTAR 2) tool, two authors extracted data and assessed the methods of included reviews. Main outcomes included reported pooled and single study point estimates for several SGLT-2 inhibitor side effects such as genital infections, bone fractures, lower limb amputations, increased blood acidity, among others. Of the reviews included in our study, 35 of the 47 reviews assessed were of low quality. Canagliflozin and dapagliflozin were associated with an increased risk of blood acidity in a 2020 review. Canagliflozin was the only agent associated with an increased amputation risk; however, this was driven by results from a single trial program. Dapagliflozin was the only agent that exhibited a significantly increased risk of urinary tract infections. Empagliflozin was associated with an increased risk of bladder cancer; however, this finding was susceptible to bias. None of the agents were associated with an increased risk of kidney injury or bone fractures.

Epistemonikos: a comprehensive database of systematic reviews for health decision-making

Background

Systematic reviews allow health decisions to be informed by the best available research evidence. However, their number is proliferating quickly, and many skills are required to identify all the relevant reviews for a specific question.

Methods and findings

We screen 10 bibliographic databases on a daily or weekly basis, to identify systematic reviews relevant for health decision-making. Using a machine-based approach developed for this project we select reviews, which are then validated by a network of more than 1000 collaborators. After screening over 1,400,000 records we have identified more than 300,000 systematic reviews, which are now stored in a single place and accessible through an easy-to-use search engine. This makes Epistemonikos the largest database of its kind.

Conclusions

Using a systematic approach, recruiting a broad network of collaborators and implementing automated methods, we developed a one-stop shop for systematic reviews relevant for health decision making.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-020-01157-x.

Can we increase the subjective well-being of the general population? An umbrella review of the evidence

INTRODUCTION

Subjective well-being (SWB) refers to being satisfied with one's life, having positive affect and having little negative affect. We may understand it as a subjective definition of good life, or in colloquial terms "happiness", and it has been associated with several important benefits such as lower mortality. In the last decades, several randomized controlled trials (RCT) have investigated the efficacy of several interventions in increasing SWB in the general population but results from different disciplines have not been integrated.

METHODS

We conducted an umbrella review of systematic reviews and meta-analyses of RCT that assess the efficacy of any kind of interventions in increasing SWB in the general population, including both positive psychology interventions (PPI) and other interventions. We (re)calculated the meta-analytic statistics needed to objectively assess the quality of the evidence of the efficacy of each type of intervention in improving each component of SWB according to the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach.

RESULTS

There was moderate-quality evidence that PPI might induce small decreases of negative affect, and low-quality evidence that they might induce moderate increases of positive affect. We found similar results for those PPI specifically consisting in conducting acts of kindness (especially spending money on or giving items to others), for which there was low-quality evidence that they might induces small increases of life satisfaction, but not for PPI specifically consisting in practicing gratitude. Quality of the evidence of the efficacy for the other interventions included in the umbrella review (yoga, resilience training, physical activity, leisure, control enhancement, psychoeducation, and miscellaneous) was very low.

CONCLUSION

There is some evidence that PPI, and specially conducting acts of kindness such as spending money on others, may increase the SWB of the general population. The quality of the evidence of the efficacy for other interventions (e.g., yoga, physical activity, or leisure) is still very low. Registration number: PROSPERO CRD42020111681.

"One more time": why replicating some syntheses of evidence relevant to COVID-19 makes sense

•Given the urgent need for credible answers to high-priority questions about the health and social impacts of COVID-19, many systematic reviewers seek to contribute their skills and expertise. •Rather than embarking on unnecessary, duplicate reviews, we encourage the evidence synthesis community to prioritise purposeful replication of systematic reviews of evidence relevant to COVID-19. •We explain why replication of systematic reviews is important, how to carry out a replication, and when to consider replication of reviews.