A new taxonomy was developed for overlap across 'overviews of systematic reviews': A meta-research study of research waste

Five aspects of research waste in biomedicine: A scoping review

BACKGROUND

The number of published journal articles has grown exponentially during the last 30 years, which may have led to some wasteful research. However, the terminology associated with research waste remains unclear. To address this, we aimed to identify, define, and categorize the aspects of research waste in published biomedical reports.

METHODS

In this scoping review, we systematically searched for biomedical literature reports from 1993 to 2023 in two databases, focusing on those addressing and defining research waste. Through data charting, we analyzed and categorized the aspects of research waste.

RESULTS

Based on 4285 initial records in the searches, a total of 832 reports were included in the analysis. The included reports were primarily narrative reviews (26%) and original reports (21%). We categorized research waste into five aspects: methodological, invisible, negligible, underreported, and structural (MINUS) research waste. More than half of the reports (56%) covered methodological research waste concerning flaws in study design, study conduct, or analysis. Invisible research waste covered nonpublication, discontinuation, and lack of data-sharing. Negligible research waste primarily concerned unnecessary repetition, for example, stemming from the absence of preceding a trial with a systematic review of the literature. Underreported research waste mainly included poor reporting, resulting in a lack of transparency. Structural research waste comprised inadequate management, collaboration, prioritization, implementation, and dissemination.

CONCLUSION

MINUS encapsulates the five main aspects of research waste. Recognizing these aspects of research waste is important for addressing and preventing further research waste and thereby ensuring efficient resource allocation and scientific integrity.

A methodological review finds mismatch between overall and pairwise overlap analysis in a sample of overviews

OBJECTIVES

Overlap of primary studies is a key methodological challenge for overviews. There are limited reports of methods used to address overlap, and there is no detailed assessment of the corrected covered area (CCA) of a representative sample of overviews. To describe the approaches used to address overlap, and to estimate the overall and pairwise CCA.

METHODS

We searched PubMed for overviews published in 2018. Two authors conducted the screening process. We described the strategy used for assessing overlap, and calculated overall and pairwise CCA for each overview.

RESULTS

We analyzed a random sample of 30 out of 89 eligible articles. Eleven did not address the overlap. Of the remainder, most frequent strategies were visual assessment and discussion of overlap as a limitation. Median overall CCA among the included overviews was 6.7%. The pairwise analysis showed that 52.8% of SR pairs had slight overlap, while 28.3% had very high overlap.

CONCLUSION

Reported strategies for addressing overlap vary considerably among overview authors. The pairwise approach for assessing the CCA revealed highly overlapped pairs of SRs in overviews with overall slight overlap and vice versa. We encourage authors to complement the overall CCA assessment with a pairwise approach.

Risk of drug use during pregnancy: master protocol for living systematic reviews and meta-analyses performed in the metaPreg project

BACKGROUND

Knowledge about the risks of drugs during pregnancy is continuously evolving due to the frequent publication of a large number of epidemiological studies. Systematic reviews and meta-analyses therefore need to be regularly updated to reflect these advances. To improve dissemination of this updated information, we developed an initiative of real-time full-scale living meta-analyses relying on an open online dissemination platform ( www.metapreg.org ).

METHOD

All living meta-analyses performed in this project will be conducted in accordance with this master protocol after adaptation of the search strategy. A systematic literature search of PubMed and Embase will be performed. All analytical studies (e.g., cohort, case-control, randomized studies) reporting original empirical findings on the association between in utero exposure to drugs and adverse pregnancy outcomes will be included. Study screening and data extraction will be performed in a semi-automation way supervised by a biocurator. A risk of bias will be assessed using the ROBINS-I tools. All clinically relevant pregnancy adverse outcomes (malformations, stillbirths, neuro-developmental disorders, pre-eclampsia, etc.) available in the included studies will be pooled through random-effects meta-analysis. Heterogeneity will be evaluated by I² statistics.

DISCUSSION

Our living systematic reviews and subsequent updates will inform the medical, regulatory, and health policy communities as the news results evolve to guide decisions on the proper use of drugs during the pregnancy.

SYSTEMATIC REVIEW REGISTRATION

Open Science Framework (OSF) registries.

Definition, harms, and prevention of redundant systematic reviews

BACKGROUND

Along with other types of research, it has been stated that the extent of redundancy in systematic reviews has reached epidemic proportions. However, it was also emphasized that not all duplication is bad, that replication in research is essential, and that it can help discover unfortunate behaviors of scientists. Thus, the question is how to define a redundant systematic review, the harmful consequences of such reviews, and what we could do to prevent the unnecessary amount of this redundancy.

MAIN BODY

There is no consensus definition of a redundant systematic review. Also, it needs to be defined what amount of overlap between systematic reviews is acceptable and not considered a redundancy. One needs to be aware that it is possible that the authors did not intend to create a redundant systematic review. A new review on an existing topic, which is not an update, is likely justified only when it can be shown that the previous review was inadequate, for example, due to suboptimal methodology. Redundant meta-analyses could have scientific, ethical, and economic questions for researchers and publishers, and thus, they should be avoided, if possible. Potential solutions for preventing redundant reviews include the following: (1) mandatory prospective registration of systematic reviews; (2) editors and peer reviewers rejecting duplicate/redundant and inadequate reviews; (3) modifying the reporting checklists for systematic reviews; (4) developing methods for evidence-based research (EBR) monitoring; (5) defining systematic reviews; (6) defining the conclusiveness of systematic reviews; (7) exploring interventions for the adoption of methodological advances; (8) killing off zombie reviews (i.e., abandoned registered reviews); (9) better prevention of duplicate reviews at the point of registration; (10) developing living systematic reviews; and (11) education of researchers.

CONCLUSIONS

Disproportionate redundancy of the same or very similar systematic reviews can lead to scientific, ethical, economic, and societal harms. While it is not realistic to expect that the creation of redundant systematic reviews can be completely prevented, some preventive measures could be tested and implemented to try to reduce the problem. Further methodological research and development in this field will be welcome.

Use of existing systematic reviews for the development of evidence-based vaccination recommendations: Guidance from the SYSVAC expert panel

National immunization technical advisory groups (NITAGs) develop immunization-related recommendations and assist policy-makers in making evidence informed decisions. Systematic reviews (SRs) that summarize the available evidence on a specific topic are a valuable source of evidence in the development of such recommendations. However, conducting SRs requires significant human, time, and financial resources, which many NITAGs lack. Given that SRs already exist for many immunization-related topics, and to prevent duplication and overlap of reviews, a more practical approach may be for NITAGs to use existing SRs. Nevertheless, it can be challenging to identify relevant SRs, to select one SR from among multiple SRs, or to critically assess and effectively use them. To support NITAGs, the London School of Hygiene and Tropical Medicine, Robert Koch Institute and collaborators developed the SYSVAC project, which consists of an online registry of systematic reviews on immunization-related topics and an e-learning course, that supports the use of them (both freely accessible at https://www.nitag-resource.org/sysvac-systematic-reviews). Drawing from the e-learning course and recommendations from an expert panel, this paper outlines methods for using existing systematic reviews when making immunization-related recommendations. With specific examples and reference to the SYSVAC registry and other resources, it offers guidance on locating existing systematic reviews; assessing their relevance to a research question, up-to-dateness, and methodological quality and/or risk of bias; and considering the transferability and applicability of their findings to other populations or settings.

Types and associated methodologies of overviews of reviews in health care: a methodological study with published examples

OBJECTIVES

To provide a descriptive insight into the different types of research questions/objectives and associated methodologies of overviews of reviews, supplemented by representative examples from the health care literature.

STUDY DESIGN AND SETTING

We searched in methodological articles for information on types and methodologies used in overviews and we explored the typology of reviews to identify similar types in literature of overviews. We categorized the types of overviews based on the research question/objective and the methodological approach used. Indicative examples for each category were selected from a sample of 2,121 overviews that were retrieved between 2000 and 2022 from MEDLINE, Scopus, and Cochrane Database of Systematic Reviews.

RESULTS

Based on type of research question, overviews were classified as overviews of reviews of interventions, associations, prediction, diagnostic accuracy, prevalence/incidence, experiences/views, economic evaluation, and measurement properties. Based on the methodological approach, we identified a variety of methods (systematic, living, rapid, scoping, evidence mapping, framework, and methodological) used in overviews.

CONCLUSION

The proposed classification and examples provide an essential starting point for future theory-building research on typologies and study designs of overviews of reviews. It is important for methodologists to make vigorous effort to create consensus-based methodological and reporting guidelines to cover these diverse types and key methodological challenges.

Kanglaite (Coix Seed Extract) as Adjunctive Therapy in Cancer: Evidence Mapping Overview Based on Systematic Reviews With Meta-Analyses

Background: Several quantitative systematic reviews of Kanglaite (KLT), an herb preparation used to treat cancer and malignant pleural effusion, have been published in recent years. However, the clinical evidence reported in these studies has not been pursued further and the methodological quality of these meta-analyses remains unknown. Therefore, an overview was designed to map the evidence landscape based on the published meta-analyses on KLT in cancer treatment. Methods: Two bibliographic databases (PubMed and Embase) were searched from inception to 25 November 2021. Two independent reviewers were involved in study selection, data abstraction, and methodological quality assessment using AMSTAR 2. The principal features of publications and the clinical outcomes of efficacy and safety were synthesized narratively, and results of methodological quality were reported as frequencies and percentages with the corresponding 95% confidence intervals. The evidence map was used to visualize the overall quality. Excel 2016 and Stata 17/SE were used for data analysis. Results: Thirteen meta-analyses published in English were included for in-depth analysis. Among them, the year of publication ranged from 2008 to 2021, and the number of included patients ranged from 488 to 2,964. Regarding the cancer type, seven articles focused on non-small cell lung cancer, two on malignant pleural effusion, and four reviews on digestive system malignancies, such as hepatocellular carcinoma and pancreatic cancer. Almost all included meta-analyses reported that KLT as adjunctive therapy could improve various efficacy outcomes (such as disease response rates, quality of life, immune indicators) and reduce the rate of occurrence of adverse reactions, such as nausea and vomiting, leukopenia, and anemia. In terms of their methodological quality, three meta-analyses were of low quality, whereas 10 studies were critically low in quality. The methodological flaws main involved items 2 ("predesigned protocol and registration informatio''), 3 ("rationale of study design for inclusion"), 4 ("comprehensive search strategy''), 5 ("literature selection in duplicate''), 7 ("list of excluded studies with reasons''), 8 ("adequate information on included studies''), 10 ("funding support for included primary studies''), and 12 ("evaluation of the potential impact of risk of bias'') based on the AMSTAR 2 tool. Conclusion: Current evidence reveals that KLT is effective and safe as an adjunctive treatment for non-small cell lung cancer, malignant pleural effusion, and digestive system malignancies (such as hepatocellular carcinoma). However, the results assessed in this overview should be further verified using well-designed and clearly reported clinical trials and meta-analyses of KLT.

Guidelines, guidelines and more guidelines for haemorrhoid treatment: A review to sort the wheat from the chaff

AIM

Guidelines benefit patients and clinicians by distilling evidence into easy-to-read recommendations. The literature around the management of haemorrhoids is immense and guidelines are invaluable to improve treatment integrity and patient outcomes. We identified current haemorrhoid guidelines and assessed them for quality and consistency.

METHODS

A systematic search of the literature from January 2011 to October 2021 was carried out. Guidelines identified were assessed for quality using the AGREE II instrument and for consistency in terms of tabulated treatment recommendations.

RESULTS

During this period nine guidelines were identified worldwide. The general quality was poor with only one guideline considered of high enough quality for use. In general, expert selection criteria for guideline development groups were vaguely defined. There were inconsistencies in the interpretation of the published evidence leading to variation in treatment recommendations.

DISCUSSION

Fewer, higher quality guidelines, with more consistent results, are needed. Particular attention should be given to defining the selection of experts involved.