The impact of different inclusion decisions on the comprehensiveness and complexity of overviews of reviews of healthcare interventions

Dietary Flavonoids Consumption and Health: An Umbrella Review

SCOPE

The current evidence between dietary flavonoids consumption and multiple health outcomes is inadequate and inconclusive. To summarize and evaluate the evidence for dietary flavonoids consumption and multiple health outcomes, an umbrella review of meta-analyses and systematic reviews is conducted.

METHODS AND RESULTS

PubMed, Ovid-EMBASE, and the Cochrane Database of Systematic Reviews are searched up to January 2024. The study includes a total of 32 articles containing 24 unique health outcomes in this umbrella review. Meta-analyses are recalculated by using a random effects model. Separate analyses are performed based on the kind of different flavonoid subclasses. The study finds some unique associations such as flavonol and gastric cancer, isoflavone and uterine fibroids and endometrial cancer, total flavonoids consumption and lung cancer, ovarian cancer, and prostate cancer. Overall, the study confirms the negative associations between dietary flavonoids consumption and type 2 diabetes mellitus, cardiovascular diseases, breast cancer, colorectal cancer, lung cancer, and mortality, while positive associations are observed for prostate cancer and uterine fibroids.

CONCLUSION

Although dietary flavonoids are significantly associated with many outcomes, firm generalizable conclusions about their beneficial or harmful effects cannot be drawn because of the low certainty of evidence for most of outcomes. More well-designed primary studies are needed.

Association between semen parameters and recurrent pregnancy loss: An umbrella review of meta-analyses

AIM

Recurrent pregnancy loss (RPL) is a common clinical reproductive problem. With research advancements, an increasing number of studies have suggested that male factors play an important role in RPL. However, the evaluation results of male sperm quality in published meta-analyses are inconsistent. We aimed to summarize the evidence of the association between semen factors and RPL and evaluate the level and validity of the evidence.

METHODS

We searched PubMed, Cochrane Library, EMBASE, Web of Science, and Scopus databases for systematic reviews or meta-analyses to evaluate the association between male semen parameters and RPL. The methodological quality of the included meta-analyses was assessed, and data and evidence were re-synthesized and stratified using a random-effects model.

RESULTS

Seven meta-analyses and nine semen parameters were included in the final analysis. The methodological quality of all publications was considered low or very low. There was highly suggestive evidence for the association between sperm DNA fragmentation (SDF), sperm progressive motility rate, and RPL (class II). The evidence level for the association between sperm concentration, normal sperm morphology, sperm deformity rate, total motility, and RPL was suggestive evidence (class III). The evidence level for the association between sperm volume and sperm count and RPL was weak (class IV). There was no significant association between sperm pH and RPL (class NS).

CONCLUSIONS

Our results suggest level II evidence for the association between male SDF and RPL, while the evidence level for the association between conventional semen routine parameters and RPL was low (classes III and IV).

The reporting completeness and transparency of systematic reviews of prognostic prediction models for COVID-19 was poor: a methodological overview of systematic reviews

OBJECTIVES

To conduct a methodological overview of reviews to evaluate the reporting completeness and transparency of systematic reviews (SRs) of prognostic prediction models (PPMs) for COVID-19.

STUDY DESIGN AND SETTING

MEDLINE, Scopus, Cochrane Database of Systematic Reviews, and Epistemonikos (epistemonikos.org) were searched for SRs of PPMs for COVID-19 until December 31, 2022. The risk of bias in systematic reviews tool was used to assess the risk of bias. The protocol for this overview was uploaded in the Open Science Framework (https://osf.io/7y94c).

RESULTS

Ten SRs were retrieved; none of them synthesized the results in a meta-analysis. For most of the studies, there was absence of a predefined protocol and missing information on study selection, data collection process, and reporting of primary studies and models included, while only one SR had its data publicly available. In addition, for the majority of the SRs, the overall risk of bias was judged as being high. The overall corrected covered area was 6.3% showing a small amount of overlapping among the SRs.

CONCLUSION

The reporting completeness and transparency of SRs of PPMs for COVID-19 was poor. Guidance is urgently required, with increased awareness and education of minimum reporting standards and quality criteria. Specific focus is needed in predefined protocol, information on study selection and data collection process, and in the reporting of findings to improve the quality of SRs of PPMs for COVID-19.

Association between pregnancy-related complications and development of type 2 diabetes and hypertension in women: an umbrella review

BACKGROUND

Despite many systematic reviews and meta-analyses examining the associations of pregnancy complications with risk of type 2 diabetes mellitus (T2DM) and hypertension, previous umbrella reviews have only examined a single pregnancy complication. Here we have synthesised evidence from systematic reviews and meta-analyses on the associations of a wide range of pregnancy-related complications with risk of developing T2DM and hypertension.

METHODS

Medline, Embase and Cochrane Database of Systematic Reviews were searched from inception until 26 September 2022 for systematic reviews and meta-analysis examining the association between pregnancy complications and risk of T2DM and hypertension. Screening of articles, data extraction and quality appraisal (AMSTAR2) were conducted independently by two reviewers using Covidence software. Data were extracted for studies that examined the risk of T2DM and hypertension in pregnant women with the pregnancy complication compared to pregnant women without the pregnancy complication. Summary estimates of each review were presented using tables, forest plots and narrative synthesis and reported following Preferred Reporting Items for Overviews of Reviews (PRIOR) guidelines.

RESULTS

Ten systematic reviews were included. Two pregnancy complications were identified. Gestational diabetes mellitus (GDM): One review showed GDM was associated with a 10-fold higher risk of T2DM at least 1 year after pregnancy (relative risk (RR) 9.51 (95% confidence interval (CI) 7.14 to 12.67) and although the association differed by ethnicity (white: RR 16.28 (95% CI 15.01 to 17.66), non-white: RR 10.38 (95% CI 4.61 to 23.39), mixed: RR 8.31 (95% CI 5.44 to 12.69)), the between subgroups difference were not statistically significant at 5% significance level. Another review showed GDM was associated with higher mean blood pressure at least 3 months postpartum (mean difference in systolic blood pressure: 2.57 (95% CI 1.74 to 3.40) mmHg and mean difference in diastolic blood pressure: 1.89 (95% CI 1.32 to 2.46) mmHg). Hypertensive disorders of pregnancy (HDP): Three reviews showed women with a history of HDP were 3 to 6 times more likely to develop hypertension at least 6 weeks after pregnancy compared to women without HDP (meta-analysis with largest number of studies: odds ratio (OR) 4.33 (3.51 to 5.33)) and one review reported a higher rate of T2DM after HDP (hazard ratio (HR) 2.24 (1.95 to 2.58)) at least a year after pregnancy. One of the three reviews and five other reviews reported women with a history of preeclampsia were 3 to 7 times more likely to develop hypertension at least 6 weeks postpartum (meta-analysis with the largest number of studies: OR 3.90 (3.16 to 4.82) with one of these reviews reporting the association was greatest in women from Asia (Asia: OR 7.54 (95% CI 2.49 to 22.81), Europe: OR 2.19 (95% CI 0.30 to 16.02), North and South America: OR 3.32 (95% CI 1.26 to 8.74)).

CONCLUSIONS

GDM and HDP are associated with a greater risk of developing T2DM and hypertension. Common confounders adjusted for across the included studies in the reviews were maternal age, body mass index (BMI), socioeconomic status, smoking status, pre-pregnancy and current BMI, parity, family history of T2DM or cardiovascular disease, ethnicity, and time of delivery. Further research is needed to evaluate the value of embedding these pregnancy complications as part of assessment for future risk of T2DM and chronic hypertension.

Associations of antidiabetic drugs with diabetic retinopathy in people with type 2 diabetes: an umbrella review and meta-analysis

Background

Diabetic retinopathy (DR) is the most frequent complication of type 2 diabetes and remains the leading cause of preventable blindness. Current clinical decisions regarding the administration of antidiabetic drugs do not sufficiently incorporate the risk of DR due to the inconclusive evidence from preceding meta-analyses. This umbrella review aimed to systematically evaluate the effects of antidiabetic drugs on DR in people with type 2 diabetes.

Methods

A systematic literature search was undertaken in Medline, Embase, and the Cochrane Library (from inception till 17th May 2022) without language restrictions to identify systematic reviews and meta-analyses of randomized controlled trials or longitudinal studies that examined the association between antidiabetic drugs and DR in people with type 2 diabetes. Two authors independently extracted data and assessed the quality of included studies using the AMSTAR-2 (A MeaSurement Tool to Assess Systematic Reviews) checklist, and evidence assessment was performed using the GRADE (Grading of recommendations, Assessment, Development and Evaluation). Random-effects models were applied to calculate relative risk (RR) or odds ratios (OR) with 95% confidence intervals (CI). This study was registered with PROSPERO (CRD42022332052).

Results

With trial evidence from 11 systematic reviews and meta-analyses, we found that the use of glucagon-like peptide-1 receptor agonists (GLP-1 RA), sodium-glucose cotransporter-2 inhibitors (SGLT-2i), or dipeptidyl peptidase-4 inhibitors (DPP-4i) was not statistically associated with the risk of DR, compared to either placebo (RR: GLP-1 RA, 0.98, 0.89-1.08; SGLT-2i, 1.00, 95% CI 0.79-1.27; DPP-4i, 1.17, 0.99-1.39) or other antidiabetic drugs. Compared to other antidiabetic drugs, meglitinides (0.34, 0.01-8.25), SGLT-2i (0.73, 0.10-5.16), thiazolidinediones (0.92, 0.67-1.26), metformin (1.15, 0.81-1.63), sulphonylureas (1.24, 0.93-1.65), and acarbose (4.21, 0.44-40.43) were not statistically associated with the risk of DR. With evidence from longitudinal studies only, insulin was found to have a higher risk of DR than other antidiabetic drugs (OR: 2.47, 95% CI: 2.04-2.99).

Conclusion

Our results indicate that antidiabetic drugs are generally safe to prescribe regarding the risk of DR among people with type 2 diabetes. Further robust and large-scale trials investigating the effects of insulin, meglitinides, and acarbose on DR are warranted.

Systematic review registration

https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=332052, identifier CRD42022332052.

Estimating pairwise overlap in umbrella reviews: Considerations for using the corrected covered area (CCA) index methodology

Umbrella reviews (reviews of systematic reviews) are increasingly used to synthesize findings from systematic reviews. One important challenge when pooling data from several systematic reviews is publication overlap, that is, the same primary publications being included in multiple reviews. Pieper et al. have proposed using the corrected covered area (CCA) index to quantify the degree of overlap between systematic reviews to be pooled in an umbrella review. Recently, this methodology has been integrated in Excel- or R-based tools for easier use. In this short letter, we highlight an important consideration for using the CCA methodology for pairwise overlap assessment, especially when reviews include varying numbers of primary publications, and we urge researchers to fine-tune this method and exercise caution when review exclusion decisions are based on its output.

Effectiveness of nonpharmacological interventions to prevent adverse events in the intensive care unit: A review of systematic reviews

BACKGROUND

Different types of interventions have been assessed for the prevention of adverse events. However, determining which patient-safety practice is most effective can be challenging when there is no systematised evidence synthesis. An overview following the best methodological standards can provide the best reliable integrative evidence.

OBJECTIVES

The objective of this study was to provide an overview of effectiveness nonpharmacological interventions aimed at preventing adverse events in the intensive care unit.

METHODS

A review of systematic reviews (SRs) was conducted according to the Cochrane Handbook and PRISMA recommendations. PubMed, CINAHL, and Cochrane Library were searched for SRs published until March 2022. Two reviewers independently assessed the study's quality, using AMSTAR-2, and extracted data on intervention characteristics and effect on prevention of adverse events.

RESULTS

Thirty-seven SRs were included, and 27 nonpharmacological interventions were identified to prevent 11 adverse events. Most of the reviews had critically low methodological quality. Among all the identified interventions, subglottic secretion drainage, semirecumbent position, and kinetic bed therapy were effective in preventing ventilator-associated pneumonia; the use of earplugs, early mobilisation, family participation, and music in reducing delirium; physical rehabilitation in improving muscle strength; use of respiratory support in preventing reintubation; the use of a computerised physician order entry system in reducing risk of medication errors; and the use of heated water humidifier was effective in reducing artificial airway occlusion.

CONCLUSIONS

Some nonpharmacological interventions reduced adverse events in the intensive care setting. These findings should be interpreted carefully due to the low methodological quality. SRs on preventing adverse events in the intensive care unit should adhere to quality assessment tools so that best evidence can be used in decision-making.

Effectiveness of physical exercise programmes in reducing complications associated with secondary lymphoedema to breast cancer: a protocol for an overview of systematic reviews

INTRODUCTION

Breast cancer-related lymphoedema (BCRL) is one of the most underestimated and debilitating complications associated with the treatment that women with breast cancer receive. Several systematic reviews (SRs) of different physical exercise programmes have been published, presenting disperse and contradictory clinical results. Therefore, there is a need for access to the best available and summarised evidence to capture and evaluate all the physical exercise programmes that focus on reducing BCRL.

OBJECTIVE

To evaluate the effectiveness of different physical exercise programmes in reducing the volume of lymphoedema, pain intensity and improving quality of life.

METHOD AND ANALYSIS

The protocol of this overview is reported following the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols, and its methodology is based on Cochrane Handbook for Systematic Reviews of Interventions. Only those SRs involving physical exercise by patients with BCRL will be included, whether on its own or combined with other exercises or other physical therapy interventions.The outcomes of interest to be considered will be lymphoedema volume, quality of life, pain intensity, grip strength, range of motion, upper limb function and any adverse event. The MEDLINE/PubMed, Lilacs, Cochrane Library, PEDro and Embase databases will be searched for reports published from database inception to April 2023.Two researchers will perform study selection, data extraction and risk of bias assessment independently. Any discrepancy will be resolved by consensus, or ultimately, by a third-party reviewer. We will use Grading of Recommendations Assessment, Development and Evaluation System to assess the overall quality of the body of evidence.

ETHICS AND DISSEMINATION

The results of this overview will be published in peer-reviewed scholarly journals and the scientific dissemination will take place in national or international conferences. This study does not require approval from an ethics committee, as it does not directly collect information from patients.

PROSPERO REGISTRATION NUMBER

CRD42022334433.

Reliability of the evidence to guide decision-making in foot ulcer prevention in diabetes: an overview of systematic reviews

Background

Reliable evidence on the effectiveness of interventions to prevent diabetes-related foot ulceration is essential to inform clinical practice. Well-conducted systematic reviews that synthesise evidence from all relevant trials offer the most robust evidence for decision-making. We conducted an overview to assess the comprehensiveness and utility of the available secondary evidence as a reliable source of robust estimates of effect with the aim of informing a cost-effective care pathway using an economic model. Here we report the details of the overview. [PROSPERO Database (CRD42016052324)].

Methods

Medline (Ovid), Embase (Ovid), Epistomonikos, Cochrane Database of Systematic Reviews (CDSR), Database of Abstracts of Reviews of Effectiveness (DARE), and the Health Technology Assessment Journals Library were searched to 17th May 2021, without restrictions, for systematic reviews of randomised controlled trials (RCTs) of preventive interventions in people with diabetes. The primary outcomes of interest were new primary or recurrent foot ulcers. Two reviewers independently extracted data and assessed the risk of bias in the included reviews.

Findings

The overview identified 30 systematic reviews of patient education, footwear and off-loading, complex and other interventions. Many are poorly reported and have fundamental methodological shortcomings associated with increased risk of bias. Most concerns relate to vague inclusion criteria (60%), weak search or selection strategies (70%) and quality appraisal methods (53%) and inexpert conduct and interpretation of quantitative and narrative evidence syntheses (57%). The 30 reviews have collectively assessed 26 largely poor-quality RCTs with substantial overlap.

Interpretation

The majority of these systematic reviews of the effectiveness of interventions to prevent diabetic foot ulceration are at high risk of bias and fail to provide reliable evidence for decision-making. Adherence to the core principles of conducting and reporting systematic reviews is needed to improve the reliability of the evidence generated to inform clinical practice.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-022-01738-y.

Adverse Events Associated with Immune Checkpoint Inhibitors: Overview of Systematic Reviews

BACKGROUND

Recognition and management of adverse events (AEs) associated with immune checkpoint inhibitor (ICI) use by cancer patients requires expertise from multiple disciplines. Greater awareness of potential AEs may result in earlier recognition, appropriate management, and better patient outcomes.

OBJECTIVE

The primary objective of this overview of systematic reviews was to synthesize and consolidate systematic review evidence describing the incidence proportion and severity of AEs associated with various ICI therapies across different cancers.

METHODS

A systematic literature search of four databases was conducted to identify systematic reviews that describe the incidence proportion and severity of AEs related to ICI therapy in cancer patients. A systematic review was eligible if it included adults with cancer; on ICI alone or in combination with another ICI, chemotherapy, or targeted therapy; severity (graded according to the Common Terminology Criteria for Adverse Events) and incidence proportion of AEs and whether it reported its eligibility criteria. AEs of interest were identified through an iterative ranking exercise by key stakeholders and knowledge users. Extraction of PICOTTS elements and quality indicators (AMSTAR-2) were used to manage overlap of primary studies across systematic reviews at the outcome level. Cancer subtypes were mapped to drug class and AE severity.

RESULTS

Overall, 129 systematic reviews met the inclusion criteria for data mapping. Systematic reviews reported incidence proportions for more than 76 AEs, of which 34 were identified as AEs of interest. After overlap assessment, 65 systematic reviews were chosen for data extraction. The three AEs with the highest median incidence were fatigue (18.3%, interquartile range [IQR] 15.0-28.0%), diarrhea (15.3%, IQR 9.7-29.2%) and rash (14.4%, IQR 10.3-19.2%). The three AEs (high-grade) with the highest median incidence were diarrhea (1.5%, IQR 1.2-6.0%), colitis (1.3%, IQR 0.6-6.1%) and neutropenia (1.2%, IQR 0.4-3.3%). Incidence proportions of high-grade AEs were often considerably lower than all-grade AEs and combination therapy (ICI combinations or combinations of ICI with chemotherapy or targeted therapy) was responsible for some of the highest incidence proportions regardless of AE. Rare AEs and certain cancer subtypes were not well reported.

CONCLUSIONS

Early recognition of AEs associated with ICIs requires expertise from diverse specialists, not just oncologists. Greater awareness of potential AEs may result in earlier recognition, appropriate management, and better patient outcomes.

PROSPERO REGISTRATION

CRD42021231593.

Implementation strategies for knowledge products in primary healthcare: a systematic review of systematic reviews (Preprint)

Background

The underuse or overuse of knowledge products leads to waste in health care, and primary care is no exception.

Objective

This study aimed to characterize which knowledge products are frequently implemented, the implementation strategies used in primary care, and the implementation outcomes that are measured.

Methods

We performed a systematic review (SR) of SRs using the Cochrane systematic approach to include eligible SRs. The inclusion criteria were any primary care contexts, health care professionals and patients, any Effective Practice and Organization of Care implementation strategies of specified knowledge products, any comparators, and any implementation outcomes based on the Proctor framework. We searched the MEDLINE, EMBASE, CINAHL, Ovid PsycINFO, Web of Science, and Cochrane Library databases from their inception to October 2019 without any restrictions. We searched the references of the included SRs. Pairs of reviewers independently performed selection, data extraction, and methodological quality assessment by using A Measurement Tool to Assess Systematic Reviews 2. Data extraction was informed by the Effective Practice and Organization of Care taxonomy for implementation strategies and the Proctor framework for implementation outcomes. We performed a descriptive analysis and summarized the results by using a narrative synthesis.

Results

Of the 11,101 records identified, 81 (0.73%) SRs were included. Of these 81, a total of 47 (58%) SRs involved health care professionals alone. Moreover, 15 SRs had a high or moderate methodological quality. Most of them addressed 1 type of knowledge product (56/81, 69%), common clinical practice guidelines (26/56, 46%) or management, and behavioral or pharmacological health interventions (24/56, 43%). Mixed strategies were used for implementation (67/81, 83%), predominantly education-based (meetings in 60/81, 74%; materials distribution in 59/81, 73%; and academic detailing in 45/81, 56%), reminder (53/81, 36%), and audit and feedback (40/81, 49%) strategies. Education meetings (P=.13) and academic detailing (P=.11) seemed to be used more when the population was composed of health care professionals alone. Improvements in the adoption of knowledge products were the most commonly measured outcome (72/81, 89%). The evidence level was reported in 12% (10/81) of SRs on 62 outcomes (including 48 improvements in adoption), of which 16 (26%) outcomes were of moderate or high level.

Conclusions

Clinical practice guidelines and management and behavioral or pharmacological health interventions are the most commonly implemented knowledge products and are implemented through the mixed use of educational, reminder, and audit and feedback strategies. There is a need for a strong methodology for the SR of randomized controlled trials to explore their effectiveness and the entire cascade of implementation outcomes.

The role of diet in renal cell carcinoma incidence: an umbrella review of meta-analyses of observational studies

BACKGROUND

Evidence associating diet with the incidence of renal cell carcinoma (RCC) is inconclusive. We aimed to summarize evidence associating dietary factors with RCC incidence and assess the strength and validity of this evidence.

METHODS

We conducted an umbrella review of systematic reviews or meta-analyses (SRoMAs) that assessed the association between diet and RCC incidence. Through April 2021, PubMed, Web of Science, Embase, The Cochrane Library, Scopus, and WCRF were searched. Two independent reviewers selected studies, extracted data, and appraised the quality of SRoMAs. According to credibility assessment criteria, evidence can be divided into five categories: convincing (class I), highly suggestive (class II), suggestive (class III), weak (class IV), and nonsignificant (class V).

RESULTS

Twenty-nine meta-analyses were obtained after screening. After excluding 7 overlapping meta-analyses, 22 meta-analyses including 502 individual studies and 64 summary hazard ratios for RCC incidence were included: dietary patterns or dietary quality indices (n = 6), foods (n = 13), beverages (n = 4), alcohol (n = 7), macronutrients (n =15), and micronutrients (n =19). No meta-analyses had high methodological quality. Five meta-analyses exhibited small study effects; one meta-analysis showed evidence of excess significance bias. No dietary factors showed convincing or highly suggestive evidence of association with RCC in the overall analysis. Two protective factors had suggestive evidence (vegetables (0.74, 95% confidence interval 0.63 to 0.86) and vitamin C (0.77, 0.66 to 0.90)) in overall analysis. One protective factor had convincing evidence (moderate drinking (0.77, 0.70 to 0.84)) in Europe and North America and one protective factor had highly suggestive evidence (cruciferous vegetables (0.78, 0.70 to 0.86)) in North America.

CONCLUSIONS

Although many meta-analyses have assessed associations between dietary factors and RCC, no high-quality evidence exists (classes I and II) in the overall analysis. Increased intake of vegetables and vitamin C is negatively associated with RCC risk. Moderate drinking might be beneficial for Europeans and North Americans, and cruciferous vegetables might be beneficial to North Americans, but the results should be interpreted with caution. More researches are needed in the future.

TRIAL REGISTRATION

PROSPERO CRD42021246619.

Managing overlap of primary study results across systematic reviews: practical considerations for authors of overviews of reviews

BACKGROUND

Overviews often identify and synthesise a large number of systematic reviews on the same topic, which is likely to lead to overlap (i.e. duplication) in primary studies across the reviews. Using a primary study result multiple times in the same analysis overstates its sample size and number of events, falsely leading to greater precision in the analysis. This paper aims to: (a) describe types of overlapping data that arise from the same primary studies reported across multiple reviews, (b) describe methods to identify and explain overlap of primary study data, and (c) present six case studies illustrating different approaches to manage overlap.

METHODS

We first updated the search in PubMed for methods from the MOoR framework relating to overlap of primary studies. One author screened the studies titles and abstracts, and any full-text articles retrieved, extracted methods data relating to overlap of primary studies and mapped it to the overlap methods from the MOoR framework. We also describe six case studies as examples of overviews that use specific overlap methods across the steps in the conduct of an overview. For each case study, we discuss potential methodological implications in terms of limitations, efficiency, usability, and resource use.

RESULTS

Nine methods studies were found and mapped to the methods identified by the MOoR framework to address overlap. Overlap methods were mapped across four steps in the conduct of an overview - the eligibility criteria step, the data extraction step, the assessment of risk of bias step, and the synthesis step. Our overview case studies used multiple methods to reduce overlap at different steps in the conduct of an overview.

CONCLUSIONS

Our study underlines that there is currently no standard methodological approach to deal with overlap in primary studies across reviews. The level of complexity when dealing with overlap can vary depending on the yield, trends and patterns of the included literature and the scope of the overview question. Choosing a method might be dependent on the number of included reviews and their primary studies. Gaps in evaluation of methods to address overlap were found and further investigation in this area is needed.

Coronavirus disease (COVID-19) pandemic: an overview of systematic reviews

BACKGROUND

Navigating the rapidly growing body of scientific literature on the SARS-CoV-2 pandemic is challenging, and ongoing critical appraisal of this output is essential. We aimed to summarize and critically appraise systematic reviews of coronavirus disease (COVID-19) in humans that were available at the beginning of the pandemic.

METHODS

Nine databases (Medline, EMBASE, Cochrane Library, CINAHL, Web of Sciences, PDQ-Evidence, WHO's Global Research, LILACS, and Epistemonikos) were searched from December 1, 2019, to March 24, 2020. Systematic reviews analyzing primary studies of COVID-19 were included. Two authors independently undertook screening, selection, extraction (data on clinical symptoms, prevalence, pharmacological and non-pharmacological interventions, diagnostic test assessment, laboratory, and radiological findings), and quality assessment (AMSTAR 2). A meta-analysis was performed of the prevalence of clinical outcomes.

RESULTS

Eighteen systematic reviews were included; one was empty (did not identify any relevant study). Using AMSTAR 2, confidence in the results of all 18 reviews was rated as "critically low". Identified symptoms of COVID-19 were (range values of point estimates): fever (82-95%), cough with or without sputum (58-72%), dyspnea (26-59%), myalgia or muscle fatigue (29-51%), sore throat (10-13%), headache (8-12%) and gastrointestinal complaints (5-9%). Severe symptoms were more common in men. Elevated C-reactive protein and lactate dehydrogenase, and slightly elevated aspartate and alanine aminotransferase, were commonly described. Thrombocytopenia and elevated levels of procalcitonin and cardiac troponin I were associated with severe disease. A frequent finding on chest imaging was uni- or bilateral multilobar ground-glass opacity. A single review investigated the impact of medication (chloroquine) but found no verifiable clinical data. All-cause mortality ranged from 0.3 to 13.9%.

CONCLUSIONS

In this overview of systematic reviews, we analyzed evidence from the first 18 systematic reviews that were published after the emergence of COVID-19. However, confidence in the results of all reviews was "critically low". Thus, systematic reviews that were published early on in the pandemic were of questionable usefulness. Even during public health emergencies, studies and systematic reviews should adhere to established methodological standards.

Measuring test-retest reliability (TRR) of AMSTAR provides moderate to perfect agreement - a contribution to the discussion of the importance of TRR in relation to the psychometric properties of assessment tools

Background

Systematic Reviews (SRs) can build the groundwork for evidence-based health care decision-making. A sound methodological quality of SRs is crucial. AMSTAR (A Measurement Tool to Assess Systematic Reviews) is a widely used tool developed to assess the methodological quality of SRs of randomized controlled trials (RCTs). Research shows that AMSTAR seems to be valid and reliable in terms of interrater reliability (IRR), but the test retest reliability (TRR) of AMSTAR has never been investigated. In our study we investigated the TRR of AMSTAR to evaluate the importance of its measurement and contribute to the discussion of the measurement properties of AMSTAR and other quality assessment tools.

Methods

Seven raters at three institutions independently assessed the methodological quality of SRs in the field of occupational health with AMSTAR. Between the first and second ratings was a timespan of approximately two years. Answers were dichotomized, and we calculated the TRR of all raters and AMSTAR items using Gwet’s AC1 coefficient. To investigate the impact of variation in the ratings over time, we obtained summary scores for each review.

Results

AMSTAR item 4 (Was the status of publication used as an inclusion criterion?) provided the lowest median TRR of 0.53 (moderate agreement). Perfect agreement of all reviewers was detected for AMSTAR-item 1 with a Gwet’s AC1 of 1, which represented perfect agreement. The median TRR of the single raters varied between 0.69 (substantial agreement) and 0.89 (almost perfect agreement). Variation of two or more points in yes-scored AMSTAR items was observed in 65% (73/112) of all assessments.

Conclusions

The high variation between the first and second AMSTAR ratings suggests that consideration of the TRR is important when evaluating the psychometric properties of AMSTAR.. However, more evidence is needed to investigate this neglected issue of measurement properties. Our results may initiate discussion of the importance of considering the TRR of assessment tools. A further examination of the TRR of AMSTAR, as well as other recently established rating tools such as AMSTAR 2 and ROBIS (Risk Of Bias In Systematic reviews), would be useful.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12874-021-01231-y.

Methods for depicting overlap in overviews of systematic reviews: An introduction to static tabular and graphical displays

BACKGROUND AND OBJECTIVE

To introduce potential static tabular and graphical techniques for visually presenting overlap between systematic reviews (SRs) included in overviews of systematic reviews (OoSRs).

METHODS

The graphical approaches described include Venn and Euler diagrams, as well as matrix-based, node-link, and aggregation-based techniques. We used fundamental concepts of mathematics from set and network theory to develop our novel graphical approaches. The graphical displays were created using R.

RESULTS

Overview authors have the flexibility to choose from a variety of visualizations, depending on the characteristics of their study. If the OoSRs include few SRs, a Venn or an Euler diagram can be used. In case of OoSRs with more SRs, Upset plots, heatmaps, and node-link graphs are more appropriate for visualizing overlapping SRs. Stacked bar plots constitute an aggregation-based technique of illustrating overlap. Strengths and limitations of each graphical approach are presented.

CONCLUSION

The degree of overlap should be explored for the entire study and for specific outcomes of interest. The proposed graphical techniques may assist methodologists and authors in identifying overlap, which in turn may improve validity and transparency in OoSRs. More research is needed to understand which technique would be most useful and easiest to understand.

Exercise/physical activity and health outcomes: an overview of Cochrane systematic reviews

BACKGROUND

Sedentary lifestyle is a major risk factor for noncommunicable diseases such as cardiovascular diseases, cancer and diabetes. It has been estimated that approximately 3.2 million deaths each year are attributable to insufficient levels of physical activity. We evaluated the available evidence from Cochrane systematic reviews (CSRs) on the effectiveness of exercise/physical activity for various health outcomes.

METHODS

Overview and meta-analysis. The Cochrane Library was searched from 01.01.2000 to issue 1, 2019. No language restrictions were imposed. Only CSRs of randomised controlled trials (RCTs) were included. Both healthy individuals, those at risk of a disease, and medically compromised patients of any age and gender were eligible. We evaluated any type of exercise or physical activity interventions; against any types of controls; and measuring any type of health-related outcome measures. The AMSTAR-2 tool for assessing the methodological quality of the included studies was utilised.

RESULTS

Hundred and fifty CSRs met the inclusion criteria. There were 54 different conditions. Majority of CSRs were of high methodological quality. Hundred and thirty CSRs employed meta-analytic techniques and 20 did not. Limitations for studies were the most common reasons for downgrading the quality of the evidence. Based on 10 CSRs and 187 RCTs with 27,671 participants, there was a 13% reduction in mortality rates risk ratio (RR) 0.87 [95% confidence intervals (CI) 0.78 to 0.96]; I2 = 26.6%, [prediction interval (PI) 0.70, 1.07], median effect size (MES) = 0.93 [interquartile range (IQR) 0.81, 1.00]. Data from 15 CSRs and 408 RCTs with 32,984 participants showed a small improvement in quality of life (QOL) standardised mean difference (SMD) 0.18 [95% CI 0.08, 0.28]; I2 = 74.3%; PI -0.18, 0.53], MES = 0.20 [IQR 0.07, 0.39]. Subgroup analyses by the type of condition showed that the magnitude of effect size was the largest among patients with mental health conditions.

CONCLUSION

There is a plethora of CSRs evaluating the effectiveness of physical activity/exercise. The evidence suggests that physical activity/exercise reduces mortality rates and improves QOL with minimal or no safety concerns.

TRIAL REGISTRATION

Registered in PROSPERO ( CRD42019120295 ) on 10th January 2019.

Guidance for overviews of reviews continues to accumulate, but important challenges remain: a scoping review

BACKGROUND

Overviews of reviews (overviews) provide an invaluable resource for healthcare decision-making by combining large volumes of systematic review (SR) data into a single synthesis. The production of high-quality overviews hinges on the availability of practical evidence-based guidance for conduct and reporting.

OBJECTIVES

Within the broad purpose of informing the development of a reporting guideline for overviews, we aimed to provide an up-to-date map of existing guidance related to the conduct of overviews, and to identify common challenges that authors face when undertaking overviews.

METHODS

We updated a scoping review published in 2016 using the search methods that had produced the highest yield: ongoing reference tracking (2014 to March 2020 in PubMed, Scopus, and Google Scholar), hand-searching conference proceedings and websites, and contacting authors of published overviews. Using a qualitative meta-summary approach, one reviewer extracted, organized, and summarized the guidance and challenges presented within the included documents. A second reviewer verified the data and synthesis.

RESULTS

We located 28 new guidance documents, for a total of 77 documents produced by 34 research groups. The new guidance helps to resolve some earlier identified challenges in the production of overviews. Important developments include strengthened guidance on handling primary study overlap at the study selection and analysis stages. Despite marked progress, several areas continue to be hampered by inconsistent or lacking guidance. There is ongoing debate about whether, when, and how supplemental primary studies should be included in overviews. Guidance remains scant on how to extract and use appraisals of quality of the primary studies within the included SRs and how to adapt GRADE methodology to overviews. The challenges that overview authors face are often related to the above-described steps in the process where evidence-based guidance is lacking or conflicting.

CONCLUSION

The rising popularity of overviews has been accompanied by a steady accumulation of new, and sometimes conflicting, guidance. While recent guidance has helped to address some of the challenges that overview authors face, areas of uncertainty remain. Practical tools supported by empirical evidence are needed to assist authors with the many methodological decision points that are encountered in the production of overviews.

Association between the reproductive health of young women and cardiovascular disease in later life: umbrella review

OBJECTIVE

To consolidate evidence from systematic reviews and meta-analyses investigating the association between reproductive factors in women of reproductive age and their subsequent risk of cardiovascular disease.

DESIGN

Umbrella review.

DATA SOURCES

Medline, Embase, and Cochrane databases for systematic reviews and meta-analyses from inception until 31 August 2019.

REVIEW METHODS

Two independent reviewers undertook screening, data extraction, and quality appraisal. The population was women of reproductive age. Exposures were fertility related factors and adverse pregnancy outcomes. Outcome was cardiovascular diseases in women, including ischaemic heart disease, heart failure, peripheral arterial disease, and stroke.

RESULTS

32 reviews were included, evaluating multiple risk factors over an average follow-up period of 7-10 years. All except three reviews were of moderate quality. A narrative evidence synthesis with forest plots and tabular presentations was performed. Associations for composite cardiovascular disease were: twofold for pre-eclampsia, stillbirth, and preterm birth; 1.5-1.9-fold for gestational hypertension, placental abruption, gestational diabetes, and premature ovarian insufficiency; and less than 1.5-fold for early menarche, polycystic ovary syndrome, ever parity, and early menopause. A longer length of breastfeeding was associated with a reduced risk of cardiovascular disease. The associations for ischaemic heart disease were twofold or greater for pre-eclampsia, recurrent pre-eclampsia, gestational diabetes, and preterm birth; 1.5-1.9-fold for current use of combined oral contraceptives (oestrogen and progesterone), recurrent miscarriage, premature ovarian insufficiency, and early menopause; and less than 1.5-fold for miscarriage, polycystic ovary syndrome, and menopausal symptoms. For stroke outcomes, the associations were twofold or more for current use of any oral contraceptive (combined oral contraceptives or progesterone only pill), pre-eclampsia, and recurrent pre-eclampsia; 1.5-1.9-fold for current use of combined oral contraceptives, gestational diabetes, and preterm birth; and less than 1.5-fold for polycystic ovary syndrome. The association for heart failure was fourfold for pre-eclampsia. No association was found between cardiovascular disease outcomes and current use of progesterone only contraceptives, use of non-oral hormonal contraceptive agents, or fertility treatment.

CONCLUSIONS

From menarche to menopause, reproductive factors were associated with cardiovascular disease in women. In this review, presenting absolute numbers on the scale of the problem was not feasible; however, if these associations are causal, they could account for a large proportion of unexplained risk of cardiovascular disease in women, and the risk might be modifiable. Identifying reproductive risk factors at an early stage in the life of women might facilitate the initiation of strategies to modify potential risks. Policy makers should consider incorporating reproductive risk factors as part of the assessment of cardiovascular risk in clinical guidelines.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42019120076.

Examining overlap of included studies in meta-reviews: Guidance for using the corrected covered area index

Overlap in meta-reviews results from the use of multiple identical primary studies in similar reviews. It is an important area for research synthesists because overlap indicates the degree to which reviews address the same or different literatures of primary research. Current guidelines to address overlap suggest that assessing and documenting the degree of overlap in primary studies, calculated via the corrected covered area (CCA) is a promising method. Yet, the CCA is a simple percentage of overlap and current guidelines do not detail ways that reviewers can use the CCA as a diagnostic tool while also comprehensively incorporating these findings into their conclusions. Furthermore, we maintain that meta-review teams must address non-independence via overlap more thoroughly than by simply estimating and reporting the CCA. Instead, we recommend and elaborate five steps to take when examining overlap, illustrating these steps through the use of an empirical example of primary study overlap in a recently conducted meta-review. This work helps to show that overlap of primary studies included in a meta-review is not necessarily a bias but often can be a benefit. We also highlight further areas of caution in this task and potential for the development of new tools to address non-independence issues.

Effects of E-Learning in a Continuing Education Context on Nursing Care: Systematic Review of Systematic Qualitative, Quantitative, and Mixed-Studies Reviews

Background

E-learning is rapidly growing as an alternative way of delivering education in nursing. Two contexts regarding the use of e-learning in nursing are discussed in the literature: (1) education among nursing students and (2) nurses’ continuing education within a life-long learning perspective. A systematic review of systematic reviews on e-learning for nursing and health professional students in an academic context has been published previously; however, no such review exists regarding e-learning for registered nurses in a continuing education context.

Objective

We aimed to systematically summarize the qualitative and quantitative evidence regarding the effects of e-learning on nursing care among nurses in a continuing education context.

Methods

We conducted a systematic review of systematic qualitative, quantitative, and mixed-studies reviews, searching within four bibliographic databases. The eligibility criteria were formulated using the population, interventions, comparisons, outcomes, and study design (PICOS) format. The included population was registered nurses. E-learning interventions were included and compared with face-to-face and any other e-learning interventions, as well as blended learning. The outcomes of interest were derived from two models: nursing-sensitive indicators from the Nursing Care Performance Framework (eg, teaching and collaboration) and the levels of evaluation from the Kirkpatrick model (ie, reaction, learning, behavior, and results).

Results

We identified a total of 12,906 records. We retrieved 222 full-text papers for detailed evaluation, from which 22 systematic reviews published between 2008 and 2018 met the eligibility criteria. The effects of e-learning on nursing care were grouped under Kirkpatrick’s levels of evaluation: (1) nurse reactions to e-learning, (2) nurse learning, (3) behavior, and (4) results. Level 2, nurse learning, was divided into three subthemes: knowledge, skills, attitude and self-efficacy. Level 4, results, was divided into patient outcomes and costs. Most of the outcomes were reported in a positive way. For instance, nurses were satisfied with the use of e-learning and they improved their knowledge. The most common topics covered by the e-learning interventions were medication calculation, preparation, and administration.

Conclusions

The effects of e-learning are mainly reported in terms of nurse reactions, knowledge, and skills (ie, the first two levels of the Kirkpatrick model). The effectiveness of e-learning interventions for nurses in a continuing education context remains unknown regarding how the learning can be transferred to change practice and affect patient outcomes. Further scientific, methodological, theoretical, and practice-based breakthroughs are needed in the fast-growing field of e-learning in nursing education, especially in a life-learning perspective.

Trial Registration

International Prospective Register of Systematic Reviews (PROSPERO) CRD42016050714; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=50714

A decision tool to help researchers make decisions about including systematic reviews in overviews of reviews of healthcare interventions

BACKGROUND

Overviews of reviews of healthcare interventions (overviews) integrate information from multiple systematic reviews (SRs) to provide a single synthesis of relevant evidence for decision-making. Overviews may identify multiple SRs that examine the same intervention for the same condition and include some, but not all, of the same primary studies. Different researchers use different approaches to manage these "overlapping SRs," but each approach has advantages and disadvantages. This study aimed to develop an evidence-based decision tool to help researchers make informed inclusion decisions when conducting overviews of healthcare interventions.

METHODS

We used a two-stage process to develop the decision tool. First, we conducted a multiple case study to obtain empirical evidence upon which the tool is based. We systematically conducted seven overviews five times each, making five different decisions about which SRs to include in the overviews, for a total of 35 overviews; we then examined the impact of the five inclusion decisions on the overviews' comprehensiveness and challenges, within and across the seven overview cases. Second, we used a structured, iterative process to transform the evidence obtained from the multiple case study into an empirically based decision tool with accompanying descriptive text.

RESULTS

The resulting decision tool contains four questions: (1) Do Cochrane SRs likely examine all relevant intervention comparisons and available data? (2) Do the Cochrane SRs overlap? (3) Do the non-Cochrane SRs overlap? (4) Are researchers prepared and able to avoid double-counting outcome data from overlapping SRs, by ensuring that each primary study's outcome data are extracted from overlapping SRs only once? Guidance is provided to help researchers answer each question, and empirical evidence is provided regarding the advantages, disadvantages, and potential trade-offs of the different inclusion decisions.

CONCLUSIONS

This evidence-based decision tool is designed to provide researchers with the knowledge and means to make informed inclusion decisions in overviews. The tool can provide practical guidance and support for overview authors by helping them consider questions that could affect the comprehensiveness and complexity of their overviews. We hope this tool will be a useful resource for researchers conducting overviews, and we welcome discussion, testing, and refinement of the proposed tool.