Methodological approaches for assessing certainty of the evidence in umbrella reviews: A scoping review

Characterising umbrella reviews in paediatrics: A systematic analysis

AIM

To characterise typical practices in conducting and reporting umbrella reviews in paediatrics.

METHODS

PubMed database was searched for umbrella reviews focusing on children (0-18 years) published from 1 January 2020 to 15 June 2024, regardless of the speciality. The included studies were assessed for reporting practices, result summarisation and critical appraisal methods.

RESULTS

Ninety-nine eligible umbrella reviews were included, of which 49 focused on interventions and 50 focused on epidemiology. The median number of included reviews in the umbrella reviews was 18 (range = 3-302), and statistical synthesis was performed in 32 studies. The most frequently used tool for critical appraisal of the included reviews was the updated Assessing the Methodological Quality of Systematic Reviews (AMSTAR-2) tool (n = 56); the original AMSTAR-1 tool was used in seven reviews, and the Risk of Bias in Systematic Reviews tool was used in 12 reviews. Evidence certainty was assessed using the Grading of Recommendations Assessment, Development and Evaluation framework in 14 reviews and by the Ioannidis criteria in 18 reviews.

CONCLUSION

Umbrella reviews have heterogeneous characteristics in terms of reporting practices. Greater adherence to guidelines on conducting and reporting umbrella reviews is needed, particularly for improving critical appraisal methods.

Risk factors for Peri-implantitis: An umbrella review of meta-analyses of observational studies and assessment of biases

OBJECTIVES

To perform a comprehensive quantitative and qualitative analysis of the findings from previously published meta-analyses and to assess existing biases.

DATA/SOURCES

A search was conducted for meta-analyses of observational studies investigating the association between any risk factor and peri‑implantitis in PubMed, Scopus, Cochrane Database of Systematic Reviews, and Epistemonikos, from inception until October 2023 (PROSPERO: CRD42024512408).

STUDY SELECTION

From a total of 5002 publications, 51 full-text articles were evaluated for eligibility, and 12 articles that described 41 unique meta-analyses evaluating the association between risk factors and periimplantitis were selected. Among 41 associations, 24 associations were significant. None of the associations were graded as convincing evidence. Two associations, presence of periodontitis (OR = 3.84 [95 % CI 2.58,5.72]) and cigarette smoking (RR=2.07 [95 % CI 1.41,3.04]) were graded as highly suggestive. Eight associations, diabetes mellitus, hyperglycaemia, lack of prophylaxis, history of chronic periodontal disease, ongoing or history of periodontal disease, implants located in the anterior region of the jaw (maxillary and mandibular), osteoprotegerin (OPG) gene polymorphisms, and lack of keratinized mucosal width were graded as suggestive evidence.

CONCLUSIONS

Periodontitis and cigarette smoking are highly suggestive risk factors for peri‑implantitis. The remaining risk factors which are suggestive require more high-quality studies to be performed to upgrade the level of evidence.

CLINICAL SIGNIFICANCE

The highly suggestive and suggestive risk factors for peri‑implantitis summarized in this umbrella review should be rigorously assessed, monitored and managed by clinicians to reduce the risk peri‑implantitis, as well as to form part of the preoperative consent process.

Identifying competencies in advanced healthcare practice: an umbrella review

The four pillars of advanced healthcare practice (AHCP) are clinical practice, leadership and management, education, and research. It is unclear, however; how competencies of AHCP as defined by individual health professions relate to these pillars. Addressing this knowledge gap will help to facilitate the operationalization of AHCP as a concept and help inform educational curricula. To identify existing competencies across AHCP literature and examine how they relate to the four pillars of a multi-professional AHCP framework. An umbrella review was conducted in accordance with JBI methodology. The electronic search for published and grey literature was completed using CINAHL, Scopus, Medline (OVID), Embase (OVID), ERIC (OVID) and Google. Secondary reviews and research syntheses of master level AHCP programs published after 1990 in either English or French were considered for inclusion and results were analyzed using a directed content analysis. Seventeen publications detailing 620 individual competencies were included. AHCP competencies were described across four professions and 22 countries, with many publications related to nursing and AHCP in the United Kingdom, Canada, and Australia. Many retrieved competencies were found to map to the four pillars of AHCP, although clinical practice and leadership and management pillars were addressed more often. Competencies of AHCP are generally consistent with the four pillars. However, the distribution of competencies is unequal across pillars, professions, and geographical regions, which may provide direction for further research. Doi: 10.17605/OSF.IO/KV2FD Published on March 07, 2023.

Timing, Indicators, and Approaches to Digital Patient Experience Evaluation: Umbrella Systematic Review

BACKGROUND

The increasing prevalence of DH applications has outpaced research and practice in digital health (DH) evaluations. Patient experience (PEx) was reported as one of the challenges facing the health system by the World Health Organization. To generate evidence on DH and promote the appropriate integration and use of technologies, a standard evaluation of PEx in DH is required.

OBJECTIVE

This study aims to systematically identify evaluation timing considerations (ie, when to measure), evaluation indicators (ie, what to measure), and evaluation approaches (ie, how to measure) with regard to digital PEx. The overall aim of this study is to generate an evaluation guide for further improving digital PEx evaluation.

METHODS

This is a 2-phase study parallel to our previous study. In phase 1, literature reviews related to PEx in DH were systematically searched from Scopus, PubMed, and Web of Science databases. Two independent raters conducted 2 rounds of paper screening, including title and abstract screening and full-text screening, and assessed the interrater reliability for 20% (round 1: 23/115 and round 2: 12/58) random samples using the Fleiss-Cohen coefficient (round 1: k1=0.88 and round 2: k2=0.80). When reaching interrater reliability (k>0.60), TW conducted the rest of the screening process, leaving any uncertainties for group discussions. Overall, 38% (45/119) of the articles were considered eligible for further thematic analysis. In phase 2, to check if there were any meaningful novel insights that would change our conclusions, we performed an updated literature search in which we collected 294 newly published reviews, of which 102 (34.7%) were identified as eligible articles. We considered them to have no important changes to our original results on the research objectives. Therefore, they were not integrated into the synthesis of this review and were used as supplementary materials.

RESULTS

Our review highlights 5 typical evaluation objectives that serve 5 stakeholder groups separately. We identified a set of key evaluation timing considerations and classified them into 3 categories: intervention maturity stages, timing of the evaluation, and timing of data collection. Information on evaluation indicators of digital PEx was identified and summarized into 3 categories (intervention outputs, patient outcomes, and health care system impact), 9 themes, and 22 subthemes. A set of evaluation theories, common study designs, data collection methods and instruments, and data analysis approaches was captured, which can be used or adapted to evaluate digital PEx.

CONCLUSIONS

Our findings enabled us to generate an evaluation guide to help DH intervention researchers, designers, developers, and program evaluators evaluate digital PEx. Finally, we propose 6 directions for encouraging further digital PEx evaluation research and practice to address the challenge of poor PEx.

Research integrity in randomized clinical trials: A scoping umbrella review

BACKGROUND

Randomized clinical trials (RCTs) are experiencing a crisis of confidence in their trustworthiness. Although a comprehensive literature search yielded several reviews on RCT integrity, an overarching overview is lacking.

OBJECTIVES

The authors undertook a scoping umbrella review of the research integrity literature concerning RCTs.

SEARCH STRATEGY AND SELECTION CRITERIA

Following prospective registration (https://osf.io/3ursn), two reviewers independently searched PubMed, Scopus, The Cochrane Library, and Google Scholar, without language or time restrictions, until November 2021. The authors included systematic reviews covering any aspect of research integrity throughout the RCT lifecycle.

DATA COLLECTION AND ANALYSIS

The authors assessed methodological quality using a modified AMSTAR 2 (A Measurement Tool to Assess Systematic Reviews) tool and collated the main findings.

MAIN RESULTS

A total of 55 relevant reviews, summarizing 6001 studies (median per review, 63; range, 8-1106) from 1964 to 2021, had an overall critically low quality of 96% (53 reviews). Topics covered included general aspects (15%), design and approval (22%), conduct and monitoring (11%), reporting (38%), postpublication concerns (2%), and future research (13%). The most common integrity issues covered were ethics (18%) and transparency (18%).

CONCLUSIONS

Low-quality reviews identified various integrity issues across the RCT lifecycle, emphasizing the importance of high ethical standards and professionalism while highlighting gaps in the integrity landscape. Multistakeholder consensus is needed to develop specific RCT integrity standards.

Prenatal exposure to ambient air pollutants and congenital heart defects: An umbrella review

BACKGROUND

Prenatal exposure to ambient air pollutants has been linked to congenital heart defects (CHD), but findings of existing systematic reviews have been mixed.

OBJECTIVE

To assess the epidemiological evidence on associations between prenatal exposure to ambient air pollutants and CHD subtypes, based on a systematic overview of reviews ("umbrella review").

METHODS

We conducted a systematic search for reviews assessing associations between prenatal exposure to criteria air pollutants and CHD. The risk of bias was evaluated using the Risk of Bias in Systematic Reviews (ROBIS) tool. The certainty of the systematic review findings was graded using the Navigation Guide methodology.

RESULTS

We identified eleven systematic reviews, including eight with meta-analyses, assessing in total 35 primary studies of prenatal exposure to criteria air pollutants and various CHD subtypes. The certainty of the findings of four meta-analyses indicating an increased risk for coarctation of the aorta associated with nitrogen dioxide exposure was rated as moderate. The certainty of findings indicating positive, inverse, or null associations for other pollutant-subtype combinations was rated as very low to low, based on low precision and high statistical heterogeneity of summary odds ratios (SOR), substantial inconsistencies between review findings, and methodological limitations of the systematic reviews.

DISCUSSION

The inconsistent findings and high statistical heterogeneity of many SOR of the included systematic reviews may partly be traced to differences in methodological approaches, and the risk of bias across included reviews (e.g., inclusion criteria, systematic search strategies, synthesis methods) and primary studies (e.g., exposure assessment, diagnostic criteria). Adherence to appropriate systematic review guidelines for environmental health research, as well as rigorous evaluation of risk of bias in primary studies, are essential for future risk assessments and policy-making. Still, our findings suggest that prenatal exposure to ambient air pollutants may increase risks for at least some CHD subtypes.

Family Mealtimes: A Systematic Umbrella Review of Characteristics, Correlates, Outcomes and Interventions

Systematic reviews have examined the multitude of studies investigating family mealtimes and their importance to child/adolescent health and psychosocial outcomes, but the focus of each is limited to specific aspects of family meals (e.g., frequency) and/or specific outcomes (e.g., nutrition). Their findings require synthesis and so a systematic umbrella review was undertaken. Databases were searched to identify systematic reviews (with or without meta-analysis/meta-synthesis) addressing at least one of the following questions: what are the characteristics and/or correlates of family mealtimes; what outcomes are associated with family mealtimes; are interventions aimed at promoting family mealtimes effective? Forty-one eligible reviews were retrieved. Their findings demonstrate that families with children/adolescents typically eat together at least a few days each week. More frequent family meals are predicted by a more positive mealtime environment, more positive attitudes towards family meals, the presence of younger children, and families having more time. Greater family meal frequency protects children/adolescents against a poorer diet, obesity, risk behaviours, poorer mental health and wellbeing, and poorer academic outcomes. Findings from interventions seeking to promote family mealtimes are mixed. This umbrella review provides a comprehensive and integrated understanding of research into family mealtimes, establishing where evidence is sound and where further research is needed.

Global prevalence of developmental disabilities in children and adolescents: A systematic umbrella review

Aim

The provisions of the United Nation's Sustainable Development Goals (SDGs) for disability-inclusive education have stimulated a growing interest in ascertaining the prevalence of children with developmental disabilities globally. We aimed to systematically summarize the prevalence estimates of developmental disabilities in children and adolescents reported in systematic reviews and meta-analyses.

Methods

For this umbrella review we searched PubMed, Scopus, Embase, PsycINFO, and Cochrane Library for systematic reviews published in English between September 2015 and August 2022. Two reviewers independently assessed study eligibility, extracted the data, and assessed risk of bias. We reported the proportion of the global prevalence estimates attributed to country income levels for specific developmental disabilities. Prevalence estimates for the selected disabilities were compared with those reported in the Global Burden of Disease (GBD) Study 2019.

Results

Based on our inclusion criteria, 10 systematic reviews reporting prevalence estimates for attention-deficit/hyperactivity disorder, autism spectrum disorder, cerebral palsy, developmental intellectual disability, epilepsy, hearing loss, vision loss and developmental dyslexia were selected from 3,456 identified articles. Global prevalence estimates were derived from cohorts in high-income countries in all cases except epilepsy and were calculated from nine to 56 countries. Sensory impairments were the most prevalent disabilities (approximately 13%) and cerebral palsy was the least prevalent disability (approximately 0.2-0.3%) based on the eligible reviews. Pooled estimates for geographical regions were available for vision loss and developmental dyslexia. All studies had a moderate to high risk of bias. GBD prevalence estimates were lower for all disabilities except cerebral palsy and intellectual disability.

Conclusion

Available estimates from systematic reviews and meta-analyses do not provide representative evidence on the global and regional prevalence of developmental disabilities among children and adolescents due to limited geographical coverage and substantial heterogeneity in methodology across studies. Population-based data for all regions using other approaches such as reported in the GBD Study are warranted to inform global health policy and intervention.

Introduction to Umbrella Reviews as a Useful Evidence-Based Practice

The umbrella review (UR) is a novel methodological approach that has been established to address the ever-expanding research volume of published systematic reviews. In this study, we examined the rationale underlying URs, the concepts and goals of URs, and their applicability in clinical settings. Additionally, we briefly assessed the process of conducting URs and discussed the current challenges in this regard. URs are used to integrate, evaluate, and synthesize the findings of related systematic reviews. By organizing and summarizing the abundant information in accordance with the level of evidence, URs can serve as a useful methodological tool and provide appropriate entry points to clinicians or decision-makers in the medical field. Considering the availability of many suitable interventions for specific conditions in a broad field, URs can enable evidence-based decision-making and offer a broad perspective for the resolution of issues in healthcare by summarizing the evidence and providing directions on a variety of topics. URs are clearly contributing to the management of the deluge of evidence in evidence-based medicine. However, despite the availability of several directions for conducting URs, some points of confusion persist, especially when determining the certainty of evidence. Therefore, advanced guidelines for the appropriate performance of URs are required to provide more reliable evidence through URs.

Autism Spectrum as an Etiologic Systemic Disorder: A Protocol for an Umbrella Review

Autism spectrum disorder (ASD) is the most common neurodevelopmental disorder with a huge prevalence increasing every year (1/44 children). Still diagnosed as a mental disorder, the last 10 years of research found possible causes, risks, genetics, environmental triggers, epigenetics, metabolic, immunological, and neurophysiological unbalances as relevant aetiology. Umbrella methodology is the highest level of scientific evidence, designed to support clinical and political decisions. A literature search for autism aetiology, pathophysiology, or causes, conducted in the last 10 years, at PubMed, Embase, Cochrane, Scopus, and the Web of Science, resulted in six umbrella reviews. Nevertheless, only one quantitative analysis reported risk factors and biomarkers but excluded genetics, experiments on animal models, and post-mortem studies. We grouped ASD’s multi-factorial causes and risks into five etiological categories: genetic, epigenetic, organic, psychogenic, and environmental. Findings suggest that autism might be evaluated as a systemic disorder instead of only through the lens of mental and behavioural. The overview implications of included studies will be qualitatively analysed under ROBIS and GRADE tools. This umbrella review can provide a rational basis for a new urgent health policy to develop better and adequate integrated care services for ASD. The methodological protocol has the register CRD42022348586 at PROSPERO.