Methodological quality assessment of paper-based systematic reviews published in oral health

Pediatric dentistry systematic reviews using the GRADE approach: methodological study

BACKGROUND

To assess the reporting of the certainty of the evidence using the GRADE approach in systematic reviews of interventions in pediatric dentistry.

METHODS

The inclusion criteria were systematic reviews of randomized clinical trials (RCTs) and non-randomized studies of interventions (NRSIs) in pediatric dentistry that reported the certainty of the evidence through the GRADE approach. Paired independent reviewers screened the studies, extracted data, and appraised the methodological quality using the Assessing the Methodological Quality of Systematic Reviews (AMSTAR 2) tool. The certainty of the evidence was extracted for each outcome. A descriptive analysis was conducted.

RESULTS

Around 28% of pediatric dentistry reviews of interventions used the GRADE approach (n = 24). Twenty reviews reported 112 evidence outcomes from RCTs and 13 from NRSIs using GRADE evidence profile tables. The methodological quality was high (16.7%), moderate (12.5%), low (37.5%), and critically low (33.3%), fulfilling the majority of the AMSTAR 2 criteria. The certainty of the evidence for outcomes generated from RCTs and NRSIs was very low (40.2% and 84.6%), low (33.1% and 7.7%), moderate (17.8% and 7.7%), and high (9.8% and 0.0%). The main reasons to downgrade the certainty were due to (for RCTs and NRSIs, respectively): risk of bias (68.8% and 84.6%), imprecision (67.8% and 100.0%), inconsistency (18.8% and 23.1%), indirectness (17.8% and 0.0%), and publication bias (7.1% and 0.0%).

CONCLUSION

The proportion of systematic reviews assessing the certainty of the evidence using the GRADE approach was considered small, considering the total initial number of published pediatric dentistry reviews of intervention. The certainty of the evidence was mainly very low and low, and the main problems for downgrading the certainty of evidence were due to risk of bias and imprecision.

REGISTRATION

PROSPERO database #CRD42022365443.

Methodological quality of systematic reviews in dentistry including animal studies: a cross-sectional study

BACKGROUND

The overall confidence in the results of systematic reviews including animal models can be heterogeneous. We assessed the methodological quality of systematic reviews including animal models in dentistry as well as the overall confidence in the results of those systematic reviews.

MATERIAL & METHODS

PubMed, Web of Science and Scopus were searched for systematic reviews including animal studies in dentistry published later than January 2010 until 18th of July 2022. Overall confidence in the results was assessed using a modified version of the A MeaSurement Tool to Assess systematic Reviews (AMSTAR-2) checklist. Checklist items were rated as yes, partial yes, no and not applicable. Linear regression analysis was used to investigate associations between systematic review characteristics and the overall adherence to the AMSTAR-2 checklist. The overall confidence in the results was calculated based on the number of critical and non-critical weaknesses presented in the AMSTAR-2 items and rated as high, moderate, low and critical low.

RESULTS

Of initially 951 retrieved systematic reviews, 190 were included in the study. The overall confidence in the results was low in 43 (22.6%) and critically low in 133 (70.0%) systematic reviews. While some AMSTAR-2 items were regularly reported (e.g. conflict of interest, selection in duplicate), others were not (e.g.

FUNDING

n = 1; 0.5%). Multivariable linear regression analysis showed that the adherence scores of AMSTAR-2 was significantly associated with publication year, journal impact factor (IF), topic, and the use of tools to assess risk of bias (RoB) of the systematic reviews.

CONCLUSION

Although the methodological quality of dental systematic reviews of animal models improved over the years, it is still suboptimal. The overall confidence in the results was mostly low or critically low. Systematic reviews, which were published later, published in a journal with a higher IF, focused on non-surgery topics, and used at least one tool to assess RoB correlated with greater adherence to the AMSTAR-2 guidelines.

Methodological assessment and overall confidence in the results of systematic reviews with meta-analyses focusing on traumatic dental injuries: A cross-sectional study

BACKGROUND/AIMS

High methodological quality is required to interpret results of systematic reviews (SRs) in a reliable and accurate manner. The primary aim of this study was to appraise the methodologic quality of SRs with meta-analysis within the field of traumatic dental injuries using the A MeaSurement Tool to Assess systematic Reviews (AMSTAR) 2 tool and assess overall confidence in their results. A secondary aim was to identify potential predictive factors associated with methodological quality.

MATERIALS AND METHODS

SRs with meta-analyses published in English in the field of traumatic dental injuries from inception to March 2023 were identified. The methodological quality of the included reviews was assessed using the AMSTAR 2 checklist. Two independent evaluators scored each AMSTAR 2 item as "yes" if it was adequately addressed, "partial yes" if it was partially addressed, and "no" if it was not addressed. The overall confidence in the results of each review was classified as "High," "Moderate," "Low," or "Critically low." Using multiple regression, the relationship between five predictor variables (journal impact factor, year of publication, number of authors, journal adherence to Preferred Reporting Items for Systematic reviews and Meta-analyses [PRISMA] guidelines and a priori protocol registration) and the total AMSTAR 2 scores was analyzed. The p-value was 5%.

RESULTS

Forty-one SRs were included. The overall confidence in the results of 13 reviews was categorized as "Critically low," 18 as "Low," 3 as "Moderate" and 7 as "High." Among the five predictor variables analyzed statistically, impact factor of the journal and year of publication significantly influenced the total AMSTAR 2 scores. The number of authors, adherence to PRISMA guidelines, and a priori protocol registration had no significant impact on AMSTAR 2 scores.

CONCLUSION

The overall confidence in the results of SRs with meta-analysis within the field of traumatic dental injuries was "Low" or "Critically Low" in the vast majority of studies (31 of 41). SRs with meta-analyses published in journals with higher impact factors and more recent publications had significantly higher methodological quality.

Methodological assessment and overall confidence in the results of systematic reviews with network meta-analyses in Endodontics

AIM

The aims of the study were to assess the methodological quality of systematic reviews with network meta-analyses (NMAs) in Endodontics using the "A MeaSurement Tool to Assess systematic Reviews" (AMSTAR 2) tool, and to evaluate the overall confidence in the results of the individual reviews included in the analysis.

METHODOLOGY

Systematic reviews with NMAs within the specialty of Endodontics published in English were identified from the PubMed, EbBSCOhost and SCOPUS databases from inception to July 2021. Two reviewers were involved independently in the selection of the reviews, data extraction, methodological quality assessment and overall confidence rating. Disagreements were resolved by discussion between the reviewers to achieve consensus; if disagreements persisted, a third reviewer made the final decision. The methodological quality of the included NMAs was appraised using the AMSTAR 2 checklist, which contains 16 items. The reviewers scored each item - 'Yes' - when the item was fully addressed, 'Partial Yes' - when the item was not fully addressed, or 'No' - when the item was not addressed. The overall confidence in the results of each review was classified as 'High', 'Moderate', 'Low' or 'Critically low' based on the criteria reported by the AMSTAR 2 developers.

RESULTS

Twelve systematic reviews with NMAs were included. All the NMAs adequately reported Item 1 ("Did the research questions and inclusion criteria for the review include the components of PICO?"), Item 8 ("Did the review authors describe the included studies in adequate detail?"), Item 9 ("Did the review authors use a satisfactory technique for assessing the risk of bias (RoB) in individual studies that were included in the review?") and Item 16 ("Did the review authors report any potential sources of conflict of interest, including any funding they received for conducting the review?") , whereas only one NMA reported Item 10 adequately ("Did the review authors report on the sources of funding for the studies included in the review?"). The overall confidence in the results of eight reviews was categorised as "Critically low", one review was "Low", two reviews were "Moderate" and one review was "High".

CONCLUSION

The overall confidence in the results for the majority of systematic reviews with NMAs in Endodontics was judged to be 'Critically low' as their methodological quality was below the necessary standard. AMSTAR 2 and PRISMA for NMA guidelines are available to guide authors to produce high quality systematic reviews with NMAs and for Editors and peer-reviewers when assessing submissions to journals.

There is still room for improvement in the completeness of abstract reporting according to the PRISMA-A checklist: a cross-sectional study on systematic reviews in periodontology

BACKGROUND

To evaluate the completeness of reporting abstracts of systematic reviews (SRs) before and after the publication of the PRISMA-A checklist in 2013 and to assess if an association exists between abstract characteristics and the completeness of reporting.

METHODS

A systematic search of the literature was conducted in the PubMed and Scopus databases in March 2020. The search focused on the SRs of evaluations of interventions published since 2002 in the field of periodontology. The abstracts of the selected SRs were divided into two groups before and after publication of the PRISMA-A checklist in 2013, and compliance with the 12 items reported in the checklist was evaluated by three calibrated evaluators.

RESULTS

A set of 265 abstracts was included in the study. The total score before (mean score, 53.78%; 95% CI, 51.56-55.90%) and after (mean score, 56.88%; 95% CI, 55.39-58.44%) the publication of the PRISMA-A statement exhibited a statistically significant improvement (P = 0.012*). Nevertheless, only the checklist items included studies and synthesis of the results displayed a statistically significant change after guideline publication. The total PRISMA-A score was higher in the meta-analysis group and in articles authored by more than four authors.

CONCLUSIONS

The impact of the PRISMA-A was statistically significant, but the majority of the items did not improve after its introduction. The editors and referees of periodontal journals should promote adherence to the checklist to improve the quality of the reports and provide readers with better insight into the characteristics of published studies.

Critical analysis of methodological quality of systematic reviews and meta-analysis of antibiotics in third molar surgeries using AMSTAR 2

The clinicians usually prescribe antibiotics to reduce post-operative complications during third molar surgeries. However, in the absence of clear conclusions regarding the use of antibiotics in third molar surgeries, present systematic review was planned to assess the quality of systematic reviews evaluating the efficiency of antibiotics in reducing post-operative complications. The literature search was done in Cochrane Library, Cochrane Central Register of Controlled Trials (CENTRAL), PUBMED, EMBASE, and Google scholar. Systematic reviews published in English during the period from January 1990 to December 2019 were included. The maxillary and mandibular third molars indicated for extraction either because of infection, orthodontic or prophylactic reasons were included. From 526 screened studies, thirteen reviews were qualified for qualitative analysis. The qualities of the included reviews were evaluated using the AMSTAR 2 tool. The included reviews were also evaluated based on the number of authors, geographic region, impact factor of the published journal, year of publication, and the number of citations for each review. One high quality, eight moderate quality, three low quality, and one critically low-quality reviews were observed in the present review. No statistically significant difference was observed between the included reviews based upon the analysis of the number of authors, geographic region, impact factor of the published journal, year of publication, and the number of citations for each review. Considering the observations form the high and moderate-quality reviews, the present systematic review concludes that antibiotics effectively aid in reducing the post-operative complications and frequency of observation of dry socket.

Barriers involved in the application of evidence-based dentistry principles: A systematic review

BACKGROUND

The authors' objective in this systematic review was to investigate the barriers involved in the application of evidence-based dentistry principles, as reported by dentists. The authors registered the protocol in the PROSPERO database.

TYPES OF STUDIES REVIEWED

Eligible studies included qualitative and quantitative approaches, constituting information about barriers, collected through interviews, questionnaires, or conversation sessions. The authors searched databases and reference lists of preselected studies. After the selection process, the authors evaluated the included studies for potential risk of bias and collected either qualitative or quantitative data.

RESULTS

After the selection process, the authors included 35 studies, of which 16 were reported in this article. The authors synthesized and classified the barriers in 4 categories: self-related, evidence-related, context-related, and patient-related barriers. Shortage of time and financial constraints were the barriers most frequently studied. However, the quantification of these barriers, as well as others, was not possible because of the variability of the results and methodological issues of the included studies.

CONCLUSIONS AND PRACTICAL IMPLICATIONS

The authors suggest the development of valid questionnaires and their use in representative samples to quantify the effects of specific barriers. The authors encourage practitioners to participate in educational programs focused on training in evidence-based dentistry abilities, in addition to seeking accessible and synthesized formats of reliable scientific knowledge.

Systematic reviews in orthodontics: Impact of the PRISMA for Abstracts checklist on completeness of reporting

INTRODUCTION

This study evaluated and compared the completeness of reporting of abstracts of orthodontics systematic reviews before and after the publication of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension for Abstracts Checklist (PRISMA-A).

METHODS

Abstracts of systematic reviews and meta-analyses in orthodontics published in PubMed, Latin American and Caribbean Health Sciences Literature, and the Cochrane Database of Systematic Reviews databases before March 23, 2018, that met the predefined inclusion and exclusion criteria, were evaluated using the 12 items of PRISMA-A, scoring each item from 0 to 2. Abstracts were classified into 2 groups: before and after publication of the PRISMA-A checklist. Three calibrated evaluators (intraclass correlation coefficient and kappa > 0.8) assessed the scores for compliance with the checklist. The number of authors, country of affiliation of the first author, performance of meta-analysis, and topic of the article were recorded. A regression analysis was performed to assess the associations between abstract characteristics and the PRISMA-A scores.

RESULTS

Of 1034 abstracts evaluated, 389 were included in the analysis. The mean PRISMA-A score was 53.39 (95% CI, 51.83-54.96). The overall score for studies published after the publication of the checklist was significantly higher than for studies published before (P ≤ 0.0001). The components returning significantly higher scores after publication of PRISMA-A were title (P = 0.024), information from databases (P = 0.026), risk of bias (P ≤ 0.0001), included studies (P ≤ 0.0001), synthesis of results (P ≤ 0.0001), interpretation of results (P = 0.035), financing and conflict of interest (P ≤ 0.0001), and registration (P ≤ 0.0001). These results showed the positive effect of PRISMA-A had on the quality of reporting of orthodontics systematic reviews. Nevertheless, the poor adherence revealed that there is still need for improvement in the quality of abstract reporting.

CONCLUSIONS

The quality of reporting of abstracts of orthodontic systematic reviews and meta-analyses increased after the introduction of PRISMA-A.

Critical appraisal of methodological quality of Systematic Reviews and Meta-analysis in Paediatric Dentistry journals

OBJECTIVE

To systematically assess the methodological quality of Systematic Reviews (SRs) and Meta-Analyses (MA) published in Paediatric Dentistry journals and to analyse the relationship between the authors, journals, country, review topic, and the year of publication to the methodological quality of SRs and MA.

DESIGN

Paediatric Dentistry journals ranked in the top five of the h5 index of Google Scholar Metrics were selected. SRs with MA were searched independently by two reviewers using PubMed and Scopus databases until December 2017. Methodological quality was assessed using A Measurement Tool to Assess Systematic Reviews (AMSTAR) tool. Statistical significance was set at P < 0.05 and Mann-Whitney U test and Kruskal-Wallis test was employed for comparing the AMSTAR score with the journal characteristics.

RESULTS

Finally, 24 SRs with MA were included. The overall AMSTAR score of SRs and MA published in paediatric dentistry journals was 7.08 ± 2.41. No statistically significant differences were found between the country, journal or focus of study to the quality of SRs except the number of authors and the year of publication (P < 0.05).

CONCLUSIONS

The quality of SRs and MA in leading Paediatric Dentistry journals were evaluated with AMSTAR tool and areas where quality could be improved were identified.

Methodological and Reporting Quality of Systematic Reviews and Meta-analyses in Endodontics

INTRODUCTION

The aim of this systematic review (SR) was to evaluate the quality of SRs and meta-analyses (MAs) in endodontics.

METHODS

A comprehensive literature search was conducted to identify relevant articles in the electronic databases from January 2000 to June 2017. Two reviewers independently assessed the articles for eligibility and data extraction. SRs and MAs on interventional studies with a minimum of 2 therapeutic strategies in endodontics were included in this SR. Methodologic and reporting quality were assessed using A Measurement Tool to Assess Systematic Reviews (AMSTAR) and Preferred Reporting Items for Systematic Review and Meta-Analyses (PRISMA), respectively. The interobserver reliability was calculated using the Cohen kappa statistic. Statistical analysis with the level of significance at P < .05 was performed using Kruskal-Wallis tests and simple linear regression analysis.

RESULTS

A total of 30 articles were selected for the current SR. Using AMSTAR, the item related to the scientific quality of studies used in conclusion was adhered by less than 40% of studies. Using PRISMA, 3 items were reported by less than 40% of studies, which were on objectives, protocol registration, and funding. No association was evident comparing the number of authors and country with quality. Statistical significance was observed when quality was compared among journals, with studies published as Cochrane reviews superior to those published in other journals. AMSTAR and PRISMA scores were significantly related.

CONCLUSIONS

SRs in endodontics showed variability in both methodologic and reporting quality.

Quality assessment of systematic reviews on total hip or knee arthroplasty using mod-AMSTAR

Background

Increasing numbers of systematic reviews (SRs) on total knee arthroplasty (TKA) and total hip arthroplasty (THA) have been published in recent years, but their quality has been unclear. The purpose of this study is to evaluate the methodological quality of SRs on TKA and THA.

Methods

We searched Ovid-Medline, Ovid-Embase, Cochrane Databases (including HTA, DARE, and CDSR), CBM, CNKI, Wang Fang, and VIP, from January 2014 to December 2015 for THA and TKA. The quality of SRs was assessed using the modified 25-item “Assessment of Multiple Systematic Reviews” (mod-AMSTAR) tool, which was based on the AMSTAR scale. A T-test, nonparametric test, and linear regression were conducted to assess the relationship between bibliographical characteristics and methodological quality.

Results

Sixty-three SRs were included, from which the majority of SRs (50, 79.4%) were conducted in Asia. Only 4 reviews were rated as high quality, and most were weak in providing a priori design (6, 9.5%), not limiting the publication type (8, 13%), providing an excluded primary studies list (4, 6.3%) and reporting support for the included primary studies (1, 1.6%). Reviews published in English journals performed better than did Chinese journals in duplicate data extraction (81.3% vs 46.7%, p = 0.017; 70.8% vs 33.3%, p = 0.009) and providing source of support for the SR (87.5% vs 33.3%, P < 0.001). Reviews published in journals with a higher impact factor were associated with a higher mod-AMSTAR score (regression coefficient: 0.38, 95%CI: 0.11–0.65; P = 0.006).

Conclusion

The methodological quality of the included SRs is far from satisfactory. Authors of SRs should conform to the recommendations outlined in the mod-AMSTAR items. Areas needing improvement were providing a priori design, not limiting the publication type, providing an excluded primary studies list, and reporting conflicts of interest.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0488-8) contains supplementary material, which is available to authorized users.

Pendulum and modified pendulum appliances for maxillary molar distalization in Class II malocclusion - a systematic review

OBJECTIVE

The main purpose of the present systematic review was to evaluate the quantitative effects of the pendulum appliance and modified pendulum appliances for maxillary molar distalization in Class II malocclusion.

MATERIALS AND METHODS

Our systematic search included MEDLINE, EMBASE, CINAHL, PsychINFO, Scopus and key journals and review articles; the date of the last search was 30 January 2017. We graded the methodological quality of the studies by means of the Quality Assessment Tool for Quantitative Studies, developed for the Effective Public Health Practice Project (EPHPP).

RESULTS

In total, 203 studies were identified for screening, and 25 studies were eligible. The quality assessment rated four (16%) of the study as being of strong quality and 21 (84%) of these studies as being of moderate quality. The pendulum appliances showed mean molar distalization of 2-6.4 mm, distal tipping of molars from 6.67° to 14.50° and anchorage loss with mean premolar and incisor mesial movement of 1.63-3.6 mm and 0.9-6.5 mm, respectively. The bone anchored pendulum appliances (BAPAs) showed mean molar distalization of 4.8-6.4 mm, distal tipping of molars from 9° to 11.3° and mean premolar distalization of 2.7-5.4 mm.

CONCLUSIONS

Pendulum and modified pendulum appliances are effective in molar distalization. Pendulum appliance with K-loop modification, implant supported pendulum appliance and BAPA significantly reduced anchorage loss of the anterior teeth and distal tipping of the molar teeth.

The Factors Affecting Long-Term Stability in Anterior Open-Bite Correction - A Systematic Review

Objective

The present systemic review was conducted with the main purpose to evaluate the quantitative effects of orthognathic surgeries, extraction versus non-extraction treatment, and the type of malocclusion in the stability of anterior open-bite (AOB) correction over the long-term.

Methods

The systematic search for studies was conducted through MEDLINE, CINAHL, EMBASE, Scopus, PsychINFO, various key journals, and review articles; November 30, 2016, was the last date for the search. The Quality Assessment Tool for Quantitative Studies was used to grade the methodological quality of the studies.

Results

The present review included 14 studies. Stability of the corrected AOB ranged from 61.9% to 100%. The studies with orthognathic surgeries showed a stability of 70-100%. The studies without orthognathic surgeries showed the stability of 61.9-96.7%. All of the studies were retrospective. The mean change in AOB before (T1) and after treatment (T2-T1) was 0.1 mm to 6.93 mm and the mean change in overbite from T2 to T3 (T3-T2) was -0.06 mm to 2.5 mm.

Conclusion

Studies with orthognathic surgeries presented with high amount of long-term stability in corrected AOB. No significant difference was noticed in relation to the type of malocclusion and extraction or non-extraction cases.