Assessment of the quality of reporting in abstracts of systematic reviews with meta-analyses in periodontology and implant dentistry

Effect of PRISMA 2009 on reporting quality in systematic reviews and meta-analyses in high-impact dental medicine journals between 1993-2018

INTRODUCTION

The PRISMA guidelines were published in 2009 to address inadequate reporting of key methodological details in systematic reviews and meta-analyses (SRs/MAs). This study sought to assess the impact of PRISMA on the quality of reporting in the full text of dental medicine journals.

METHODS

This study assessed the impact of PRISMA (2009) on thirteen methodological details in SRs/MAs published in the highest-impact dental medicine journals between 1993-2009 (n = 211) and 2012-2018 (n = 618). The study further examined the rate of described use of PRISMA in the abstract or full text of included studies published post- PRISMA and the impact of described use of PRISMA on level of reporting. This study also examined potential effects of inclusion of PRISMA in Instructions for Authors, along with study team characteristics.

RESULTS

The number of items reported in SRs/MAs increased following the publication of PRISMA (pre-PRISMA: M = 7.83, SD = 3.267; post-PRISMA: M = 10.55, SD = 1.4). Post-PRISMA, authors rarely mention PRISMA in abstracts (8.9%) and describe the use of PRISMA in the full text in 59.87% of SRs/MAs. The described use of PRISMA within the full text indicates that its intent (guidance for reporting) is not well understood, with over a third of SRs/MAs (35.6%) describing PRISMA as guiding the conduct of the review. However, any described use of PRISMA was associated with improved reporting. Among author team characteristics examined, only author team size had a positive relationship with improved reporting.

CONCLUSION

Following the 2009 publication of PRISMA, the level of reporting of key methodological details improved for systematic reviews/meta-analyses published in the highest-impact dental medicine journals. The positive relationship between reference to PRISMA in the full text and level of reporting provides further evidence of the impact of PRISMA on improving transparent reporting in dental medicine SRs/MAs.

THE REPORTING QUALITY OF SYSTEMATIC REVIEW ABSTRACTS IN LEADING GENERAL DENTAL JOURNALS: A METHODOLOGICAL STUDY

OBJECTIVE

To assess the reporting quality of systematic review (SR) abstracts published in leading general dental journals according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Abstracts (PRISMA-A) guidelines, and to identify factors associated with overall reporting quality.

METHODS

We identified SR abstracts published in 10 leading general dental journals and assessed their reporting quality. For each abstract, an overall reporting score (ORS, range: 0-13) was calculated. Risk ratio (RR) was calculated to compare the reporting quality of abstracts in Pre-PRISMA (2011-2012) and Post-PRISMA (2017-2018) periods. Univariable and multivariable linear regression analyses were performed to identify factors associated with reporting quality.

RESULTS

A total of 104 eligible abstracts were included. The mean ORS was 5.59 (SD = 1.48) and 6.97 (1.74) respectively in the Pre- and Post-PRISMA abstracts, with statistically significant difference (mean difference = 1.38; 95% CI: 0.70, 2.05). Reporting of the exact P-value (B = 1.22; 95% CI: 0.45, 1.99) was a significant predictor of higher reporting quality.

CONCLUSION

The reporting quality of SR abstracts published in leading general dental journals improved after the release of PRISMA-A guidelines, but is still suboptimal. Relevant stakeholders need to work together to enhance the reporting quality of SR abstracts in dentistry.

Reporting quality of abstracts of systematic reviews/meta-analyses: An appraisal of Arab Journal of Urology across 12 years: the PRISMA-Abstracts checklist

Objective

We appraised the reporting quality of abstracts of systematic reviews/meta-analyses (SR/MAs) published in one urology journal and explored associations between abstract characteristics and completeness of reporting.

Methods

The Arab Journal of Urology (AJU) was searched for SR/MAs published between January 2011 and 31 May 2022. SR/MAs with structured abstract and quantitative synthesis were eligible. Two reviewers simultaneously together selected the SR/MAs by title, screened the abstracts, and included those based on inclusion/exclusion criteria. Data of a range of characteristics were extracted from each SR/MAs into a spreadsheet. To gauge completeness of reporting, the PRISMA-Abstract checklist (12 items) was used to appraise the extent to which abstracts adhered to the checklist. For each abstract, we computed item, section, and overall adherence. Chi-square and t-tests compared the adherence scores. Univariate and multivariate analyses identified the abstract characteristics associated with overall adherence.

Results

In total, 66 SR/MAs published during the examined period; 62 were included. Partial reporting was not uncommon. In terms of adherence to the 12 PRISMA-A items were: two items exhibited 100% adherence (title, objectives); five items had 80% to <100% adherence (interpretation, included studies, synthesis of results, eligibility criteria, and information sources); two items displayed 40% to <80% adherence (description of the effect, strengths/limitations of evidence); and three items had adherence that fell between 0% and 1.6% (risk of bias, funding/conflict of interest, registration). Multivariable regression revealed two independent predictors of overall adherence: single-country authorship (i.e. no collaboration) was associated with higher overall adherence (P = 0.046); and abstracts from South America were associated with lower overall adherence (P = 0.04).

Conclusion

This study is the first to appraise abstracts of SR/MAs in urology. For high-quality abstracts, improvements are needed in the quality of reporting. Adoption/better adherence to PRISMA-A checklist by editors/authors could improve the reporting quality and completeness of SR/MAs abstracts.

There is still room for improvement in the completeness of abstract reporting according to the PRISMA-A checklist: a cross-sectional study on systematic reviews in periodontology

BACKGROUND

To evaluate the completeness of reporting abstracts of systematic reviews (SRs) before and after the publication of the PRISMA-A checklist in 2013 and to assess if an association exists between abstract characteristics and the completeness of reporting.

METHODS

A systematic search of the literature was conducted in the PubMed and Scopus databases in March 2020. The search focused on the SRs of evaluations of interventions published since 2002 in the field of periodontology. The abstracts of the selected SRs were divided into two groups before and after publication of the PRISMA-A checklist in 2013, and compliance with the 12 items reported in the checklist was evaluated by three calibrated evaluators.

RESULTS

A set of 265 abstracts was included in the study. The total score before (mean score, 53.78%; 95% CI, 51.56-55.90%) and after (mean score, 56.88%; 95% CI, 55.39-58.44%) the publication of the PRISMA-A statement exhibited a statistically significant improvement (P = 0.012*). Nevertheless, only the checklist items included studies and synthesis of the results displayed a statistically significant change after guideline publication. The total PRISMA-A score was higher in the meta-analysis group and in articles authored by more than four authors.

CONCLUSIONS

The impact of the PRISMA-A was statistically significant, but the majority of the items did not improve after its introduction. The editors and referees of periodontal journals should promote adherence to the checklist to improve the quality of the reports and provide readers with better insight into the characteristics of published studies.

Has the Reporting Quality of Systematic Review Abstracts in Nursing Improved Since the Release of PRISMA for Abstracts? A Survey of High-Profile Nursing Journals

BACKGROUND

The PRISMA for Abstracts (PRISMA-A) was developed to guide authors to present a structured abstract. However, the adherence of abstracts to these guidelines in some areas was of concern.

AIMS

To determine whether the publication of PRISMA-A resulted in an improvement in the abstracts reported with nursing systematic reviews (SRs).

METHODS

This was a cross-sectional study. We searched PubMed for randomized controlled trials-based SRs published in top-tier nursing journals. A PRISMA-A checklist was used to assess abstracts in the SR included. Total score on checklists, comparison of total scores between two periods, and effect factors were analyzed.

RESULTS

Overall, abstract reporting compliance with PRISMA-A has not improved significantly with the time span. Of the 81 SRs, 74.1% were structured. About half reported eligibility criteria, information sources, and description of the effect as recommended. Registration status was reported only in 4.9%. The reporting quality was significantly higher for journals with higher impact factors (p < .001).

LINKING EVIDENCE TO ACTION

Although not inclusive of all SRs in the nursing field, our sample reflects the general trend that there was no significant improvement in the compliance of SR abstracts reported in nursing with the release of PRISMA-A. There is room for improvement, as most items have not been fully reported.

Biomedical conferences’ author instructions rarely mention guidelines for reporting abstracts of trials and systematic reviews

Aim: To analyze whether instructions for authors of biomedical conference abstracts mention guidelines for writing randomized controlled trial and systematic review abstracts and to evaluate reasons for their absence from instructions. Materials & methods: We analyzed instructions for authors of biomedical conferences advertized in 2019 and assessed whether they mentioned Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Abstracts and Consolidated Standards of Reporting Trials for Abstracts guidelines. We surveyed contact persons from abstract/publication committees of selected conferences to analyze why relevant guidelines were missing. Results: Instructions for abstracts were available for 819 conferences. Only two (0.2%) had reporting instructions for randomized controlled trial/systematic review authors. Almost half of the contacted conference organizers whose response we received were not aware of Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Abstracts and Consolidated Standards of Reporting Trials for Abstracts guidelines. Conclusion: Conference organizers do not require and are not familiar enough with reporting guidelines.

Systematic reviews in orthodontics: Impact of the PRISMA for Abstracts checklist on completeness of reporting

INTRODUCTION

This study evaluated and compared the completeness of reporting of abstracts of orthodontics systematic reviews before and after the publication of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension for Abstracts Checklist (PRISMA-A).

METHODS

Abstracts of systematic reviews and meta-analyses in orthodontics published in PubMed, Latin American and Caribbean Health Sciences Literature, and the Cochrane Database of Systematic Reviews databases before March 23, 2018, that met the predefined inclusion and exclusion criteria, were evaluated using the 12 items of PRISMA-A, scoring each item from 0 to 2. Abstracts were classified into 2 groups: before and after publication of the PRISMA-A checklist. Three calibrated evaluators (intraclass correlation coefficient and kappa > 0.8) assessed the scores for compliance with the checklist. The number of authors, country of affiliation of the first author, performance of meta-analysis, and topic of the article were recorded. A regression analysis was performed to assess the associations between abstract characteristics and the PRISMA-A scores.

RESULTS

Of 1034 abstracts evaluated, 389 were included in the analysis. The mean PRISMA-A score was 53.39 (95% CI, 51.83-54.96). The overall score for studies published after the publication of the checklist was significantly higher than for studies published before (P ≤ 0.0001). The components returning significantly higher scores after publication of PRISMA-A were title (P = 0.024), information from databases (P = 0.026), risk of bias (P ≤ 0.0001), included studies (P ≤ 0.0001), synthesis of results (P ≤ 0.0001), interpretation of results (P = 0.035), financing and conflict of interest (P ≤ 0.0001), and registration (P ≤ 0.0001). These results showed the positive effect of PRISMA-A had on the quality of reporting of orthodontics systematic reviews. Nevertheless, the poor adherence revealed that there is still need for improvement in the quality of abstract reporting.

CONCLUSIONS

The quality of reporting of abstracts of orthodontic systematic reviews and meta-analyses increased after the introduction of PRISMA-A.

Conference abstracts describing systematic reviews on pain were selectively published, not reliable, and poorly reported

OBJECTIVE

The objective of the study was to determine the reporting quality of systematic review (SR) abstracts presented at World Congresses on Pain (WCPs) and to quantify agreement in results presented in those abstracts with their corresponding full-length publications.

STUDY DESIGN AND SETTING

We screened abstracts of five WCPs held from 2008 to 2016 to find abstracts describing SRs. Two authors searched for corresponding full publications using PubMed and Google Scholar in April 2018. Methods and outcomes extracted from abstracts were compared with their corresponding full publications. The reporting quality of abstracts was evaluated against the PRISMA for Abstracts (PRISMA-A) checklist.

RESULTS

We identified 143 conference abstracts describing SRs. Of these, 90 (63%) were published as full-length articles in peer-reviewed journals by April 2018, with a median time from conference presentation to publication of 5 months (interquartile range: -0.25 to 14 months). Among 79 abstract-publication pairs evaluable for discordance, there was some form of discordance in 40% of pairs. Qualitative discordance (different direction of the effect) was found in 13 analyzed pairs (16%). The median adherence by abstracts to each PRISMA-A checklist item was 33% (interquartile range: 29% to 42%).

CONCLUSION

Conference abstracts of pain SRs are selectively published, not reliable, and poorly reported.

Reporting Quality of Systematic Review Abstracts Published in Leading Neurosurgical Journals: A Research on Research Study

BACKGROUND

Systematic review (SR) abstracts are frequently relied upon to guide clinical decision-making. However, there is mounting evidence that the quality of abstract reporting in the medical literature is suboptimal.

OBJECTIVE

To appraise SR abstract reporting quality in neurosurgical journals and identify factors associated with improved reporting.

METHODS

This study systematically surveyed SR abstracts published in 8 leading neurosurgical journals between 8 April 2007 and 21 August 2017. Abstracts were identified through a search of the MEDLINE database and their reporting quality was determined in duplicate using a tool derived from the Preferred Reporting Items for Systematic Reviews and Meta-analyses for Abstracts (PRISMA-A) statement. All SR abstracts that provided comparison between treatment strategies were eligible for inclusion. Descriptive statistics were utilized to identify factors associated with improved reporting.

RESULTS

A total of 257 abstracts were included in the analysis, with a mean of 22.8 (±25.3) included studies. The overall quality of reporting in included abstracts was suboptimal, with a mean score of 53.05% (±11.18). Reporting scores were higher among abstracts published after the release of the PRISMA-A guidelines (M = 56.52; 21.74-73.91) compared with those published beforehand (M = 47.83; 8.70-69.57; U = 4346.00, z = -4.61, P < .001). Similarly, both word count (r = 0.338, P < .001) and journal impact factor (r = 0.199, P = .001) were associated with an improved reporting score.

CONCLUSION

This study demonstrates that the overall reporting quality of abstracts in leading neurosurgical journals requires improvement. Strengths include the large number abstracts assessed, and its weaknesses include the fact that only neurosurgery-specific journals were surveyed. We recommend that attention be turned toward strengthening abstract submission and peer-review processes.

PRISMA for abstracts: best practice for reporting abstracts of systematic reviews in Endodontology

An abstract is a brief overview of a scientific, clinical or review manuscript as well as a stand-alone summary of a conference abstract. Scientists, clinician-scientists and clinicians rely on the summary information provided in the abstracts of systematic reviews to assist in subsequent clinical decision-making. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) for Abstracts checklist was developed to improve the quality, accuracy and completeness of abstracts associated with systematic reviews and meta-analyses. The PRISMA for Abstracts checklist provides a framework for authors to follow, which helps them provide in the abstract the key information from the systematic review that is required by stakeholders. The PRISMA for Abstracts checklist contains 12 items (title, objectives, eligibility criteria, information sources, risk of bias, included studies, synthesis of results, description of the effect, strength and limitations, interpretation, funding and systematic review registration) under six sections (title, background, methods, results, discussion, other). The current article highlights the relevance and importance of the items in the PRISMA for Abstracts checklist to the specialty of Endodontology, while offering explanations and specific examples to assist authors when writing abstracts for systematic reviews when reported in manuscripts or submitted to conferences. Strict adherence to the PRISMA for Abstracts checklist by authors, reviewers, and journal editors will result in the consistent publication of high-quality abstracts within Endodontology. This article is protected by copyright. All rights reserved.

Quality of abstract of systematic reviews and meta-analyses in paediatric dentistry journals

AIM

To systematically evaluate the reporting quality of the abstract of systematic reviews and meta-analyses in paediatric dentistry journals.

MATERIALS AND METHODS

Systematic reviews with meta-analyses in paediatric dentistry were searched in PubMed and Scopus databases from inception to December 2017. Selection of studies by title and abstract screening followed by full-text assessment was independently done by two reviewers. The quality of abstracts was assessed by PRISMA-Abstract checklist comprising of 12 items; one each for title and objective, three items for methods, three items for results, two items for discussion and two items for others. PRISMA-A median scores were calculated and compared with the article characteristics. Statistical significance was set at p < 0.05 and multi-variate analysis was performed using Kruskal-Wallis test.

RESULTS

A total of 24 studies were included in the analysis. The mean PRISMA-Abstract score was 7.46 ± 1.19. None of the studies were of high quality (score 10-12), 20 were of moderate (score 7-9), and 4 were of low quality (score 1-6). Journals that adhered to PRISMA guidelines showed significantly higher quality (p < 0.05). No association was found between the quality and the number of authors, country, journals, year of publication, word count and focus of study.

CONCLUSION

Majority of abstracts of systematic reviews and meta-analyses in paediatric dentistry journals were of moderate quality. Adoption and adherence to PRISMA-Abstract checklist by the journal editors and authors will enhance the reporting quality of abstracts.

Quality of flow diagram in systematic review and/or meta-analysis

Systematic reviews and/or meta-analyses generally provide the best evidence for medical research. Authors are recommended to use flow diagrams to present the review process, allowing for better understanding among readers. However, no studies as of yet have assessed the quality of flow diagrams in systematic review/meta-analyses. Our study aims to evaluate the quality of systematic review/meta-analyses over a period of ten years, by assessing the quality of the flow diagrams, and the correlation to the methodological quality. Two hundred articles of "systematic review" and/or "meta-analysis" from January 2004 to August 2015 were randomly retrieved in Pubmed to be assessed for the flow diagram and methodological qualities. The flow diagrams were evaluated using a 16-grade scale corresponding to the four stages of PRISMA flow diagram. It composes four parts: Identification, Screening, Eligibility and Inclusion. Of the 200 articles screened, 154 articles were included and were assessed with AMSTAR checklist. Among them, 78 articles (50.6%) had the flow diagram. Over ten years, the proportion of papers with flow diagram available had been increasing significantly with regression coefficient beta = 5.649 (p = 0.002). However, the improvement in quality of the flow diagram increased slightly but not significantly (regression coefficient beta = 0.177, p = 0.133). Our analysis showed high variation in the proportion of articles that reported flow diagram components. The lowest proportions were 1% for reporting methods of duplicates removal in screening phase, followed by 6% for manual search in identification phase, 22% for number of studies for each specific/subgroup analysis, 27% for number of articles retrieved from each database, and 31% for number of studies included in qualitative analysis. The flow diagram quality was correlated with the methodological quality with the Pearson's coefficient r = 0.32 (p = 0.0039). Therefore, this review suggests that the reporting quality of flow diagram is less satisfactory, hence not maximizing the potential benefit of the flow diagrams. A guideline with standardized flow diagram is recommended to improve the quality of systematic reviews, and to enable better reader comprehension of the review process.

Evaluating the Risk of Bias of a Study

OBJECTIVE

This first article of a series of 4 is aimed at guiding dental practitioners on how to evaluate the internal validity (risk of bias,) of randomized controlled trials (RCT). All RCT's contain different areas and potential sources of bias. Understanding risk of bias (ROB) will allow dental practitioners to improve the quality of dental treatments.

METHODS

The following areas of bias were elucidated: sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective reporting, and "other bias". The reader determines the ROB level by evaluating the areas or potential source of bias in the first phase. Normally, ROB levels are classified as low, high and unclear ROB.

RESULTS

This article reported the concepts and methods of evaluation of ROB in several areas of an RCT. An RCT with low ROB in all evaluated areas gives the dental practitioners more certainty and confidence that a specific clinical procedure is in fact effective and relevant to the patient.

CONCLUSIONS

The information provided here may guide dental practitioners in the evaluation of ROB in an RCT. The correct evaluation of ROB may improve the quality of dental treatments.

Quality and clarity in systematic review abstracts: an empirical study

BACKGROUND

Systematic review (SR) abstracts are important for disseminating evidence syntheses to inform medical decision making. We assess reporting quality in SR abstracts using PRISMA for Abstracts (PRISMA-A), Cochrane Handbook, and Agency for Healthcare Research & Quality guidance.

METHODS

We evaluated a random sample of 200 SR abstracts (from 2014) comparing interventions in the general medical literature. We assessed adherence to PRISMA-A criteria, problematic wording in conclusions, and whether "positive" studies described clinical significance.

RESULTS

On average, abstracts reported 60% of PRISMA-A checklist items (mean 8.9 ± 1.7, range 4 to 12). Eighty percent of meta-analyses reported quantitative measures with a confidence interval. Only 49% described effects in terms meaningful to patients and clinicians (e.g., absolute measures), and only 43% mentioned strengths/limitations of the evidence base. Average abstract word count was 274 (SD 89). Word count explained only 13% of score variability. PRISMA-A scores did not differ between Cochrane and non-Cochrane abstracts (mean difference 0.08, 95% confidence interval -1.16 to 1.00). Of 275 primary outcomes, 48% were statistically significant, 32% were not statistically significant, and 19% did not report significance or results. Only one abstract described clinical significance for positive findings. For "negative" outcomes, we identified problematic simple restatements (20%), vague "no evidence of effect" wording (9%), and wishful wording (8%).

CONCLUSIONS

Improved SR abstract reporting is needed, particularly reporting of quantitative measures (for meta-analysis), easily interpretable units, strengths/limitations of evidence, clinical significance, and clarifying whether negative results reflect true equivalence between treatments. Copyright © 2016 John Wiley & Sons, Ltd.

Reporting quality in abstracts of meta-analyses of depression screening tool accuracy: a review of systematic reviews and meta-analyses

OBJECTIVE

Concerns have been raised regarding the quality and completeness of abstract reporting in evidence reviews, but this had not been evaluated in meta-analyses of diagnostic accuracy. Our objective was to evaluate reporting quality and completeness in abstracts of systematic reviews with meta-analyses of depression screening tool accuracy, using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) for Abstracts tool.

DESIGN

Cross-sectional study.

INCLUSION CRITERIA

We searched MEDLINE and PsycINFO from 1 January 2005 through 13 March 2016 for recent systematic reviews with meta-analyses in any language that compared a depression screening tool to a diagnosis based on clinical or validated diagnostic interview.

DATA EXTRACTION

Two reviewers independently assessed quality and completeness of abstract reporting using the PRISMA for Abstracts tool with appropriate adaptations made for studies of diagnostic test accuracy. Bivariate associations of number of PRISMA for Abstracts items complied with (1) journal abstract word limit and (2) A Measurement Tool to Assess Systematic Reviews (AMSTAR) scores of meta-analyses were also assessed.

RESULTS

We identified 21 eligible meta-analyses. Only two of 21 included meta-analyses complied with at least half of adapted PRISMA for Abstracts items. The majority met criteria for reporting an appropriate title (95%), result interpretation (95%) and synthesis of results (76%). Meta-analyses less consistently reported databases searched (43%), associated search dates (33%) and strengths and limitations of evidence (19%). Most meta-analyses did not adequately report a clinically meaningful description of outcomes (14%), risk of bias (14%), included study characteristics (10%), study eligibility criteria (5%), registration information (5%), clear objectives (0%), report eligibility criteria (0%) or funding (0%). Overall meta-analyses quality scores were significantly associated with the number of PRISMA for Abstracts scores items reported adequately (r=0.45).

CONCLUSIONS

Quality and completeness of reporting were found to be suboptimal. Journal editors should endorse PRISMA for Abstracts and allow for flexibility in abstract word counts to improve quality of abstracts.

The use of systemic antibiotics in the treatment of refractory periodontitis: A systematic review

BACKGROUND

The goal in treating refractory periodontitis (RP) is to arrest or slow disease progression, which usually has included the use of systemic antibiotics adjunct to conventional mechanical debridement. The aim of this systematic review was to evaluate the evidence that the association of systemic antibiotics with conventional mechanical debridement increases the efficacy of periodontal therapy in the treatment of RP.

TYPES OF STUDIES REVIEWED

The authors searched for studies in PubMed MEDLINE, Cochrane Central Register of Controlled Trials, Thomson Reuters Web of Science, Scopus, Latin American and Caribbean Center on Health Sciences Information, and Scientific Electronic Library Online electronic databases by using selected key words from the earliest records up through October 31, 2014. Only clinical intervention studies in which investigators compared the treatment of participants with RP with either mechanical debridement alone or associated with systemic antibiotics were eligible for selection. Two authors independently assessed the risk of bias of each selected study.

RESULTS

The authors identified 13 articles and included 6 of them. Investigators in all studies reported greater reductions in probing depth or in loss of clinical attachment level after adjunct systemic antibiotic therapy when compared with mechanical debridement alone. Antibiotics tested included metronidazole, clindamycin, tetracycline hydrochloride, amoxicillin, and amoxicillin and potassium clavulanate. Five studies presented a high risk of bias, and 1 study presented an unclear risk.

CONCLUSIONS AND PRACTICAL IMPLICATIONS

The overall quality of the evidence does not allow the conclusion that adjunct systemic antibiotics are of additional benefit to conventional mechanical debridement alone.

Methodological quality assessment of paper-based systematic reviews published in oral health

OBJECTIVES

This study aimed to conduct a methodological assessment of paper-based systematic reviews (SR) published in oral health using a validated checklist. A secondary objective was to explore temporal trends on methodological quality.

MATERIAL AND METHODS

Two electronic databases (OVID Medline and OVID EMBASE) were searched for paper-based SR of interventions published in oral health from inception to October 2014. Manual searches of the reference lists of paper-based SR were also conducted. Methodological quality of included paper-based SR was assessed using an 11-item questionnaire, Assessment of Multiple Systematic Reviews (AMSTAR) checklist. Methodological quality was summarized using the median and inter-quartile range (IQR) of the AMSTAR score over different categories and time periods.

RESULTS

A total of 643 paper-based SR were included. The overall median AMSTAR score was 4 (IQR 2-6). The highest median score (5) was found in the pain dentistry and periodontology fields, while the lowest median score (3) was found in implant dentistry, restorative dentistry, oral medicine, and prosthodontics. The number of paper-based SR per year and the median AMSTAR score increased over time (median score in 1990s was 2 (IQR 2-3), 2000s was 4 (IQR 2-5), and 2010 onwards was 5 (IQR 3-6)).

CONCLUSION

Although the methodological quality of paper-based SR published in oral health has improved in the last few years, there is still scope for improving quality in most evaluated dental specialties.

CLINICAL RELEVANCE

Large-scale assessment of methodological quality of dental SR highlights areas of methodological strengths and weaknesses that can be targeted in future publications to encourage better quality review methodology.

Effect of Risk of Bias on the Effect Size of Meta-Analytic Estimates in Randomized Controlled Trials in Periodontology and Implant Dentistry

Background

Risk of bias (ROB) may threaten the internal validity of a clinical trial by distorting the magnitude of treatment effect estimates, although some conflicting information on this assumption exists.

Objective

The objective of this study was evaluate the effect of ROB on the magnitude of treatment effect estimates in randomized controlled trials (RCTs) in periodontology and implant dentistry.

Methods

A search for Cochrane systematic reviews (SRs), including meta-analyses of RCTs published in periodontology and implant dentistry fields, was performed in the Cochrane Library in September 2014. Random-effect meta-analyses were performed by grouping RCTs with different levels of ROBs in three domains (sequence generation, allocation concealment, and blinding of outcome assessment). To increase power and precision, only SRs with meta-analyses including at least 10 RCTs were included. Meta-regression was performed to investigate the association between ROB characteristics and the magnitudes of intervention effects in the meta-analyses.

Results

Of the 24 initially screened SRs, 21 SRs were excluded because they did not include at least 10 RCTs in the meta-analyses. Three SRs (two from periodontology field) generated information for conducting 27 meta-analyses. Meta-regression did not reveal significant differences in the relationship of the ROB level with the size of treatment effect estimates, although a trend for inflated estimates was observed in domains with unclear ROBs.

Conclusion

In this sample of RCTs, high and (mainly) unclear risks of selection and detection biases did not seem to influence the size of treatment effect estimates, although several confounders might have influenced the strength of the association.

Reporting of sources of funding in systematic reviews in periodontology and implant dentistry

Industry-supported clinical trials may present better outcomes than those supported by other sources. The aim of this paper was to assess whether systematic reviews (SRs) published in periodontology and implant dentistry report and discuss the influence of funding sources on study results. Two reviewers conducted a comprehensive search in PubMed and the Cochrane Database of Systematic Reviews independently and in duplicate to identify SRs published up to 11 November 2012. Speciality dental journals and the reference lists of included SRs were also scrutinised. Information on the reporting and discussion of funding sources of primary studies included in the SRs was extracted independently and in duplicate. Any disagreement regarding SR selection or data extraction was discussed until consensus was achieved. Of 146 SRs included in the assessment, only 45 (31%) reported the funding sources of primary studies. Fourteen (10%) SRs discussed the potential influence of funding sources on study results, that is, sponsorship bias. Funding sources are inadequately reported and discussed in SRs in periodontology and implant dentistry. Assessment, reporting, and critical appraisal of potential sponsorship bias of meta-analytic estimates are paramount to provide proper guidance for clinical treatments.

Use of methodological tools for assessing the quality of studies in periodontology and implant dentistry: a systematic review

OBJECTIVE

To evaluate the methodological approaches used to assess the quality of studies included in systematic reviews (SRs) in periodontology and implant dentistry.

MATERIALS & METHODS

Two electronic databases (PubMed and Cochrane Database of Systematic Reviews) were searched independently to identify SRs examining interventions published through 2 September 2013. The reference lists of included SRs and records of 10 specialty dental journals were searched manually. Methodological approaches were assessed using seven criteria based on the Cochrane Handbook for Systematic Reviews of Interventions. Temporal trends in methodological quality were also explored.

RESULTS

Of the 159 SRs with meta-analyses included in the analysis, 44 (28%) reported the use of domain-based tools, 15 (9%) reported the use of checklists and 7 (4%) reported the use of scales. Forty-two (26%) SRs reported use of more than one tool. Criteria were met heterogeneously; authors of 15 (9%) publications incorporated the quality of evidence of primary studies into SRs, whereas 69% of SRs reported methodological approaches in the Materials/Methods section. Reporting of four criteria was significantly better in recent (2010-2013) than in previous publications.

CONCLUSION

The analysis identified several methodological limitations of approaches used to assess evidence in studies included in SRs in periodontology and implant dentistry.

Bias due to selective inclusion and reporting of outcomes and analyses in systematic reviews of randomised trials of healthcare interventions

BACKGROUND

Systematic reviews may be compromised by selective inclusion and reporting of outcomes and analyses. Selective inclusion occurs when there are multiple effect estimates in a trial report that could be included in a particular meta-analysis (e.g. from multiple measurement scales and time points) and the choice of effect estimate to include in the meta-analysis is based on the results (e.g. statistical significance, magnitude or direction of effect). Selective reporting occurs when the reporting of a subset of outcomes and analyses in the systematic review is based on the results (e.g. a protocol-defined outcome is omitted from the published systematic review).

OBJECTIVES

To summarise the characteristics and synthesise the results of empirical studies that have investigated the prevalence of selective inclusion or reporting in systematic reviews of randomised controlled trials (RCTs), investigated the factors (e.g. statistical significance or direction of effect) associated with the prevalence and quantified the bias.

SEARCH METHODS

We searched the Cochrane Methodology Register (to July 2012), Ovid MEDLINE, Ovid EMBASE, Ovid PsycINFO and ISI Web of Science (each up to May 2013), and the US Agency for Healthcare Research and Quality (AHRQ) Effective Healthcare Program's Scientific Resource Center (SRC) Methods Library (to June 2013). We also searched the abstract books of the 2011 and 2012 Cochrane Colloquia and the article alerts for methodological work in research synthesis published from 2009 to 2011 and compiled in Research Synthesis Methods.

SELECTION CRITERIA

We included both published and unpublished empirical studies that investigated the prevalence and factors associated with selective inclusion or reporting, or both, in systematic reviews of RCTs of healthcare interventions. We included empirical studies assessing any type of selective inclusion or reporting, such as investigations of how frequently RCT outcome data is selectively included in systematic reviews based on the results, outcomes and analyses are discrepant between protocol and published review or non-significant outcomes are partially reported in the full text or summary within systematic reviews.

DATA COLLECTION AND ANALYSIS

Two review authors independently selected empirical studies for inclusion, extracted the data and performed a risk of bias assessment. A third review author resolved any disagreements about inclusion or exclusion of empirical studies, data extraction and risk of bias. We contacted authors of included studies for additional unpublished data. Primary outcomes included overall prevalence of selective inclusion or reporting, association between selective inclusion or reporting and the statistical significance of the effect estimate, and association between selective inclusion or reporting and the direction of the effect estimate. We combined prevalence estimates and risk ratios (RRs) using a random-effects meta-analysis model.

MAIN RESULTS

Seven studies met the inclusion criteria. No studies had investigated selective inclusion of results in systematic reviews, or discrepancies in outcomes and analyses between systematic review registry entries and published systematic reviews. Based on a meta-analysis of four studies (including 485 Cochrane Reviews), 38% (95% confidence interval (CI) 23% to 54%) of systematic reviews added, omitted, upgraded or downgraded at least one outcome between the protocol and published systematic review. The association between statistical significance and discrepant outcome reporting between protocol and published systematic review was uncertain. The meta-analytic estimate suggested an increased risk of adding or upgrading (i.e. changing a secondary outcome to primary) when the outcome was statistically significant, although the 95% CI included no association and a decreased risk as plausible estimates (RR 1.43, 95% CI 0.71 to 2.85; two studies, n = 552 meta-analyses). Also, the meta-analytic estimate suggested an increased risk of downgrading (i.e. changing a primary outcome to secondary) when the outcome was statistically significant, although the 95% CI included no association and a decreased risk as plausible estimates (RR 1.26, 95% CI 0.60 to 2.62; two studies, n = 484 meta-analyses). None of the included studies had investigated whether the association between statistical significance and adding, upgrading or downgrading of outcomes was modified by the type of comparison, direction of effect or type of outcome; or whether there is an association between direction of the effect estimate and discrepant outcome reporting.Several secondary outcomes were reported in the included studies. Two studies found that reasons for discrepant outcome reporting were infrequently reported in published systematic reviews (6% in one study and 22% in the other). One study (including 62 Cochrane Reviews) found that 32% (95% CI 21% to 45%) of systematic reviews did not report all primary outcomes in the abstract. Another study (including 64 Cochrane and 118 non-Cochrane reviews) found that statistically significant primary outcomes were more likely to be completely reported in the systematic review abstract than non-significant primary outcomes (RR 2.66, 95% CI 1.81 to 3.90). None of the studies included systematic reviews published after 2009 when reporting standards for systematic reviews (Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement, and Methodological Expectations of Cochrane Intervention Reviews (MECIR)) were disseminated, so the results might not be generalisable to more recent systematic reviews.

AUTHORS' CONCLUSIONS

Discrepant outcome reporting between the protocol and published systematic review is fairly common, although the association between statistical significance and discrepant outcome reporting is uncertain. Complete reporting of outcomes in systematic review abstracts is associated with statistical significance of the results for those outcomes. Systematic review outcomes and analysis plans should be specified prior to seeing the results of included studies to minimise post-hoc decisions that may be based on the observed results. Modifications that occur once the review has commenced, along with their justification, should be clearly reported. Effect estimates and CIs should be reported for all systematic review outcomes regardless of the results. The lack of research on selective inclusion of results in systematic reviews needs to be addressed and studies that avoid the methodological weaknesses of existing research are also needed.

Search strategies in systematic reviews in periodontology and implant dentistry

OBJECTIVE

To perform an overview of literature search strategies in systematic reviews (SRs) published in periodontology and implant dentistry.

MATERIALS AND METHODS

Two electronic databases (PubMed and Cochrane Database of SRs) were searched, independently and in duplicate, for SRs with meta-analyses on interventions, with the last search performed on 11 November 2012. Manual searches of the reference lists of included SRs and 10 specialty dental journals were conducted. Methodological issues of the search strategies of included SRs were assessed with Cochrane collaboration guidelines and AMSTAR recommendations. The search strategies employed in Cochrane and paper-based SRs were compared.

RESULTS

A total of 146 SRs with meta-analyses were included, including 19 Cochrane and 127 paper-based SRs. Some issues, such as "the use of keywords," were reported in most of the SRs (86%). Other issues, such as "search of grey literature" and "language restriction," were not fully reported (34% and 50% respectively). The quality of search strategy reporting in Cochrane SRs was better than that of paper-based SRs for seven of the eight criteria assessed.

CONCLUSION

There is room for improving the quality of reporting of search strategies in SRs in periodontology and implant dentistry, particularly in SRs published in paper-based journals.