Reporting quality of systematic review abstracts in leading oral implantology journals

Effect of PRISMA 2009 on reporting quality in systematic reviews and meta-analyses in high-impact dental medicine journals between 1993-2018

INTRODUCTION

The PRISMA guidelines were published in 2009 to address inadequate reporting of key methodological details in systematic reviews and meta-analyses (SRs/MAs). This study sought to assess the impact of PRISMA on the quality of reporting in the full text of dental medicine journals.

METHODS

This study assessed the impact of PRISMA (2009) on thirteen methodological details in SRs/MAs published in the highest-impact dental medicine journals between 1993-2009 (n = 211) and 2012-2018 (n = 618). The study further examined the rate of described use of PRISMA in the abstract or full text of included studies published post- PRISMA and the impact of described use of PRISMA on level of reporting. This study also examined potential effects of inclusion of PRISMA in Instructions for Authors, along with study team characteristics.

RESULTS

The number of items reported in SRs/MAs increased following the publication of PRISMA (pre-PRISMA: M = 7.83, SD = 3.267; post-PRISMA: M = 10.55, SD = 1.4). Post-PRISMA, authors rarely mention PRISMA in abstracts (8.9%) and describe the use of PRISMA in the full text in 59.87% of SRs/MAs. The described use of PRISMA within the full text indicates that its intent (guidance for reporting) is not well understood, with over a third of SRs/MAs (35.6%) describing PRISMA as guiding the conduct of the review. However, any described use of PRISMA was associated with improved reporting. Among author team characteristics examined, only author team size had a positive relationship with improved reporting.

CONCLUSION

Following the 2009 publication of PRISMA, the level of reporting of key methodological details improved for systematic reviews/meta-analyses published in the highest-impact dental medicine journals. The positive relationship between reference to PRISMA in the full text and level of reporting provides further evidence of the impact of PRISMA on improving transparent reporting in dental medicine SRs/MAs.

Do pooled estimates from orthodontic meta-analyses change depending on the meta-analysis approach? A meta-epidemiological study

BACKGROUND

In meta-analyses involving a few trials, appropriate measures should be employed to assess between-study heterogeneity. When the number of studies is less than five and heterogeneity is evident, the Hartung and Knapp (HK) correction should be used. The aim of this study was to compare the reported estimates of published orthodontic meta-analyses with the pooled effect size estimates and prediction intervals (PI) calculated using eight heterogeneity estimators and corrected using the HK correction.

MATERIAL AND METHODS

Systematic reviews (SRs) published between 2017 and 2022 in four orthodontic journals and the Cochrane Database of Systematic Reviews with a meta-analysis of at least three studies were sourced. Study characteristics were extracted at the SR and the outcome/meta-analysis levels. All selected meta-analyses were re-analysed by fitting a random-effects model using eight different heterogeneity estimators, both with and without the HK correction. For each meta-analysis, the overall estimate, along with its standard error, the P-value, and the corresponding 95% confidence interval (CI), the between-study variance (tau2), the I2 statistic, and the PI were calculated.

RESULTS

One-hundred-six SRs were analysed. The most prevalent type of SR was non-Cochrane (95.3%), and the most used meta-analyses synthesis model was the random effect (83.0%). The median number of primary studies was 6 (interquartile range: 5, range: 3-45). The between-study variance was reported in most of the eligible meta-analyses (91.5%), but the type of heterogeneity estimator was reported in only one of them (0.9%). In 5 of 106 meta-analyses (4.7%), the HK correction was applied to adjust the CI of the pooled estimate. The percentage of statistically significant results, which became statistically non-significant, ranged from 16.7% to 25%, depending on the heterogeneity estimator. As the number of studies in a meta-analysis increased, the difference between corrected and uncorrected CIs reduced. Based on the PIs, more than half of the meta-analyses having statistically significant results are likely to change in the future, suggesting the result of the meta-analysis is not conclusive.

CONCLUSIONS

The statistical significance of pooled estimates from meta-analyses with at least three studies is sensitive to the HK correction, the heterogeneity variance estimator, and PIs. Clinicians should be aware of the clinical implications of not appropriately assessing the effect of the small number of studies and the between-study heterogeneity when interpreting results from meta-analyses.

THE REPORTING QUALITY OF SYSTEMATIC REVIEW ABSTRACTS IN LEADING GENERAL DENTAL JOURNALS: A METHODOLOGICAL STUDY

OBJECTIVE

To assess the reporting quality of systematic review (SR) abstracts published in leading general dental journals according to the Preferred Reporting Items for Systematic Reviews and Meta-Analyses for Abstracts (PRISMA-A) guidelines, and to identify factors associated with overall reporting quality.

METHODS

We identified SR abstracts published in 10 leading general dental journals and assessed their reporting quality. For each abstract, an overall reporting score (ORS, range: 0-13) was calculated. Risk ratio (RR) was calculated to compare the reporting quality of abstracts in Pre-PRISMA (2011-2012) and Post-PRISMA (2017-2018) periods. Univariable and multivariable linear regression analyses were performed to identify factors associated with reporting quality.

RESULTS

A total of 104 eligible abstracts were included. The mean ORS was 5.59 (SD = 1.48) and 6.97 (1.74) respectively in the Pre- and Post-PRISMA abstracts, with statistically significant difference (mean difference = 1.38; 95% CI: 0.70, 2.05). Reporting of the exact P-value (B = 1.22; 95% CI: 0.45, 1.99) was a significant predictor of higher reporting quality.

CONCLUSION

The reporting quality of SR abstracts published in leading general dental journals improved after the release of PRISMA-A guidelines, but is still suboptimal. Relevant stakeholders need to work together to enhance the reporting quality of SR abstracts in dentistry.

Research transparency in dental research: A programmatic analysis

We assessed adherence to five transparency practices-data sharing, code sharing, conflict of interest disclosure, funding disclosure, and protocol registration-in articles in dental journals. We searched and exported the full text of all research articles from PubMed-indexed dental journals available in the Europe PubMed Central database until the end of 2021. We programmatically assessed their adherence to the five transparency practices using a validated and automated tool. Journal- and article-related information was retrieved from ScimagoJR and Journal Citation Reports. Of all 329,784 articles published in PubMed-indexed dental journals, 10,659 (3.2%) were available to download. Of those, 77% included a conflict of interest disclosure, and 62% included a funding disclosure. Seven percent of the articles had a registered protocol. Data sharing (2.0%) and code sharing (0.1%) were rarer. Sixteen percent of articles did not adhere to any of the five transparency practices, 29% adhered to one, 48% adhered to two, 7.0% adhered to three, 0.3% adhered to four, and no article adhered to all five practices. Adherence to transparency practices increased over time; however, data and code sharing especially remained rare. Coordinated efforts involving all stakeholders are needed to change current transparency practices in dental research.

Reporting completeness in abstracts of systematic reviews of diagnostic test accuracy studies in cardiovascular diseases is suboptimal

OBJECTIVE

Journal abstracts are crucial for the identification and initial assessment of content of studies. We evaluated whether authors in the field of cardiovascular diseases (CVDs) reported Diagnostic Test Accuracy Systematic Reviews (DTA SRs) abstracts adequately, as defined by the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA)-DTA guidelines.

METHODS

SRs of DTA studies in CVDs published in general and specialized medical journals were identified in a MEDLINE search between 2010-2020. Adherence to 12 PRISMA-DTA for abstracts items was assessed independently by two reviewers and compared by journal's type. Moreover, the association of reporting completeness with different characteristics was investigated.

RESULTS

We included 72 abstracts. Studies published in general medical journals had higher mean reporting score than those in specialized journals (6.2 vs 5.3 out of 12 items; mean difference: 0.88; 95% confidence interval: 0.21, 1.55). PRISMA-DTA adherence was higher in journals that adopted this guideline and in articles with structured abstracts. However, number of participants analysed, funding and registration were the least-reported items in the identified abstracts.

CONCLUSION

The reporting of abstracts of DTA reports in CVDs is suboptimal according to PRISMA-DTA guidelines. Abstract reporting could be improved with the use of higher word count limits and the adoption of PRISMA-DTA guidelines especially in specialized journals.

Factors associated with the reporting quality of low back pain systematic review abstracts in physical therapy: a methodological study

BACKGROUND

Abstracts of systematic reviews (SR) are frequently used to guide clinical decision-making. However, if the abstract is inadequately reported, key information may be missing and it may not accurately summarize the results of the review.

OBJECTIVE

We aimed to investigate 1) if abstracts are fully reported; 2) if abstract reporting is associated with review/journal characteristics in physical therapy for low back pain (LBP); and 3) if these abstracts are consistent with the corresponding full texts.

METHODS

We searched the Physiotherapy Evidence Database for SRs in physical therapy for LBP published between 2015 and 2017. Associations between abstract reporting quality and review/journal characteristics were explored with linear regression. Abstract reporting was assessed with the 12 item Preferred Reporting Items for Systematic Reviews and Meta-Analyses for abstracts (PRISMA-A) checklist. Consistency of reporting between abstracts and the full text was evaluated by comparing responses to each item of the PRISMA-A using Kappa coefficients. Methodological quality of the reviews was assessed with A MeaSurement Tool to Assess systematic Reviews (AMSTAR-2).

RESULTS

We included 66 SRs, 9 Cochrane and 57 non-Cochrane. Review methodological quality ranged from 'high' (8%) to 'critically low' (76%). The mean ± SD of the "total number of PRISMA-A fully reported items" (range 0-12 points for fully reported items) was 4.1 ± 1.9 points for non-Cochrane review abstracts and 9.9 ± 1.1 points for Cochrane abstracts. Factors associated with reporting quality of abstracts were: journal impact factor (ß 0.20; 95% CI: 0.06, 0.35), number of words in abstract (ß 0.01; 95% CI: 0.00, 0.01) and review methodological quality ('critically low' with ß -3.06; 95% CI: -5.30, -0.82; with 'high' as reference variable). There was typically inconsistent reporting between abstract and full text, with most Kappa values lower than 0.60.

CONCLUSIONS

The abstracts of SRs in physical therapy for LBP were poorly reported and inconsistent with the full text. The reporting quality of abstracts was higher in journals with a higher impact factor, in abstracts with a greater number of words, and when the review was of higher methodological quality.

Do not make clinical decisions based on abstracts of healthcare research: A systematic review

OBJECTIVE

To summarize the reporting quality of healthcare abstracts and inconsistencies between abstracts and full texts.

STUDY DESIGN AND SETTING

This systematic review included overviews of randomized controlled trials (RCTs) and systematic reviews (SRs) that summarized data of healthcare abstracts on reporting of abstracts and consistency of abstracts with the full text. Searches were performed in PubMed, CENTRAL, Cochrane Library and EMBASE databases from 1900 to February 2019. Two authors screened the overviews and extracted the data. All analyses were descriptive and divided into two main groups: abstracts' reporting quality and abstracts' consistency with the full text. Abstracts were considered poorly reported and inconsistent with the full text if more than 5% of abstracts' information was not fully reported or not consistent with the full text.

RESULTS

27 overviews analyzing 5,194 RCTs and 866 SRs were retrieved for reporting quality of abstracts. A total of 22 overviews analyzing 2,025 RCTs and 551 SRs were included for consistency of abstracts with the full text. Abstracts across all healthcare areas presented poor reporting quality and were inconsistent with the full texts, with results and conclusions as the most inconsistent sections.

CONCLUSION

Abstracts of healthcare RCTs and SRs have shown a large room for improvement in reporting quality and consistency with the full text. Authors, journal editors and reviewers need to give the highest priority to this matter.

Reporting, interpretation, and extrapolation issues (SPIN) in abstracts of orthodontic meta-analyses published from 2000 to 2020

AIM

To assess the prevalence of and identify factors associated with SPIN in abstracts of orthodontic meta-analyses.

MATERIALS AND METHODS

Electronic search was performed within the contents of five orthodontic journals and the Cochrane Database of Systematic Reviews (CDSR) to identify meta-analyses of studies involving humans, from 1 January 2000 until 31 August 2020. Inclusion of SPIN in the abstract of meta-analyses, defined as misleading reporting, misleading interpretation, and inappropriate extrapolation of the findings, was documented. Extent of SPIN and associations with journal and year of publication, type of study, number of authors, continent of authorship, methodologist involvement, funding, and significance of the primary outcome were investigated.

RESULTS

One hundred and nine meta-analyses were identified, with the highest proportion being published in the European Journal of Orthodontics (EJO: 31/109; 28.4%). Inclusion of SPIN, in at least one domain, was recorded in nearly half (53/109; 48.6%) of the studies, of which 30 (56.6%) included 2 or more domains of SPIN. Meta-analyses of observational studies presented 1.66 times higher risk for including SPIN in their abstracts compared with interventional ones [95% confidence intervals (CIs): 1.14, 2.40; P = 0.007], after adjusting for a number of predictors. Studies with a large number of authors (≥6) presented 1.76 times higher risk of SPIN (≥6 versus 1-3: 95% CIs: 1.04, 2.97; Wald test, P = 0.021), conditional on the pre-defined predictors.

CONCLUSIONS

Flaws in the reporting and interpretation of the findings of abstracts of meta-analyses, as framed by inclusion of SPIN are persistent in orthodontic research, being more prevalent in meta-analyses of observational studies. Consistent, multidirectional efforts should be endorsed to improve the quality of the disseminated research findings.

There is still room for improvement in the completeness of abstract reporting according to the PRISMA-A checklist: a cross-sectional study on systematic reviews in periodontology

BACKGROUND

To evaluate the completeness of reporting abstracts of systematic reviews (SRs) before and after the publication of the PRISMA-A checklist in 2013 and to assess if an association exists between abstract characteristics and the completeness of reporting.

METHODS

A systematic search of the literature was conducted in the PubMed and Scopus databases in March 2020. The search focused on the SRs of evaluations of interventions published since 2002 in the field of periodontology. The abstracts of the selected SRs were divided into two groups before and after publication of the PRISMA-A checklist in 2013, and compliance with the 12 items reported in the checklist was evaluated by three calibrated evaluators.

RESULTS

A set of 265 abstracts was included in the study. The total score before (mean score, 53.78%; 95% CI, 51.56-55.90%) and after (mean score, 56.88%; 95% CI, 55.39-58.44%) the publication of the PRISMA-A statement exhibited a statistically significant improvement (P = 0.012*). Nevertheless, only the checklist items included studies and synthesis of the results displayed a statistically significant change after guideline publication. The total PRISMA-A score was higher in the meta-analysis group and in articles authored by more than four authors.

CONCLUSIONS

The impact of the PRISMA-A was statistically significant, but the majority of the items did not improve after its introduction. The editors and referees of periodontal journals should promote adherence to the checklist to improve the quality of the reports and provide readers with better insight into the characteristics of published studies.

Dental Research Waste in Design, Analysis, and Reporting: A Scoping Review

Research waste is highly prevalent across biomedical investigations. We aimed to assess the evidence on the extent of research waste in dental research. We performed a scoping review of empirical evaluations of dental studies assessing the prevalence and impact of limitations in design, conduct, analysis, and reporting of research. PubMed was searched using specific terms to retrieve studies dealing with design, conduct, analysis, and reporting of studies in dentistry, with no year or language restrictions. Of the 1,807 publications identified from the search and from manual searches, 71 were included in this review. The topic and article selection was based on the expert opinion of the authors. The existing evidence suggests that, although there are improvements over time, substantial deficiencies in all areas (design, conduct, analysis, reporting) were prevalent in dental research publications. Waste in research is a multifaceted problem without a simple solution. However, an appreciation of optimal research design and execution is a prerequisite and should be underpinned by policies that include appropriate training in research methods and properly aligned incentives.

Reporting quality of systematic review abstracts in operative dentistry: An assessment using the PRISMA for Abstracts guidelines

OBJECTIVES

To assess and compare the reporting quality of systematic review (SR) abstracts in operative dentistry published before and after the release of Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Abstracts (PRISMA-A), and to identify factors associated with reporting quality.

METHODS

PubMed was searched for abstracts published during 2010-2012 (Pre-PRISMA period) and 2017-2019 (Post-PRISMA period). Reporting quality was assessed and scored using a modified 13-item PRSIMA-A checklist. Risk ratio (RR) was used to compare the adequate reporting rate of each item between the two periods. Univariable and multivariable linear regression analyses were performed to identify factors associated with reporting quality.

RESULTS

A total of 160 abstracts were included and assessed. Only four items ('objective', 'results of main outcomes', 'description of the effect' and 'interpretation') were adequately reported in most abstracts (>75 %). According to the multivariable analysis, greater word count (P = 0.001), being published in the Post-PRISMA period (P = 0.025) and geographic origin from Asia (P = 0.025) or South America (P = 0.015) were significantly associated with higher reporting quality.

CONCLUSIONS/CLINICAL SIGNIFICANCE

The reporting quality of SR abstracts in operative dentistry had improved significantly after the publication of PRISMA-A, but was still suboptimal. Researchers, reviewers and journal editors in operative dentistry need to be familiar with the PRISMA-A checklist, and make concerted efforts to improve the reporting of SR abstracts.

An Evaluation of the Presence of Spin in the Abstracts of Tonsillectomy Systematic Reviews

OBJECTIVES/HYPOTHESIS

Spin-the practice of adding or omitting information intentionally or unintentionally to make the results of a study more favorable-may influence clinical decision making, especially when present in study abstracts. Here, we quantify and characterize the presence of spin in the abstracts of systematic reviews regarding tonsillectomy.

METHODS

This study is an analysis of systematic review abstracts. Searches were conducted on September 23, 2019 on PubMed and Embase using the advanced search feature to retrieve systematic reviews regarding tonsillectomies. The nine most severe forms of spin were then evaluated. Spin was classified by two investigators in parallel, with each blinded to the classifications of the other. Study characteristics were also recorded in duplicate. Consensus meetings between investigators were held to resolve disagreements.

RESULTS

In the 85 included systematic reviews, at least one form of spin was present in 44.7% (38/85) of abstracts. Journals with higher impact factors were less likely to contain spin in the abstracts of systematic reviews (point biserial correlation coefficient of -0.30). No statistically significant associations were found between the presence of spin and intervention type (P = .56) or adherence to Preferred Reporting Items for Systematic Reviews and Meta-Analyses (P = .08); however, there was a significant association between spin and funding source (P = .03).

CONCLUSIONS

Spin was common in the abstracts of our sample of tonsillectomy systematic reviews. Researchers, clinicians, and peer reviewers could benefit from learning to recognize spin in medical literature. Further research is needed into the effects of spin on clinical decision making.

LEVEL OF EVIDENCE

NA Laryngoscope, 2020.

Trends in the level of evidence and impact of clinical studies published in leading oral implantology journals: 2008-2018

OBJECTIVES

To present the characteristics and level of evidence (LOE) of clinical studies published in leading oral implantology journals during 2008-2018 and to explore whether the LOE of a study is associated with its scientific and social impact.

MATERIALS AND METHODS

Clinical studies with direct relevance to the evaluation of healthcare interventions published in 2008, 2013, and 2018 in six oral implantology journals were identified via hand searches. A modified 4-level Oxford 2011 LOE tool was used to assess the LOE of all eligible studies. The citation count and Altmetric Attention Score (AAS) of each study were extracted from Web of Science and Altmetric Explorer, respectively. Thereafter, multivariable generalized estimation equation analyses were used to investigate the association between LOE, citation counts, and AAS, adjusting for potential confounding factors and clustering effects.

RESULTS

A total of 763 clinical studies were included, among which the proportion of level-1, level-2, level-3, and level-4 studies was 2.4%, 30.4%, 40.2%, and 27.0%, respectively. During 2008-2018, the proportion of high LOE studies (level-1 and level-2) increased substantially from 24.6% to 43.1%, although the number of systematic reviews that only include randomized controlled trials has remained limited. According to multivariable analyses, the citation count (p = .002) and AAS (p = .005) of high LOE studies were both significantly greater than those of low LOE studies.

CONCLUSIONS

During the past decade, the proportion of high LOE studies has increased substantially in the field of oral implantology. Clinical studies with higher LOE tend to have greater scientific and social impact.

Has the Reporting Quality of Systematic Review Abstracts in Nursing Improved Since the Release of PRISMA for Abstracts? A Survey of High-Profile Nursing Journals

BACKGROUND

The PRISMA for Abstracts (PRISMA-A) was developed to guide authors to present a structured abstract. However, the adherence of abstracts to these guidelines in some areas was of concern.

AIMS

To determine whether the publication of PRISMA-A resulted in an improvement in the abstracts reported with nursing systematic reviews (SRs).

METHODS

This was a cross-sectional study. We searched PubMed for randomized controlled trials-based SRs published in top-tier nursing journals. A PRISMA-A checklist was used to assess abstracts in the SR included. Total score on checklists, comparison of total scores between two periods, and effect factors were analyzed.

RESULTS

Overall, abstract reporting compliance with PRISMA-A has not improved significantly with the time span. Of the 81 SRs, 74.1% were structured. About half reported eligibility criteria, information sources, and description of the effect as recommended. Registration status was reported only in 4.9%. The reporting quality was significantly higher for journals with higher impact factors (p < .001).

LINKING EVIDENCE TO ACTION

Although not inclusive of all SRs in the nursing field, our sample reflects the general trend that there was no significant improvement in the compliance of SR abstracts reported in nursing with the release of PRISMA-A. There is room for improvement, as most items have not been fully reported.

The Reporting Quality of Systematic Reviews in Japanese Physical Therapy Journals

OBJECTIVES

The purpose of this study was to evaluate the quality of reporting of systematic reviews published in Japanese in the field of physical therapy.

METHODS

The study design was a bibliometric analysis of systematic reviews. Two Japanese physical therapy journals (Physical Therapy Japan and Rigakuryoho Kagaku) were analysed using J-STAGE. The inclusion criterion was that articles were systematic reviews. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist was used to score the reporting quality of eligible systematic reviews. The quality assessment was performed by two reviewers independently.

RESULTS

Of the 1578 articles identified, thirteen articles were included in this study. The median score of checklist items adequately adhered to across the included studies was 12 (range, 7-17). None of the studies adhered to the structured summary or additional analysis PRISMA items. The intention of bias assessment across studies was reported in only three studies (23%), and only two of these three reported the results.

CONCLUSIONS

The reporting quality of systematic reviews published in Japanese physical therapy journals was suboptimal. Therefore, readers should critically appraise the contents of systematic reviews. It is recommended that journals should strictly require their authors to adhere to reporting guidelines.

Eight in Every 10 Abstracts of Low Back Pain Systematic Reviews Presented Spin and Inconsistencies With the Full Text: An Analysis of 66 Systematic Reviews

STUDY DESIGN

Overview study.

BACKGROUND

Abstracts of systematic reviews have presented 'spin' (i.e. overstated interpretation of study results) and inconsistency with the full text.

OBJECTIVES

1. Do abstracts of low back pain reviews contain spin? 2. Do these abstracts consistently represent the full text? 3. Is abstract spin associated with the type of conclusion?

METHODS

We searched the Physiotherapy Evidence Database (PEDro) on 10th January 2018. Data were extracted from systematic reviews of physiotherapy interventions for low back pain, published between 2015 and 2017. Spin was assessed using a 7-item checklist. We evaluated consistency by comparing information contained in the abstract and the full text using the 7-item checklist with Kappa coefficient analysis. We used logistic regression analysis to evaluate the association between spin in the abstract and type of conclusion. We evaluated methodological quality using the AMSTAR-2 (A MeaSurement Tool to Assess systematic Reviews).

RESULTS

We included 66 eligible systematic reviews, subdivided into Cochrane (n=9) and non-Cochrane (n=57) reviews. There was some form of spin in 80% of abstracts. Abstracts of non-Cochrane reviews were not consistent with the full text (fair to moderate agreement). Cochrane review abstracts had substantial to almost perfect agreement with the full text. Spin was not associated with the type of conclusion in all systematic reviews (P < 0.05). The methodological quality ranged from 'high' to 'critically low'.

CONCLUSIONS

The abstracts of systematic reviews evaluating physiotherapy interventions for low back pain need improvement. J Orthop Sports Phys Ther, Epub 23 Aug 2019. doi:10.2519/jospt.2020.8962.

Systematic reviews in orthodontics: Impact of the PRISMA for Abstracts checklist on completeness of reporting

INTRODUCTION

This study evaluated and compared the completeness of reporting of abstracts of orthodontics systematic reviews before and after the publication of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) extension for Abstracts Checklist (PRISMA-A).

METHODS

Abstracts of systematic reviews and meta-analyses in orthodontics published in PubMed, Latin American and Caribbean Health Sciences Literature, and the Cochrane Database of Systematic Reviews databases before March 23, 2018, that met the predefined inclusion and exclusion criteria, were evaluated using the 12 items of PRISMA-A, scoring each item from 0 to 2. Abstracts were classified into 2 groups: before and after publication of the PRISMA-A checklist. Three calibrated evaluators (intraclass correlation coefficient and kappa > 0.8) assessed the scores for compliance with the checklist. The number of authors, country of affiliation of the first author, performance of meta-analysis, and topic of the article were recorded. A regression analysis was performed to assess the associations between abstract characteristics and the PRISMA-A scores.

RESULTS

Of 1034 abstracts evaluated, 389 were included in the analysis. The mean PRISMA-A score was 53.39 (95% CI, 51.83-54.96). The overall score for studies published after the publication of the checklist was significantly higher than for studies published before (P ≤ 0.0001). The components returning significantly higher scores after publication of PRISMA-A were title (P = 0.024), information from databases (P = 0.026), risk of bias (P ≤ 0.0001), included studies (P ≤ 0.0001), synthesis of results (P ≤ 0.0001), interpretation of results (P = 0.035), financing and conflict of interest (P ≤ 0.0001), and registration (P ≤ 0.0001). These results showed the positive effect of PRISMA-A had on the quality of reporting of orthodontics systematic reviews. Nevertheless, the poor adherence revealed that there is still need for improvement in the quality of abstract reporting.

CONCLUSIONS

The quality of reporting of abstracts of orthodontic systematic reviews and meta-analyses increased after the introduction of PRISMA-A.

Assessment of the abstract reporting of systematic reviews of dose-response meta-analysis: a literature survey

Background

There is an increasing number of published systematic reviews (SR) of dose-response meta-analyses (DRMAs) over the past decades. However, the quality of abstract reporting of these SR-DRMAs remains to be understood. We conducted a literature survey to investigate the abstract reporting of SR-DRMAs.

Methods

Medline, Embase, and Wiley online Library were searched for eligible SR-DRMAs. The reporting quality of SR-DRMAs was assessed by the modified PRISMA-for-Abstract checklist (14 items). We summarized the adherence rate of each item and categorized them as well complied (adhered by 80% or above), moderately complied (50 to 79%), and poorly complied (less than 50%). We used total score to reflect the abstract quality and regression analysis was employed to explore the potential influence factors for it.

Results

We included 529 SR-DRMAs. Eight of 14 items were moderately (3 items) or poorly complied (5 items) while only 6 were well complied by these SR-DRMAs. Most of the SR-DRMAs failed to describe the methods for risk of bias assessment (30.2, 95% CI: 26.4, 34.4%) and the results of bias assessment (48.8, 95% CI: 44.4, 53.1%). Few SR-DRMAs reported the funding (2.3, 95% CI: 1.2, 3.9%) and registration (0.6, 95% CI: 0.1, 1.6%) information in the abstract. Multivariable regression analysis suggested word number of abstracts [> 250 vs. ≤ 250 (estimated ß = 0.31; 95% CI: 0.02, 0.61; P = 0.039)] was positively associated with the abstract reporting quality.

Conclusion

The abstract reporting of SR-DRMAs is suboptimal, substantial effort is needed to improve the reporting. More word number may benefit for the abstract reporting. Given that reporting of abstract largely depends on the reporting and conduct of the SR-DRMA, review authors should also focus on the completeness of SR-DRMA itself.

Electronic supplementary material

The online version of this article (10.1186/s12874-019-0798-5) contains supplementary material, which is available to authorized users.

Reporting Quality of Systematic Review Abstracts Published in Leading Neurosurgical Journals: A Research on Research Study

BACKGROUND

Systematic review (SR) abstracts are frequently relied upon to guide clinical decision-making. However, there is mounting evidence that the quality of abstract reporting in the medical literature is suboptimal.

OBJECTIVE

To appraise SR abstract reporting quality in neurosurgical journals and identify factors associated with improved reporting.

METHODS

This study systematically surveyed SR abstracts published in 8 leading neurosurgical journals between 8 April 2007 and 21 August 2017. Abstracts were identified through a search of the MEDLINE database and their reporting quality was determined in duplicate using a tool derived from the Preferred Reporting Items for Systematic Reviews and Meta-analyses for Abstracts (PRISMA-A) statement. All SR abstracts that provided comparison between treatment strategies were eligible for inclusion. Descriptive statistics were utilized to identify factors associated with improved reporting.

RESULTS

A total of 257 abstracts were included in the analysis, with a mean of 22.8 (±25.3) included studies. The overall quality of reporting in included abstracts was suboptimal, with a mean score of 53.05% (±11.18). Reporting scores were higher among abstracts published after the release of the PRISMA-A guidelines (M = 56.52; 21.74-73.91) compared with those published beforehand (M = 47.83; 8.70-69.57; U = 4346.00, z = -4.61, P < .001). Similarly, both word count (r = 0.338, P < .001) and journal impact factor (r = 0.199, P = .001) were associated with an improved reporting score.

CONCLUSION

This study demonstrates that the overall reporting quality of abstracts in leading neurosurgical journals requires improvement. Strengths include the large number abstracts assessed, and its weaknesses include the fact that only neurosurgery-specific journals were surveyed. We recommend that attention be turned toward strengthening abstract submission and peer-review processes.

PRISMA for abstracts: best practice for reporting abstracts of systematic reviews in Endodontology

An abstract is a brief overview of a scientific, clinical or review manuscript as well as a stand-alone summary of a conference abstract. Scientists, clinician-scientists and clinicians rely on the summary information provided in the abstracts of systematic reviews to assist in subsequent clinical decision-making. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) for Abstracts checklist was developed to improve the quality, accuracy and completeness of abstracts associated with systematic reviews and meta-analyses. The PRISMA for Abstracts checklist provides a framework for authors to follow, which helps them provide in the abstract the key information from the systematic review that is required by stakeholders. The PRISMA for Abstracts checklist contains 12 items (title, objectives, eligibility criteria, information sources, risk of bias, included studies, synthesis of results, description of the effect, strength and limitations, interpretation, funding and systematic review registration) under six sections (title, background, methods, results, discussion, other). The current article highlights the relevance and importance of the items in the PRISMA for Abstracts checklist to the specialty of Endodontology, while offering explanations and specific examples to assist authors when writing abstracts for systematic reviews when reported in manuscripts or submitted to conferences. Strict adherence to the PRISMA for Abstracts checklist by authors, reviewers, and journal editors will result in the consistent publication of high-quality abstracts within Endodontology. This article is protected by copyright. All rights reserved.

Evaluations of the uptake and impact of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement and extensions: a scoping review

BACKGROUND

The PRISMA Statement is a reporting guideline designed to improve transparency of systematic reviews (SRs) and meta-analyses. Seven extensions to the PRISMA Statement have been published to address the reporting of different types or aspects of SRs, and another eight are in development. We performed a scoping review to map the research that has been conducted to evaluate the uptake and impact of the PRISMA Statement and extensions. We also synthesised studies evaluating how well SRs published after the PRISMA Statement was disseminated adhere to its recommendations.

METHODS

We searched for meta-research studies indexed in MEDLINE® from inception to 31 July 2017, which investigated some component of the PRISMA Statement or extensions (e.g. SR adherence to PRISMA, journal endorsement of PRISMA). One author screened all records and classified the types of evidence available in the studies. We pooled data on SR adherence to individual PRISMA items across all SRs in the included studies and across SRs published after 2009 (the year PRISMA was disseminated).

RESULTS

We included 100 meta-research studies. The most common type of evidence available was data on SR adherence to the PRISMA Statement, which has been evaluated in 57 studies that have assessed 6487 SRs. The pooled results of these studies suggest that reporting of many items in the PRISMA Statement is suboptimal, even in the 2382 SRs published after 2009 (where nine items were adhered to by fewer than 67% of SRs). Few meta-research studies have evaluated the adherence of SRs to the PRISMA extensions or strategies to increase adherence to the PRISMA Statement and extensions.

CONCLUSIONS

Many studies have evaluated how well SRs adhere to the PRISMA Statement, and the pooled result of these suggest that reporting of many items is suboptimal. An update of the PRISMA Statement, along with a toolkit of strategies to help journals endorse and implement the updated guideline, may improve the transparency of SRs.

Quality and clarity in systematic review abstracts: an empirical study

BACKGROUND

Systematic review (SR) abstracts are important for disseminating evidence syntheses to inform medical decision making. We assess reporting quality in SR abstracts using PRISMA for Abstracts (PRISMA-A), Cochrane Handbook, and Agency for Healthcare Research & Quality guidance.

METHODS

We evaluated a random sample of 200 SR abstracts (from 2014) comparing interventions in the general medical literature. We assessed adherence to PRISMA-A criteria, problematic wording in conclusions, and whether "positive" studies described clinical significance.

RESULTS

On average, abstracts reported 60% of PRISMA-A checklist items (mean 8.9 ± 1.7, range 4 to 12). Eighty percent of meta-analyses reported quantitative measures with a confidence interval. Only 49% described effects in terms meaningful to patients and clinicians (e.g., absolute measures), and only 43% mentioned strengths/limitations of the evidence base. Average abstract word count was 274 (SD 89). Word count explained only 13% of score variability. PRISMA-A scores did not differ between Cochrane and non-Cochrane abstracts (mean difference 0.08, 95% confidence interval -1.16 to 1.00). Of 275 primary outcomes, 48% were statistically significant, 32% were not statistically significant, and 19% did not report significance or results. Only one abstract described clinical significance for positive findings. For "negative" outcomes, we identified problematic simple restatements (20%), vague "no evidence of effect" wording (9%), and wishful wording (8%).

CONCLUSIONS

Improved SR abstract reporting is needed, particularly reporting of quantitative measures (for meta-analysis), easily interpretable units, strengths/limitations of evidence, clinical significance, and clarifying whether negative results reflect true equivalence between treatments. Copyright © 2016 John Wiley & Sons, Ltd.

Reporting quality in abstracts of meta-analyses of depression screening tool accuracy: a review of systematic reviews and meta-analyses

OBJECTIVE

Concerns have been raised regarding the quality and completeness of abstract reporting in evidence reviews, but this had not been evaluated in meta-analyses of diagnostic accuracy. Our objective was to evaluate reporting quality and completeness in abstracts of systematic reviews with meta-analyses of depression screening tool accuracy, using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) for Abstracts tool.

DESIGN

Cross-sectional study.

INCLUSION CRITERIA

We searched MEDLINE and PsycINFO from 1 January 2005 through 13 March 2016 for recent systematic reviews with meta-analyses in any language that compared a depression screening tool to a diagnosis based on clinical or validated diagnostic interview.

DATA EXTRACTION

Two reviewers independently assessed quality and completeness of abstract reporting using the PRISMA for Abstracts tool with appropriate adaptations made for studies of diagnostic test accuracy. Bivariate associations of number of PRISMA for Abstracts items complied with (1) journal abstract word limit and (2) A Measurement Tool to Assess Systematic Reviews (AMSTAR) scores of meta-analyses were also assessed.

RESULTS

We identified 21 eligible meta-analyses. Only two of 21 included meta-analyses complied with at least half of adapted PRISMA for Abstracts items. The majority met criteria for reporting an appropriate title (95%), result interpretation (95%) and synthesis of results (76%). Meta-analyses less consistently reported databases searched (43%), associated search dates (33%) and strengths and limitations of evidence (19%). Most meta-analyses did not adequately report a clinically meaningful description of outcomes (14%), risk of bias (14%), included study characteristics (10%), study eligibility criteria (5%), registration information (5%), clear objectives (0%), report eligibility criteria (0%) or funding (0%). Overall meta-analyses quality scores were significantly associated with the number of PRISMA for Abstracts scores items reported adequately (r=0.45).

CONCLUSIONS

Quality and completeness of reporting were found to be suboptimal. Journal editors should endorse PRISMA for Abstracts and allow for flexibility in abstract word counts to improve quality of abstracts.

Evaluation of risk of bias assessment of trials in systematic reviews of oral health interventions, 1991-2014: A methodology study

BACKGROUND

The authors aimed to describe how often and by what means investigators assessed the risk of bias of clinical trials in systematic reviews of oral health interventions and to identify factors associated with risk of bias assessments.

METHODS

The authors selected therapeutic oral health systematic reviews published from 1991 through 2014. They extracted data related to the tools used for risk of bias assessment of primary studies and data related to other review characteristics. They descriptively analyzed the data and used multivariate logistic regression.

RESULTS

The authors identified 1,114 oral health systematic reviews (130 Cochrane reviews and 984 non-Cochrane reviews). The investigators of the primary studies assessed risk of bias in 61.4% of the reviews, and the risk of bias assessments occurred more often in Cochrane reviews than in non-Cochrane reviews (100% versus 56.3%; P < .001) and in reviews published after the dissemination of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement (odds ratio [OR], 1.55; 95% confidence interval [CI], 1.17-2.06). Compared with the investigators of reviews of public oral health interventions, investigators of reviews of oral surgery were less likely to assess risk of bias (OR, 0.41; 95% CI, 0.25-0.67). Furthermore, the investigators of systematic reviews published in dental journals were less likely to assess risk of bias of individual trials (OR, 0.28; 95% CI, 0.19-0.41) compared with the investigators of reviews published in nondental journals.

CONCLUSIONS

The investigators of primary studies did not undertake risk of bias assessment in a considerable portion of non-Cochrane oral health systematic reviews. The investigators of reviews published in dental journals were less likely to assess risk of bias than the investigators of reviews published in nondental journals. The results of this study provide evidence of the need for improving the conduct and reporting of oral health systematic reviews with respect to risk of bias assessment.

PRACTICAL IMPLICATIONS

Clinicians should determine to what extent the findings of a systematic review are valid on the basis of whether the investigators assessed and considered risk of bias during the interpretation of findings.

Deficiencies in the publication and reporting of the results of systematic reviews presented at scientific medical conferences

OBJECTIVES

To evaluate the publication and quality of reporting of abstracts of systematic reviews presented at scientific medical conferences.

STUDY DESIGN AND SETTING

We included all abstracts of systematic reviews published in the proceedings of nine leading international conferences in 2010. For each conference abstract, we searched PubMed (January 1, 2010, to June 2013) to identify their corresponding full publication. We assessed the extent to which conference abstracts and their corresponding journal abstract reported items included in the Preferred Reporting Items for Systematic reviews and Meta-Analysis for Abstracts checklist and recorded any important discrepancies between sources.

RESULTS

We identified 197 abstracts of systematic reviews, representing <1% of the total number of conference abstracts presented. Of these 53% were published in full, the median time to publication was 14 months (interquartile range, 6.6-20.1 months). Although most conference and journal abstracts reported details of included studies (conference n = 83 of 103; 81% vs. journal n = 81 of 103; 79%), size and direction of effect (76% vs. 75%), and conclusions (79% vs. 81%), many failed to report the date of search (27% vs. 25%), assessment of risk of bias (18% vs. 12%), and the result for the main efficacy outcome(s) including the number of studies (37% vs. 31%) and participants (30% vs. 20%), harms(s) (17% vs. 17%), strengths (17% vs. 13%) and limitations (36% vs. 30%) of the evidence, or funding source (1% vs. 0%). There were discrepancies between journal and corresponding conference abstracts including deletion of studies (13%), changes in reported efficacy (11%), and harm (10%) outcome(s) and changes in the nature or direction of conclusions (24%).

CONCLUSION

Despite the importance of systematic reviews in the delivery of evidence-based health care, very few are presented at scientific conferences and only half of those presented are published in full. Serious deficiencies in the reporting of abstracts of systematic reviews make it difficult for readers to reliably assess their findings.

Discrepancies in Outcome Reporting Exist Between Protocols and Published Oral Health Cochrane Systematic Reviews

Objectives

To assess discrepancies in the analyzed outcomes between protocols and published reviews within Cochrane oral health systematic reviews (COHG) on the Cochrane Database of Systematic Reviews (CDSR).

Study Design and Setting

All COHG systematic reviews on the CDSR and the corresponding protocols were retrieved in November 2014 and information on the reported outcomes was recorded. Data was collected at the systematic review level by two reviewers independently.

Results

One hundred and fifty two reviews were included. In relation to primary outcomes, 11.2% were downgraded to secondary outcomes, 9.9% were omitted altogether in the final publication and new primary outcomes were identified in 18.4% of publications. For secondary outcomes, 2% were upgraded to primary, 12.5% were omitted and 30.9% were newly introduced in the publication. Overall, 45.4% of reviews had at least one discrepancy when compared to the protocol; these were reported in 14.5% reviews. The number of review updates appears to be associated with discrepancies between final review and protocol (OR: 3.18, 95% CI: 1.77, 5.74, p<0.001). The risk of reporting significant results was lower for both downgraded outcomes [RR: 0.52, 95% CI: 0.17, 1.58, p = 0.24] and upgraded or newly introduced outcomes [RR: 0.77, 95% CI: 0.36, 1.64, p = 0.50] compared to outcomes with no discrepancies. The risk of reporting significant results was higher for upgraded or newly introduced outcomes compared to downgraded outcomes (RR = 1.19, 95% CI: 0.65, 2.16, p = 0.57). None of the comparisons reached statistical significance.

Conclusion

While no evidence of selective outcome reporting was found in this study, based on the present analysis of SRs published within COHG systematic reviews, discrepancies between outcomes in pre-published protocols and final reviews continue to be common. Solutions such as the use of standardized outcomes to reduce the prevalence of this issue may need to be explored.

The Quality of the Evidence According to GRADE Is Predominantly Low or Very Low in Oral Health Systematic Reviews

Objectives

The main objective was to assess the credibility of the evidence using Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) in oral health systematic reviews on the Cochrane Database of Systematic Reviews (CDSR) and elsewhere.

Study Design and Setting

Systematic Reviews or meta-analyses (January 2008-December 2013) from 14 high impact general dental and specialty dental journals and the Cochrane Database of Systematic Reviews were screened for meta-analyses. Data was collected at the systematic review, meta-analysis and trial level. Two reviewers applied and agreed on the GRADE rating for the selected meta-analyses.

Results

From the 510 systematic reviews initially identified 91 reviews (41 Cochrane and 50 non-Cochrane) were eligible for inclusion. The quality of evidence was high in 2% and moderate in 18% of the included meta-analyses with no difference between Cochrane and non-Cochrane reviews, journal impact factor or year of publication. The most common domains prompting downgrading of the evidence were study limitations (risk of bias) and imprecision (risk of play of chance).

Conclusion

The quality of the evidence in oral health assessed using GRADE is predominantly low or very low suggesting a pressing need for more randomised clinical trials and other studies of higher quality in order to inform clinical decisions thereby reducing the risk of instituting potentially ineffective and/or harmful therapies.

The reporting quality of meta-analysis results of systematic review abstracts in periodontology and implant dentistry is suboptimal

ARTICLE TITLE AND BIBLIOGRAPHIC INFORMATION

Assessment of the quality of reporting in abstracts of systematic reviews with meta-analyses in periodontology and implant dentistry. Faggion CM Jr., Liu J, Huda F, Atieh M. J Periodontal Res 2014; 49(2):137-42.

REVIEWER

Argy Polychronopoulou, DDS, MS, ScM, ScD PURPOSE/QUESTION: What is the reporting quality of meta-analysis results of abstracts of systematic reviews in periodontology and implant dentistry journals?

SOURCE OF FUNDING

The authors received no funding for this study

TYPE OF STUDY/DESIGN

Meta-epidemiological study

LEVEL OF EVIDENCE

Level 3: Other evidence

STRENGTH OF RECOMMENDATION GRADE

Not applicable.

Reporting quality of systematic reviews/meta-analyses of acupuncture

BACKGROUND

The QUOROM and PRISMA statements were published in 1999 and 2009, respectively, to improve the consistency of reporting systematic reviews (SRs)/meta-analyses (MAs) of clinical trials. However, not all SRs/MAs adhere completely to these important standards. In particular, it is not clear how well SRs/MAs of acupuncture studies adhere to reporting standards and which reporting criteria are generally ignored in these analyses.

OBJECTIVES

To evaluate reporting quality in SRs/MAs of acupuncture studies.

METHODS

We performed a literature search for studies published prior to 2014 using the following public archives: PubMed, EMBASE, Web of Science, the Cochrane Database of Systematic Reviews (CDSR), the Chinese Biomedical Literature Database (CBM), the Traditional Chinese Medicine (TCM) database, the Chinese Journal Full-text Database (CJFD), the Chinese Scientific Journal Full-text Database (CSJD), and the Wanfang database. Data were extracted into pre-prepared Excel data-extraction forms. Reporting quality was assessed based on the PRISMA checklist (27 items).

RESULTS

Of 476 appropriate SRs/MAs identified in our search, 203, 227, and 46 were published in Chinese journals, international journals, and the Cochrane Database, respectively. In 476 SRs/MAs, only 3 reported the information completely. By contrast, approximately 4.93% (1/203), 8.81% (2/227) and 0.00% (0/46) SRs/Mas reported less than 10 items in Chinese journals, international journals and CDSR, respectively. In general, the least frequently reported items (reported≤50%) in SRs/MAs were "protocol and registration", "risk of bias across studies", and "additional analyses" in both methods and results sections.

CONCLUSIONS

SRs/MAs of acupuncture studies have not comprehensively reported information recommended in the PRISMA statement. Our study underscores that, in addition to focusing on careful study design and performance, attention should be paid to comprehensive reporting standards in SRs/MAs on acupuncture studies.