Retrieving Cochrane reviews is sometimes challenging and their reporting is not always optimal

Cochrane review abstracts are getting longer, but this has no large impact on the reporting quality

OBJECTIVE

To evaluate the reporting of abstracts of Cochrane Reviews according to PRISMA-A and to investigate a possible association with the abstract´s length.

METHODS

This is a retrospective, observational study based on all Cochrane Reviews indexed in Medline (via PubMed) until November 18, 2022. In the second part, a random sample of 440 abstracts was drawn, in which PRISMA-A adherence was assessed by two independent reviewers. Analyses were stratified by the year of publication and the number of words.

RESULTS

Overall, the median number of words of the 15,188 included abstracts was 469 (IQR 389-686 words), steadily increasing from 353 words in 2000 to 838 words in 2022, with less than one percent of the abstracts in 2022 having ≤ 300 words (in 2000: 30.7%). Analyses on PRISMA-A adherence in the random sample showed a mean score of 6.1 out of 12 fully reported items. Stratified by year, PRISMA-A adherence increased, with higher word counts in 2000-2010 and 2011-2015, while there was no difference in PRISMA-A adherence by abstract length in 2016-2022.

CONCLUSION

Over the years, abstracts of Cochrane Reviews have become longer, running up to 1000 words. This conflicts with the Cochrane Handbook, which recommends a maximum length of 400 until it was aligned with MECIR in 2019, which has recommended a length of <700 words since 2012 but allows up to 1000 words. It is debatable whether such long abstracts meet the key goals of abstracts of being informative, accurate, appealing, and concise.

Reporting according to the preferred reporting items for systematic reviews and meta-analyses for abstracts (PRISMA-A) depends on abstract length

OBJECTIVE

To evaluate reporting of abstracts of systematic reviews according to the preferred reporting items for systematic reviews and meta-analyses for abstracts (PRISMA-A) 2013 checklist.

STUDY DESIGN AND SETTING

A random sample of 534 systematic reviews on effectiveness indexed in PubMed between 2000 and 2019 was assessed. Adherence of abstracts to PRISMA-A was analysed using descriptive statistics. Results were stratified by number of words, structure, and year of publication.

RESULT

The mean score of fully reported PRISMA-A items was 5.4 of 12, with adherence varying widely between items (0% to 98.8%). Cochrane reviews received higher mean total scores than non-Cochrane reviews (6.3 vs. 5.2). Adherence to PRISMA-A increased linearly with increasing word count. In non-Cochrane reviews, authors of structured abstracts more often adhered to PRISMA-A than those of unstructured abstracts. No improvements in reporting of abstracts were found after the implementation of PRISMA-A in 2013.

CONCLUSION

Adherence to PRISMA-A shows great potential for improvement. Therefore, authors, editors, and reviewers should be made aware of PRISMA-A by referring to it in the journal submission guidelines. As adherence to PRISMA-A increases with the number of words, journals should consider to increase the word limit to 250-300 words.

Restrictions and their reporting in systematic reviews of effectiveness: an observational study

BACKGROUND

Restrictions in systematic reviews (SRs) can lead to bias and may affect conclusions. Therefore, it is important to report whether and which restrictions were used. This study aims to examine the use of restrictions regarding language, publication period, and study type, as well as the transparency of reporting in SRs of effectiveness.

METHODS

A retrospective observational study was conducted with a random sample of 535 SRs of effectiveness indexed in PubMed between 2000 and 2019. The use of restrictions and their reporting were analysed using descriptive statistics.

RESULTS

Of the total 535 SRs included, four out of every ten (41.3%) lacked information on at least one of the three restrictions considered (language, publication period, or study type). Overall, 14.6% of SRs did not provide information on restrictions regarding publication period, 19.1% regarding study type, and 18.3% regarding language. Of all included SRs, language was restricted in 46.4%, and in more than half of the SRs with restricted language (130/248), it was unclear whether the restriction was applied during either the search or the screening process, or both. The restrictions were justified for publication period in 22.2% of the respective SRs (33/149), study type in 6.5% (28/433), and language in 3.2% (8/248). Differences in reporting were found between countries as well as between Cochrane and non-Cochrane reviews.

CONCLUSIONS

This study suggests that there is a lack of transparency in reporting on restrictions in SRs. Authors as well as editors and reviewers should be encouraged to improve the reporting and justification of restrictions to increase the transparency of SRs.