Quality of reporting of systematic reviews and meta-analyses of surgical randomized clinical trials

The Value of Systematic Reviews and Meta-Analyses in Surgery

BACKGROUND

Systematic reviews and meta-analyses are generally regarded as sitting atop the hierarchy of clinical evidence. The unbiased summary of current evidence that a systematic review provides, along with the increased statistical power from larger numbers of patients, is invaluable in guiding clinical decision-making and development of practice guidelines. Surgical specialties have historically lagged behind other areas of medicine in the application of evidence-based medicine, perhaps due to the unique challenges faced in the conduct of surgical clinical trials. These challenges extend to the conduct of systematic reviews, due to issues with the quality and heterogeneity of the underlying literature.

SUMMARY

Recent years have seen an improvement in the quality of randomized controlled trials in surgical topics and an explosion in the publication of systematic reviews. This review explores recent trends in systematic reviews in surgery and discussed some of the aspects in conducting and interpreting reviews that are unique to surgical topics, including blinding, surgical heterogeneity and learning curves, patient and clinician preference, and industry involvement. Key Messages: Clinical trials, and therefore systematic reviews, of surgical interventions pose unique challenges which are important to consider when conducting them or applying the findings to clinical practice. Despite the challenges, systematic reviews still represent the best level of evidence for development of surgical practice guidelines.

Clinical Epidemiology in China series. Paper 3: The methodological and reporting quality of systematic reviews and meta-analyses published by China' researchers in English-language is higher than those published in Chinese-language

OBJECTIVE

To assess the methodological and reporting quality of Chinese- and English -language systematic reviews and meta-analyses (SRs/MAs) published by Chinese authors between 2016 and 2018.

STUDY DESIGN AND SETTING

We searched MEDLINE and Chinese Science Citation Database (CSCD) for SRs/MAs led by Chinese authors published between 2016 and 2018. We used random sampling to select 10% of the eligible SRs/MAs published in each year from CSCD, and then matched the same number of SRs/MAs in MEDLINE. Reporting quality was evaluated using Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) and methodological quality using the Assessment of Multiple Systematic Reviews (AMSTAR-2) tool. Stratified analyses were conducted to compare the differences of quality between Chinese- and English language SRs/MAs.

RESULTS

We identified 336 SRs/MAs (168 in Chinese and 168 in English). The reporting quality in Chinese-language SRs/MAs was slightly lower than English-language SRs/MAs (mean PRISMA scores: 20.58 vs. 21.71 in 2016, 19.87 vs. 21.24 in 2017, and 21.29 vs. 22.38 in 2018). Less than half of both Chinese- and English-language SRs/MAs complied with item 5 (protocol and registration), item 7 (information sources), item 8 (search) and item 27 (funding)). The methodological quality in Chinese -language SRs/MAs was also slightly lower than English -language SRs/MAs (mean AMSTAR-2 scores: 8.07 vs. 9.36 in 2016; 9.21 vs. 10.26 in 2017; 8.86 vs. 9.28 in 2018). Three items (item 2: established a protocol; item 4: use a comprehensive literature search; and item 10: report the sources of funding) were adhered to by less than 10% of both Chinese- and English -language SRs/MAs. Only one (0.6%) Chinese-language SRs/MA and nine (5.4%) English-language SRs/MAs were rated as high methodological quality.

CONCLUSION

The reporting and methodological quality of English-language SRs/MAs conducted by authors from China between 2016 and 2018 were slightly better than those of Chinese -language SRs/MAs.

Evaluation of spin in the abstracts of systematic reviews and meta-analyses focused on cataract therapies

PURPOSE

Spin-the misrepresentation of study findings such that the beneficial effects of an intervention are magnified beyond what the results actually show-is a reporting practice that has been shown to influence perceptions of treatment efficacy and clinical decision making. We evaluated the extent of spin in the abstracts of systematic reviews of cataract surgery and its complications. We also evaluated whether particular study attributes were associated with spin.

DESIGN

Cross-sectional study.

METHODS

We searched MEDLINE and Embase for systematic reviews and meta-analyses relating to cataract treatment. From these search records, screening for eligible studies was done in duplicate. Using a previously developed classification system for spin, we assessed the systematic reviews that met our eligibility criteria for the occurrence of the 9 most severe forms of spin. We performed the evaluation of spin, extracted study characteristics, and appraised the methodological quality of each study using the 16-question AMSTAR-2 scale in duplicate.

RESULTS

Searches retrieved 2,059 studies, of which 110 were eligible for data extraction. We found at least 1 form of spin in 30.0% of included systematic reviews (33/110). Six of the 9 types of spin were identified in our sample, the most common being type 3 in 18.2% (20/110) of abstracts. We found no significant association between spin in abstracts, AMSTAR-2 appraisal, and any of the extracted study characteristics.

CONCLUSION

Spin was evident in approximately one-third of the abstracts of evaluated systematic reviews and meta-analyses of cataract surgery and associated complications.