Adopting AMSTAR 2 critical appraisal tool for systematic reviews: speed of the tool uptake and barriers for its adoption

Frequency of use and adequacy of Cochrane risk of bias tool 2 in non-Cochrane systematic reviews published in 2020: Meta-research study

Risk of bias (RoB) assessment is essential to the systematic review methodology. The new version of the Cochrane RoB tool for randomized trials (RoB 2) was published in 2019 to address limitations identified since the first version of the tool was published in 2008 and to increase the reliability of assessments. This study analyzed the frequency of usage of the RoB 2 and the adequacy of reporting the RoB 2 assessments in non-Cochrane reviews published in 2020. This meta-research study included non-Cochrane systematic reviews of interventions published in 2020. For the reviews that used the RoB 2 tool, we analyzed the reporting of the RoB 2 assessment. Among 3880 included reviews, the Cochrane RoB 1 tool was the most frequently used (N = 2228; 57.4%), followed by the Cochrane RoB 2 tool (N = 267; 6.9%). From 267 reviews that reported using the RoB 2 tool, 213 (79.8%) actually used it. In 26 (12.2%) reviews, erroneous statements were used to indicate the RoB 2 assessment. Only 20 (9.4%) reviews presented a complete RoB 2 assessment with a detailed table of answers to all signaling questions. The judgment of risk of bias by the RoB 2 tool was not justified by a comment in 158 (74.2%) reviews. Only in 33 (14.5%) of reviews the judgment in all domains was justified in the accompanying comment. In most reviews (81.7%), the RoB was inadequately assessed at the study level. In conclusion, the majority of non-Cochrane reviews published in 2020 still used the Cochrane RoB 1 tool. Many reviews used the RoB 2 tool inadequately. Further studies about the uptake and the use of the RoB 2 tool are needed.

Quality of systematic reviews on the treatment of vesiculobullous skin diseases. A meta-epidemiological study

BACKGROUND

Systematic reviews of Randomized Controlled Trials (RCTs) are considered high-level evidence to support a decision on therapeutic interventions, and their methodological quality is essential to provide reliable and applicable results.

OBJECTIVE

This meta-epidemiological study aimed to map and critically appraise systematic reviews assessing treatments for vesiculobullous skin diseases.

METHODS

We conducted a comprehensive search strategy on MEDLINE (via Pubmed) in December 2022 without restrictions to find systematic reviews evaluating pharmacological interventions for vesiculobullous skin diseases. The methodological quality was assessed using the AMSTAR-2 tool, and additional information was extracted. We identified nine systematic reviews published between 2002 and 2021, seven assessing pemphigus.

RESULTS

According to the AMSTAR-2 tool, 55.6% were classified as critically low quality, 22.2% as moderate quality, 11.1% as low and 11.1% as high quality. No review assessed the certainty of the evidence (GRADE); 86% of pemphigus reviews had at least two overlapping RCTs. There were some limitations regarding methodological flaws and the AMSTAR-2 tool use CONCLUSIONS: These findings reveal a frail methodological quality of systematic reviews about vesiculobullous diseases treatment that may impact the results. Therefore, methodological rigor is mandatory for future systematic reviews to avoid duplication of effort and increase the certainty of the evidence supporting decision-making.

Author instructions in biomedical journals infrequently address systematic review reporting and methodology: a cross-sectional study

OBJECTIVES

We aimed to analyze how instructions for authors in journals indexed in MEDLINE address systematic review (SR) reporting and methodology.

STUDY DESIGN AND SETTING

We analyzed instructions for authors in 20% of MEDLINE-indexed journals listed in the online catalog of the National Library of Medicine on July 27, 2021. We extracted data only from the instructions published in English. We extracted data on the existence of instructions for reporting and methodology of SRs.

RESULTS

Instructions from 1,237 journals mentioned SRs in 45% (n = 560) of the cases. Systematic review (SR) registration was mentioned in 104/1,237 (8%) of instructions. Guidelines for reporting SR protocols were found in 155/1,237 (13%) of instructions. Guidelines for reporting SRs were explicitly mentioned in 461/1,237 (37%), whereas the EQUATOR (Enhancing the Quality and Transparency of Health Research) network was referred to in 474/1,237 (38%) of instructions. Less than 2% (n = 20) of instructions mentioned risk of bias and meta-analyses; less than 1% mentioned certainty of evidence assessment, methodological expectations, updating of SRs, overviews of SRs, or scoping reviews.

CONCLUSION

Journals indexed in MEDLINE rarely provide instructions for authors regarding SR reporting and methodology. Such instructions could potentially raise authors' awareness and improve how SRs are prepared and reported.

Quality of systematic reviews with meta-analyses of resveratrol: A methodological systematic review

Recently, several meta-analyses (MAs) have focused on the health effects of resveratrol. However, the methodological and reporting quality of these MAs has not yet been fully evaluated so far. Therefore, the present study evaluated the quality of these MAs through a methodological systematic review. Systematic searches were conducted in PubMed, Embase, Web of Science, and Cochrane Library from inception until May 20, 2022, and PubMed was used to update the search until September 6, 2023. The methodological and reporting quality of the selected MAs was evaluated using AMSTAR-2 and PRISMA 2009. Fifty-one MAs published during 2013-2023 were included. In each review, the number of primary studies ranged from 3 to 37, and the number of participants ranged from 50 to 2114. Among the first-listed primary outcomes, only 23 (45.10%) were "positive." As for the methodological quality, most MAs (44, 86.27%) on resveratrol were rated critically low. Inadequate reporting of the included MAs mainly involved items 2 ("Structured summary"), 5 ("Protocol and registration"), 8 ("Search"), 9 ("Study selection"), 10 ("Data collection process"), 12 ("Risk of bias in individual studies"), and 24 ("Summary of evidence") based on the PRISMA 2009. Additionally, journal's impact factor, number of authors, and funding support were positively associated with the overall methodological quality but were not statistically significant (p > 0.05). Future MAs on resveratrol require better design, implementation, and reporting by following the Cochrane Handbook, AMSTAR-2, and PRISMA.

Most systematic reviews reporting adherence to AMSTAR 2 had critically low methodological quality: a cross-sectional meta-research study

OBJECTIVES

To analyze the methodological quality and characteristics of systematic reviews (SRs) that reported they were conducted in line with the AMSTAR 2 (A MeaSurement Tool to Assess Systematic Reviews).

STUDY DESIGN AND SETTING

This was a cross-sectional meta-research study. We searched MEDLINE and Embase. We included full reports of SRs reporting the study was conducted, prepared, or designed in line with the AMSTAR 2. Eligible SRs were those published from January 1, 2018, until May 3, 2022. We assessed the methodological quality of the included SRs using AMSTAR 2.

RESULTS

We included a total of 45 records. There were 43 SRs and 2 SR protocols. Among them, most were SRs of interventions that included primary studies on humans. More than half had a meta-analysis. According to our overall AMSTAR 2 assessments of included SRs, 35 SRs were of critically low confidence, 7 SRs were of low confidence, and one SR was of high confidence. There were no SRs of moderate confidence.

CONCLUSION

Even when authors indicate in their manuscripts that the SR was conducted/prepared/designed in line with the AMSTAR 2, it does not necessarily imply it is of high or even moderate confidence according to AMSTAR 2. A self-assessment with AMSTAR 2 could be required for submission and carefully checked by the editors/peer reviewers.

Efficacy and safety of modular versus monoblock stems in revision total hip arthroplasty: a systematic review and meta-analysis

BACKGROUND

Both modular and monoblock tapered fluted titanium (TFT) stems are increasingly being used for revision total hip arthroplasty (rTHA). However, the differences between the two designs in clinical outcomes and complications are not yet clear. Here, we intend to compare the efficacy and safety of modular versus monoblock TFT stems in rTHA.

METHODS

PubMed, Embase, Web of Science, and Cochrane Library databases were searched to include studies comparing modular and monoblock implants in rTHA. Data on the survivorship of stems, postoperative hip function, and complications were extracted following inclusion criteria. Inverse variance and Mantel-Haenszel methods in Review Manager (version 5.3 from Cochrane Collaboration) were used to evaluate differences between the two groups.

RESULTS

Ten studies with a total of 2188 hips (1430 modular and 758 monoblock stems) were finally included. The main reason for the revision was aseptic loosening. Paprosky type III was the most common type in both groups. Both stems showed similar re-revision rates (modular vs monoblock: 10.3% vs 9.5%, P = 0.80) and Harris Hip Scores (WMD = 0.43, P = 0.46) for hip function. The intraoperative fracture rate was 11.6% and 5.0% (P = 0.0004) for modular and monoblock stems, respectively. The rate of subsidence > 10 mm was significantly higher in the monoblock group (4.5% vs 1.0%, P = 0.003). The application of extended trochanteric osteotomy was more popular in monoblock stems (22.7% vs 17.5%, P = 0.003). The incidence of postoperative complications such as periprosthetic femoral fracture and dislocation was similar between both stems.

CONCLUSIONS

No significant difference was found between modular and monoblock tapered stems as regards postoperative hip function, re-revision rates, and complications. Severe subsidence was more frequent in monoblock stems while modular ones were at higher risk of intraoperative fracture.

LEVEL OF EVIDENCE

Level III, systematic review of randomized control and non-randomized studies.

TRIAL REGISTRATION

We registered our study in the international prospective register of systematic reviews (PROSPERO) (CRD42020213642).