Quality assessment of systematic reviews on total hip or knee arthroplasty using mod-AMSTAR

Evaluating the quality of systematic reviews of comparative studies in autograft-based anterior cruciate ligament reconstruction using the AMSTAR-2 tool: A systematic umbrella review

PURPOSE

There remains a lack of consensus around autograft selection in anterior cruciate ligament reconstruction (ACLR), though there is a large body of overlapping systematic reviews and meta-analyses. Systematic reviews and their methodological quality were aimed to be further assessed, using a validated tool known as assessing the methodological quality of systematic reviews (AMSTAR-2).

METHODS

MEDLINE, Embase and CENTRAL were searched from inception to 23 April 2023 for systematic reviews (with/without meta-analysis) comparing primary ACLR autografts. A final quality rating from AMSTAR-2 was provided for each study ('critically low', 'low', 'moderate' or 'high' quality). Correlational analyses were conducted for ratings in relation to study characteristics.

RESULTS

Two thousand five hundred and ninety-eight studies were screened, and 50 studies were ultimately included. Twenty-four studies (48%) were rated as 'critically low', 17 (34%) as 'low', seven (14%) as 'moderate' and two (4%) as 'high' quality. The least followed domains were reporting on sources of funding (1/50 studies), the impact of risk of bias on results of meta-analyses (11/36 studies) and publication bias (17/36 studies). There was a significant increase in the frequency of studies graded as 'moderate' compared to 'low' or 'critically low' quality over time (p = 0.020).

CONCLUSION

The methodological quality of systematic reviews comparing autografts in ACLR is low, with many studies being rated lower due to commonly absent aspects of systematic review methodology such as investigating sources of funding and publication bias. More recent studies were generally more likely to be of higher quality. Authors are advised to consult AMSTAR-2 prior to conducting systematic reviews in ACLR.

LEVEL OF EVIDENCE

Level IV.

Methodological quality of systematic reviews in dentistry including animal studies: a cross-sectional study

BACKGROUND

The overall confidence in the results of systematic reviews including animal models can be heterogeneous. We assessed the methodological quality of systematic reviews including animal models in dentistry as well as the overall confidence in the results of those systematic reviews.

MATERIAL & METHODS

PubMed, Web of Science and Scopus were searched for systematic reviews including animal studies in dentistry published later than January 2010 until 18th of July 2022. Overall confidence in the results was assessed using a modified version of the A MeaSurement Tool to Assess systematic Reviews (AMSTAR-2) checklist. Checklist items were rated as yes, partial yes, no and not applicable. Linear regression analysis was used to investigate associations between systematic review characteristics and the overall adherence to the AMSTAR-2 checklist. The overall confidence in the results was calculated based on the number of critical and non-critical weaknesses presented in the AMSTAR-2 items and rated as high, moderate, low and critical low.

RESULTS

Of initially 951 retrieved systematic reviews, 190 were included in the study. The overall confidence in the results was low in 43 (22.6%) and critically low in 133 (70.0%) systematic reviews. While some AMSTAR-2 items were regularly reported (e.g. conflict of interest, selection in duplicate), others were not (e.g.

FUNDING

n = 1; 0.5%). Multivariable linear regression analysis showed that the adherence scores of AMSTAR-2 was significantly associated with publication year, journal impact factor (IF), topic, and the use of tools to assess risk of bias (RoB) of the systematic reviews.

CONCLUSION

Although the methodological quality of dental systematic reviews of animal models improved over the years, it is still suboptimal. The overall confidence in the results was mostly low or critically low. Systematic reviews, which were published later, published in a journal with a higher IF, focused on non-surgery topics, and used at least one tool to assess RoB correlated with greater adherence to the AMSTAR-2 guidelines.

The clinical effectiveness and safety of intravenous unfractionated heparin following digital replantation and revascularization: A narrative systematic review

BACKGROUND

Digital replants and revascularization (DRV) have been performed since the 1960s but there are no recognized standard peri-operative anticoagulation practices. A narrative systematic review of the clinical effectiveness and safety of therapeutic peri-operative unfractionated heparin following DRV was undertaken.

METHODS

A review of the literature from 1985 to March 2022 was conducted using Medline, Embase, CINAHL and EBM reviews. Unfractionated heparin (UFH) use following DRV was compared to low-molecular weight heparin, other anticoagulants or no anticoagulation. Randomized trials, observational studies as well as guidelines were selected and independently screened. The Revised Cochrane risk-of-bias (RoB 2) tool and ROBINS-I were used to appraise risk of bias.

RESULTS

While the search strategy identified 1490 references, only six studies met the inclusion criteria. Significant heterogeneity and the low methodological quality of the evidence precluded a meta-analysis. Among the four studies that documented the surgical success rate associated with the use of a therapeutic dose of UFH post DRV, only two reported improved clinical outcomes. Evidence of a higher complication rate related to UFH use was found in four studies. Low quality evidence suggests that a therapeutic dose of unfractionated heparin leads to a higher risk of complications when compared with heparin given as an intermittent bolus of unfractionated heparin or subcutaneous heparin, or prostaglandin E1 or no heparin.

CONCLUSIONS

Current evidence suggests that IV UFH use following DRV has no significant impact on the success of the intervention. Heparin use may not be innocuous as some studies showed increased bleeding complications.

Traduction franco-canadienne de l’Assessment of Systematic Reviews Revised (AMSTAR 2) : validation transculturelle et fidélité interjuges

Objective: Produce a French-Canadian translation of AMSTAR 2, affirm its content validity, and examine interrater reliability. Methods: Based on Vallerand's methodological approach, we conducted forward and parallel inverse-translations. Subsequently, an expert panel evaluated the translations to create a preliminary experimental French-Canadian version. A second expert panel examined this version and proposed additional modifications. Twenty future health professionals then rated the second experimental version for ambiguity on a scale (from 1 to 7). The principal co-investigators then reviewed the problematic elements and proposed a pre-official version. To ascertain content validity, a final back-translation was conducted resulting in the official version. Four judges evaluated 13 systematic reviews using the official French-Canadian version of AMSTAR 2. The Kappa coefficient was used to evaluate interrater reliability. Results: This rigorous adaptation enabled the development of a Franco-Canadian version of AMSTAR 2. Its application demonstrated low ambiguity (mean 1.15; SD 0.26) as well as good overall interrater reliability (total κ > 0.64) across all items. Conclusion: The French-Canadian version of AMSTAR 2 can now support francophone clinicians, educators, and managers in Canada as they undertake evidence-based practice.

Methodological steps used by authors of systematic reviews and meta-analyses of clinical trials: a cross-sectional study

Background

The quality of systematic reviews and meta-analyses (SR/MAs) depends on the extent of the methods used. We investigated the methodological steps used by authors of SR/MAs of clinical trials via an author survey.

Methods

We conducted an email-based cross-sectional study by contacting corresponding authors of SR/MAs that were published in 2015 and 2016 and retrieved through the PubMed database. The 27-item questionnaire was developed to study the methodological steps used by authors when conducting a SR/MA and the demographic characteristics of the respondent. Besides the demographic characteristics, methodological questions regarding the source, extraction and synthesis of data were included.

Results

From 10,292 emails sent, 384 authors responded and were included in the final analysis. Manual searches were carried out by 69.2% of authors, while 87.3% do updated searches, 49.2% search grey literature, 74.9% use the Cochrane tool for risk of bias assessment, 69.8% assign more than two reviewers for data extraction, 20.5% use digital software to extract data from graphs, 57.9% use raw data in the meta-analysis, and 43.8% meta-analyze both adjusted and non-adjusted data. There was a positive correlation of years of experience in conducting of SR/MAs with both searching grey literature (P = 0.0003) and use of adjusted and non-adjusted data (P = 0.006).

Conclusions

Many authors still do not carry out many of the vital methodological steps to be taken when performing any SR/MA. The experience of the authors in SR/MAs is highly correlated with use of the recommended tips for SR/MA conduct. The optimal methodological approach for researchers conducting a SR/MA should be standardized.

Electronic supplementary material

The online version of this article (10.1186/s12874-019-0780-2) contains supplementary material, which is available to authorized users.