Over half of clinical practice guidelines use non-systematic methods to inform recommendations: A methods study

Similarities, reliability and gaps in assessing the quality of conduct of systematic reviews using AMSTAR-2 and ROBIS: systematic survey of nutrition reviews

BACKGROUND

AMSTAR-2 ('A Measurement Tool to Assess Systematic Reviews, version 2') and ROBIS ('Risk of Bias in Systematic Reviews') are independent instruments used to assess the quality of conduct of systematic reviews/meta-analyses (SR/MAs). The degree of overlap in methodological constructs together with the reliability and any methodological gaps have not been systematically assessed and summarized in the field of nutrition.

METHODS

We performed a systematic survey of MEDLINE, EMBASE, and the Cochrane Library for SR/MAs published between January 2010 and November 2018 that examined the effects of any nutritional intervention/exposure for cancer prevention. We followed a systematic review approach including two independent reviewers at each step of the process. For AMSTAR-2 (16 items) and ROBIS (21 items), we assessed the similarities, the inter-rater reliability (IRR) and any methodological limitations of the instruments. Our protocol for the survey was registered in PROSPERO (CRD42019121116).

RESULTS

We found 4 similar domain constructs based on 11 comparisons from a total of 12 AMSTAR-2 and 14 ROBIS items. Ten comparisons were considered fully overlapping. Based on Gwet's agreement coefficients, six comparisons provided almost perfect (> 0.8), three substantial (> 0.6), and one a moderate level of agreement (> 0.4). While there is considerable overlap in constructs, AMSTAR-2 uniquely addresses explaining the selection of study designs for inclusion, reporting on excluded studies with justification, sources of funding of primary studies, and reviewers' conflict of interest. By contrast, ROBIS uniquely addresses appropriateness and restrictions within eligibility criteria, reducing risk of error in risk of bias (RoB) assessments, completeness of data extracted for analyses, the inclusion of all necessary studies for analyses, and adherence to predefined analysis plan.

CONCLUSIONS

Among the questions on AMSTAR-2 and ROBIS, 70.3% (26/37 items) address the same or similar methodological constructs. While the IRR of these constructs was moderate to perfect, there are unique methodological constructs that each instrument independently addresses. Notably, both instruments do not address the reporting of absolute estimates of effect or the overall certainty of the evidence, items that are crucial for users' wishing to interpret the importance of SR/MA results.

Stretching for Recovery from Groin Pain or Injury in Athletes: A Critical and Systematic Review

Stretching is usually used as part of rehabilitation protocols for groin pain or injury, but its specific contribution to and within multimodal recovery protocols is unclear. Our goal was to systematically review the effects of stretching for the recovery from groin pain or injury. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed, with eligibility criteria defined according to PICOS: (Participants) athletes with groin pain or injuries; (Interventions) interventions with stretching as the differentiating factor; (Comparators) comparators not applying stretching; (Outcomes) symptom remission or improvement and/or time to return to sport and/or return to play; (Study design) randomized controlled trials. Searches were performed on 26 March 2021, in CINAHL, Cochrane Library, EBSCO, EMBASE, PEDro, PubMed, Scielo, Scopus, SPORTDiscus, and Web of Science, with no limitations regarding language or date, and no filters. Of 117 retrieved results, 65 were duplicates and 49 were excluded at the screening stage. The three articles eligible for full-text analysis failed to comply with one or more inclusion criteria (participants, intervention and/or comparators). We then went beyond the protocol and searched for non-randomized trials and case series, but no intervention was found where stretching was the differentiating factor. We found no trials specifically assessing the effects of stretching on recovery or improvement of groin pain or injury in athletes. Currently, the efficacy of these interventions is unknown, and more research is warranted.