Do guidelines for treating chest disease in children use Cochrane Reviews effectively? A systematic review

Over half of clinical practice guidelines use non-systematic methods to inform recommendations: A methods study

INTRODUCTION

Assessing the process used to synthesize the evidence in clinical practice guidelines enables users to determine the trustworthiness of the recommendations. Clinicians are increasingly dependent on guidelines to keep up with vast quantities of medical literature, and guidelines are followed to avoid malpractice suits. We aimed to assess whether systematic methods were used when synthesizing the evidence for guidelines; and to determine the type of review cited in support of recommendations.

METHODS

Guidelines published in 2017 and 2018 were retrieved from the TRIP and Epistemonikos databases. We randomly sorted and sequentially screened clinical guidelines on all topics to select the first 50 that met our inclusion criteria. Our primary outcomes were the number of guidelines using either a systematic or non-systematic process to gather, assess, and synthesise evidence; and the numbers of recommendations within guidelines based on different types of evidence synthesis (systematic or non-systematic reviews). If a review was cited, we looked for evidence that it was critically appraised, and recorded which quality assessment tool was used. Finally, we examined the relation between the use of the GRADE approach, systematic review process, and type of funder.

RESULTS

Of the 50 guidelines, 17 (34%) systematically synthesised the evidence to inform recommendations. These 17 guidelines clearly reported their objectives and eligibility criteria, conducted comprehensive search strategies, and assessed the quality of the studies. Of the 29/50 guidelines that included reviews, 6 (21%) assessed the risk of bias of the review. The quality of primary studies was reported in 30/50 (60%) guidelines.

CONCLUSIONS

High quality, systematic review products provide the best available evidence to inform guideline recommendations. Using non-systematic methods compromises the validity and reliability of the evidence used to inform guideline recommendations, leading to potentially misleading and untrustworthy results.

Methodological evaluation of reviews that support recommendations from three consensus workshops in periodontology

OBJECTIVE

This study aimed to evaluate comprehensiveness and reproducibility of reviews that support consensus guidelines in periodontology.

METHODS

We included the reviews that support consensus guidelines from three workshops in periodontology, which were overseen by likely the two most important organisations in the field: the European Federation of Periodontology and the American Academy of Periodontology. We independently evaluated the comprehensiveness of literature searches by determining whether authors had searched reference lists, journals, registries and grey literature and whether the searches were limited to only one or a few languages. We evaluated whether review authors reported the eligibility criteria, the search strategies, and the list of included/excluded articles. We tested whether the search and selection of articles in one major database was reproducible.

RESULTS

Twenty-nine reviews were evaluated. Two (7%) reviews reported grey literature searches, and more than two-thirds of the reviews did not report hand-searching. Almost half of the reviews did not report whether there was language restriction for the literature searches. Two-thirds of the reviews reported the use of keywords only (without Boolean operators). One-fourth of the reviews reported the presence of a list of excluded articles after the full-text assessment. None of the reviews reported a detailed list of excluded articles after screening of titles/abstracts. None of the reviews reported enough information to allow reproduction of the findings of the PubMed search.

CONCLUSIONS

There is room to improve the reporting of the methodologies that are used in reviews that support periodontology consensus guidelines, although heterogeneity in reporting was found across all the reviews.

Gaps in the evidence for treatment decisions in cystic fibrosis: a systematic review

INTRODUCTION

Cystic fibrosis (CF) is a multisystem disorder. Treatment is complex and evidence for treatment decisions may be absent. Characterising gaps in the research evidence will highlight treatment uncertainties and help prioritise research questions. We systematically identified the evidence gaps for treatment decisions in CF.

METHODS

We searched for systematic reviews and guidelines on treatment interventions in CF. Two researchers identified eligible reviews with arbitration from a third. Using a structured framework, we extracted and characterised evidence gaps.

RESULTS

There were 73 reviews and 21 guidelines that met our inclusion criteria. From these, we identified 148 evidence gaps across a range of treatment areas. We found 111 evidence gaps through systematic reviews and a further 37 from guidelines. The reason for an evidence gap could only be reliably characterised for systematic reviews. In most cases, there was more than one explanation-most commonly few or no trials (97/111 evidence gaps). Other important factors leading to evidence gaps were small sample size (49/111), inadequate duration of follow-up (38/111) or intervention (37/111) and factors relating to outcomes (35/111). Evidence gaps from both systematic reviews and guidelines fell into the following categories: Respiratory (91); Gastrointestinal (20); PhysiotherapyandExercise (16); Musculoskeletal (6); Endocrine (4); Basic defect of CF (8); Psychosocial (2); Ears, Nose and Throat (1).

CONCLUSIONS

We have compiled an up-to-date list of treatment uncertainties in CF and the reasons for these uncertainties. These can be used as a resource to aid researchers and funders when planning future trials.

PROSPERO REGISTRATION NUMBER

Pre-results; CRD42015030111.