A practitioner's guide to developing critical appraisal skills: reviews of research

A typology of systematic reviews for synthesising evidence on health care

OBJECTIVES

The objectives of this paper are to (a) Review published references to systematic reviews; (b) offer a typology of systematic reviews for synthesising evidence on health care; and (c) summarise the guides for designing, reporting and appraising the reviews.

BACKGROUND

Systematic reviews play a role in finding, synthesising, transferring and implementing evidence for healthcare policy, practice guidelines and allocation of health resources. They have been particularly successful in confirming or synthesising evidence for health care by meta-analysing aggregated data from multiple randomised controlled trials. However, concerns about the limitations of evidence from controlled trials have prompted interest in other review methods capable of locating and appraising evidence from more diverse, and possibly more realistic, healthcare situations.

METHODS

An iterative citation-tracking process with Google Search and grey literature identified 204 papers on previous typologies and methods of systematic reviews.

RESULTS AND CONCLUSIONS

There are six types of systematic reviews: narrative; meta-analysis; scoping; qualitative; umbrella; and realist. Each type has distinct objectives, characteristics and attributes, but with much overlapping of methods and guides. Sensitivity to the need for qualitative evidence on complex human responses to ill-health and health care has broadened the objectives and methods of health-related systematic reviews to find, appraise and synthesis useful evidence for practice guidelines, healthcare policy and allocation of health resources.

Systematic review versus structured critical analysis

Objective: This analysis discusses common problems with systematic reviews (SRs) and presents a novel solution, the structured critical analysis (SCA) that can be incorporated into a SR or used as an alternative literature review design.Methods: A cross-sectional survey of current SRs related to interdisciplinary restorative dentistry was obtained by evaluating 100 current SRs for their self-reported methodological quality and its effect on scientific validity.Results: The preferred reporting items for systematic reviews and meta-analyses protocol (PRISMA) was used in 99/100 SRs, but only 8/100 reported a low risk of bias. High statistical heterogeneity precluding meta-analysis was found in 44/100 SRs. Only 94 paragraphs/100 SRs provided critical analysis.Discussion: Significant problems were found with current SRs that can compromise their reliability as the premier level of evidence for clinical science research. The reader must be aware of these deficiencies to correctly interpret the SR and cannot rely on the format alone.

Evidence-based dentistry for planning restorative treatments: barriers and potential solutions

INTRODUCTION

Evidence-based dentistry (EBD) can help provide the best treatment option for every patient, however, its implementation in restorative dentistry is very limited.

OBJECTIVE

This study aimed at assessing the barriers preventing the implementation of EBD among dental undergraduate and graduate students in Montreal, and explore possible solutions to overcome these barriers.

MATERIALS AND METHODS

A cross-sectional survey was conducted by means of a paper format self-administrated questionnaire distributed among dental students. The survey assessed the barriers and potential solutions for implementation of an evidence-based practice.

RESULTS

Sixty-one students completed the questionnaire. Forty-one percent of respondents found evidence-based literature to be the most reliable source of information for restorative treatment planning, however, only 16% used it. They considered that finding reliable information was difficult and they sometimes encountered conflicting information when consulting different sources. Dental students had positive attitudes towards the need for better access to evidence-based literature to assist learning and decision making in restorative treatment planning and to improve treatment outcomes. Even for dentists trained in EBD, online searching takes too much time, and even though it can provide information of better quality than personal intuition, it might not be enough to identify the best available evidence.

CONCLUSIONS

Even though dental students are aware of the importance of EBD in restorative dentistry they rarely apply the concept, mainly due to time constraints. For this reason, implementation of EBD would probably require faster access to evidence-based knowledge.

Limitations of A Measurement Tool to Assess Systematic Reviews (AMSTAR) and suggestions for improvement

A Measurement Tool to Assess Systematic Reviews (AMSTAR) is a commonly used tool to assess the quality of systematic reviews; however, modifications are needed to improve its usability, reliability, and validity. In this commentary, we summarize our experience and the experiences of others who have used AMSTAR and provide suggestions for its improvement. We propose that AMSTAR should modify a number of individual items and their instructions and responses to make them more congruent with an assessment of the methodologic quality of systematic reviews. We recommend adding new items and modifying existing items to assess the quality of the body of evidence and to address subgroup and sensitivity analyses. More detailed instructions are needed for scoring individual items across multiple reviewers, and we recommend that a total score should not be calculated. These suggestions need to be empirically tested prior to implementation.