Grading the quality of evidence and the strength of recommendations in clinical dentistry: a critical review of 2 prominent approaches

Evaluating the Risk of Bias of a Study

OBJECTIVE

This first article of a series of 4 is aimed at guiding dental practitioners on how to evaluate the internal validity (risk of bias,) of randomized controlled trials (RCT). All RCT's contain different areas and potential sources of bias. Understanding risk of bias (ROB) will allow dental practitioners to improve the quality of dental treatments.

METHODS

The following areas of bias were elucidated: sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, selective reporting, and "other bias". The reader determines the ROB level by evaluating the areas or potential source of bias in the first phase. Normally, ROB levels are classified as low, high and unclear ROB.

RESULTS

This article reported the concepts and methods of evaluation of ROB in several areas of an RCT. An RCT with low ROB in all evaluated areas gives the dental practitioners more certainty and confidence that a specific clinical procedure is in fact effective and relevant to the patient.

CONCLUSIONS

The information provided here may guide dental practitioners in the evaluation of ROB in an RCT. The correct evaluation of ROB may improve the quality of dental treatments.

Failure of single-unit restorations on root filled posterior teeth: a systematic review

This systematic review investigated the failure rate of conventional single-unit restorations in root filled posterior permanent teeth. Two reviewers independently applied eligibility criteria, extracted data and assessed the quality of the evidence of each included study according to the Cochrane Collaboration's procedures for randomized control trials (RCTs) and the STROBE criteria for observational studies. The MEDLINE (via Ovid), EMBASE (via Ovid), Cochrane Oral Health Group Trials Register and CENTRAL (via Cochrane Library) databases were searched electronically (January 1993 to week 1, February 2015). This was complemented by an additional hand search of selected journals and the references of relevant studies. Clinical studies published on root filled single-unit restorative treatments with a mean follow-up period of at least 3 years were selected. The outcome measured was clinical or radiological failure. Overall, the four RCTs and the single observational study included were of low and high quality, respectively. Therefore, a meta-analysis was not possible. The pooled mean failure rates were reported according to the type of treatment and remaining coronal tooth structure. The current evidence suggested that the failure rates of the treatments may depend on the amount of remaining tooth structure and type of treatment. Post-retained crowns were associated with the most favourable outcome in teeth with one to two remaining coronal tooth wall(s), whereas post-free crowns were superior when greater tooth structure was available. Restorations in teeth without ferrules had such a high rate of failure that other treatment options should be considered.

Strength of recommendation for single-visit root canal treatment: grading the body of the evidence using a patient-centred approach

AIM

To identify, search for and critically interpret the results from clinical studies on single- versus multiple-visit root canal treatment in the light of an evidence-based paradigm. For that purpose, the quality of the available body of evidence was assessed using the SORT 'grade' - Strength of Recommendation Taxonomy.

METHODOLOGY

A protocol was followed including all aspects of the review methods: (i) search strategy, (ii) inclusion criteria for studies, (iii) screening methods, (iv) quality assessment, (v) data synthesis of the selected studies, and (vi) the assessment of quality of the body of evidence available by the means of the SORT 'grade'. An extensive search of recent biomedical literature was performed in PubMed (up to May, 2014), EMBASE (dating from 1980 to May 2014) and Cochrane databases with appropriate headings and keywords related to single- and multiple-visit root canal treatment. Selected studies were stratified according to their level of evidence using the SORT criteria: (i) quality - Level A high-quality evidence, Level B medium/low-quality evidence and Level C no evidence; (ii) degree of consistency - consistent, when most studies found a similar conclusion, or inconsistent, when there was considerable variation amongst study findings.

RESULTS

After the digital and manual searches, 246 studies were indentified. Two hundred and seven studies were eliminated by exclusion criteria, resulting in a yield of 39 articles that were selected for retrieval. Of the 39 articles, only 11 could be classified as Level 2 (B); the other 28 articles did not obey the criteria to be considered as real patient-oriented evidence; thus, they were classified as Level 3 (C). No studies were ranked as Level 1 because they did not obey the minimum standard to be considered as both good-quality research and patient-oriented evidence. Despite this, all of the selected Level B studies reported no significant differences between single- and multiple-visit treatments.

CONCLUSION

There is B-level (mid-level) evidence to confirm there is no difference between the two different treatments, based on research addressing clinical outcomes and using some consistent but limited-quality methods of scientific investigation. More studies focused on evaluating patient-centred outcomes are urgently required.

The Quality of the Evidence According to GRADE Is Predominantly Low or Very Low in Oral Health Systematic Reviews

Objectives

The main objective was to assess the credibility of the evidence using Grades of Recommendation, Assessment, Development, and Evaluation (GRADE) in oral health systematic reviews on the Cochrane Database of Systematic Reviews (CDSR) and elsewhere.

Study Design and Setting

Systematic Reviews or meta-analyses (January 2008-December 2013) from 14 high impact general dental and specialty dental journals and the Cochrane Database of Systematic Reviews were screened for meta-analyses. Data was collected at the systematic review, meta-analysis and trial level. Two reviewers applied and agreed on the GRADE rating for the selected meta-analyses.

Results

From the 510 systematic reviews initially identified 91 reviews (41 Cochrane and 50 non-Cochrane) were eligible for inclusion. The quality of evidence was high in 2% and moderate in 18% of the included meta-analyses with no difference between Cochrane and non-Cochrane reviews, journal impact factor or year of publication. The most common domains prompting downgrading of the evidence were study limitations (risk of bias) and imprecision (risk of play of chance).

Conclusion

The quality of the evidence in oral health assessed using GRADE is predominantly low or very low suggesting a pressing need for more randomised clinical trials and other studies of higher quality in order to inform clinical decisions thereby reducing the risk of instituting potentially ineffective and/or harmful therapies.

The shortened dental arch revisited: from evidence to recommendations by the use of the GRADE approach

Clinicians must frequently decide whether or not to treat patients with loss of posterior teeth, a condition called the shortened dental arch (SDA). Although many studies have been reported, there are no clear recommendations for the management of SDA cases. In this work, therefore, an innovative system, the Grading of Recommendations Assessment, Development and Evaluation (GRADE) approach, was used to grade the evidence and strength of recommendations for clinical intervention. An extensive literature search for longitudinal studies was conducted on 17 November 2010 in the PubMed and LILACS electronic databases using the term shortened dental arch. A 'snowballing' strategy, for example, manual searching of the reference lists of included papers, was also conducted. Unpublished and published studies were sought in ClinicalTrials.gov and in the search engine 'Google' (Scholar) in English, French, German, Italian, Portuguese and Spanish. Finally, grey literature was searched in OpenSIGLE (System for Information on Grey Literature in Europe). Titles and abstracts of 133 articles were initially assessed. Nine studies were finally included. Although there was no difference between the effectiveness of restorative and non-restorative approaches for SDA, fixed partial dentures seem better than removable prostheses. The overall body of evidence was, however, graded as low quality. Two different clinical scenarios are used to illustrate recommendations in the management of SDA cases by the use of the GRADE system. The GRADE approach may improve transparency in a shared decision-making process, mainly under conditions in which the quality of evidence is low or unclear.

Recommendations by Cochrane Review Groups for assessment of the risk of bias in studies

BACKGROUND

Assessing the risk of bias in individual studies in a systematic review can be done using individual components or by summarizing the study quality in an overall score.

METHODS

We examined the instructions to authors of the 50 Cochrane Review Groups that focus on clinical interventions for recommendations on methodological quality assessment of studies.

RESULTS

Forty-one of the review groups (82%) recommended quality assessment using components and nine using a scale. All groups recommending components recommended to assess concealment of allocation, compared to only two of the groups recommending scales (P < 0.0001). Thirty-five groups (70%) recommended assessment of sequence generation and 21 groups (42%) recommended assessment of intention-to-treat analysis. Only 28 groups (56%) had specific recommendations for using the quality assessment of studies analytically in reviews, with sensitivity analysis, quality as an inclusion threshold and subgroup analysis being the most commonly recommended methods. The scales recommended had problems in the individual items and some of the groups recommending components recommended items not related to bias in their quality assessment.

CONCLUSION

We found that recommendations by some groups were not based on empirical evidence and many groups had no recommendations on how to use the quality assessment in reviews. We suggest that all Cochrane Review Groups refer to the Cochrane Handbook for Systematic Reviews of Interventions, which is evidence-based, in their instructions to authors and that their own guidelines are kept to a minimum and describe only how methodological topics that are specific to their fields should be handled.