Application of the Composite Quality Score (CQS-2B) versus Cochrane's Risk of Bias tool (Version 2) in systematic reviews of clinical trials - an exploratory study

Objectives

To explore whether systematic review conclusions generated from Cochrane's second version of its Risk of Bias tool (RoB 2) for trial appraisal differ when the Composite Quality Score, Version 2.B (CQS-2B) is used instead and to develop a testable hypothesis based on these findings.

Methods

PubMed was searched for one single systematic review. From the review's accepted trials, data concerning effect estimates and overall bias risk according to the RoB 2 tool were extracted. All trial reports were appraised again using the CQS-2B. Datasets were stratified according to overall bias risk (RoB 2) or corroboration (C-) level (CQS-2B). The effect estimates from trials with 'low bias risk' (RoB 2) and with highest C-level (CQS-2B) were pooled separately. These pooled effect estimates were statistically and all clinical conclusions qualitatively compared.

Results

The pooled effect estimates for trials with 'low bias risk' (RoB 2) were -0.07, 95% CI: -0.10 to -0.04 (I2 = 0.0%) and for the highest C-levels (CQS-2B) 0.08, 95% CI: -0.12 to -0.04 (I2 = 57.0%). The difference was statistically not significant (p = 0.70). Contrary to the RoB 2 tool, no clinical conclusions in line with the CQS-2B were made, because the effect estimates were judged to be erroneously overestimated, due to high risk of bias.

Conclusion

A testable hypothesis was generated suggesting that trial appraisal using the CQS-2B may provide more conservative conclusions based on similar data than with the RoB 2 tool.

Trial characteristics and treatment effect estimates in randomized controlled trials of Chinese herbal medicine: A meta-epidemiological study

BACKGROUND

Previously published meta-epidemiological studies focused on Western medicine have identified some trial characteristics that impact the treatment effect of randomized controlled trials (RCTs). Nevertheless, it remains unclear if similar associations exist in RCTs on Chinese herbal medicine (CHM). Further, Chinese medicine-related characteristics have not been explored yet.

OBJECTIVE

To investigate trial characteristics related to treatment effect estimates on CHM RCTs.

SEARCH STRATEGY

This meta-epidemiological study searched 5 databases for systematic reviews on CHM treatment published between January 2011 and July 2021.

INCLUSION CRITERIA

An eligible systematic review should only include RCTs of CHM and conduct at least one meta-analysis.

DATA EXTRACTION AND ANALYSIS

Two reviewers independently conducted data extraction on general characteristics of systematic reviews, meta-analyses and included RCTs. They also assessed the risk of bias of RCTs using the Cochrane risk of bias tool. A two-step approach was used for data analyses. The ratio of odds ratios (ROR) and difference in standardized mean differences (dSMD) with 95% confidence interval (CI) were applied to present the difference in effect estimates for binary and continuous outcomes, respectively.

RESULTS

Ninety-one systematic reviews, comprising 1338 RCTs were identified. For binary outcomes, RCTs incorporated with syndrome differentiation (ROR: 1.23; 95 % CI: [1.07, 1.39]), adopting Chinese medicine formula (ROR: 1.19; 95% CI: [1.03, 1.34]), with low risk of bias on incomplete outcome data (ROR: 1.29; 95% CI: [1.06, 1.52]) and selective outcome reporting (ROR: 1.12; 95% CI: [1.01, 1.24]), as well as a trial size ≥ 100 (ROR: 1.23; 95% CI: [1.04, 1.42]) preferred to show larger effect estimates. As for continuous outcomes, RCTs with Chinese medicine diagnostic criteria (dSMD: 0.23; 95% CI: [0.06, 0.41]), judged as high/unclear risk of bias on allocation concealment (dSMD: -0.70; 95% CI: [-0.99, -0.42]), with low risk of bias on incomplete outcome data (dSMD: 0.30; 95% CI: [0.18, 0.43]), conducted at a single center (dSMD: -0.33; 95% CI: [-0.61, -0.05]), not using intention-to-treat analysis (dSMD: -0.75; 95% CI: [-1.43, -0.07]), and without funding support (dSMD: -0.22; 95% CI: [-0.41, -0.02]) tended to show larger effect estimates.

CONCLUSION

This study provides empirical evidence for the development of a specific critical appraisal tool for risk of bias assessments on CHM RCTs. Please cite this article as: Wang BH, Lin YL, Gao YY, Song JL, Qin L, Li LQ, Liu WQ, Zhong CCW, Jiang MY, Mao C, Yang XB, Chung VCH, Wu IXY. Trial characteristics and treatment effect estimates in randomized controlled trials of Chinese herbal medicine: A meta-epidemiological study. J Integr Med. 2024; 22(3): 223-234.

Education and learning: potential methodological and ethical issues in systematic reviews containing a meta-analysis: some critical reading suggestions for junior doctors

Junior doctors make clinical decisions regularly; therefore, they need to adequately interpret the evidence supporting these decisions. Patients can be harmed if clinical treatments are supported by biased or unreliable evidence. Systematic reviews that contain meta-analyses of randomized controlled trials are a relatively low-biased type of evidence to support clinical interventions. Therefore, it is reasonable to think that doctors will likely select this type of study to answer clinical questions. In this article, doctors are informed about potential methodological and ethical issues in systematic reviews that contain a meta-analysis that are sometimes not easily identified or even overlooked by the current tools developed to assess their methodological quality or risk of bias. The article presents a discussion of topics related to data extraction, accuracy in reporting, reproducibility, heterogeneity, quality assessment of primary studies included in the systematic review, sponsorship, and conflict of interest. It is expected that the information reported will be useful for junior doctors when they are reading and interpreting evidence from systematic reviews containing meta-analyses of therapeutic interventions, mainly those doctors unfamiliar with methodological principles.

The impact of bias of underlying literature in guidelines on its recommendations: assessment of the German fluoride guideline

PURPOSE

The significance of the underlying literature in clinical guidelines can be weakened by the risk of bias, which could negatively affect the recommendations. Especially in controversial matters, such as fluoride use for caries prevention in children, biased results may be not reliable and lead to incorrect conclusions. This study was performed to detect bias in underlying literature of the German guideline for caries prevention using fluoride in children, where no consensus was reached between paediatricians and paediatric dentists.

METHODS

Three tools used for risk of bias assessments of different study designs were RoB 2 for RCTs, ROBINS-I for non-randomized studies, and ROBIS for systematic reviews. For each study cited in the guideline two independent risk of bias assessments were performed. Disagreements were resolved by consensus.

RESULTS

Out of 58 papers, 48.3% (n = 28) showed high risk of bias, with the majority in sections regarding fluoride tablets, fluoridated toothpaste, and paediatricians' recommendations. 9 out of 20 recommendations and statements were based on studies with high risk of bias, all of which were in these three controversial sections. 13 out of 29 RCTs showed high risk of bias (44.8%), as all 13 non-randomized trials did, while only 2 of 16 (12.5%) systematic reviews had high risk of bias.

CONCLUSION

Considering risk of bias of cited studies in clinical guidelines may result in substantial changes in its recommendations and aid in reaching consensus. Efforts should be made to assess risk of bias of underlying literature in future clinical guidelines.

Compelling evidence from meta-epidemiological studies demonstrates overestimation of effects in randomized trials that fail to optimize randomization and blind patients and outcome assessors

OBJECTIVES

To investigate the impact of potential risk of bias elements on effect estimates in randomized trials.

STUDY DESIGN AND SETTING

We conducted a systematic survey of meta-epidemiological studies examining the influence of potential risk of bias elements on effect estimates in randomized trials. We included only meta-epidemiological studies that either preserved the clustering of trials within meta-analyses (compared effect estimates between trials with and without the potential risk of bias element within each meta-analysis, then combined across meta-analyses; between-trial comparisons), or preserved the clustering of substudies within trials (compared effect estimates between substudies with and without the element, then combined across trials; within-trial comparisons). Separately for studies based on between- and within-trial comparisons, we extracted ratios of odds ratios (RORs) from each study and combined them using a random-effects model. We made overall inferences and assessed certainty of evidence based on Grading of Recommendations, Assessment, development, and Evaluation and Instrument to assess the Credibility of Effect Modification Analyses.

RESULTS

Forty-one meta-epidemiological studies (34 of between-, 7 of within-trial comparisons) proved eligible. Inadequate random sequence generation (ROR 0.94, 95% confidence interval [CI] 0.90-0.97) and allocation concealment (ROR 0.92, 95% CI 0.88-0.97) probably lead to effect overestimation (moderate certainty). Lack of patients blinding probably overestimates effects for patient-reported outcomes (ROR 0.36, 95% CI 0.28-0.48; moderate certainty). Lack of blinding of outcome assessors results in effect overestimation for subjective outcomes (ROR 0.69, 95% CI 0.51-0.93; high certainty). The impact of patients or outcome assessors blinding on other outcomes, and the impact of blinding of health-care providers, data collectors, or data analysts, remain uncertain. Trials stopped early for benefit probably overestimate effects (moderate certainty). Trials with imbalanced cointerventions may overestimate effects, while trials with missing outcome data may underestimate effects (low certainty). Influence of baseline imbalance, compliance, selective reporting, and intention-to-treat analysis remain uncertain.

CONCLUSION

Failure to ensure random sequence generation or adequate allocation concealment probably results in modest overestimates of effects. Lack of patients blinding probably leads to substantial overestimates of effects for patient-reported outcomes. Lack of blinding of outcome assessors results in substantial effect overestimation for subjective outcomes. For other elements, though evidence for consistent systematic overestimate of effect remains limited, failure to implement these safeguards may still introduce important bias.

Effectiveness, acceptability, and completeness and quality of intervention reporting of psychological interventions for people with dementia or mild cognitive impairment: protocol for a mixed-methods systematic review

INTRODUCTION

Mental health difficulties such as anxiety and depression have negative impacts on psychological well-being and are common in people with dementia and mild cognitive impairment. However, access to psychological treatments is limited. This mixed-method systematic review will: (1) examine the effectiveness of psychological interventions to improve mental health and psychological well-being in people with dementia or mild cognitive impairment; (2) examine the effectiveness of these psychological interventions to improve mental health and psychological well-being in informal caregivers; (3) examine potential clinical and methodological moderators associated with effectiveness; (4) explore factors associated with the acceptability of psychological interventions from the perspective of key stakeholders; and (5) examine the completeness and quality of intervention reporting.

METHODS AND ANALYSIS

Electronic databases (ASSIA, CENTRAL, CINAHL, EMBASE, PsycINFO and MEDLINE) will be systematically searched and supplemented with expert contact, reference and citation checking, and grey literature searches. If possible, we will conduct a meta-analysis to examine the overall effectiveness of psychological interventions to improve mental health and psychological well-being in people with dementia or mild cognitive impairment and their informal caregivers; and examine potential clinical and methodological moderators associated with effectiveness. We will conduct a deductive framework synthesis, informed by the theoretical framework of acceptability, to explore factors associated with the acceptability of psychological interventions from the perspective of key stakeholders. In accordance with Joanna Briggs Institute guidance, we will adopt a convergent segregated approach to data synthesis and integration of quantitative and qualitative findings. We will examine the completeness and quality of intervention reporting according to the Template for Intervention Description and Replication checklist and guide.

ETHICS AND DISSEMINATION

No primary data will be collected, and therefore, ethical approval is not required. Results will be disseminated through a peer-reviewed publication, academic conferences, and plain language summaries.

PROSPERO REGISTRATION NUMBER

CRD42023400514.

Allocation concealment appraisal of clinical therapy trials using the extended Composite Quality Score (CQS-2)-An empirically based update

Objectives

The objective of this study was to revise CQS-2/Criterion II concerning allocation concealment appraisal for prospective, controlled clinical therapy trials.

Methods

Meta-analyses of trials with inadequate allocation concealment were tested for in-between trial heterogeneity (I² > 0) due to imbalances in baseline variables. Meta-analyses with positive test results were used as a basis to deduce criteria for adequate allocation concealment. The CQS-2/Criterion II was reformulated in line with the findings.

Result

One suitable meta-analysis was identified. Two forest plots with data from five and four trials with inadequate/unclear allocation concealment were selected for testing. In addition, a total of five trials with adequate allocation concealment were identified. The meta-analysis test results were positive, and keywords for the judgment of adequate allocation concealment were extracted verbatim from the text of the meta-analysis. The extracted keywords indicated "central allocation" as the main criterion for adequate allocation concealment. Criterion II of the CQS-2 was revised accordingly.

Conclusion

Criterion II of the CQS-2 trial appraisal tool was revised. The revised appraisal tool was specified as version CQS-2B.

Extension of the Composite Quality Score (CQS) as an appraisal tool for prospective, controlled clinical therapy trials-A systematic review of meta-epidemiological evidence

AIM

To conduct a survey of current meta-epidemiological studies to identify additional trial design characteristics that may be associated with significant over- or underestimation of the treatment effect and to use such identified characteristics as a basis for the formulation of new CQS appraisal criteria.

MATERIALS AND METHODS

We retrieved eligible studies from two systematic reviews on this topic (latest search May 2015) and searched the databases PubMed and Embase for further studies from June 2015 -March 2022. All data were extracted by one author and verified by another. Sufficiently homogeneous estimates from single studies were pooled using random-effects meta-analysis. Trial design characteristics associated with statistically significant estimates from single datasets (which could not be pooled) and meta-analyses were used as a basis to formulate new or amend existing CQS criteria.

RESULTS

A total of 38 meta-epidemiological studies were identified. From these, seven trial design characteristics associated with statistically significant over- or underestimation of the true therapeutic effect were found.

CONCLUSION

One new criterion concerning double-blinding was added to the CQS, and the original criteria for concealing the random allocation sequence and for minimum sample size were amended.

Efficacy of biologics in root coverage and gingival augmentation therapy: An American Academy of Periodontology best evidence systematic review and network meta-analysis

BACKGROUND

The aim of this systematic review was to assess the efficacy of three biologics, namely autologous blood-derived products (ABPs), enamel matrix derivatives (EMD) and recombinant human platelet-derived growth factor BB (rhPDGF-BB), in root coverage and gingival augmentation therapy.

METHODS

The protocol of this PRISMA 2020-compliant systematic review was registered in PROSPERO (CRD42021285917). After study selection, data of interest were extracted. A network meta-analysis (NMA) was conducted to assess the effect of different surgical interventions on the main clinical outcomes of interest (i.e., mean root coverage [MRC%], complete root coverage [CRC%], keratinized tissue width [KTW], gingival thickness [GT] change, and recession depth [RD] reduction).

RESULTS

A total of 48 trials reported in 55 articles were selected. All studies reported on the treatment of gingival recession defects for root coverage purposes. Forty-six treatment arms from 24 trials were included in the NMA. These arms consisted of treatment with coronally advanced flap (CAF) alone, EMD + CAF, platelet-rich fibrin (PRF) + CAF, and subepithelial connective tissue graft (SCTG) + CAF. Regarding MRC%, SCTG+CAF was associated with a significant higher estimate (13.41%, 95% CI [8.06-18.75], P <  0.01), while EMD+CAF (6.68%, 95% CI [-0.03 to 13.4], P = 0.061) and PRF+CAF (1.03%, 95% CI [-5.65 to 7.72], P = 0.71) failed to show statistically significant differences compared with CAF alone (control group) or with each other. Similarly, only SCTG+CAF led to a significantly higher CRC% (14.41%, 95% CI [4.21 to 24.61], P < 0.01), while treatment arms EMD + CAF (13.48%, 95% CI [-3.34 to 30.32], P = 0.11) and PRF+CAF (-0.91%, 95% CI [-15.38, 13.57], p = 0.81) did not show significant differences compared with CAF alone or with each other. Differences in the CI of PRF+CAF (symmetrical around a zero adjunctive effect) and EMD+CAF (non-symmetrical) suggest that EMD could have some additional value compared with PRF. Treatment with SCTG+CAF led to a statistically significant higher RD reduction (-0.39 mm, 95% CI [-0.55 to 0.22], P < 0.01), however EMD+CAF (-0.13 mm, 95% CI [-0.29 to 0.01], P = 0.08) and PRF+CAF (-0.06 mm, 95% CI [-0.23 to 0.09], P = 0.39) failed to show significant differences compared with CAF or with each other. While SCTG+CAF was associated with a statistically significant higher gain of KTW (0.71 mm, 95% CI [0.48 to 0.93], P < 0.01), EMD+CAF (0.24 mm, 95% CI [-0.02 to 0.51], P = 0.08) and PRF+CAF (0.08 mm, 95% CI [-0.23 to 0.41], P = 0.58) did not result into significant changes compared with CAF alone or with each other. Regarding the use of rhPDGF-BB+CAF, although available studies have reported equivalent results compared with SCTG+CAF, evidence is very limited.

CONCLUSIONS

The use of ABPs, EMD, or rhPDGF-BB in conjunction with a CAF for root coverage purposes is safe and generally promotes significant improvements respective to baseline clinical parameters. However, the adjunctive use of ABPs and EMD does not provide substantial additional improvements in terms of clinical outcomes and patient-reported outcome measures to those achieved using CAF alone, when baseline KTW is >2 mm. Both PRF+CAF and EMD+CAF rendered inferior MRC%, CRC%, RD reduction, and KTW gain compared with SCTG+CAF, which should still be considered the gold-standard in root coverage therapy. Although some studies have reported equivalent results for rhPDGF-BB+CAF compared with the gold-standard intervention, limited evidence precludes formal comparisons with CAF or SCTG+CAF that could be extrapolated to guide clinical practice.

Selection, Confounding, and Attrition Biases in Randomized Controlled Trials of Rehabilitation Interventions: What Are They and How Can They Affect Randomized Controlled Trials Results? Basic Information for Junior Researchers and Clinicians

ABSTRACT

A thorough knowledge of biases in intervention studies and how they influence study results is essential for the practice of evidence-based medicine. The objective of this review was to provide a basic knowledge and understanding of the concept of biases and associated influence of these biases on treatment effects, focusing on the area of rehabilitation research. This article provides a description of selection biases, confounding, and attrition biases. In addition, useful recommendations are provided to identify, avoid, or control these biases when designing and conducting rehabilitation trials. The literature selected for this review was obtained mainly by compiling the information from several reviews looking at biases in rehabilitation. In addition, separate searches by biases and looking at reference lists of selected studies as well as using Scopus forward citation for relevant references were used. If not addressed appropriately, biases related to intervention research are a threat to internal validity and consequently to external validity. By addressing these biases, ensuring appropriate randomization, allocation concealment, appropriate retention techniques to avoid dropouts, appropriate study design and statistical analysis, among others, will generate more accurate treatment effects. Based on their impact on clinical results, a proper understanding of these concepts is central for researchers, rehabilitation clinicians, and other stakeholders working on this field.

Bias in Hand Surgical Randomized Controlled Trials: Systematic Review and Meta-Epidemiological Study

PURPOSE

Inappropriately reported or conducted studies may decrease the quality of care due to under- or overestimation of the benefits or harms of interventions. Our aim was to evaluate how often hand surgical randomized controlled trials (RCTs) use and report adequate methods to ensure internal validity, and whether inadequate reporting or methods are associated with the magnitude of treatment effect estimates.

METHODS

Data Sources were the Cochrane Central Register of Controlled Trials, MEDLINE, and Embase databases until November 2020. We included published RCTs investigating the effects of any surgical intervention in the hand and wrist region. We assessed internal validity using the Cochrane Risk of Bias (RoB) tool for 6 domains: selection, performance, detection, attrition, selective reporting, and "other" bias. We extracted the primary outcome and calculated the effect size for each study. We used mixed-effect meta-regression to assess whether the RoB modified the magnitude of the effects.

RESULTS

For 207 assessed trials, the RoB was unclear or high for 72% in selection, 93% in performance, 88% in detection, 25% in attrition, 22% in selective reporting, and 34% in the "other" bias domain. Trials with a high or unclear risk of selection bias yielded 0.28 standardized mean difference (95% confidence interval, 0.02-0.55) larger effect sizes compared to studies with a low risk. Risks of bias for other domains did not modify the intervention effects. The risk for selection bias declined over time: the odds ratio for a high or unclear RoB was 0.90 (95% confidence interval, 0.85-0.95) per additional year of publication CONCLUSIONS: The internal validity and credibility of hand surgical RCTs can be improved by using established methods to achieve true randomization, blinding of the participants and study personnel, publishing the trial protocol and avoiding selective reporting of the outcomes, and reporting the trial as recommended in the Consolidated Standards of Reporting Trials statement.

CLINICAL RELEVANCE

Clinicians should be aware that RCTs that do not use or report proper randomization and allocation concealment may overestimate the treatment effects.

Combining meta-epidemiological study datasets on commercial funding of randomised clinical trials: Database, methods, and descriptive results of the COMFIT study

Randomised trials are often funded by commercial companies and methodological studies support a widely held suspicion that commercial funding may influence trial results and conclusions. However, these studies often have a risk of confounding and reporting bias. The risk of confounding is markedly reduced in meta-epidemiological studies that compare fairly similar trials within meta-analyses, and risk of reporting bias is reduced with access to unpublished data. Therefore, we initiated the COMmercial Funding In Trials (COMFIT) study aimed at investigating the impact of commercial funding on estimated intervention effects in randomised clinical trials based on a consortium of researchers who agreed to share meta-epidemiological study datasets with information on meta-analyses and trials included in meta-epidemiological studies. Here, we describe the COMFIT study, its database, and descriptive results. We included meta-epidemiological studies with published or unpublished data on trial funding source and results or conclusions. We searched five bibliographic databases and other sources. We invited authors of eligible meta-epidemiological studies to join the COMFIT consortium and to share data. The final construction of the COMFIT database involves checking data quality, identifying trial references, harmonising variable categories, and removing non-informative meta-analyses as well as correlated meta-analyses and trial results. We included data from 17 meta-epidemiological studies, covering 728 meta-analyses and 6841 trials. Seven studies (405 meta-analyses, 3272 trials) had not published analyses on the impact of commercial funding, but shared unpublished data on funding source. On this basis, we initiated the construction of a combined database. Once completed, the database will enable comprehensive analyses of the impact of commercial funding on trial results and conclusions with increased statistical power and a markedly reduced risk of confounding and reporting bias.

The influence of flap design on patients' experiencing pain, swelling, and trismus after mandibular third molar surgery: a scoping systematic review

Third molar removal surgery usually comes accompanied by postoperative discomfort, which could be influenced by the surgical approach chosen. This scoping systematic review aimed at compiling the available evidence focused on the influence of flap design, including envelope flap (EF), triangular flap (TF), and modified triangular flap (MTF), on postoperative pain, swelling, and trismus, as primary outcome measures, and any result mentioning healing promotion or delay, as secondary outcome measure, after mandibular third molar extraction surgery. An electronic search, complemented by a manual search, of articles published from 1999 to 2020 was conducted in the Medline (PubMed), EMBASE and Web of Science databases including human randomized controlled trials, prospective, and retrospective studies with at least 15 patients. The risk of bias of the included studies was assessed either with the Cochrane’s Risk of Bias tool or with the Newcastle-Ottawa scale. Every step of the review was performed independently and in duplicate. The initial electronic search recovered 2102 articles. After applying the inclusion criteria, 12 articles were included. For patient’s perceived postoperative pain, TF and MTF frequently reported better results than EF. For swelling, the literature is divided, despite a trend favoring EF. For trismus, data showed that its occurrence is mostly associated with the duration of the surgery rather than with the chosen flap. For healing, the limited data is inconclusive. Finally, randomized studies showed a high risk of bias, whereas nonrandomized studies were mostly of good quality and low risk of bias. Although there was no clear consensus regarding the influence of different flap designs for third mandibular molar extraction on postoperative clinical morbidities; the surgeon’s experience, estimated surgical difficulty, molar position and orientation, and surg ery duration should be considered when choosing among the different flap designs.

Influence of Sponsorship Bias on Treatment Effect Size Estimates in Randomized Trials of Oral Health Interventions: A Meta-epidemiological Study

BACKGROUND

In this meta-epidemiological study, we aimed to examine associations between treatment effect size estimates and sponsorship bias in oral health randomized clinical trials.

METHODS

We selected oral health related meta-analyses that included a minimum of five randomized controlled trials. We extracted data, in duplicate, related to influence of sponsorship bias. We quantified the extent of bias associated with influence of sponsorship on the magnitude of effect size estimates of continuous variables using a two-level meta-meta-analytic approach with random-effects models to allow for intra- and inter-meta-analysis heterogeneity.

RESULTS

We initially identified 540 randomized trials included in 64 meta-analyses. Risk of sponsorship bias was judged as being "unclear" in 72.8% (n = 393) of the trials, while it was assessed as "low" in 16.7% (n = 90) and as "high" in 10.6% (n = 57) of the trials. Using a meta-epidemiological analysis (37 meta-analyses, including 328 trials that analyzed 85,934 patients), we identified statistically significant larger treatment effect size estimates in trials that had "high or unclear" risk of sponsorship bias (difference in treatment effect size estimates=0.10; 95% confidence intervals: 0.02 to 0.19) than in trials that had "low" risk of sponsorship bias.

CONCLUSIONS

We identified significant differences in treatment effect size estimates between dental trials based on sponsorship bias. Treatment effect size estimates were 0.10 larger in trials with "high or unclear" risk of sponsorship bias.

PRACTICAL IMPLICATIONS

Clinicians should have an adequate knowledge of sponsorship bias in a clinical trial and be able to estimate the degree to which the conclusions of a systematic review are synthesized and interpreted, based on trials with low risk of sponsorship bias.

The reporting quality of split-mouth trials in orthodontics according to CONSORT guidelines: 2015-19

OBJECTIVES

To assess the reporting quality of split-mouth trials (SMTs) in orthodontic journals, and to identify factors associated with better reporting.

MATERIALS AND METHODS

Seven leading orthodontic journals were hand searched for SMTs published during 2015-19. The CONSORT 2010 guideline and CONSORT for within-person trial (WPT) extension were used to assess the trial reporting quality (TRQ) and WPT-specific reporting quality (WRQ) of included SMTs, respectively. A binary score (0 or 1) was given to each item of the guidelines, and total scores were calculated for TRQ (score range, 0-32) and WRQ (score range, 0-15). Univariable and multivariable linear regression analyses were performed to identify factors associated with TRQ and WRQ.

RESULTS

A total of 42 SMTs were included. The mean overall scores for TRQ and WRQ were 16.8 [standard deviation (SD) 7.1] and 5.6 (SD 2.3), respectively. Only 11 SMTs (26.2%) presented the rationale for using a split-mouth design. Key methodological items including random sequence generation (22/42, 52.4%), allocation concealment (9/42, 21.4%), and blinding (20/42, 47.6%) were poorly reported. Only six SMTs (14.3%) used a paired method for sample size calculation, and half (21/42, 50.0%) considered the dependent nature of data in statistical analysis. In multivariable analyses, higher TRQ and WRQ were both significantly associated with journal, reported use of CONSORT and funding status.

CONCLUSIONS

The reporting quality of SMTs in orthodontics has much room for improvement. Joint efforts from relevant stakeholders are needed to improve the reporting quality of SMTs and reduce relevant avoidable research waste.

Degree of Risk of Bias in Randomized Controlled Trials: Does It Have an Impact on Root Coverage Outcomes?

OBJECTIVE

Improvement over time in mean (MRC) and complete root coverage (CRC) reported in randomized controlled trials (RCTs) has been documented. Our objective was to assess the effect that the ROB has on relevant outcomes reported in RCTs evaluating the efficacy of a commonly performed root coverage procedure.

METHODS

RCTs that involved the use of a bilaminar technique of coronally advanced flap with connective tissue graft were selected. The following data were extracted: MRC, CRC, whether conflict of interest was addressed (yes/no), adequacy of random sequence generation, allocation concealment, and blinding of outcome assessment. Trials were categorized into four groups as per different time periods: before Consolidated Standards of Reporting Trials (CONSORT) (before 1996), CONSORT (1997 to 2001), CONSORT 2001 (2002 to 2010), and CONSORT 2010 (2011 to 2019). Differences between group means were assessed using statistical analyses.

RESULTS

The search yielded a total of 47 RCTs published between 1993 and 2019 that met the inclusion criteria. A trend toward lower ROB over time in the selected RCTs was observed. However, differences in MRC and CRC with respect to the degree ROB of included trials were not significant.

CONCLUSIONS

Degree of ROB did not influence MRC and CRC reported in the RCTs included in this investigation. This finding suggests that the observed improvement of clinical outcomes over time should be attributed to other factors, such as refinement of surgical techniques.

Effect of Cold Application on Bruising and Pain Following Heparin Subcutaneous Injection: A Systematic Review and Meta-Analysis

PURPOSE

We performed this review systematically to assess the effect of cold application at the heparin subcutaneous injection site on incidence and size of bruising, hematoma, and pain intensity.

DESIGN

A systematic review and meta-analysis were utilized as the study design.

DATA SOURCES

To retrieve the relevant randomized controlled trials (RCTs) published up to July 2019, the databases of PubMed, Scopus, Web of Science, ScienceDirect, Cochrane Library, Google Scholar, TRIP, and Elmnet.ir were searched.

REVIEW METHODS

Those RCTs were selected in which the participants had received any type of heparin via subcutaneous injections at least once a day, as were those comparing the effect of cold application (i.e., moist or dry ice packs or vapocoolant spray) on injection sites with no interventions or placebos. The types of outcomes measured included pain intensity, bruising, and hematoma at injection sites. Furthermore, odds ratios (ORs) and mean differences (MDs) with 95% confidence intervals (CIs) were employed to measure the pooled data.

RESULTS

A total of nine studies recruiting 896 patients with a mean age range of 37 to 62 years were ultimately included in this study. The findings revealed that eight studies had used low-molecular-weight heparin, but unfractionated heparin had been administered in one study. The quality of the studies was also assessed in seven domains. Overall, risk of bias in the included articles was low to moderate. Pooling data of six studies showed that the patients had experienced less pain intensity once cold had been applied at the heparin injection sites (MD = -1.67; 95% CI -3.03 to -0.31; p = .02; I² = 92%). Bruising had been correspondingly reported in eight and six studies in terms of size and incidence, respectively. In particular, the results had demonstrated no statistically significant difference between either group in terms of bruising size or incidence at 48 to 72 hr following heparin injection. Furthermore, hematoma incidence had been reported in two studies, and one trial had reported hematoma size only. The results had suggested that the patients had smaller hematomas (MD = -0.87; 95% CI -1.63 to -0.11; p = .03) and a lower incidence of hematomas (OR = 0.35; 95% CI 0.16 to 0.76; p = .008) in the cold application group 48 hr after injection.

CONCLUSIONS

The results of this systematic review and meta-analysis showed that patients could experience less pain and fewer hematomas once the cold had been applied for 3 to 5 min at the injection sites before or after subcutaneous heparin administration. During this period, clinicians should also consider the patient's comfort; if some patients feel discomfort at the cold application site, the minimum possible time should be considered.

CLINICAL REVERENCE

This study's results could be used as evidence for all hospitalized and outpatients who need to receive any form of heparin and may suffer from local complications of this medicine.

Quality of Split-Mouth Trials in Dentistry: 1998, 2008, and 2018

The objectives of this study were to assess the reporting quality and methodological quality of split-mouth trials (SMTs) published during the past 2 decades and to determine whether there has been an improvement in their quality over time. We searched the MEDLINE database via PubMed to identify SMTs published in 1998, 2008, and 2018. For each included SMT, we used the CONsolidated Standards Of Reporting Trials (CONSORT) 2010 guideline, CONSORT for within-person trial (WPT) extension, and a new 3-item checklist to assess its trial reporting quality (TRQ), WPT-specific reporting quality (WRQ), and SMT-specific methodological quality (SMQ), respectively. Multivariable generalized linear models were performed to analyze the quality of SMTs over time, adjusting for potential confounding factors. A total of 119 SMTs were included. The mean overall score for the TRQ (score range, 0 to 32), WRQ (0 to 15), and SMQ (0 to 3) was 15.77 (SD 4.51), 6.06 (2.06), and 1.12 (0.70), respectively. The primary outcome was clearly defined in only 28 SMTs (23.5%), and only 27 (22.7%) presented a replicable sample size calculation. Only 45 SMTs (37.8%) provided the rationale for using a split-mouth design. The correlation between body sites was reported in only 5 studies (4.2%) for sample size calculation and 4 studies (3.4%) for statistical results. Only 2 studies (1.7%) performed an appropriate sample size calculation, and 46 (38.7%) chose appropriate statistical methods, both accounting for the correlation among treatment groups and the clustering/multiplicity of measurements within an individual. Results of regression analyses suggested that the TRQ of SMTs improved significantly with time ( P < 0.001), while there was no evidence of improvement in WRQ or SMQ. Both the reporting quality and methodological quality of SMTs still have much room for improvement. Concerted efforts are needed to improve the execution and reporting of SMTs.

Patient morbidity after impacted third molar extraction with different flap designs. A systematic review and meta-analysis

BACKGROUND

To review the literature on the effect of different surgical flaps upon patient morbidity (pain perception, trismus, swelling and osteitis) after impacted third molar extraction.

MATERIAL AND METHODS

An electronic and complementary search of main databases and grey literature was performed up to January 2019 to retrieve randomized clinical trials. The Cochrane risk of bias assessment tool was used for methodological appraisal. A random-effects meta-analysis was conducted of pain perception and trismus.

RESULTS

From the initially 1314 screened studies, only 11 were included in the qualitative synthesis, and 5 in the meta-analysis. There were no statistically significant differences in pain between the envelope and triangular flap designs over time, except on the sixth postoperative day, when the envelope flap proved more painful. Regarding trismus, statistically significant differences were observed on the seventh postoperative day, with greater mouth opening in the envelope flap group than in the triangular flap group. There were no clear differences in swelling and osteitis among the flap designs.

CONCLUSIONS

Despite its limitations, the present meta-analysis found no clear differences in patient morbidity between the different flap designs.

The effectiveness of integrative healthcare for chronic disease: A systematic review

BACKGROUND

The past few decades have witnessed a surge in consumer, clinician and academic interest in the field of integrative healthcare (IHC). Yet, there is still uncertainty regarding the effectiveness of IHC for complex, long-term health conditions.

OBJECTIVE

To assess the effectiveness of IHC for the management of any chronic health condition.

METHODS

Seven databases and four clinical trial registries were searched from inception through to May 2018 for comparative/controlled clinical trials investigating the effectiveness of IHC for any chronic disease, and assessing any outcome. Risk of bias was assessed using the Cochrane Collaboration Risk of Bias tool.

RESULTS

The search yielded 6,926 results. Eight studies met the inclusion criteria. All studies had at least three design features that carried an uncertain/high risk of bias. Differences in physiological, psychological and functional outcomes, and quality of life between patients receiving IHC and patients receiving conventional/usual care were varied and inconsistent. Changes in patient satisfaction with care were inconclusive. No studies reported the effectiveness of IHC on workforce- or administration-related parameters. Evidence from one trial suggested IHC may be more cost-effective than conventional care.

CONCLUSIONS

The findings indicate some promising effects for the use of IHC to manage chronic disease. However, the uncertain/high risk of bias across multiple domains, diverse and inconsistent findings, and heterogeneity of outcome measures and study populations prevents firm conclusions from being reached. Along with conducting further well-designed, long-term studies in this field, there is a need to ensure interventions closely align with the definition/principles of IHC.

Influence of blinding on treatment effect size estimate in randomized controlled trials of oral health interventions

Background

Recent methodologic evidence suggests that lack of blinding in randomized trials can result in under- or overestimation of the treatment effect size. The objective of this study is to quantify the extent of bias associated with blinding in randomized controlled trials of oral health interventions.

Methods

We selected all oral health meta-analyses that included a minimum of five randomized controlled trials. We extracted data, in duplicate, related to nine blinding-related criteria, namely: patient blinding, assessor blinding, care-provider blinding, investigator blinding, statistician blinding, blinding of both patients and assessors, study described as “double blind”, blinding of patients, assessors, and care providers concurrently, and the appropriateness of blinding. We quantified the impact of bias associated with blinding on the magnitude of effect size using a two-level meta-meta-analytic approach with a random effects model to allow for intra- and inter-meta-analysis heterogeneity.

Results

We identified 540 randomized controlled trials, included in 64 meta-analyses, analyzing data from 137,957 patients. We identified significantly larger treatment effect size estimates in trials that had inadequate patient blinding (difference in treatment effect size = 0.12; 95% CI: 0.00 to 0.23), lack of blinding of both patients and assessors (difference = 0.19; 95% CI: 0.06 to 0.32), and lack of blinding of patients, assessors, and care-providers concurrently (difference = 0.14; 95% CI: 0.03 to 0.25). In contrast, assessor blinding (difference = 0.06; 95% CI: -0.06 to 0.18), caregiver blinding (difference = 0.02; 95% CI: -0.04 to 0.09), principal-investigator blinding (difference = − 0.02; 95% CI: -0.10 to 0.06), describing a trial as “double-blind” (difference = 0.09; 95% CI: -0.05 to 0.22), and lack of an appropriate method of blinding (difference = 0.06; 95% CI: -0.06 to 0.18) were not associated with over- or underestimated treatment effect size.

Conclusions

We found significant differences in treatment effect size estimates between oral health trials based on lack of patient and assessor blinding. Treatment effect size estimates were 0.19 and 0.14 larger in trials with lack of blinding of both patients and assessors and blinding of patients, assessors, and care-providers concurrently. No significant differences were identified in other blinding criteria. Investigators of oral health systematic reviews should perform sensitivity analyses based on the adequacy of blinding in included trials.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0491-0) contains supplementary material, which is available to authorized users.

Randomized clinical trials in dentistry: Risks of bias, risks of random errors, reporting quality, and methodologic quality over the years 1955-2013

OBJECTIVES

To examine the risks of bias, risks of random errors, reporting quality, and methodological quality of randomized clinical trials of oral health interventions and the development of these aspects over time.

METHODS

We included 540 randomized clinical trials from 64 selected systematic reviews. We extracted, in duplicate, details from each of the selected randomized clinical trials with respect to publication and trial characteristics, reporting and methodologic characteristics, and Cochrane risk of bias domains. We analyzed data using logistic regression and Chi-square statistics.

RESULTS

Sequence generation was assessed to be inadequate (at unclear or high risk of bias) in 68% (n = 367) of the trials, while allocation concealment was inadequate in the majority of trials (n = 464; 85.9%). Blinding of participants and blinding of the outcome assessment were judged to be inadequate in 28.5% (n = 154) and 40.5% (n = 219) of the trials, respectively. A sample size calculation before the initiation of the study was not performed/reported in 79.1% (n = 427) of the trials, while the sample size was assessed as adequate in only 17.6% (n = 95) of the trials. Two thirds of the trials were not described as double blinded (n = 358; 66.3%), while the method of blinding was appropriate in 53% (n = 286) of the trials. We identified a significant decrease over time (1955-2013) in the proportion of trials assessed as having inadequately addressed methodological quality items (P < 0.05) in 30 out of the 40 quality criteria, or as being inadequate (at high or unclear risk of bias) in five domains of the Cochrane risk of bias tool: sequence generation, allocation concealment, incomplete outcome data, other sources of bias, and overall risk of bias.

CONCLUSIONS

The risks of bias, risks of random errors, reporting quality, and methodological quality of randomized clinical trials of oral health interventions have improved over time; however, further efforts that contribute to the development of more stringent methodology and detailed reporting of trials are still needed.