The importance of allocation concealment and patient blinding in osteoarthritis trials: a meta-epidemiologic study

Are Neonatal Trials Better Conducted and Reported over the Last 6 Decades? An Analysis on Their Risk-of-Bias Status in Cochrane Reviews

BACKGROUND

The introduction of Neonatology as a subspecialty in 1960 has stimulated an enormous amount of neonatal research. A large proportion of neonatal randomized-controlled trials (RCTs) have been included in the Cochrane reviews, within which methodological quality or risk-of-bias (ROB) assessment is an integral feature.

OBJECTIVES

We described the ROB profile of neonatal RCTs published since the 1950s.

METHODS

We analyzed individual studies within the Cochrane Neonatal reviews published up to December 2016. We extracted the reviewers' judgments on the ROB domains including random sequence generation, allocation concealment, blinding, incomplete outcome data, and selective reporting. We evaluated blinding of personnel in trials in which blinding was considered feasible.

RESULTS

We assessed 1980 RCTs published between 1952 and 2016 from 294 Cochrane Neonatal systematic reviews, with full ROB assessments performed in 848 trials (42.8%). Among the ROB domains, the highest proportion of trials (73%) were judged as satisfactory ("low risk") in handling incomplete outcome data, while fewest trials achieved blinding of outcome assessor (38.4%). In the last 6 decades, a progressive increase has been observed in the proportion of trials that were rated as low risk in random sequence generation, allocation concealment, and selective reporting. However, blinding was achieved in less than half of the trials with no clear improvement across decades (23-44% since the 1980s).

CONCLUSIONS

Despite steady improvement in the overall quality of neonatal RCTs over the last 6 decades, blinding remained unsatisfactory in the majority of the trials.

Effects of Meditative Movements on Major Depressive Disorder: A Systematic Review and Meta-Analysis of Randomized Controlled Trials

BACKGROUND

Tai Chi, Qigong, and Yoga are recognized as the most popular complementary approaches for alleviating musculoskeletal pain, improving sleep quality, and reducing blood pressure. The therapeutic effects of these meditative movements for treating major depressive disorder (MDD) is yet to be determined. Therefore, we examined whether meditative movements (Tai Chi, Qigong, and Yoga) are effective for treating MDD. Seven electronic databases (SPORTDiscus, PubMed, PsycINFO, Cochrane Library, Web of Science, CNKI, and Wanfang) were used to search relevant articles. Randomized controlled trials (RCT) using Tai Chi, Qigong or Yoga as intervention for MDD were considered for the meta-analysis (standardized mean difference: SMD).

RESULTS

Meta-analysis on 15 fair-to-high quality RCTs showed a significant benefit in favor of meditative movement on depression severity (SMD = -0.56, 95% CI -0.76 to -0.37, p < 0.001, I² = 35.76%) and on anxiety severity (SMD = -0.46, 95% CI -0.71 to -0.21, p < 0.001, I² = 1.17%). Meditative movement interventions showed significantly improved treatment remission rate (OR = 6.7, 95% CI 2.38 to 18.86, p < 0.001) and response rate (OR = 5.2, 95% CI 1.73 to 15.59, p < 0.001) over passive controls.

CONCLUSIONS

Emphasizing the therapeutic effects of meditative movements for treating MDD is critical because it may provide a useful alternative to existing mainstream treatments (drug therapy and psychotherapy) for MDD. Given the fact that meditative movements are safe and easily accessible, clinicians may consider recommending meditative movements for symptomatic management in this population.

Effectiveness and adequacy of blinding in the moderation of pain outcomes: Systematic review and meta-analyses of dry needling trials

BACKGROUND

Blinding is critical to clinical trials because it allows for separation of specific intervention effects from bias, by equalising all factors between groups except for the proposed mechanism of action. Absent or inadequate blinding in clinical trials has consistently been shown in large meta-analyses to result in overestimation of intervention effects. Blinding in dry needling trials, particularly blinding of participants and therapists, is a practical challenge; therefore, specific effects of dry needling have yet to be determined. Despite this, dry needling is widely used by health practitioners internationally for the treatment of pain. This review presents the first empirical account of the influence of blinding on intervention effect estimates in dry needling trials. The aim of this systematic review was to determine whether participant beliefs about group allocation relative to actual allocation (blinding effectiveness), and/or adequacy of blinding procedures, moderated pain outcomes in dry needling trials.

METHODS

Twelve databases (MEDLINE, EMBASE, AMED, Scopus, CINAHL, PEDro, The Cochrane Library, Trove, ProQuest, trial registries) were searched from inception to February 2016. Trials that compared active dry needling with a sham that simulated dry needling were included. Two independent reviewers performed screening, data extraction, and critical appraisal. Available blinding effectiveness data were converted to a blinding index, a quantitative measurement of blinding, and meta-regression was used to investigate the influence of the blinding index on pain. Adequacy of blinding procedures was based on critical appraisal, and subgroup meta-analyses were used to investigate the influence of blinding adequacy on pain. Meta-analytical techniques used inverse-variance random-effects models.

RESULTS

The search identified 4,894 individual publications with 24 eligible for inclusion in the quantitative syntheses. In 19 trials risk of methodological bias was high or unclear. Five trials were adequately blinded, and blinding was assessed and sufficiently reported to compute the blinding index in 10 trials. There was no evidence of a moderating effect of blinding index on pain. For short-term and long-term pain assessments pooled effects for inadequately blinded trials were statistically significant in favour of active dry needling, whereas there was no evidence of a difference between active and sham groups for adequately blinded trials.

DISCUSSION

The small number and size of included trials meant there was insufficient evidence to conclusively determine if a moderating effect of blinding effectiveness or adequacy existed. However, with the caveats of small sample size, generally unclear risk of bias, statistical heterogeneity, potential publication bias, and the limitations of subgroup analyses, the available evidence suggests that inadequate blinding procedures could lead to exaggerated intervention effects in dry needling trials.

Influence of blinding on treatment effect size estimate in randomized controlled trials of oral health interventions

Background

Recent methodologic evidence suggests that lack of blinding in randomized trials can result in under- or overestimation of the treatment effect size. The objective of this study is to quantify the extent of bias associated with blinding in randomized controlled trials of oral health interventions.

Methods

We selected all oral health meta-analyses that included a minimum of five randomized controlled trials. We extracted data, in duplicate, related to nine blinding-related criteria, namely: patient blinding, assessor blinding, care-provider blinding, investigator blinding, statistician blinding, blinding of both patients and assessors, study described as “double blind”, blinding of patients, assessors, and care providers concurrently, and the appropriateness of blinding. We quantified the impact of bias associated with blinding on the magnitude of effect size using a two-level meta-meta-analytic approach with a random effects model to allow for intra- and inter-meta-analysis heterogeneity.

Results

We identified 540 randomized controlled trials, included in 64 meta-analyses, analyzing data from 137,957 patients. We identified significantly larger treatment effect size estimates in trials that had inadequate patient blinding (difference in treatment effect size = 0.12; 95% CI: 0.00 to 0.23), lack of blinding of both patients and assessors (difference = 0.19; 95% CI: 0.06 to 0.32), and lack of blinding of patients, assessors, and care-providers concurrently (difference = 0.14; 95% CI: 0.03 to 0.25). In contrast, assessor blinding (difference = 0.06; 95% CI: -0.06 to 0.18), caregiver blinding (difference = 0.02; 95% CI: -0.04 to 0.09), principal-investigator blinding (difference = − 0.02; 95% CI: -0.10 to 0.06), describing a trial as “double-blind” (difference = 0.09; 95% CI: -0.05 to 0.22), and lack of an appropriate method of blinding (difference = 0.06; 95% CI: -0.06 to 0.18) were not associated with over- or underestimated treatment effect size.

Conclusions

We found significant differences in treatment effect size estimates between oral health trials based on lack of patient and assessor blinding. Treatment effect size estimates were 0.19 and 0.14 larger in trials with lack of blinding of both patients and assessors and blinding of patients, assessors, and care-providers concurrently. No significant differences were identified in other blinding criteria. Investigators of oral health systematic reviews should perform sensitivity analyses based on the adequacy of blinding in included trials.

Electronic supplementary material

The online version of this article (10.1186/s12874-018-0491-0) contains supplementary material, which is available to authorized users.

Association Between Risk-of-Bias Assessments and Results of Randomized Trials in Cochrane Reviews: The ROBES Meta-Epidemiologic Study

Flaws in the design of randomized trials may bias intervention effect estimates and increase between-trial heterogeneity. Empirical evidence suggests that these problems are greatest for subjectively assessed outcomes. For the Risk of Bias in Evidence Synthesis (ROBES) Study, we extracted risk-of-bias judgements (for sequence generation, allocation concealment, blinding, and incomplete data) from a large collection of meta-analyses published in the Cochrane Library (issue 4; April 2011). We categorized outcome measures as mortality, other objective outcome, or subjective outcome, and we estimated associations of bias judgements with intervention effect estimates using Bayesian hierarchical models. Among 2,443 randomized trials in 228 meta-analyses, intervention effect estimates were, on average, exaggerated in trials with high or unclear (versus low) risk-of-bias judgements for sequence generation (ratio of odds ratios (ROR) = 0.91, 95% credible interval (CrI): 0.86, 0.98), allocation concealment (ROR = 0.92, 95% CrI: 0.86, 0.98), and blinding (ROR = 0.87, 95% CrI: 0.80, 0.93). In contrast to previous work, we did not observe consistently different bias for subjective outcomes compared with mortality. However, we found an increase in between-trial heterogeneity associated with lack of blinding in meta-analyses with subjective outcomes. Inconsistency in criteria for risk-of-bias judgements applied by individual reviewers is a likely limitation of routinely collected bias assessments. Inadequate randomization and lack of blinding may lead to exaggeration of intervention effect estimates in randomized trials.

Randomized controlled trials of simulation-based interventions in Emergency Medicine: a methodological review

The number of trials assessing Simulation-Based Medical Education (SBME) interventions has rapidly expanded. Many studies show that potential flaws in design, conduct and reporting of randomized controlled trials (RCTs) can bias their results. We conducted a methodological review of RCTs assessing a SBME in Emergency Medicine (EM) and examined their methodological characteristics. We searched MEDLINE via PubMed for RCT that assessed a simulation intervention in EM, published in 6 general and internal medicine and in the top 10 EM journals. The Cochrane Collaboration risk of Bias tool was used to assess risk of bias, intervention reporting was evaluated based on the "template for intervention description and replication" checklist, and methodological quality was evaluated by the Medical Education Research Study Quality Instrument. Reports selection and data extraction was done by 2 independents researchers. From 1394 RCTs screened, 68 trials assessed a SBME intervention. They represent one quarter of our sample. Cardiopulmonary resuscitation (CPR) is the most frequent topic (81%). Random sequence generation and allocation concealment were performed correctly in 66 and 49% of trials. Blinding of participants and assessors was performed correctly in 19 and 68%. Risk of attrition bias was low in three-quarters of the studies (n = 51). Risk of selective reporting bias was unclear in nearly all studies. The mean MERQSI score was of 13.4/18.4% of the reports provided a description allowing the intervention replication. Trials assessing simulation represent one quarter of RCTs in EM. Their quality remains unclear, and reproducing the interventions appears challenging due to reporting issues.

Blinding in trials of interventional procedures is possible and worthwhile

In this paper, we use evidence from our earlier review of surgical randomised controlled trials with a placebo arm to show that blinding in trials of interventional procedures is feasible. We give examples of ingenious strategies that have been used to simulate the active procedure and to make the placebo control indistinguishable from the active treatment. We discuss why it is important to blind of patients, assessors, and caregivers and what types of bias that may occur in interventional trials. Finally, we describe the benefits of blinding, from the obvious ones such as avoiding bias, as well as less evident benefits such as avoiding patient drop out in the control arm.

Dexamethasone as an adjuvant to peripheral nerve block

BACKGROUND

Peripheral nerve block (infiltration of local anaesthetic around a nerve) is used for anaesthesia or analgesia. A limitation to its use for postoperative analgesia is that the analgesic effect lasts only a few hours, after which moderate to severe pain at the surgical site may result in the need for alternative analgesic therapy. Several adjuvants have been used to prolong the analgesic duration of peripheral nerve block, including perineural or intravenous dexamethasone.

OBJECTIVES

To evaluate the comparative efficacy and safety of perineural dexamethasone versus placebo, intravenous dexamethasone versus placebo, and perineural dexamethasone versus intravenous dexamethasone when added to peripheral nerve block for postoperative pain control in people undergoing surgery.

SEARCH METHODS

We searched the Cochrane Central Register of Controlled Trials, MEDLINE, Embase, DARE, Web of Science and Scopus from inception to 25 April 2017. We also searched trial registry databases, Google Scholar and meeting abstracts from the American Society of Anesthesiologists, the Canadian Anesthesiologists' Society, the American Society of Regional Anesthesia, and the European Society of Regional Anaesthesia.

SELECTION CRITERIA

We included all randomized controlled trials (RCTs) comparing perineural dexamethasone with placebo, intravenous dexamethasone with placebo, or perineural dexamethasone with intravenous dexamethasone in participants receiving peripheral nerve block for upper or lower limb surgery.

DATA COLLECTION AND ANALYSIS

We used standard methodological procedures expected by Cochrane.

MAIN RESULTS

We included 35 trials of 2702 participants aged 15 to 78 years; 33 studies enrolled participants undergoing upper limb surgery and two undergoing lower limb surgery. Risk of bias was low in 13 studies and high/unclear in 22. Perineural dexamethasone versus placeboDuration of sensory block was significantly longer in the perineural dexamethasone group compared with placebo (mean difference (MD) 6.70 hours, 95% confidence interval (CI) 5.54 to 7.85; participants1625; studies 27). Postoperative pain intensity at 12 and 24 hours was significantly lower in the perineural dexamethasone group compared with control (MD -2.08, 95% CI -2.63 to -1.53; participants 257; studies 5) and (MD -1.63, 95% CI -2.34 to -0.93; participants 469; studies 9), respectively. There was no significant difference at 48 hours (MD -0.61, 95% CI -1.24 to 0.03; participants 296; studies 4). The quality of evidence is very low for postoperative pain intensity at 12 hours and low for the remaining outcomes. Cumulative 24-hour postoperative opioid consumption was significantly lower in the perineural dexamethasone group compared with placebo (MD 19.25 mg, 95% CI 5.99 to 32.51; participants 380; studies 6). Intravenous dexamethasone versus placeboDuration of sensory block was significantly longer in the intravenous dexamethasone group compared with placebo (MD 6.21, 95% CI 3.53 to 8.88; participants 499; studies 8). Postoperative pain intensity at 12 and 24 hours was significantly lower in the intravenous dexamethasone group compared with placebo (MD -1.24, 95% CI -2.44 to -0.04; participants 162; studies 3) and (MD -1.26, 95% CI -2.23 to -0.29; participants 257; studies 5), respectively. There was no significant difference at 48 hours (MD -0.21, 95% CI -0.83 to 0.41; participants 172; studies 3). The quality of evidence is moderate for duration of sensory block and postoperative pain intensity at 24 hours, and low for the remaining outcomes. Cumulative 24-hour postoperative opioid consumption was significantly lower in the intravenous dexamethasone group compared with placebo (MD -6.58 mg, 95% CI -10.56 to -2.60; participants 287; studies 5). Perinerual versus intravenous dexamethasoneDuration of sensory block was significantly longer in the perineural dexamethasone group compared with intravenous by three hours (MD 3.14 hours, 95% CI 1.68 to 4.59; participants 720; studies 9). We found that postoperative pain intensity at 12 hours and 24 hours was significantly lower in the perineural dexamethasone group compared with intravenous, however, the MD did not surpass our pre-determined minimally important difference of 1.2 on the Visual Analgue Scale/Numerical Rating Scale, therefore the results are not clinically significant (MD -1.01, 95% CI -1.51 to -0.50; participants 217; studies 3) and (MD -0.77, 95% CI -1.47 to -0.08; participants 309; studies 5), respectively. There was no significant difference in severity of postoperative pain at 48 hours (MD 0.13, 95% CI -0.35 to 0.61; participants 227; studies 3). The quality of evidence is moderate for duration of sensory block and postoperative pain intensity at 24 hours, and low for the remaining outcomes. There was no difference in cumulative postoperative 24-hour opioid consumption (MD -3.87 mg, 95% CI -9.93 to 2.19; participants 242; studies 4). Incidence of severe adverse eventsFive serious adverse events were reported. One block-related event (pneumothorax) occurred in one participant in a trial comparing perineural dexamethasone and placebo; however group allocation was not reported. Four non-block-related events occurred in two trials comparing perineural dexamethasone, intravenous dexamethasone and placebo. Two participants in the placebo group required hospitalization within one week of surgery; one for a fall and one for a bowel infection. One participant in the placebo group developed Complex Regional Pain Syndrome Type I and one in the intravenous dexamethasone group developed pneumonia. The quality of evidence is very low due to the sparse number of events.

AUTHORS' CONCLUSIONS

Low- to moderate-quality evidence suggests that when used as an adjuvant to peripheral nerve block in upper limb surgery, both perineural and intravenous dexamethasone may prolong duration of sensory block and are effective in reducing postoperative pain intensity and opioid consumption. There is not enough evidence to determine the effectiveness of dexamethasone as an adjuvant to peripheral nerve block in lower limb surgeries and there is no evidence in children. The results of our review may not apply to participants at risk of dexamethasone-related adverse events for whom clinical trials would probably be unsafe.There is not enough evidence to determine the effectiveness of dexamethasone as an adjuvant to peripheral nerve block in lower limb surgeries and there is no evidence in children. The results of our review may not be apply to participants who at risk of dexamethasone-related adverse events for whom clinical trials would probably be unsafe. The nine ongoing trials registered at ClinicalTrials.gov may change the results of this review.

Influence of allocation concealment and intention-to-treat analysis on treatment effects of physical therapy interventions in low back pain randomised controlled trials: a protocol of a meta-epidemiological study

INTRODUCTION

Meta-epidemiological studies examining the influence of methodological characteristics, such as allocation concealment and intention-to-treat analysis have been performed in a large number of healthcare areas. However, there are no studies investigating these characteristics in physical therapy interventions for patients with low back pain. The aim of this study is to investigate the influence of allocation concealment and the use of intention-to-treat analysis on estimates of treatment effects of physical therapy interventions in low back pain clinical trials.

METHODS AND ANALYSIS

Searches on PubMed, Embase, Cochrane Database of Systematic Reviews, Physiotherapy Evidence Database (PEDro) and CINAHL databases will be performed. We will search for systematic reviews that include a meta-analysis of randomised controlled trials that compared physical therapy interventions in patients with low back pain with placebo or no intervention, and have pain intensity or disability as the primary outcomes. Information about selection (allocation concealment) and attrition bias (intention-to-treat analysis) will be extracted from the PEDro database for each included trial. Information about bibliographic data, study characteristics, participants' characteristics and study results will be extracted. A random-effects model will be used to provide separate estimates of treatment effects for trials with and without allocation concealment and with and without intention-to-treat analysis (eg, four estimates). A meta-regression will be performed to measure the association between methodological features and treatment effects from each trial. The dependent variable will be the treatment effect (the mean between-group differences) for the primary outcomes (pain or disability), while the independent variables will be the methodological features of interest (allocation concealment and intention-to-treat analysis). Other covariates will include sample size and sequence generation.

ETHICS AND DISSEMINATION

No ethical approval will be required for this study. The study findings will be published in a peer-reviewed journal and presented at international conferences.

REGISTRATION NUMBER

International Prospective Register of Systematic Reviews (CRD42016052347).

Blinding in trials of interventional procedures is possible and worthwhile

In this paper, we use evidence from our earlier review of surgical randomised controlled trials with a placebo arm to show that blinding in trials of interventional procedures is feasible. We give examples of ingenious strategies that have been used to simulate the active procedure and to make the placebo control indistinguishable from the active treatment. We discuss why it is important to blind of patients, assessors, and caregivers and what types of bias that may occur in interventional trials. Finally, we describe the benefits of blinding, from the obvious ones such as avoiding bias, as well as less evident benefits such as avoiding patient drop out in the control arm.

Impact of Selection Bias on Treatment Effect Size Estimates in Randomized Trials of Oral Health Interventions: A Meta-epidemiological Study

Emerging evidence suggests that design flaws of randomized controlled trials can result in over- or underestimation of the treatment effect size (ES). The objective of this study was to examine associations between treatment ES estimates and adequacy of sequence generation, allocation concealment, and baseline comparability among a sample of oral health randomized controlled trials. For our analysis, we selected all meta-analyses that included a minimum of 5 oral health randomized controlled trials and used continuous outcomes. We extracted data, in duplicate, related to items of selection bias (sequence generation, allocation concealment, and baseline comparability) in the Cochrane Risk of Bias tool. Using a 2-level meta-meta-analytic approach with a random effects model to allow for intra- and inter-meta-analysis heterogeneity, we quantified the impact of selection bias on the magnitude of ES estimates. We identified 64 meta-analyses, including 540 randomized controlled trials analyzing 137,957 patients. Sequence generation was judged to be adequate (at low risk of bias) in 32% ( n = 173) of trials, and baseline comparability was judged to be adequate in 77.8% of trials. Allocation concealment was unclear in the majority of trials ( n = 458, 84.8%). We identified significantly larger treatment ES estimates in trials that had inadequate/unknown sequence generation (difference in ES = 0.13; 95% CI: 0.01 to 0.25) and inadequate/unknown allocation concealment (difference in ES = 0.15; 95% CI: 0.02 to 0.27). In contrast, baseline imbalance (difference in ES = 0.01, 95% CI: –0.09 to 0.12) was not associated with inflated or underestimated ES. In conclusion, treatment ES estimates were 0.13 and 0.15 larger in trials with inadequate/unknown sequence generation and inadequate/unknown allocation concealment, respectively. Therefore, authors of systematic reviews using oral health randomized controlled trials should perform sensitivity analyses based on the adequacy of sequence generation and allocation concealment.

Guidelines for standardizing and increasing the transparency in the reporting of biomedical research

There is a lack of awareness about the guidelines for standardized and transparent reporting of biomedical research, among the medical professionals. This paper aims to familiarize the clinical researchers and practitioners regarding the issues related to transparency and the evolving guidelines for standardizing them, in the reporting of biomedical research. A narrative review method is adopted here, primarily based on the EQUATOR and SAMPL guidelines for reporting studies and statistical analyses. As study methods and statistical approaches support each other, their reporting practices as per the standardized guidelines have been dealt here in a congruous manner.

Cost-effectiveness of adjunct non-pharmacological interventions for osteoarthritis of the knee

BACKGROUND

There is limited information on the costs and benefits of alternative adjunct non-pharmacological treatments for knee osteoarthritis and little guidance on which should be prioritised for commissioning within the NHS. This study estimates the costs and benefits of acupuncture, braces, heat treatment, insoles, interferential therapy, laser/light therapy, manual therapy, neuromuscular electrical stimulation, pulsed electrical stimulation, pulsed electromagnetic fields, static magnets and transcutaneous electrical nerve Stimulation (TENS), based on all relevant data, to facilitate a more complete assessment of value.

METHODS

Data from 88 randomised controlled trials including 7,507 patients were obtained from a systematic review. The studies reported a wide range of outcomes. These were converted into EQ-5D index values using prediction models, and synthesised using network meta-analysis. Analyses were conducted including firstly all trials and secondly only trials with low risk of selection bias. Resource use was estimated from trials, expert opinion and the literature. A decision analytic model synthesised all evidence to assess interventions over a typical treatment period (constant benefit over eight weeks or linear increase in effect over weeks zero to eight and dissipation over weeks eight to 16).

RESULTS

When all trials are considered, TENS is cost-effective at thresholds of £20-30,000 per QALY with an incremental cost-effectiveness ratio of £2,690 per QALY vs. usual care. When trials with a low risk of selection bias are considered, acupuncture is cost-effective with an incremental cost-effectiveness ratio of £13,502 per QALY vs. TENS. The results of the analysis were sensitive to varying the intensity, with which interventions were delivered, and the magnitude and duration of intervention effects on EQ-5D.

CONCLUSIONS

Using the £20,000 per QALY NICE threshold results in TENS being cost-effective if all trials are considered. If only higher quality trials are considered, acupuncture is cost-effective at this threshold, and thresholds down to £14,000 per QALY.

PRISMA-combined α-blockers and antimuscarinics for ureteral stent-related symptoms: A meta-analysis

BACKGROUND

As a monotherpay, a-blockers and anti-muscarinics are both efficacy for ureteral stent-related symptoms (SRS). The aim of the study was to systematically evaluate their efficacy of a combination therapy for SRS.

METHODS

Relevant studies investigating α-blockers and/or anti-muscarinics for SRS were identified though searching online databases including PubMed, EMBASE, Cochrane Library, and other sources up to March 2016. The RevMan software was used for data analysis, and senesitivity analysis and inverted funnel plot were also adopted.

RESULTS

Seven randomized controlled trials (RCTs) and 1 prospective controlled trial including 545 patients were selected. Compared with α-blockers, the combination group achieved significant improvements in total International Prostate Symptom Score (IPSS) [-3.93 (2.89, 4.96), P < 0.00001], obstructive subscore [-1.29 (0.68, 1.89), P < 0.0001], irritative subscore [-2.93 (2.18, 3.68), P < 0.00001], and quality of life score [-0.99 (0.42, 1.55), P < 0.001]. Compared with antimuscarinics, there were also significant differences in total IPSS [-3.49 (2.43, 4.55), P < 0.00001], obstructive subscore [-1.40 (0.78, 2.01), P < 0.00001], irritative subscore [-2.10 (1.30, 2.90), P < 0.00001], and quality of life score [-1.18 (0.58, 1.80), P < 0.001] in favor of combination group. No significant difference was found in the visual analog pain score and the urinary symptoms score in Ureteral Stent Symptom Questionnaire (USSQ). No significant difference in complications was found.

CONCLUSIONS

Current analysis shows significant advantages of combination therapy compared with monotherapy of α-blockers or antimuscarinics alone mainly based on IPSS. More RCTs adopting validated USSQ as outcome measures are warranted to support the finding.

Do virtual reality games improve mobility skills and balance measurements in community-dwelling older adults? Systematic review and meta-analysis

OBJECTIVE

To summarize evidence on the effectiveness of virtual reality games and conventional therapy or no-intervention for fall prevention in the elderly.

DATA SOURCES

An electronic data search (last searched December 2016) was performed on 10 databases (Web of Science, EMBASE, PUBMED, CINAHL, LILACS, SPORTDiscus, Cochrane Library, Scopus, SciELO, PEDro) and retained only randomized controlled trials.

REVIEW METHOD

Sample characteristics and intervention parameters were compared, focusing on clinical homogeneity of demographic characteristics, type/duration of interventions, outcomes (balance, reaction time, mobility, lower limb strength and fear of falling) and low risk of bias. Based on homogeneity, a meta-analysis was considered. Two independent reviewers assessed the risk of bias.

RESULTS

A total of 28 studies met the inclusion criteria and were appraised ( n: 1121 elderly participants). We found that virtual reality games presented positive effects on balance and fear of falling compared with no-intervention. Virtual reality games were also superior to conventional interventions for balance improvements and fear of falling. The six studies included in the meta-analysis demonstrated that virtual reality games significantly improved mobility and balance after 3-6 and 8-12 weeks of intervention when compared with no-intervention. The risk of bias revealed that less than one-third of the studies correctly described the random sequence generation and allocation concealment procedures.

CONCLUSION

Our review suggests positive clinical effects of virtual reality games for balance and mobility improvements compared with no-treatment and conventional interventions. However, owing to the high risk of bias and large variability of intervention protocols, the evidence remains inconclusive and further research is warranted.

Acupuncture for chronic pain and depression in primary care: a programme of research

Background

There has been an increase in the utilisation of acupuncture in recent years, yet the evidence base is insufficiently well established to be certain about its clinical effectiveness and cost-effectiveness. Addressing the questions related to the evidence base will reduce uncertainty and help policy- and decision-makers with regard to whether or not wider access is appropriate and provides value for money.

Aim

Our aim was to establish the most reliable evidence on the clinical effectiveness and cost-effectiveness of acupuncture for chronic pain by drawing on relevant evidence, including recent high-quality trials, and to develop fresh evidence on acupuncture for depression. To extend the evidence base we synthesised the results of published trials using robust systematic review methodology and conducted a randomised controlled trial (RCT) of acupuncture for depression.

Methods and results

We synthesised the evidence from high-quality trials of acupuncture for chronic pain, consisting of musculoskeletal pain related to the neck and low back, osteoarthritis of the knee, and headache and migraine, involving nearly 18,000 patients. In an individual patient data (IPD) pairwise meta-analysis, acupuncture was significantly better than both sham acupuncture (p < 0.001) and usual care (p < 0.001) for all conditions. Using network meta-analyses, we compared acupuncture with other physical therapies for osteoarthritis of the knee. In both an analysis of all available evidence and an analysis of a subset of better-quality trials, using aggregate-level data, we found acupuncture to be one of the more effective therapies. We developed new Bayesian methods for analysing multiple individual patient-level data sets to evaluate heterogeneous continuous outcomes. An accompanying cost-effectiveness analysis found transcutaneous electrical nerve stimulation (TENS) to be cost-effective for osteoarthritis at a threshold of £20,000 per quality-adjusted life-year when all trials were synthesised. When the analysis was restricted to trials of higher quality with adequate allocation concealment, acupuncture was cost-effective. In a RCT of acupuncture or counselling compared with usual care for depression, in which half the patients were also experiencing comorbid pain, we found acupuncture and counselling to be clinically effective and acupuncture to be cost-effective. For patients in whom acupuncture is inappropriate or unavailable, counselling is cost-effective.

Conclusion

We have provided the most robust evidence from high-quality trials on acupuncture for chronic pain. The synthesis of high-quality IPD found that acupuncture was more effective than both usual care and sham acupuncture. Acupuncture is one of the more clinically effective physical therapies for osteoarthritis and is also cost-effective if only high-quality trials are analysed. When all trials are analysed, TENS is cost-effective. Promising clinical and economic evidence on acupuncture for depression needs to be extended to other contexts and settings. For the conditions we have investigated, the drawing together of evidence on acupuncture from this programme of research has substantially reduced levels of uncertainty. We have identified directions for further research. Our research also provides a valuable basis for considering the potential role of acupuncture as a referral option in health care and enabling providers and policy-makers to make decisions based on robust sources of evidence.

Trial registration

Current Controlled Trials ISRCTN63787732.

Funding

The National Institute for Health Research Programme Grants for Applied Research programme.

Mechanisms and direction of allocation bias in randomised clinical trials

BACKGROUND

Selective allocation of patients into the compared groups of a randomised trial may cause allocation bias, but the mechanisms behind the bias and its directionality are incompletely understood. We therefore analysed the mechanisms and directionality of allocation bias in randomised clinical trials.

METHODS

Two systematic reviews and a theoretical analysis. We conducted one systematic review of empirical studies of motives/methods for deciphering patient allocation sequences; and another review of methods publications commenting on allocation bias. We theoretically analysed the mechanisms of allocation bias and hypothesised which main factors predicts its direction.

RESULTS

Three empirical studies addressed motives/methods for deciphering allocation sequences. Main motives included ensuring best care for patients and ensuring best outcome for the trial. Main methods included various manipulations with randomisation envelopes. Out of 57 methods publications 11 (19 %) mentioned explicitly that allocation bias can go in either direction. We hypothesised that the direction of allocation bias is mainly decided by the interaction between the patient allocators' motives and treatment preference.

CONCLUSION

Inadequate allocation concealment may exaggerate treatment effects in some trials while underestimate effects in others. Our hypothesis provides a theoretical overview of the main factors responsible for the direction of allocation bias.

Intra-Articular Therapies in the Treatment of Hand Osteoarthritis: A Systematic Literature Review

Background

Local treatments to alleviate symptoms in hand osteoarthritis (OA) are preferred, especially in elderly patients with comorbidities. Therefore, we have summarized the benefits and harms of intra-articular (IA) therapies.

Methods

We conducted a systematic literature review until May 2015, including all controlled trials investigating efficacy or safety of any IA therapy in carpometacarpal (CMC) and interphalangeal (IP) OA compared with placebo or other treatments. Two authors independently selected trials and assessed risk of bias using the Cochrane tool. The main efficacy outcome was pain. We performed meta-analysis where appropriate.

Results

A total of 13 trials (864 patients) studying CMC (n = 11) and IP OA (n = 2) were included, comparing corticosteroids or hyaluronic acid (HA) versus placebo (n = 4 and n = 3), and corticosteroids versus HA (n = 6). Single studies investigated infliximab, dextrose, and different HAs. The overall risk of bias was unclear or high in most trials. Meta-analysis of two trials comparing corticosteroids with placebo in CMC OA showed no improvement in pain [mean difference −3.56, 95 % confidence interval (CI) −13.87 to 6.75, scale 0–100). HA also appeared not efficacious compared with placebo in CMC OA. One trial comparing corticosteroids with placebo in IP OA demonstrated significantly improved pain during movement. No convincing evidence for efficacy of corticosteroids or HA over the other or alternative therapies was found. Only local adverse events were reported. No specific IA therapy appeared more harmful than another.

Conclusion

Despite a beneficial short-term safety profile, IA corticosteroids or HA do not appear more effective than placebo in CMC OA. The suggestion that IA corticosteroids might be efficacious in IP OA requires confirmation.

Electronic supplementary material

The online version of this article (doi:10.1007/s40266-015-0330-5) contains supplementary material, which is available to authorized users.

Empirical Evidence of Study Design Biases in Randomized Trials: Systematic Review of Meta-Epidemiological Studies

OBJECTIVE

To synthesise evidence on the average bias and heterogeneity associated with reported methodological features of randomized trials.

DESIGN

Systematic review of meta-epidemiological studies.

METHODS

We retrieved eligible studies included in a recent AHRQ-EPC review on this topic (latest search September 2012), and searched Ovid MEDLINE and Ovid EMBASE for studies indexed from Jan 2012-May 2015. Data were extracted by one author and verified by another. We combined estimates of average bias (e.g. ratio of odds ratios (ROR) or difference in standardised mean differences (dSMD)) in meta-analyses using the random-effects model. Analyses were stratified by type of outcome ("mortality" versus "other objective" versus "subjective"). Direction of effect was standardised so that ROR < 1 and dSMD < 0 denotes a larger intervention effect estimate in trials with an inadequate or unclear (versus adequate) characteristic.

RESULTS

We included 24 studies. The available evidence suggests that intervention effect estimates may be exaggerated in trials with inadequate/unclear (versus adequate) sequence generation (ROR 0.93, 95% CI 0.86 to 0.99; 7 studies) and allocation concealment (ROR 0.90, 95% CI 0.84 to 0.97; 7 studies). For these characteristics, the average bias appeared to be larger in trials of subjective outcomes compared with other objective outcomes. Also, intervention effects for subjective outcomes appear to be exaggerated in trials with lack of/unclear blinding of participants (versus blinding) (dSMD -0.37, 95% CI -0.77 to 0.04; 2 studies), lack of/unclear blinding of outcome assessors (ROR 0.64, 95% CI 0.43 to 0.96; 1 study) and lack of/unclear double blinding (ROR 0.77, 95% CI 0.61 to 0.93; 1 study). The influence of other characteristics (e.g. unblinded trial personnel, attrition) is unclear.

CONCLUSIONS

Certain characteristics of randomized trials may exaggerate intervention effect estimates. The average bias appears to be greatest in trials of subjective outcomes. More research on several characteristics, particularly attrition and selective reporting, is needed.

Evaluation of risk of bias assessment of trials in systematic reviews of oral health interventions, 1991-2014: A methodology study

BACKGROUND

The authors aimed to describe how often and by what means investigators assessed the risk of bias of clinical trials in systematic reviews of oral health interventions and to identify factors associated with risk of bias assessments.

METHODS

The authors selected therapeutic oral health systematic reviews published from 1991 through 2014. They extracted data related to the tools used for risk of bias assessment of primary studies and data related to other review characteristics. They descriptively analyzed the data and used multivariate logistic regression.

RESULTS

The authors identified 1,114 oral health systematic reviews (130 Cochrane reviews and 984 non-Cochrane reviews). The investigators of the primary studies assessed risk of bias in 61.4% of the reviews, and the risk of bias assessments occurred more often in Cochrane reviews than in non-Cochrane reviews (100% versus 56.3%; P < .001) and in reviews published after the dissemination of the Preferred Reporting Items for Systematic Reviews and Meta-Analyses statement (odds ratio [OR], 1.55; 95% confidence interval [CI], 1.17-2.06). Compared with the investigators of reviews of public oral health interventions, investigators of reviews of oral surgery were less likely to assess risk of bias (OR, 0.41; 95% CI, 0.25-0.67). Furthermore, the investigators of systematic reviews published in dental journals were less likely to assess risk of bias of individual trials (OR, 0.28; 95% CI, 0.19-0.41) compared with the investigators of reviews published in nondental journals.

CONCLUSIONS

The investigators of primary studies did not undertake risk of bias assessment in a considerable portion of non-Cochrane oral health systematic reviews. The investigators of reviews published in dental journals were less likely to assess risk of bias than the investigators of reviews published in nondental journals. The results of this study provide evidence of the need for improving the conduct and reporting of oral health systematic reviews with respect to risk of bias assessment.

PRACTICAL IMPLICATIONS

Clinicians should determine to what extent the findings of a systematic review are valid on the basis of whether the investigators assessed and considered risk of bias during the interpretation of findings.

Global Postural Reeducation for patients with musculoskeletal conditions: a systematic review of randomized controlled trials

OBJECTIVES

To systematically review randomized controlled trials that assessed the effects of Global Postural Reeducation (GPR) on patient-reported outcomes in conditions of the musculoskeletal system.

METHOD

An electronic search of MEDLINE (via PubMed), EMBASE, Cochrane CENTRAL, and SciELO was performed from their inception to June 2015. Randomized controlled trials that analyzed pain and patient-reported outcomes were included in this review. The Cochrane Collaboration's Risk of Bias Tool was used to evaluate risk of bias, and the quality of evidence was rated following the GRADE approach. There were no language restrictions.

RESULTS

Eleven trials were included totaling 383 patients. Overall, the trials had high risk of bias. GPR was superior to no treatment but not to other forms of treatment for pain and disability. No placebo-controlled trials were found.

CONCLUSION

GPR is not superior to other treatments; however, it is superior to no treatment. Due to the lack of studies, it is unknown if GPR is better than placebo. The quality of the available evidence ranges from low to very low, therefore future studies may change the effect estimates of GPR in musculoskeletal conditions.

Preoperative progressive explosive-type resistance training is feasible and effective in patients with hip osteoarthritis scheduled for total hip arthroplasty--a randomized controlled trial

OBJECTIVE

To investigate the efficacy and feasibility of progressive explosive-type resistance training (RT) in patients with osteoarthritis (OA) of the hip scheduled for total hip arthroplasty (THA).

METHOD

Randomized controlled trial (1:1) in patients diagnosed with hip OA and scheduled for THA. The intervention group (IG) performed supervised preoperative progressive explosive-type RT twice a week for 10 weeks; four exercises (hip/thigh) performed in three series each (8-12 repetition maximum). The control group (CG) received 'care as usual'. Efficacy was reported as the between-group difference in the Hip Osteoarthritis Outcome Score (HOOS) (primary endpoint; ADL function), and leg muscle power at post intervention follow-up immediate before surgery. Intention-to-treat analyses were performed in a multilevel regression model adjusting for baseline, sex, age and weight. Feasibility was reported as adherence, exercise related pain and adverse effects. Post-surgical follow up will be reported separately. ClinicalTrials.gov registration: NCT01164111.

RESULTS

Eighty patients (age 70.4 ± 7.6 years, BMI 27.8 ± 4.6, 52 females (65%) were included. Adherence was high (93%) with acceptable exercise related pain (VAS score ≤ 5) reported in 83% of sessions and no adverse events. Changes in HOOS 'function' was 10.0 points 95%CI [4.7; 15.3] higher in IG compared to CG (P < 0.001). For all the remaining HOOS subscales IG scored significantly better (P < 0.03) and had higher leg extension muscle power (P < 0.0001) compared to CG.

CONCLUSION

Progressive explosive-type RT was feasible in the included group of hip OA patients scheduled for THA and resulted in significant improvement in self-reported outcomes and increased leg muscle power.

Risk of selection bias in randomised trials

BACKGROUND

Selection bias occurs when recruiters selectively enrol patients into the trial based on what the next treatment allocation is likely to be. This can occur even if appropriate allocation concealment is used if recruiters can guess the next treatment assignment with some degree of accuracy. This typically occurs in unblinded trials when restricted randomisation is implemented to force the number of patients in each arm or within each centre to be the same. Several methods to reduce the risk of selection bias have been suggested; however, it is unclear how often these techniques are used in practice.

METHODS

We performed a review of published trials which were not blinded to assess whether they utilised methods for reducing the risk of selection bias. We assessed the following techniques: (a) blinding of recruiters; (b) use of simple randomisation; (c) avoidance of stratification by site when restricted randomisation is used; (d) avoidance of permuted blocks if stratification by site is used; and (e) incorporation of prognostic covariates into the randomisation procedure when restricted randomisation is used. We included parallel group, individually randomised phase III trials published in four general medical journals (BMJ, Journal of the American Medical Association, The Lancet, and New England Journal of Medicine) in 2010.

RESULTS

We identified 152 eligible trials. Most trials (98%) provided no information on whether recruiters were blind to previous treatment allocations. Only 3% of trials used simple randomisation; 63% used some form of restricted randomisation, and 35% did not state the method of randomisation. Overall, 44% of trials were stratified by site of recruitment; 27% were not, and 29% did not report this information. Most trials that did stratify by site of recruitment used permuted blocks (58%), and only 15% reported using random block sizes. Many trials that used restricted randomisation also included prognostic covariates in the randomisation procedure (56%).

CONCLUSIONS

The risk of selection bias could not be ascertained for most trials due to poor reporting. Many trials which did provide details on the randomisation procedure were at risk of selection bias due to a poorly chosen randomisation methods. Techniques to reduce the risk of selection bias should be more widely implemented.

What is the influence of randomisation sequence generation and allocation concealment on treatment effects of physical therapy trials? A meta-epidemiological study

OBJECTIVE

To determine if adequacy of randomisation and allocation concealment is associated with changes in effect sizes (ES) when comparing physical therapy (PT) trials with and without these methodological characteristics.

DESIGN

Meta-epidemiological study.

PARTICIPANTS

A random sample of randomised controlled trials (RCTs) included in meta-analyses in the PT discipline were identified.

INTERVENTION

Data extraction including assessments of random sequence generation and allocation concealment was conducted independently by two reviewers. To determine the association between sequence generation, and allocation concealment and ES, a two-level analysis was conducted using a meta-meta-analytic approach.

PRIMARY AND SECONDARY OUTCOME MEASURES

association between random sequence generation and allocation concealment and ES in PT trials.

RESULTS

393 trials included in 43 meta-analyses, analysing 44,622 patients contributed to this study. Adequate random sequence generation and appropriate allocation concealment were accomplished in only 39.7% and 11.5% of PT trials, respectively. Although trials with inappropriate allocation concealment tended to have an overestimate treatment effect when compared with trials with adequate concealment of allocation, the difference was non-statistically significant (ES=0.12; 95% CI -0.06 to 0.30). When pooling our results with those of Nuesch et al, we obtained a pooled statistically significant value (ES=0.14; 95% CI 0.02 to 0.26). There was no difference in ES in trials with appropriate or inappropriate random sequence generation (ES=0.02; 95% CI -0.12 to 0.15).

CONCLUSIONS

Our results suggest that when evaluating risk of bias of primary RCTs in PT area, systematic reviewers and clinicians implementing research into practice should pay attention to these biases since they could exaggerate treatment effects. Systematic reviewers should perform sensitivity analysis including trials with low risk of bias in these domains as primary analysis and/or in combination with less restrictive analyses. Authors and editors should make sure that allocation concealment and random sequence generation are properly reported in trial reports.

PEDro or Cochrane to Assess the Quality of Clinical Trials? A Meta-Epidemiological Study

OBJECTIVE

There is debate on how the methodological quality of clinical trials should be assessed. We compared trials of physical therapy (PT) judged to be of adequate quality based on summary scores from the Physiotherapy Evidence Database (PEDro) scale with trials judged to be of adequate quality by Cochrane Risk of Bias criteria.

DESIGN

Meta-epidemiological study within Cochrane Database of Systematic Reviews.

METHODS

Meta-analyses of PT trials were identified in the Cochrane Database of Systematic Reviews. For each trial PeDro and Cochrane assessments were extracted from the PeDro and Cochrane databases. Adequate quality was defined as adequate generation of random sequence, concealment of allocation, and blinding of outcome assessors (Cochrane criteria) or as trials with a PEDro summary score ≥5 or ≥6 points. We combined trials of adequate quality using random-effects meta-analysis.

RESULTS

Forty-one Cochrane reviews and 353 PT trials were included. All meta-analyses included trials with PEDro scores ≥5, 37 (90.2%) included trials with PEDro scores ≥6 and only 22 (53.7%) meta-analyses included trials of adequate quality according to the Cochrane criteria. Agreement between PeDro and Cochrane was poor for PeDro scores of ≥5 points (kappa = 0.12; 95% CI 0.07 to 0.16) and slight for ≥6 points (kappa 0.24; 95% CI 0.16-0.32). When combining effect sizes of trials deemed to be of adequate quality according to PEDro or Cochrane criteria, we found that a substantial difference in the combined effect size (≥0.15) was evident in 9 (22%) out of the 41 meta-analyses for PEDro cutoff ≥5 and 10 (24%) for cutoff ≥6.

CONCLUSIONS

The PeDro and Cochrane approaches lead to different sets of trials of adequate quality, and different combined treatment estimates from meta-analyses of these trials. A consistent approach to assessing RoB in trials of physical therapy should be adopted.

Avoidable waste of research related to inadequate methods in clinical trials

OBJECTIVE

To assess the waste of research related to inadequate methods in trials included in Cochrane reviews and to examine to what extent this waste could be avoided. A secondary objective was to perform a simulation study to re-estimate this avoidable waste if all trials were adequately reported.

DESIGN

Methodological review and simulation study.

DATA SOURCES

Trials included in the meta-analysis of the primary outcome of Cochrane reviews published between April 2012 and March 2013.

DATA EXTRACTION AND SYNTHESIS

We collected the risk of bias assessment made by the review authors for each trial. For a random sample of 200 trials with at least one domain at high risk of bias, we re-assessed risk of bias and identified all related methodological problems. For each problem, possible adjustments were proposed that were then validated by an expert panel also evaluating their feasibility (easy or not) and cost. Avoidable waste was defined as trials with at least one domain at high risk of bias for which easy adjustments with no or minor cost could change all domains to low risk. In the simulation study, after extrapolating our re-assessment of risk of bias to all trials, we considered each domain rated as unclear risk of bias as missing data and used multiple imputations to determine whether they were at high or low risk.

RESULTS

Of 1286 trials from 205 meta-analyses, 556 (43%) had at least one domain at high risk of bias. Among the sample of 200 of these trials, 142 were confirmed as high risk; in these, we identified 25 types of methodological problem. Adjustments were possible in 136 trials (96%). Easy adjustments with no or minor cost could be applied in 71 trials (50%), resulting in 17 trials (12%) changing to low risk for all domains. So the avoidable waste represented 12% (95% CI 7% to 18%) of trials with at least one domain at high risk. After correcting for incomplete reporting, avoidable waste due to inadequate methods was estimated at 42% (95% CI 36% to 49%).

CONCLUSIONS

An important burden of wasted research is related to inadequate methods. This waste could be partly avoided by simple and inexpensive adjustments.

Inducing a blind spot: blinding data collectors in an investigation of experimental pain

OBJECTIVE

Methods of blinding have been infrequently investigated. This study assessed data collector blinding within an investigation of endogenous pain modulation.

DESIGN

Participants (N = 33; 52% women) were randomly assigned to an order of an exercise bout with intramuscular control injections, quiet rest with intramuscular algesic injections, and a control condition. Data collectors (N = 4; 50% women) recorded participants' pain responses and then left the room before the injection and exercise procedures were administered by other personnel. Immediately after the procedures, the data collectors returned to collect additional pain responses and record their suspicions regarding the assigned condition, their confidence level in their suspicions (0-100 mm scale), and the reason/s for their suspicions.

RESULTS

Data collectors correctly identified control and algesic injections in 90.4 and 73.1% of the sessions, respectively, and quiet rest and exercise in 94.2 and 69.2% of the sessions, respectively. The confidence of the data collectors in their suspicions was 60.17 mm (SD = 34.54) for injections and 62.19 mm (SD = 34.95) for exercise. However, data collectors only had correct suspicions and confidence of at least 90 mm for 26.2% of the injections and 29.4% of the exercise bouts. Participants' pain responses were the primary reason for data collectors' suspicions. Neither the amount of experience nor the sexual composition of the participant-data collector dyad influenced the blinding.

CONCLUSIONS

Collection of participants' pain responses led to frequent episodes of unblinding. However, it may be misleading to only consider the frequency of correct suspicions as successful or unsuccessful blinding.

Efficacy and safety of ginger in osteoarthritis patients: a meta-analysis of randomized placebo-controlled trials

The aim of this study was to assess the clinical efficacy and safety of oral ginger for symptomatic treatment of osteoarthritis (OA) by carrying out a systematic literature search followed by meta-analyses on selected studies. Inclusion criteria were randomized controlled trials (RCTs) comparing oral ginger treatment with placebo in OA patients aged >18 years. Outcomes were reduction in pain and reduction in disability. Harm was assessed as withdrawals due to adverse events. The efficacy effect size was estimated using Hedges' standardized mean difference (SMD), and safety by risk ratio (RR). Standard random-effects meta-analysis was used, and inconsistency was evaluated by the I-squared index (I(2)). Out of 122 retrieved references, 117 were discarded, leaving five trials (593 patients) for meta-analyses. The majority reported relevant randomization procedures and blinding, but an inadequate intention-to-treat (ITT) analysis. Following ginger intake, a statistically significant pain reduction SMD = -0.30 ([95% CI: [(-0.50, -0.09)], P = 0.005]) with a low degree of inconsistency among trials (I(2) = 27%), and a statistically significant reduction in disability SMD = -0.22 ([95% CI: ([-0.39, -0.04)]; P = 0.01; I(2) = 0%]) were seen, both in favor of ginger. Patients given ginger were more than twice as likely to discontinue treatment compared to placebo ([RR = 2.33; 95% CI: (1.04, 5.22)]; P = 0.04; I(2) = 0%]). Ginger was modestly efficacious and reasonably safe for treatment of OA. We judged the evidence to be of moderate quality, based on the small number of participants and inadequate ITT populations. Prospero: CRD42011001777.

Assessing bias in osteoarthritis trials included in Cochrane reviews: protocol for a meta-epidemiological study

INTRODUCTION

The validity of systematic reviews and meta-analysis depends on methodological quality and unbiased dissemination of trials. Our objective is to evaluate the association of estimates of treatment effects with different bias-related study characteristics in meta-analyses of interventions used for treating pain in osteoarthritis (OA). From the findings, we hope to consolidate guidance on interpreting OA trials in systematic reviews based on empirical evidence from Cochrane reviews.

METHODS AND ANALYSIS

Only systematic reviews that compare experimental interventions with sham, placebo or no intervention control will be considered eligible. Bias will be assessed with the risk of bias tool, used according to the Cochrane Collaboration's recommendations. Furthermore, center status, trial size and funding will be assessed. The primary outcome (pain) will be abstracted from the first appearing forest plot for overall pain in the Cochrane review. Treatment effect sizes will be expressed as standardised mean differences (SMDs), where the difference in mean values available from the forest plots is divided by the pooled SD. To empirically assess the risk of bias in treatment benefits, we will perform stratified analyses of the trials from the included meta-analyses and assess the interaction between trial characteristics and treatment effect. A relevant study-level covariate is defined as one that decreases the between-study variance (τ(2), estimated as Tau-squared) as a consequence of inclusion in the mixed effects statistical model.

ETHICS AND DISSEMINATION

Meta-analyses and randomised controlled trials provide the most reliable basis for treatment of patients with OA, but the actual impact of bias is unclear. This study will systematically examine the methodological quality in OA Cochrane reviews and explore the effect estimates behind possible bias. Since our study does not collect primary data, no formal ethical assessment and informed consent are required.

TRIAL REGISTRATION NUMBER

PROSPERO (CRD42013006924).

Meta-epidemiology

The concept of meta-epidemiology has been introduced with considering the methodological limitations of systematic review for intervention trials. The paradigm of meta-epidemiology has shifted from a statistical method into a new methodology to close gaps between evidence and practice. Main interest of meta-epidemiology is to control potential biases in previous quantitative systematic reviews and draw appropriate evidences for establishing evidence-base guidelines. Nowadays, the network meta-epidemiology was suggested in order to overcome some limitations of meta-epidemiology. To activate meta-epidemiologic studies, implementation of tools for risk of bias and reporting guidelines such as the Consolidated Standards for Reporting Trials (CONSORT) should be done.

A theoretical analysis showed that blinding cannot eliminate potential for bias associated with beliefs about allocation in randomized clinical trials

OBJECTIVES

To explore the theoretical justification for blinding in randomized trials and make recommendations concerning the implementation and interpretation of blinded randomized trials.

STUDY DESIGN AND SETTING

A theoretical analysis was conducted of the potential for bias in randomized trials with successful blinding (ie, trials in which beliefs about allocation to treatment or control groups are independent of actual allocation). The analysis identified conditions that must be satisfied to ensure that blinding eliminates the potential for bias associated with beliefs about allocation.

RESULTS

Even when beliefs about allocation are independent of actual allocation, they can still cause bias. The potential for bias is eliminated when the belief is uniformly one of complete ambivalence about allocation.

CONCLUSION

Even when blinding succeeds in making beliefs about allocation independent of actual allocation, beliefs about allocation may still cause bias. It is difficult to determine the extent of bias in any particular trial. Bias could be eliminated by establishing a state of complete ambivalence about the allocation of every trial participant, but universal ambivalence may be difficult to achieve and may reduce the generalizability of the trial's findings.

Bias due to lack of patient blinding in clinical trials. A systematic review of trials randomizing patients to blind and nonblind sub-studies

BACKGROUND

Blinding patients in clinical trials is a key methodological procedure, but the expected degree of bias due to nonblinded patients on estimated treatment effects is unknown.

METHODS

Systematic review of randomized clinical trials with one sub-study (i.e. experimental vs control) involving blinded patients and another, otherwise identical, sub-study involving nonblinded patients. Within each trial, we compared the difference in effect sizes (i.e. standardized mean differences) between the sub-studies. A difference <0 indicates that nonblinded patients generated a more optimistic effect estimate. We pooled the differences with random-effects inverse variance meta-analysis, and explored reasons for heterogeneity.

RESULTS

Our main analysis included 12 trials (3869 patients). The average difference in effect size for patient-reported outcomes was -0.56 (95% confidence interval -0.71 to -0.41), (I(2)=60%, P=0.004), i.e. nonblinded patients exaggerated the effect size by an average of 0.56 standard deviation, but with considerable variation. Two of the 12 trials also used observer-reported outcomes, showing no indication of exaggerated effects due lack of patient blinding. There was a larger effect size difference in 10 acupuncture trials [-0.63 (-0.77 to -0.49)], than in the two non-acupuncture trials [-0.17 (-0.41 to 0.07)]. Lack of patient blinding also increased attrition and use of co-interventions: ratio of control group attrition risk 1.79 (1.18 to 2.70), and ratio of control group co-intervention risk 1.55 (0.99 to 2.43).

CONCLUSIONS

This study provides empirical evidence of pronounced bias due to lack of patient blinding in complementary/alternative randomized clinical trials with patient-reported outcomes.

Methodological characteristics and treatment effect sizes in oral health randomised controlled trials: Is there a relationship? Protocol for a meta-epidemiological study

INTRODUCTION

It is fundamental that randomised controlled trials (RCTs) are properly conducted in order to reach well-supported conclusions. However, there is emerging evidence that RCTs are subject to biases which can overestimate or underestimate the true treatment effect, due to flaws in the study design characteristics of such trials. The extent to which this holds true in oral health RCTs, which have some unique design characteristics compared to RCTs in other health fields, is unclear. As such, we aim to examine the empirical evidence quantifying the extent of bias associated with methodological and non-methodological characteristics in oral health RCTs.

METHODS AND ANALYSIS

We plan to perform a meta-epidemiological study, where a sample size of 60 meta-analyses (MAs) including approximately 600 RCTs will be selected. The MAs will be randomly obtained from the Oral Health Database of Systematic Reviews using a random number table; and will be considered for inclusion if they include a minimum of five RCTs, and examine a therapeutic intervention related to one of the recognised dental specialties. RCTs identified in selected MAs will be subsequently included if their study design includes a comparison between an intervention group and a placebo group or another intervention group. Data will be extracted from selected trials included in MAs based on a number of methodological and non-methodological characteristics. Moreover, the risk of bias will be assessed using the Cochrane Risk of Bias tool. Effect size estimates and measures of variability for the main outcome will be extracted from each RCT included in selected MAs, and a two-level analysis will be conducted using a meta-meta-analytic approach with a random effects model to allow for intra-MA and inter-MA heterogeneity.

ETHICS AND DISSEMINATION

The intended audiences of the findings will include dental clinicians, oral health researchers, policymakers and graduate students. The aforementioned will be introduced to the findings through workshops, seminars, round table discussions and targeted individual meetings. Other opportunities for knowledge transfer will be pursued such as key dental conferences. Finally, the results will be published as a scientific report in a dental peer-reviewed journal.

Funding source and the quality of reports of chronic wounds trials: 2004 to 2011

BACKGROUND

Critical commentaries suggest that wound care randomised controlled trials (RCTs) are often poorly reported with many methodological flaws. Furthermore, interventions in chronic wounds, rather than being drugs, are often medical devices for which there are no requirements for RCTs to bring products to market. RCTs in wounds trials therefore potentially represent a form of marketing. This study presents a methodological overview of chronic wound trials published between 2004 and 2011 and investigates the influence of industry funding on methodological quality.

METHODS

A systematic search for RCTs for the treatment of chronic wounds published in the English language between 2004 and 2011 (inclusive) in the Cochrane Wounds Group Specialised Register of Trials was carried out.Data were extracted on aspects of trial design, conduct and quality including sample size, duration of follow-up, specification of a primary outcome, use of surrogate outcomes, and risks of bias. In addition, the prevalence of industry funding was assessed and its influence on the above aspects of trial design, conduct and quality was assessed.

RESULTS

A total of 167 RCTs met our inclusion criteria. We found chronic wound trials often have short durations of follow-up (median 12 weeks), small sample sizes (median 63), fail to define a primary outcome in 41% of cases, and those that do define a primary outcome, use surrogate measures of healing in 40% of cases. Only 40% of trials used appropriate methods of randomisation, 25% concealed allocation and 34% blinded outcome assessors. Of the included trials, 41% were wholly or partially funded by industry, 33% declared non-commercial funding and 26% did not report a funding source. Industry funding was not statistically significantly associated with any measure of methodological quality, though this analysis was probably underpowered.

CONCLUSIONS

This overview confirms concerns raised about the methodological quality of RCTs in wound care and illustrates that greater efforts must be made to follow international standards for conducting and reporting RCTs. There is currently minimal evidence of an influence of industry funding on methodological quality although analyses had limited power and funding source was not reported for a quarter of studies.

A look at the potential association between PICOT framing of a research question and the quality of reporting of analgesia RCTs

BACKGROUND

Methodologists have proposed the formation of a good research question to initiate the process of developing a research protocol that will guide the design, conduct and analysis of randomized controlled trials (RCTs), and help improve the quality of reporting such studies. Five constituents of a good research question based on the PICOT framing include: Population, Intervention, Comparator, Outcome, and Time-frame of outcome assessment. The aim of this study was to analyze if the presence a structured research question, in PICOT format, in RCTs used within a 2010 meta-analysis investigating the effectiveness of femoral nerve blocks after total knee arthroplasty, is independently associated with improved quality of reporting.

METHODS

Twenty-three RCT reports were assessed for the quality of reporting and then examined for the presence of the five constituents of a structured research question based on PICOT framing. We created a PICOT score (predictor variable), with a possible score between 0 and 5; one point for every constituent that was included. Our outcome variable was a 14 point overall reporting quality score (OQRS) and a 3 point key methodological items score (KMIS) based on the proper reporting of allocation concealment, blinding and numbers analysed using the intention-to-treat principle. Both scores, OQRS and KMIS, are based on the Consolidated Standards for Reporting Trials (CONSORT) statement. A multivariable regression analysis was conducted to determine if PICOT score was independently associated with OQRS and KMIS.

RESULTS

A completely structured PICOT score question was found in 2 of the 23 RCTs evaluated. Although not statistically significant, higher PICOT was associated with higher OQRS [IRR: 1.267; 95% confidence interval (CI): 0.984, 1.630; p = 0.066] but not KMIS (1.061 (0.515, 2.188); 0.872). These results are comparable to those from a similar study in terms of the direction and range of IRRs estimates. The results need to be interpreted cautiously due to the small sample size.

CONCLUSIONS

This study showed that PICOT framing of a research question in anesthesia-related RCTs is not often followed. Even though a statistically significant association with higher OQRS was not found, PICOT framing of a research question is still an important attribute within all RCTs.

Comparing Haemophilus influenzae type b conjugate vaccine schedules: a systematic review and meta-analysis of vaccine trials

BACKGROUND

The optimal schedule and the need for a booster dose are unclear for Haemophilus influenzae type b (Hib) conjugate vaccines. We systematically reviewed relative effects of Hib vaccine schedules.

METHODS

We searched 21 databases to May 2010 or June 2012 and selected randomized controlled trials or quasi-randomized controlled trials that compared different Hib schedules (3 primary doses with no booster dose [3p+0], 3p+1 and 2p+1) or different intervals in primary schedules and between primary and booster schedules. Outcomes were clinical efficacy, nasopharyngeal carriage and immunological response. Results were combined in random-effects meta-analysis.

RESULTS

Twenty trials from 15 countries were included; 16 used vaccines conjugated to tetanus toxoid (polyribosylribitol phosphate conjugated to tetanus toxoid). No trials assessed clinical or carriage outcomes. Twenty trials examined immunological outcomes and found few relevant differences. Comparing polyribosylribitol phosphate conjugated to tetanus toxoid 3p+0 with 2p+0, there was no difference in seropositivity at the 1.0 μg/mL threshold by 6 months after the last primary dose (combined risk difference -0.02; 95% confidence interval: -0.10, 0.06). Only small differences were seen between schedules starting at different ages, with different intervals between primary doses, or with different intervals between primary and booster doses. Individuals receiving a booster were more likely to be seropositive than those at the same age who did not.

CONCLUSIONS

There is no clear evidence from trials that any 2p+1, 3p+0 or 3p+1 schedule of Hib conjugate vaccine is likely to provide better protection against Hib disease than other schedules. Until more data become available, scheduling is likely to be determined by epidemiological and programmatic considerations in individual settings.

How should we evaluate the risk of bias of physical therapy trials?: a psychometric and meta-epidemiological approach towards developing guidelines for the design, conduct, and reporting of RCTs in Physical Therapy (PT) area: a study protocol

BACKGROUND

Numerous tools and items have been developed in all health areas to assess the risk of bias of randomized controlled trials (RCTs). The Cochrane Collaboration (CC) released a new tool to assess bias in RCTs, based on empirical evidence quantifying the association between some design features and estimates of treatment effects (TEs). However, this evidence is limited to medicine and investigating a selected set of components. No such studies have been conducted in other health areas such as Physical Therapy (PT) and allied health professions. Evidence specific to the PT area is needed to understand and quantify the association between design features and TE estimates to inform practice and decision-making in this field. The overall goal of this project is to provide direction for the design, conduct, reporting and bias assessment of PT RCTs. We will achieve this through the following specific objectives and methods.

METHODS/DESIGN

1) to measure the association between methodological components and other factors (for example, PT area, type of intervention, type of outcomes) and TE estimates in RCTs in PT, 40 randomly selected meta-analyses of RCTs involving PT interventions will be identified from the Cochrane Database of Systematic Reviews. Trials will be evaluated independently by two reviewers using the most commonly used tools in the PT field. A two-level analysis will be conducted using a meta-meta-analytic approach; 2) to identify relevant items to evaluate risk of bias of PT trials, an exploratory factor analysis (EFA) will be used to identify the latent structure of the items; 3) to develop guidelines for the design, conduct, reporting, and risk of bias assessment of PT RCTs, items obtained from the factor analysis and the meta-epidemiological approach will be further evaluated by experts in PT through a web-based survey following a Delphi procedure.

DISCUSSION

The results of this project will have a direct impact on research and practice in PT and are valuable to a number of stakeholders: researchers when designing, conducting, and reporting trials; systematic reviewers and meta-analysts when synthesizing trial results; physiotherapists when making day-to-day treatment decision; and, other healthcare decision-makers, such as those developing policy or practice guidelines.

Combining follow-up and change data is valid in meta-analyses of continuous outcomes: a meta-epidemiological study

OBJECTIVE

To investigate whether it is valid to combine follow-up and change data when conducting meta-analyses of continuous outcomes.

STUDY DESIGN AND SETTING

Meta-epidemiological study of randomized controlled trials in patients with osteoarthritis of the knee/hip, which assessed patient-reported pain. We calculated standardized mean differences (SMDs) based on follow-up and change data, and pooled within-trial differences in SMDs. We also derived pooled SMDs indicating the largest treatment effect within a trial (optimistic selection of SMDs) and derived pooled SMDs from the estimate indicating the smallest treatment effect within a trial (pessimistic selection of SMDs).

RESULTS

A total of 21 meta-analyses with 189 trials with 292 randomized comparisons in 41,256 patients were included. On average, SMDs were 0.04 standard deviation units more beneficial when follow-up values were used (difference in SMDs: -0.04; 95% confidence interval: -0.13, 0.06; P=0.44). In 13 meta-analyses (62%), there was a relevant difference in clinical and/or significance level between optimistic and pessimistic pooled SMDs.

CONCLUSION

On average, there is no relevant difference between follow-up and change data SMDs, and combining these estimates in meta-analysis is generally valid. Decision on which type of data to use when both follow-up and change data are available should be prespecified in the meta-analysis protocol.

Comparison of treatment effect estimates for pharmacological randomized controlled trials enrolling older adults only and those including adults: a meta-epidemiological study

CONTEXT

Older adults are underrepresented in clinical research. To assess therapeutic efficacy in older patients, some randomized controlled trials (RCTs) include older adults only.

OBJECTIVE

To compare treatment effects between RCTs including older adults only (elderly RCTs) and RCTs including all adults (adult RCTs) by a meta-epidemiological approach.

METHODS

All systematic reviews published in the Cochrane Library (Issue 4, 2011) were screened. Eligible studies were meta-analyses of binary outcomes of pharmacologic treatment including at least one elderly RCT and at least one adult RCT. For each meta-analysis, we compared summary odds ratios for elderly RCTs and adult RCTs by calculating a ratio of odds ratios (ROR). A summary ROR was estimated across all meta-analyses.

RESULTS

We selected 55 meta-analyses including 524 RCTs (17% elderly RCTs). The treatment effects differed beyond that expected by chance for 7 (13%) meta-analyses, showing more favourable treatment effects in elderly RCTs in 5 cases and in adult RCTs in 2 cases. The summary ROR was 0.91 (95% CI, 0.77-1.08, p = 0.28), with substantial heterogeneity (I(2) = 51% and τ(2) = 0.14). Sensitivity and subgroup analyses by type-of-age RCT (elderly RCTs vs RCTs excluding older adults and vs RCTs of mixed-age adults), type of outcome (mortality or other) and type of comparator (placebo or active drug) yielded similar results.

CONCLUSIONS

The efficacy of pharmacologic treatments did not significantly differ, on average, between RCTs including older adults only and RCTs of all adults. However, clinically important discrepancies may occur and should be considered when generalizing evidence from all adults to older adults.

Methods to convert continuous outcomes into odds ratios of treatment response and numbers needed to treat: meta-epidemiological study

BACKGROUND

Clinicians find standardized mean differences (SMDs) calculated from continuous outcomes difficult to interpret. Our objective was to determine the performance of methods in converting SMDs or means to odds ratios of treatment response and numbers needed to treat (NNTs) as more intuitive measures of treatment effect.

METHODS

Meta-epidemiological study of large-scale trials (≥ 100 patients per group) comparing active treatment with placebo, sham or non-intervention control. Trials had to use pain or global symptoms as continuous outcomes and report both the percentage of patients with treatment response and mean pain or symptom scores per group. For each trial, we calculated odds ratios of observed treatment response and NNTs and approximated these estimates from SMDs or means using all five currently available conversion methods by Hasselblad and Hedges (HH), Cox and Snell (CS), Furukawa (FU), Suissa (SU) and Kraemer and Kupfer (KK). We compared observed and approximated values within trials by deriving pooled ratios of odds ratios (RORs) and differences in NNTs. ROR <1 and positive differences in NNTs imply that approximations are more conservative than estimates calculated from observed treatment response. As measures of agreement, we calculated intraclass correlation coefficients.

RESULTS

A total of 29 trials in 13 654 patients were included. Four out of five methods were suitable (HH, CS, FU, SU), with RORs between 0.92 for SU [95% confidence interval (95% CI), 0.86-0.99] and 0.97 for HH (95% CI, 0.91-1.04) and differences in NNTs between 0.5 (95% CI, -0.1 to -1.6) and 1.3 (95% CI, 0.4-2.1). Intraclass correlation coefficients were ≥ 0.90 for these four methods, but ≤ 0.76 for the fifth method by KK (P for differences ≤ 0.027).

CONCLUSIONS

The methods by HH, CS, FU and SU are suitable to convert summary treatment effects calculated from continuous outcomes into odds ratios of treatment response and NNTs, whereas the method by KK is unsuitable.

Completeness of reporting in randomized controlled trials of 3 vaccines: a review of adherence to the CONSORT checklist

BACKGROUND

Clear reporting of randomized controlled trials (RCTs) of vaccines is important for understanding results and assessing their validity. The CONsolidated Standards of Reporting Trials (CONSORT) statement provides guidance to help authors reporting RCTs. The objective was to assess the completeness of reporting of RCTs of vaccines based on the CONSORT 2010 checklist.

METHODS

We collected data about items required by the CONSORT checklist or specific to trials of vaccines. We used publications of RCTs identified in 3 systematic reviews of pneumococcal polysaccharide, pneumococcal conjugate and rotavirus vaccines. We included the first journal publication that reported clinical, carriage or immunological data for each trial and summarized results descriptively.

RESULTS

We included 70 publications from 19 journals. Of these, 14 publications (20%) stated in the title that the trial was randomized and 26 publications (37%) nominated at least 1 primary outcome. The method for generating the random allocation sequence was fully reported in 24 publications (34%), the method of allocation concealment in 9 publications (13%) and 30 publications (43%) included a flow diagram. Trial registration numbers were reported in all articles published in 2010 to 2011. Actual age at vaccination was reported in 20% of trials of childhood schedules. Eleven of 19 journals endorsed the CONSORT statement.

CONCLUSIONS

The reporting of RCTs of vaccines is incomplete, with important methodological details missing from most reports. Journals could play a leading role in implementing changes. Improved reporting would make publications of vaccine trials easier to find, the findings easier to interpret and aid the incorporation of findings into policy.

The quality of reporting of RCTs used within a postoperative pain management meta-analysis, using the CONSORT statement

Background

Randomized controlled trials (RCTs) are routinely used in systematic reviews and meta-analyses that help inform healthcare and policy decision making. The proper reporting of RCTs is important because it acts as a proxy for health care providers and researchers to appraise the quality of the methodology, conduct and analysis of an RCT. The aims of this study are to analyse the overall quality of reporting in 23 RCTs that were used in a meta-analysis by assessing 3 key methodological items, and to determine factors associated with high quality of reporting. It is hypothesized that studies with larger sample sizes, that have funding reported, that are published in journals with a higher impact factor and that are in journals that have adopted or endorsed the CONSORT statement will be associated with better overall quality of reporting and reporting of key methodological items.

Methods

We systematically reviewed RCTs used within an anesthesiology related post-operative pain management meta-analysis. We included all of the 23 RCTs used, all of which were parallel design that addressed the use of femoral nerve block in improving outcomes after total knee arthroplasty. Data abstraction was done independently by two reviewers. The two main outcomes were: 1) 15 point overall quality of reporting score (OQRS) based on the Consolidated Standards for Reporting Trials (CONSORT) and 2) 3 point key methodological item score (KMIS) based on allocation concealment, blinding and intention-to-treat analysis.

Results

Twenty-three RCTs were included. The median OQRS was 9.0 (Interquartile Range = 3). A multivariable regression analysis did not show any significant association between OQRS or KMIS and our four predictor variables hypothesized to improve reporting. The direction and magnitude of our results when compared to similar studies suggest that the sample size and impact factor are associated with improved key methodological item reporting.

Conclusions

The quality of reporting of RCTs used within an anesthesia related meta-analysis is poor to moderate. The information gained from this study should be used by journals to register the urgency for RCTs to be clear and transparent in reporting to help make literature accessible and comparable.