Randomisation to protect against selection bias in healthcare trials

Risk of bias assessment of sequence generation: a study of 100 systematic reviews of trials

BACKGROUND

Systematic reviews of randomised trials guide policy and healthcare decisions. Yet, we observed that some reviews judge randomised trials as high or unclear risk of bias (ROB) for sequence generation, potentially introducing bias. However, to date, the extent of this issue has not been well examined. We evaluated the consistency in the ROB assessment for sequence generation of randomised trials in Cochrane and non-Cochrane reviews, and explored the reviewers' judgement of the quality of evidence for the related outcomes.

METHODS

Cochrane intervention reviews (01/01/2017-31/03/2017) were retrieved from the Cochrane Database of Systematic Reviews. We also searched for systematic reviews in ten general medical journals with highest impact factors (01/01/2016-31/03/2017). We examined the proportion of reviews that rated the sequence generation domain as high, low or unclear risk of selection bias. For reviews that had rated any randomised trials as high or unclear risk of bias, we examined the proportion that had assessed the quality of evidence.

RESULTS

Overall, 100 systematic reviews were included in our analysis. We evaluated 64 Cochrane reviews which comprised of 984 randomised trials; 0.8% (n = 8) and 52.2% (n = 514) were rated as high and unclear ROB for sequence generation respectively. We further evaluated 36 non-Cochrane reviews which comprised of 1376 trials; 5.8% (n = 80) and 39.6% (n = 545) were rated as high and unclear ROB respectively. Ninety percent (n = 10) of non-Cochrane reviews which rated randomised trials as high ROB for sequence generation did not report an underlying reason. All Cochrane reviews assessed the quality of evidence (GRADE). For the non-Cochrane reviews, only just over half had assessed the quality of evidence.

CONCLUSION

Systematic reviews of interventions frequently rate randomised trials as high or unclear ROB for sequence generation. In general, Cochrane reviews were more transparent than non-Cochrane reviews in ROB and quality of evidence assessment. The scientific community should more strongly promote consistent ROB assessment for sequence generation to minimise selection bias and support transparent quality of evidence assessment. Consistency ensures that appropriate conclusions are drawn from the data.

Part I: Minimum Quality Threshold in Preclinical Sepsis Studies (MQTiPSS) for Study Design and Humane Modeling Endpoints

Preclinical animal studies are mandatory before new treatments can be tested in clinical trials. However, their use in developing new therapies for sepsis has been controversial because of limitations of the models and inconsistencies with the clinical conditions. In consideration of the revised definition for clinical sepsis and septic shock (Sepsis-3), a Wiggers-Bernard Conference was held in Vienna in May 2017 to propose standardized guidelines on preclinical sepsis modeling. The participants conducted a literature review of 260 most highly cited scientific articles on sepsis models published between 2003 and 2012. The review showed, for example, that mice were used in 79% and euthanasia criteria were defined in 9% of the studies. Part I of this report details the recommendations for study design and humane modeling endpoints that should be addressed in sepsis models. The first recommendation is that survival follow-up should reflect the clinical time course of the infectious agent used in the sepsis model. Furthermore, it is recommended that therapeutic interventions should be initiated after the septic insult replicating clinical care. To define an unbiased and reproducible association between a new treatment and outcome, a randomization and blinding of treatments as well as inclusion of all methodological details in scientific publications is essential. In all preclinical sepsis studies, the high standards of animal welfare must be implemented. Therefore, development and validation of specific criteria for monitoring pain and distress, and euthanasia of septic animals, as well as the use of analgesics are recommended. A set of four considerations is also proposed to enhance translation potential of sepsis models. Relevant biological variables and comorbidities should be included in the study design and sepsis modeling should be extended to mammalian species other than rodents. In addition, the need for source control (in case of a defined infection focus) should be considered. These recommendations and considerations are proposed as "best practices" for animal models of sepsis that should be implemented.

Strategies and challenges associated with recruiting retirement village communities and residents into a group exercise intervention

BACKGROUND

Randomized controlled trials (RCTs) provide the highest level of scientific evidence, but successful participant recruitment is critical to ensure the external and internal validity of results. This study describes the strategies associated with recruiting older adults at increased falls risk residing in retirement villages into an 18-month cluster RCT designed to evaluate the effects of a dual-task exercise program on falls and physical and cognitive function.

METHODS

Recruitment of adults aged ≥65 at increased falls risk residing within retirement villages (size 60-350 residents) was initially designed to occur over 12 months using two distinct cohorts (C). Recruitment occurred via a three-stage approach that included liaising with: 1) village operators, 2) independent village managers, and 3) residents. To recruit residents, a variety of different approaches were used, including distribution of information pack, on-site presentations, free muscle and functional testing, and posters displayed in common areas.

RESULTS

Due to challenges with recruitment, three cohorts were established between February 2014 and April 2015 (14 months). Sixty retirement villages were initially invited, of which 32 declined or did not respond, leaving 28 villages that expressed interest. A total of 3947 individual letters of invitation were subsequently distributed to residents of these villages, from which 517 (13.1%) expressions of interest (EOI) were received. Across three cohorts with different recruitment strategies adopted there were only modest differences in the number of EOI received (10.5 to 15.3%), which suggests that no particular recruitment approach was most effective. Following the initial screening of these residents, 398 (77.0%) participants were deemed eligible to participate, but a final sample of 300 (58.0% of the 517 EOI) consented and was randomized; 7.6% of the 3947 residents invited. Principal reasons for not participating, despite being eligible, were poor health, lack of time and no GP approval.

CONCLUSION

This study highlights that there are significant challenges associated with recruiting sufficient numbers of older adults from independent living retirement villages into an exercise intervention designed to improve health and well-being.

TRIAL REGISTRATION

Australian New Zealand Clinical Trials Registry: ACTRN12613001 161718 . Date registered 23rd October 2013.

Recommendations for the design of small population clinical trials

Background

Orphan drug development faces numerous challenges, including low disease prevalence, patient population heterogeneity, and strong presence of paediatric patient populations. Consequently, clinical trials for orphan drugs are often smaller than those of non-orphan drugs, and they require the development of efficient trial designs relevant to small populations to gain the most information from the available data. The International Rare Diseases Research Consortium (IRDiRC) is aimed at promoting international collaboration and advance rare diseases research worldwide, and has as one of its goals to contribute to 1000 new therapies for rare diseases. IRDiRC set up a Small Population Clinical Trials (SPCT) Task Force in order to address the shortcomings of our understanding in carrying out clinical trials in rare diseases.

Results

The IRDiRC SPCT Task Force met in March 2016 to discuss challenges faced in the design of small studies for rare diseases and present their recommendations, structured around six topics: different study methods/designs and their relation to different characteristics of medical conditions, adequate safety data, multi-arm trial designs, decision analytic approaches and rational approaches to adjusting levels of evidence, extrapolation, and patients’ engagement in study design.

Conclusions

Recommendations have been issued based on discussions of the Small Population Clinical Trials Task Force that aim to contribute towards successful therapy development and clinical use. While randomised clinical trials are still considered the gold standard, it is recommended to systematically take into consideration alternative trial design options when studying treatments for a rare disease. Combining different sources of safety data is important to give a fuller picture of a therapy’s safety profile. Multi-arm trials should be considered an opportunity for rare diseases therapy development, and funders are encouraged to support such trial design via international networks. Patient engagement is critical in trial design and therapy development, a process which sponsors are encouraged to incorporate when conducting trials and clinical studies. Input from multiple regulatory agencies is recommended early and throughout clinical development. Regulators are often supportive of new clinical trial designs, provided they are well thought through and justified, and they also welcome discussions and questions on this topic. Parallel advice for multiregional development programs should also be considered.

Evaluating the importance of sham-controlled trials in the investigation of medical devices in interventional cardiology

Cardiovascular medicine is one of the specialties that has relied most heavily on evidence from randomised clinical trials in determining best practice for the management of common disease conditions. When comparing treatment approaches, trials incorporating random allocation are the most appropriate method for protecting against treatment allocation bias. In order to protect against performance and ascertainment bias, trial designs including placebo control are preferable where feasible. In contrast to testing of medicines, treatments based on procedures or use of medical devices are more challenging to assess, as sham procedures are necessary to facilitate blinding of participants. However, in many cases, ethical concerns exist, as individual patients allocated to sham procedure are exposed only to risk without potential for benefit. Accordingly, the potential benefits to the general patient population must be carefully weighed against the risks of the exposed individuals. For this reason, trial design and study conduct are critically important to ensure that the investigation has the best chance of answering the study question at hand. In the current manuscript, we aim to review issues relating to the conduct of sham-controlled trials and discuss a number of recent examples in the field of interventional cardiology.

Discontinuation of non-anti-TNF drugs for rheumatoid arthritis in interventional versus observational studies: a systematic review and meta-analysis

PURPOSE

Although randomized controlled trials (RCTs) are the gold standard for the assessment of clinical outcomes, long-term extension trials (LTEs) and observational cohorts may help generate evidence. Our goal was to compare the discontinuation rates of abatacept, rituximab, and tocilizumab in rheumatoid arthritis (RA) reported in different study designs.

METHODS

A systematic review was conducted with searches in PubMed, Scopus, and the Cochrane Library, plus a manual search, for RCTs, LTEs, and observational cohorts reporting discontinuation rates by any of three causes (all-cause, inefficacy, adverse events). Meta-analyses with sensitivity analyses and meta-regressions were conducted.

RESULTS

Of the 111 studies included, 74 were RCTs (n = 55) or LTEs (n = 17) reporting data on abatacept (n = 33), rituximab (n = 10), and tocilizumab (n = 31) and 37 were observational cohort studies (abatacept = 11, rituximab = 8, tocilizumab = 18). The follow-up duration did not differ among the study designs. Discontinuation rates were similar among the drugs but varied among the study designs. Discontinuation rates were significantly higher in cohort studies than those in interventional studies for the three drugs. Sensitivity analyses could not identify patient characteristics associated with these differences. Meta-regression analyses demonstrated no correlation between study follow-up duration and discontinuation rates.

CONCLUSIONS

The discontinuation rates reported for non-anti-TNF drugs varied relative to the study design in which they were investigated. Regulatory agencies, price-setting entities, and evidence-gathering researchers should consider the effect of the real-life environment in their decisions and conclusions.

Study design and baseline characteristics of a combined educational and environmental intervention trial to lower sodium intake in Swiss employees

BACKGROUND

Blood pressure is a primary cardiovascular disease risk factor. Population-wide governmental strategies aim to reduce lifestyle and dietary risk factors for hypertension, one of which is an unbalanced diet with high sodium and low potassium intakes. Nutrition interventions in the workplace are considered a promising approach in encouraging health-promoting behaviors. We developed and conducted the health promoting sodium reduction trial "Healthful & Tasty: Sure!" in worksites in the German-speaking part of Switzerland from May 2015 to Nov 2016, for which we present the study protocol and baseline characteristics.

METHODS

Healthful & Tasty, a cluster nonrandomized single-arm trial with calibration arm, aimed to demonstrate the effectiveness of a combined educational and environmental intervention in the workplace in reducing employees' average daily sodium/salt intake by 15%. To this end, health and food literacy of employees and guideline compliance among the catering facility team needed to be improved. The primary outcome measure was sodium/salt intake estimated from sodium excretion in a 24-h urine sample. Secondary outcome measures included changes in the overall qualitative diet composition, blood pressure, anthropometric indices, and health and food literacy. Of eight organizations with catering facilities, seven organizations took part in the nutrition education and catering salt reduction interventions, and one organization participated as a control. Overall, 145 consenting employees were included in the staggered, one-year four-phase trial, of which 132 participated in the intervention group. In addition to catering surveys and food sampling, the trial included five follow-up health assessments including questionnaires, blood pressure measurements, anthropometrics, and sodium, potassium, and iodine intake measurements obtained from 24-h and spot urine samples, and a food record checklist. Exploratory and hypothesis generating baseline statistical analysis included 141 participants with adequate 24-h urine samples.

DISCUSSION

Despite practice-driven limitations to the study design and small cluster and participant numbers, this trial has methodological strength and will provide important insights into the effectiveness of a combined educational and environmental intervention to reduce salt intake among female and male Swiss employees.

TRIAL REGISTRATION

German Clinical Trials Register, DRKS00006790 . Registered 23 September 2014.

A systematic review of the effects of acupuncture on xerostomia and hyposalivation

BACKGROUND

Saliva is fundamental to our oral health and our well-being. Many factors can impair saliva secretion, such as adverse effects of prescribed medication, auto-immune diseases (for example Sjögren's syndrome) and radiotherapy for head and neck cancers. Several studies have suggested a positive effect of acupuncture on oral dryness.

METHODS

Pubmed and Web of Science were electronically searched. Reference lists of the included studies and relevant reviews were manually searched. Studies that met the inclusion criteria were systematically evaluated. Two reviewers assessed each of the included studies to confirm eligibility and assessing the risk of bias.

RESULTS

Ten randomized controlled trials investigating the effect of acupuncture were included. Five trials compared acupuncture to sham/placebo acupuncture. Four trials compared acupuncture to oral hygiene/usual care. Only one clinical trial used oral care sessions as control group. For all the included studies, the quality for all the main outcomes has been assessed as low. Although some publications suggest a positive effect of acupuncture on either salivary flow rate or subjective dry mouth feeling, the studies are inconclusive about the potential effects of acupuncture.

CONCLUSIONS

Insufficient evidence is available to conclude whether acupuncture is an evidence-based treatment option for xerostomia/hyposalivation. Further well-designed, larger, double blinded trials are required to determine the potential benefit of acupuncture. Sample size calculations should be performed before before initiating these studies.

Key Concepts for Informed Health Choices: a framework for helping people learn how to assess treatment claims and make informed choices

Many claims about the effects of treatments, though well intentioned, are wrong. Indeed, they are sometimes deliberately misleading to serve interests other than the well-being of patients and the public. People need to know how to spot unreliable treatment claims so that they can protect themselves and others from harm. The ability to assess the trustworthiness of treatment claims is often lacking. Acquiring this ability depends on being familiar with, and correctly applying, some key concepts, for example, that' association is not the same as causation.' The Informed Health Choices (IHC) Project has identified 36 such concepts and shown that people can be taught to use them in decision making. A randomised trial in Uganda, for example, showed that primary school children with poor reading skills could be taught to apply 12 of the IHC Key Concepts. The list of IHC Key Concepts has proven to be effective in providing a framework for developing and evaluating IHC resources to help children to think critically about treatment claims. The list also provides a framework for retrieving, coding and organising other teaching and learning materials for learners of any age. It should help teachers, researchers, clinicians, and patients to structure critical thinking about the trustworthiness of claims about treatment effects.

Catalogue of bias: allocation bias

This article is part of a series of articles featuring the Catalogue of Bias introduced in this volume of BMJ Evidence-Based Medicine that describes allocation bias and outlines its potential impact on research studies and the preventive steps to minimise its risk. Allocation bias is a type of selection bias and is relevant to clinical trials of interventions. Knowledge of interventions prior to group allocation can result in systematic differences in important characteristics that could influence study findings. Allocation bias can overestimate effect size by up to 30%-40%. Sequentially numbered, opaque, sealed envelopes; containers; pharmacy-controlled randomisation and central computer randomisation are methods to minimise allocation bias.

The Research Cycle: Improving Care and Outcomes for Newborn Infants

Ensuring that policies and practice in perinatal care are informed by evidence from high-quality research is fundamental to improving outcomes for newborn infants and their families. Effective interventions in the perinatal period can have a life-long impact disproportionate to their costs. Many of the major advances in care that have transformed outcomes for preterm and sick newborn infants have been informed by empirical and applied health research. Conversely, there are examples of life-long adverse consequences for infants and families that are a legacy of practices based on poor-quality evidence. The challenge in the 21st century is to maintain the trajectory of improvements in care and outcomes. This will most likely be achieved via marginal gains from new or improved care practices underpinned by a range of research approaches, from preclinical and laboratory-based empirical studies that uncover pathogenic pathways or therapeutic mechanisms, to large-scale, applied research such as multicentre, randomised controlled trials. This will involve the coordination and collaboration of research efforts globally. Strategies to develop and prioritise research questions need to involve parents and families. Given the context in which much perinatal research is conducted, particularly in emergency situations around the time of birth, robust and transparent ethics and governance frameworks are essential to maintain the trust and engagement of communities. An ethical imperative exists to ensure that research output is disseminated effectively, and that effective and cost-effective interventions are implemented and integrated within a cycle that audits and benchmarks good practice and outcomes, and informs research evidence-based continuous quality improvement. This is the first in a series of articles on research methodology in neonatal medicine to be published in Neonatology, in response to a request from trainee researchers. We introduce the series by describing the research cycle, in particular how it is applied in neonatal medicine. Subsequent articles will cover translational research, clinical trials, diagnostic tests, global challenges, and the ethical issues relating to neonatal/perinatal research.

Progesterone in women with arrested premature labor, a report of a randomised clinical trial and updated meta-analysis

BACKGROUND

Progesterone may be effective in prevention of premature birth in some high risk populations. Women with arrested premature labor are at risk of recurrent labor and maintenance therapy with standard tocolytics has not been successful.

METHODS

Randomized double blinded clinical trial of daily treatment with 200 mg vaginal progesterone in women with arrested premature labor and an updated meta-analysis.

RESULTS

The clinical trial was terminated early after 41 women were enrolled. Vaginal progesterone treatment did not change the median gestational age at delivery: 36+2 weeks versus 36+4 weeks, p = .865 nor increase the mean latency to delivery: 44.5 days versus 46.6 days, p = .841. In the updated meta-analysis, progesterone treatment did reduce delivery <37 weeks gestation and increase latency to delivery, but this treatment effect was not evident in the high quality trials: (OR 1.23, 95% CI 0.91, 1.67) and (-0.95 days, 95% CI -5.54, 3.64) respectively.

CONCLUSION

Progesterone is not effective for preventing preterm birth following arrested preterm labor.

Guidelines for reporting meta-epidemiological methodology research

Published research should be reported to evidence users with clarity and transparency that facilitate optimal appraisal and use of evidence and allow replication by other researchers. Guidelines for such reporting are available for several types of studies but not for meta-epidemiological methodology studies. Meta-epidemiological studies adopt a systematic review or meta-analysis approach to examine the impact of certain characteristics of clinical studies on the observed effect and provide empirical evidence for hypothesised associations. The unit of analysis in meta-epidemiological studies is a study, not a patient. The outcomes of meta-epidemiological studies are usually not clinical outcomes. In this guideline, we adapt items from the Preferred Reporting Items for Systematic Reviews and Meta-analyses (PRISMA) to fit the context of meta-epidemiological studies.

Low-molecular-weight heparins or heparinoids versus standard unfractionated heparin for acute ischaemic stroke

BACKGROUND

Low-molecular-weight heparins (LMWHs) and heparinoids are anticoagulants that may have more powerful antithrombotic effects than standard unfractionated heparin (UFH) but a lower risk of bleeding complications. This is an update of the original Cochrane Review of these agents, first published in 2001 and last updated in 2008.

OBJECTIVES

To determine whether antithrombotic therapy with LMWHs or heparinoids is associated with a reduction in the proportion of people who are dead or dependent for activities in daily living compared with UFH.

SEARCH METHODS

We searched the Cochrane Stroke Group Trials Register (last searched February 2017), the Cochrane Central Register of Controlled Trials (CENTRAL: the Cochrane Library Issue 1, 2017), MEDLINE (1966 to February 2017), and Embase (1980 to February 2017). We also searched trials registers to February 2017: ClinicalTrials.gov, EU Clinical Trials Register, Stroke Trials Registry, ISRCTN Registry and the World Health Organization (WHO) International Clinical Trials Registry Platform.

SELECTION CRITERIA

Unconfounded randomised trials comparing LMWH or heparinoids with standard UFH in people with acute ischaemic stroke, in which participants were recruited within 14 days of stroke onset.

DATA COLLECTION AND ANALYSIS

Two review authors independently chose studies for inclusion, assessed risk of bias and trial quality, extracted and analysed the data. Differences were resolved by discussion.

MAIN RESULTS

We included nine trials involving 3137 participants. We did not identify any new trials for inclusion in this updated review. None of the studies reported data on the primary outcome in sufficient detail to enable analysis for the review. Overall, there was a moderate risk of bias in the included studies. Compared with UFH, there was no evidence of an effect of LMWH or heparinoids on death from all causes during the treatment period (96/1616 allocated LMWH/heparinoid versus 78/1486 allocated UFH; odds ratio (OR) 1.06, 95% CI 0.78 to 1.47; 8 trials, 3102 participants, low quality evidence). LMWH or heparinoid were associated with a significant reduction in deep vein thrombosis (DVT) compared with UFH (OR 0.55, 95% CI 0.44 to 0.70, 7 trials, 2585 participants, low quality evidence). However, the number of the major clinical events such as pulmonary embolism (PE) and intracranial haemorrhage was too small to provide a reliable estimate of the effects.

AUTHORS' CONCLUSIONS

Treatment with a LMWH or heparinoid after acute ischaemic stroke appears to decrease the occurrence of DVT compared with standard UFH, but there are too few data to provide reliable information on their effects on other important outcomes, including functional outcome, death and intracranial haemorrhage.

Systematic review of public-targeted communication interventions to improve antibiotic use

Background

Excessive use of antibiotics accelerates the acquisition/spread of antimicrobial resistance. A systematic review was conducted to identify the components of successful communication interventions targeted at the general public to improve antibiotic use.

Methods

The databases MEDLINE, EMBASE, CINAHL, Web of Science and Cochrane Library were searched. Search terms were related to the population (public, community), intervention (campaign, mass media) and outcomes (antibiotic, antimicrobial resistance). References were screened for inclusion by one author with a random subset of 10% screened by a second author. No date restrictions were applied and only articles in the English language were considered. Studies had to have a control group or be an interrupted time-series. Outcomes had to measure change in antibiotic-related prescribing/consumption and/or the public's knowledge, attitudes or behaviour. Two reviewers assessed the quality of studies. Narrative synthesis was performed.

Results

Fourteen studies were included with an estimated 74-75 million participants. Most studies were conducted in the United States or Europe and targeted both the general public and clinicians. Twelve of the studies measured changes in antibiotic prescribing. There was quite strong ( P  < 0·05 to ≥ 0·01) to very strong ( P  < 0·001) evidence that interventions that targeted prescribing for RTIs were associated with decreases in antibiotic prescribing; the majority of these studies reported reductions of greater than -14% with the largest effect size reaching -30%.

Conclusion

Multi-faceted communication interventions that target both the general public and clinicians can reduce antibiotic prescribing in high-income countries but the sustainability of reductions in antibiotic prescribing is unclear.

Time to publication for publicly funded clinical trials in Australia: an observational study

OBJECTIVE

To examine the length of time between receiving funding and publishing the protocol and main paper for randomised controlled trials.

DESIGN

An observational study using survival analysis.

SETTING

Publicly funded health and medical research in Australia.

PARTICIPANTS

Randomised controlled trials funded by the National Health and Medical Research Council of Australia between 2008 and 2010.

MAIN OUTCOME MEASURES

Time from funding to the protocol paper and main results paper. Multiple variable survival models examining whether study characteristics predicted publication times.

RESULTS

We found 77 studies with a total funding of $A59 million. The median time to publication of the protocol paper was 6.4 years after funding (95% CI 4.1 to 8.1). The proportion with a published protocol paper 8 years after funding was 0.61 (95% CI 0.48 to 0.74). The median time to publication of the main results paper was 7.1 years after funding (95% CI 6.3 to 7.6). The proportion with a published main results paper 8 years after funding was 0.72 (95% CI 0.56 to 0.87). The HRs for how study characteristics might influence timing were generally close to one with narrow CIs, the notable exception was that a longer study length lengthened the time to the main paper (HR=0.62 per extra study year, 95% CI 0.43 to 0.89).

CONCLUSIONS

Despite the widespread registration of clinical trials, there remain serious concerns of trial results not being published or being published with a long delay. We have found that these same concerns apply to protocol papers, which should be publishable soon after funding. Funding agencies could set a target of publishing the protocol paper within 18 months of funding.

Methodological Quality of Studies of Endstage Renal Disease in Lupus Nephritis, 1970 to 2015

OBJECTIVE

To examine trends in methodological quality of studies of endstage renal disease risk in lupus nephritis, 1970-2015.

METHODS

We assessed quality using the Newcastle-Ottawa scale for observational studies, and the Cochrane Collaboration's risk-of-bias tool for trials.

RESULTS

In observational studies, description of enrollment criteria was high but decreased over time. Adequacy of followup was low but improved. Inception cohorts and community-based studies were uncommon. Trials had low risk of bias in blinding and selective outcome reporting but most had unclear risks in sequence generation and allocation concealment.

CONCLUSION

Methodological quality was mixed, with limited improvement over time.

Single-incision versus standard multi-incision laparoscopic colectomy in patients with malignant or benign colonic disease: a systematic review, meta-analysis and assessment of the evidence

BACKGROUND

Single-incision laparoscopic colectomy (SILC) requires only one umbilical port site and (depending on technique) a specimen extraction site. The aim of this study was the assessment of the available evidence for the comparison of SILC to conventional multi-port laparoscopic colectomy (MLC) in adult patients, in whom elective colectomy is indicated because of malignant or benign disease. First, previous meta-analyses on this topic were assessed. Secondly, a systematic review and meta-analysis of randomised controlled trials, was performed.

METHODS

Electronic literature searches (CENTRAL, MEDLINE and EMBASE; up to March 2016) were performed. Additionally, we searched clinical trials registries and abstracts from surgical society meetings. For meta-analysis, risk ratios (RR) or mean differences (MD) with 95 % confidence intervals were calculated and pooled. The quality of previous meta-analyses was evaluated against established criteria (AMSTAR) and their reported results were investigated for consistency.

RESULTS

We identified 6 previous meta-analyses of mostly low methodological quality (AMSTAR total score: 2 - 5 out of 11 items). To fill the evidence gaps, all these meta-analyses had included non-randomised studies, but usually without assessing their risk of bias. In our systematic review and meta-analysis of randomised controlled trials exclusively, we included two randomised controlled trials with a total of 82 colorectal cancer patients. There was insufficient evidence to clarify whether SILC leads to less local complications (RR = 0.52, 95 % CI 0.14 - 1.94) or lower mortality (1 death per treatment group). Length of hospital stay was significantly shorter in the SILC group (MD = -1.20 days, 95 % CI -1.95 to -0.44). One of the two studies found postoperative pain intensity to be lower at the first day. We also identified 7 ongoing trials with a total sample size of over 1000 patients.

CONCLUSION

The currently available study results are too sparse to detect (or rule out) relevant differences between SILC and MLC. The quality of the current evidence is low, and the additional analysis of non-randomised data attempts, but does not solve this problem. SILC should still be considered as an experimental procedure, since the evidence of well-designed randomised controlled trials is too sparse to allow any recommendation.

Mechanisms and direction of allocation bias in randomised clinical trials

BACKGROUND

Selective allocation of patients into the compared groups of a randomised trial may cause allocation bias, but the mechanisms behind the bias and its directionality are incompletely understood. We therefore analysed the mechanisms and directionality of allocation bias in randomised clinical trials.

METHODS

Two systematic reviews and a theoretical analysis. We conducted one systematic review of empirical studies of motives/methods for deciphering patient allocation sequences; and another review of methods publications commenting on allocation bias. We theoretically analysed the mechanisms of allocation bias and hypothesised which main factors predicts its direction.

RESULTS

Three empirical studies addressed motives/methods for deciphering allocation sequences. Main motives included ensuring best care for patients and ensuring best outcome for the trial. Main methods included various manipulations with randomisation envelopes. Out of 57 methods publications 11 (19 %) mentioned explicitly that allocation bias can go in either direction. We hypothesised that the direction of allocation bias is mainly decided by the interaction between the patient allocators' motives and treatment preference.

CONCLUSION

Inadequate allocation concealment may exaggerate treatment effects in some trials while underestimate effects in others. Our hypothesis provides a theoretical overview of the main factors responsible for the direction of allocation bias.

Effect of Telehealth Interventions on Hospitalization Indicators: A Systematic Review

BACKGROUND

Telehealth has been defined as the remote delivery of healthcare services using information and communication technology. Where resource-limited health systems face challenges caused by the increasing burden of chronic diseases and an aging global population, telehealth has been advocated as a solution for changing and improving the paradigm of healthcare delivery to cope with these issues. The aim of this systematic review is to investigate the effect of telehealth interventions on two indicators: hospitalization rate and length of stay.

MATERIALS AND METHODS

The reviewers searched the PubMed, ScienceDirect, and Springer electronic databases from January 2005 to November 2013. A search strategy was developed using a combination of the following search keywords: impact, effect, telehealth, telemedicine, telecare, hospitalization, length of stay, and resource utilization. Both randomized controlled trials and observational studies were included in the review. To be included in the review, articles had to be written in English. The results of study were compiled, reviewed, and analyzed on the basis of the review aims.

RESULTS

This systematic review examined 22 existing studies with a total population of 19,086 patients. The effect of telehealth on all-cause hospitalization was statistically significant in 40 percent of the related studies, whereas it was not statistically significant in 60 percent. Similarly, the effect of telehealth on the all-cause length of stay was statistically significant in 36 percent of the studies and nonsignificant in 64 percent.

CONCLUSION

Considering the fact that hospitalization rate and length of stay can be confounded by factors other than telehealth intervention, studies examining the effect of the intervention on these indicators must take into account all other factors influencing them. Otherwise any judgment on the effect of telehealth on these indicators cannot be valid.

Surgical stabilization for open tibial fractures in children: External fixation or elastic stable intramedullary nail - which method is optimal?

BACKGROUND

Management of open tibial fractures is well documented in adults, with existing protocols outlining detailed treatment strategies. No clear guidelines exist for children. Surgical stabilization of tibial fractures in the pediatric population requires implants that do not disrupt the open epiphyses (growth plate). Both elastic stable intramedullary nails and external fixation can be used. The objective of this study was to identify the optimal method of surgical stabilization in the treatment of open tibial fractures in children.

MATERIALS AND METHODS

MEDLINE and Embase were searched from their inception to March 2014 using the following advanced search terms (Key words): "open tibia fracture," "fracture fixation," "external fixation," "intramedullary," and "bone nail." Only studies in English and pertaining to children with open fractures treated with elastic stable intramedullary nails or external fixation between 1994 and 2014 were included. Twelve clinical studies were critically appraised.

RESULTS

Due to a paucity in the literature coupled with a nonsystematic presentation of results, it proved to be very difficult in extracting relevant results from the studies. This was further added by a variation in outcome measures. Consequently, the results we obtained were difficult to draw conclusions from.

CONCLUSION

There is no conclusive evidence or best practice guidelines for their management. Thus, as is highlighted in this study, more research is needed to determine the optimum treatment strategy for this common pediatric injury. The existing literature is of poor quality; consisting mainly of retrospective reviews of patients' medical records, charts, and radiographs. Carefully designed, high-quality prospective cohort studies utilizing a nationalized multi-hospital approach are needed to improve understanding before protocols and guidelines can be developed and implemented.

A protocol for a systematic review of non-randomised evaluations of strategies to improve participant recruitment to randomised controlled trials

BACKGROUND

Randomised controlled trials guard against selection bias and therefore offer the fairest way of evaluating healthcare interventions such as medicinal products, devices and services. Recruitment to trials can be extremely difficult, and poor recruitment can lead to extensions to both time and budget and may result in an underpowered study which does not satisfactorily answer the original research question. In the worst cases, a trial may be abandoned, causing huge waste. The evidence to support the choice of recruitment interventions is currently weak. Non-randomised evaluations of recruitment interventions are currently rejected on grounds of poor methodological quality, but systematic evaluation and assessment of this substantial body of work (using Grading of Recommendations Assessment, Development and Evaluation (GRADE) where possible) may provide useful information to support and inform the recruitment decisions of trialists and the research priorities of methodology researchers.

METHODS

The following databases will be searched for relevant studies: Cochrane Methodology Register, MEDLINE, EMBASE, CINAHL and PsycINFO. Any non-randomised study that includes a comparison of two or more interventions to improve recruitment to randomised controlled trials will be included. We will not apply any restrictions on publication date, language or journal. The primary outcome will be the number of individuals or centres recruited into a randomised controlled trial. The secondary outcome will be cost per recruit. Two reviewers will independently screen abstracts for eligible studies, and then, full texts of potentially relevant records will be reviewed. Disagreements will be resolved through discussion. The methodological quality of studies will be assessed using the Cochrane risk of bias tool for non-randomised studies, and the GRADE system will be used if studies are pooled.

DISCUSSION

This review aims to summarise the evidence on methods used to improve recruitment to randomised controlled trials. Carrying out a systematic review including only data from non-randomised studies is a novel approach, and one which some may argue is futile. However, we believe that the systematic evaluation of what is likely to be a substantial amount of research activity is necessary, worthwhile, and will yield valuable results for the clinical trials community regardless of whether the outcomes find in favour of one or more interventions. Should the results of this review suggest that non-randomised evaluations do have something to offer trialists planning their recruitment strategies, the review may be combined in the future with the Cochrane review of randomised evaluations to produce a full review of recruitment strategies encompassing both randomised and non-randomised evaluation methods.

SYSTEMATIC REVIEW REGISTRATION

PROSPERO CRD42016037718.

Hypnosis for pain management during labour and childbirth

BACKGROUND

This review is one in a series of Cochrane reviews investigating pain management for childbirth. These reviews all contribute to an overview of systematic reviews of pain management for women in labour, and share a generic protocol. This review updates an earlier version of the review of the same title.

OBJECTIVES

To examine the effectiveness and safety of hypnosis for pain management during labour and childbirth.

SEARCH METHODS

We searched the Cochrane Pregnancy and Childbirth Group's Trials Register (30 September 2015) and the reference lists of primary studies and review articles.

SELECTION CRITERIA

Randomised controlled trials (RCTs) and quasi-RCTS comparing preparation for labour using hypnosis and/or use of hypnosis during labour, with or without concurrent use of pharmacological or non-pharmacological pain relief methods versus placebo, no treatment or any analgesic drug or technique.

DATA COLLECTION AND ANALYSIS

Two review authors independently extracted data and assessed trial quality. Where possible we contacted study authors seeking additional information about data and methodology.

MAIN RESULTS

We included nine trials randomising a total of 2954 women. The risk of bias in trials was variable, there were several well-designed large trials and some trials where little was reported about trial design. Although eight of the nine trials assessed antenatal hypnotherapy, there were considerable differences between these trials in timing and technique. One trial provided hypnotherapy during labour. In this updated review we compared hypnosis interventions with all control groups (main comparison) and also with specific control conditions: standard care (nine RCTs), supportive counselling (two RCTs) and relaxation training (two RCTs).In the main comparison, women in the hypnosis group were less likely to use pharmacological pain relief or analgesia than those in the control groups, (average risk ratio (RR) 0.73, 95% CI 0.57 to 0.94, eight studies, 2916 women; very low-quality evidence; random-effects model). There were no clear differences between women in the hypnosis group and those in the control groups for most of the other primary outcomes. There were no clear differences for sense of coping with labour (MD 0.22, 95% CI -0.14 to 0.58, one study, 420 women; low-quality evidence) or spontaneous vaginal birth (average RR 1.12, 95% CI 0.96 to 1.32, six studies, 2361 women; low-quality evidence; random-effects model). There were no clear differences for satisfaction with pain relief (measured on a seven-point scale two weeks postnatally) for women in the hypnosis group who also received pethidine (MD 0.41, 95% CI -0.45 to 1.27; one study, 72 women), Entonox (MD 0.19, 95% CI -0.19 to 0.57; one study, 357 women), self-hypnosis (MD 0.28, 95% CI -0.32 to 0.88; one study, 160 women), or epidural (MD -0.03, 95% CI -0.40 to 0.34; one study, 127 women), but a slight benefit in favour of hypnosis was seen for women who received water immersion (MD 0.52, 95% CI 0.04 to 1.00; one study, 174 women (all low-quality evidence). There were no clear differences for satisfaction with pain relief when it was measured as the number of women who reported they had adequate pain relief (risk ratio (RR) 1.06, 95% confidence interval (CI) 0.94 to 1.20, one study, 264 women; low-quality evidence). It should be noted that for pharmacological pain relief and spontaneous vaginal birth, there was evidence of considerable statistical heterogeneity, which could not be fully explained by subgroup analysis.For this review's secondary outcomes, no clear differences were found between women in the hypnosis group and women in the control groups for most outcomes where data were available. There was mixed evidence regarding benefits for women in the hypnosis group compared with all control groups for pain intensity, satisfaction with childbirth experience and postnatal depression. For each of these outcomes, data from more than one trial were available for analysis but could not be combined due to differences in measurement methods. There was evidence that fewer women in the hypnosis group stayed in hospital for more than two days after the birth but this finding was based on one small study (RR 0.11, 95% CI 0.02 to 0.83). No clear differences between women in the hypnosis group and the control groups were found for the other secondary outcomes where data were available.In the comparisons of hypnosis with specific types of control conditions: standard care, supportive counselling and relaxation training, there were no clear differences found between women in the hypnosis group and those in the standard care control groups or the relaxation control groups for the primary outcomes. Compared with the women in the supportive counselling control group, women in the hypnosis group were less likely to use pharmacological analgesia (average RR 0.48, 95% CI 0.32 to 0.73, two studies, 562 women). They were also more likely to have a spontaneous vaginal birth (RR 2.42, 95% CI 1.43 to 4.07), although this finding was based on the results of one small study. Overall these new comparisons displayed much less statistical heterogeneity than the comparison including all control groups.

AUTHORS' CONCLUSIONS

There are still only a relatively small number of studies assessing the use of hypnosis for labour and childbirth. Hypnosis may reduce the overall use of analgesia during labour, but not epidural use. No clear differences were found between women in the hypnosis group and those in the control groups for satisfaction with pain relief, sense of coping with labour or spontaneous vaginal birth. Not enough evidence currently exists regarding satisfaction with pain relief or sense of coping with labour and we would encourage any future research to prioritise the measurement of these outcomes. The evidence for the main comparison was assessed using GRADE as being of low quality for all the primary outcomes with downgrading decisions due to concerns regarding inconsistency of the evidence, limitations in design and imprecision. Further research is needed in the form of large, well-designed randomised controlled trials to assess whether hypnosis is of value for pain management during labour and childbirth.

Rehabilitation following carpal tunnel release

BACKGROUND

Various rehabilitation treatments may be offered following carpal tunnel syndrome (CTS) surgery. The effectiveness of these interventions remains unclear. This is the first update of a review first published in 2013.

OBJECTIVES

To review the effectiveness and safety of rehabilitation interventions following CTS surgery compared with no treatment, placebo, or another intervention.

SEARCH METHODS

On 29 September 2015, we searched the Cochrane Neuromuscular Specialised Register, the Cochrane Register of Controlled Trials (CENTRAL), MEDLINE, EMBASE, CINAHL Plus, AMED, LILACS, and PsycINFO. We also searched PEDro (3 December 2015) and clinical trials registers (3 December 2015).

SELECTION CRITERIA

Randomised or quasi-randomised clinical trials that compared any postoperative rehabilitation intervention with either no intervention, placebo, or another postoperative rehabilitation intervention in individuals who had undergone CTS surgery.

DATA COLLECTION AND ANALYSIS

Two review authors independently selected trials for inclusion, extracted data, assessed risk of bias, and assessed the quality of the body of evidence for primary outcomes using the GRADE (Grades of Recommendation, Assessment, Development and Evaluation) approach according to standard Cochrane methodology.

MAIN RESULTS

In this review we included 22 trials with a total of 1521 participants. Two of the trials were newly identified at this update. We studied different rehabilitation treatments including immobilisation using a wrist orthosis, dressings, exercise, controlled cold therapy, ice therapy, multi-modal hand rehabilitation, laser therapy, electrical modalities, scar desensitisation, and arnica. Three trials compared a rehabilitation treatment to a placebo, four compared rehabilitation to a no treatment control, three compared rehabilitation to standard care, and 15 compared various rehabilitation treatments to one another.Overall, the included studies were very low in quality. Thirteen trials explicitly reported random sequence generation; of these, five adequately concealed the allocation sequence. Four trials achieved blinding of both participants and outcome assessors. Five were at high risk of bias from incompleteness of outcome data at one or more time intervals, and eight had high risk of selective reporting bias.These trials were heterogeneous in terms of treatments provided, duration of interventions, the nature and timing of outcomes measured, and setting. Therefore, we were not able to pool results across trials.Four trials reported our primary outcome, change in self reported functional ability at three months or more. Of these, three trials provided sufficient outcome data for inclusion in this review. One small high-quality trial studied a desensitisation programme compared with standard treatment and revealed no statistically significant functional benefit based on the Boston Carpal Tunnel Questionnaire (BCTQ) (mean difference (MD) -0.03, 95% confidence interval (CI) -0.39 to 0.33). One low-quality trial assessed participants six months post surgery using the Disabilities of the Arm, Shoulder and Hand (DASH) questionnaire and found no significant difference between a no formal therapy group and a group given a two-week course of multi-modal therapy commenced at five to seven days post surgery (MD 1.00, 95% CI -4.44 to 6.44). One very low-quality quasi-randomised trial found no statistically significant difference in function on the BCTQ at three months post surgery with early immobilisation (plaster wrist orthosis worn until suture removal) compared with a splint and late mobilisation (MD 0.39, 95% CI -0.45 to 1.23).Differences between treatments for secondary outcome measures (change in self reported functional ability measured at less than three months; change in CTS symptoms; change in CTS-related impairment measures; presence of iatrogenic symptoms from surgery; return to work or occupation; and change in neurophysiological parameters) were generally small and not statistically significant. Few studies reported adverse events.

AUTHORS' CONCLUSIONS

There is limited and, in general, low quality evidence for the benefit of the reviewed interventions. People who have undergone CTS surgery should be informed about the limited evidence of effectiveness of postoperative rehabilitation interventions. Until researchers provide results of more high-quality trials that assess the effectiveness and safety of various rehabilitation treatments, the decision to provide rehabilitation following CTS surgery should be based on the clinician's expertise, the patient's preferences and the context of the rehabilitation environment. It is important for researchers to identify patients who respond to a particular treatment and those who do not, and to undertake high-quality studies that evaluate the severity of iatrogenic symptoms from surgery, measure function and return-to-work rates, and control for confounding variables.

Design Characteristics Influence Performance of Clinical Prediction Rules in Validation: A Meta-Epidemiological Study

BACKGROUND

Many new clinical prediction rules are derived and validated. But the design and reporting quality of clinical prediction research has been less than optimal. We aimed to assess whether design characteristics of validation studies were associated with the overestimation of clinical prediction rules' performance. We also aimed to evaluate whether validation studies clearly reported important methodological characteristics.

METHODS

Electronic databases were searched for systematic reviews of clinical prediction rule studies published between 2006 and 2010. Data were extracted from the eligible validation studies included in the systematic reviews. A meta-analytic meta-epidemiological approach was used to assess the influence of design characteristics on predictive performance. From each validation study, it was assessed whether 7 design and 7 reporting characteristics were properly described.

RESULTS

A total of 287 validation studies of clinical prediction rule were collected from 15 systematic reviews (31 meta-analyses). Validation studies using case-control design produced a summary diagnostic odds ratio (DOR) 2.2 times (95% CI: 1.2-4.3) larger than validation studies using cohort design and unclear design. When differential verification was used, the summary DOR was overestimated by twofold (95% CI: 1.2 -3.1) compared to complete, partial and unclear verification. The summary RDOR of validation studies with inadequate sample size was 1.9 (95% CI: 1.2 -3.1) compared to studies with adequate sample size. Study site, reliability, and clinical prediction rule was adequately described in 10.1%, 9.4%, and 7.0% of validation studies respectively.

CONCLUSION

Validation studies with design shortcomings may overestimate the performance of clinical prediction rules. The quality of reporting among studies validating clinical prediction rules needs to be improved.

A 'reluctant' critical review: 'Manual for evidence-based clinical practice (2015)'

BACKGROUND

The Users' Guides to the Medical Literature Manual has been a major influence on the teaching and practice of health care globally.

METHODS

The 3rd edition of the multi-authored Manual was reviewed using the principles outlined in Evidence-based Medicine (EBM) texts. One 'clinical scenario' was selected for critical appraisal, as were several chapters; objectivity was enhanced by citing references to support opinions. RESULTS (SUMMARY OF THE APPRAISAL): (1) Strengths: Clinical pearls, too numerous to list.

EXAMPLES

(i) evidence is never enough to drive clinical decision making; (ii) do not rush to adopt new interventions; and (iii) question efficacy data based only on surrogate markers. (2) Weaknesses: The Manual shares shortcomings of textbooks discussed by Straus et al.: (i) references may not be current, important ones may be excluded and citations may be selective; (ii) often, opinion-based; and (iii) delays between revisions. (3) Notable omissions: Little or no discussion of: (i) important segments of the population: those <18 years of age, >65 years of age and those with multimorbidity; (ii) surgical disciplines; (iii) Greenhalgh et al.'s essay on EBM; (iv) alternate views on the hierarchy of evidence; and (vi) critical thinking. (4) Additional issues: (i) Omission of important references on dabigatran (clinical scenario: chapter 13.1); (ii) authors' advice (Chapter 13.3) to 'bypass the discussion section of published research'; and (iii) the advocacy of pre-appraised sources of evidence and network meta-analysis without warnings about limitations, are critiqued.

CONCLUSION

The Manual has several clinical pearls but readers should also be aware of shortcomings.

Risk of selection bias in randomised trials

BACKGROUND

Selection bias occurs when recruiters selectively enrol patients into the trial based on what the next treatment allocation is likely to be. This can occur even if appropriate allocation concealment is used if recruiters can guess the next treatment assignment with some degree of accuracy. This typically occurs in unblinded trials when restricted randomisation is implemented to force the number of patients in each arm or within each centre to be the same. Several methods to reduce the risk of selection bias have been suggested; however, it is unclear how often these techniques are used in practice.

METHODS

We performed a review of published trials which were not blinded to assess whether they utilised methods for reducing the risk of selection bias. We assessed the following techniques: (a) blinding of recruiters; (b) use of simple randomisation; (c) avoidance of stratification by site when restricted randomisation is used; (d) avoidance of permuted blocks if stratification by site is used; and (e) incorporation of prognostic covariates into the randomisation procedure when restricted randomisation is used. We included parallel group, individually randomised phase III trials published in four general medical journals (BMJ, Journal of the American Medical Association, The Lancet, and New England Journal of Medicine) in 2010.

RESULTS

We identified 152 eligible trials. Most trials (98%) provided no information on whether recruiters were blind to previous treatment allocations. Only 3% of trials used simple randomisation; 63% used some form of restricted randomisation, and 35% did not state the method of randomisation. Overall, 44% of trials were stratified by site of recruitment; 27% were not, and 29% did not report this information. Most trials that did stratify by site of recruitment used permuted blocks (58%), and only 15% reported using random block sizes. Many trials that used restricted randomisation also included prognostic covariates in the randomisation procedure (56%).

CONCLUSIONS

The risk of selection bias could not be ascertained for most trials due to poor reporting. Many trials which did provide details on the randomisation procedure were at risk of selection bias due to a poorly chosen randomisation methods. Techniques to reduce the risk of selection bias should be more widely implemented.

Development of an algorithm to provide awareness in choosing study designs for inclusion in systematic reviews of healthcare interventions: a method study

OBJECTIVES

To develop an algorithm that aims to provide guidance and awareness for choosing multiple study designs in systematic reviews of healthcare interventions.

DESIGN

Method study: (1) To summarise the literature base on the topic. (2) To apply the integration of various study types in systematic reviews. (3) To devise decision points and outline a pragmatic decision tree. (4) To check the plausibility of the algorithm by backtracking its pathways in four systematic reviews.

RESULTS

(1) The results of our systematic review of the published literature have already been published. (2) We recaptured the experience from our four previously conducted systematic reviews that required the integration of various study types. (3) We chose length of follow-up (long, short), frequency of events (rare, frequent) and types of outcome as decision points (death, disease, discomfort, disability, dissatisfaction) and aligned the study design labels according to the Cochrane Handbook. We also considered practical or ethical concerns, and the problem of unavailable high-quality evidence. While applying the algorithm, disease-specific circumstances and aims of interventions should be considered. (4) We confirmed the plausibility of the pathways of the algorithm.

CONCLUSIONS

We propose that the algorithm can assist to bring seminal features of a systematic review with multiple study designs to the attention of anyone who is planning to conduct a systematic review. It aims to increase awareness and we think that it may reduce the time burden on review authors and may contribute to the production of a higher quality review.

Failure of a numerical quality assessment scale to identify potential risk of bias in a systematic review: a comparison study

BACKGROUND

Assessing methodological quality of primary studies is an essential component of systematic reviews. Following a systematic review which used a domain based system [United States Preventative Services Task Force (USPSTF)] to assess methodological quality, a commonly used numerical rating scale (Downs and Black) was also used to evaluate the included studies and comparisons were made between quality ratings assigned using the two different methods. Both tools were used to assess the 20 randomized and quasi-randomized controlled trials examining an exercise intervention for chronic musculoskeletal pain which were included in the review. Inter-rater reliability and levels of agreement were determined using intraclass correlation coefficients (ICC). Influence of quality on pooled effect size was examined by calculating the between group standardized mean difference (SMD).

RESULTS

Inter-rater reliability indicated at least substantial levels of agreement for the USPSTF system (ICC 0.85; 95% CI 0.66, 0.94) and Downs and Black scale (ICC 0.94; 95% CI 0.84, 0.97). Overall level of agreement between tools (ICC 0.80; 95% CI 0.57, 0.92) was also good. However, the USPSTF system identified a number of studies (n = 3/20) as "poor" due to potential risks of bias. Analysis revealed substantially greater pooled effect sizes in these studies (SMD -2.51; 95% CI -4.21, -0.82) compared to those rated as "fair" (SMD -0.45; 95% CI -0.65, -0.25) or "good" (SMD -0.38; 95% CI -0.69, -0.08).

CONCLUSIONS

In this example, use of a numerical rating scale failed to identify studies at increased risk of bias, and could have potentially led to imprecise estimates of treatment effect. Although based on a small number of included studies within an existing systematic review, we found the domain based system provided a more structured framework by which qualitative decisions concerning overall quality could be made, and was useful for detecting potential sources of bias in the available evidence.

Catheter Ablation for Atrial Fibrillation in Heart Failure Patients: A Meta-Analysis of Randomized Controlled Trials

BACKGROUND

Rhythm control with antiarrhythmic drugs (AADs) is not superior to rate control in patients with heart failure (HF) and atrial fibrillation (AF), but AF ablation may be more successful at achieving rhythm control than AADs. However, risks for both ablation and AADs are likely higher and success rates lower in patients with HF.

OBJECTIVE

To compare rate control versus AF catheter ablation strategies in patients with AF and HF.

METHODS

We conducted a meta-analysis of trials which randomized HF patients (LVEF<50%) with AF to a rate control or AF catheter ablation strategy and reported change in LVEF, quality of life, 6-minute walk test, or peak oxygen consumption. Study quality and heterogenity were assessed using Jadad scores and Cochran's Q statistics, respectively. Mantel Haenszel relative risks and mean differences were calculated using random effect models.

RESULTS

Four trials (N=224) met inclusion criteria; 82.5% (n=185) had persistent AF. AF ablation was associated with an increase in LVEF (mean difference 8.5%; 95%CI 6.4,10.7%; P<0.001) compared to rate control. AF ablation was superior in improving quality of life by Minnesota Living with Heart Failure (MLWHF) questionnaire scores (mean difference -11.9; 95%CI -17.1, -6.6; P<0.001). Peak oxygen consumption and 6-minute walk distance increased in AF ablation compared to rate control patients (mean difference 3.2; 95%CI 1.1,5.2; P=0.003; mean difference 34.8; 95%CI 2.9, 66.7; P = 0.03, respectively). In the persistent AF subgroup LVEF and MLWHF were significantly improved with AF ablation. Major adverse event rates (RR 1.3; 95% CI, 0.4, 3.9; p=0.64) were not significantly different. No significant heterogeneity was evident.

CONCLUSIONS

In patients with HF and AF, AF catheter ablation is superior to rate control in improving LVEF, quality of life and functional capacity. Prior to accepting a rate control strategy in HF patients with persistent or drug refractory AF, consideration should be given to AF ablation.

Experiences of recruiting to a pilot trial of Cardiac Rehabilitation In patients with Bowel cancer (CRIB) with an embedded process evaluation: lessons learned to improve recruitment

Background

Recruitment to randomised controlled trials (RCTs) is a perennial problem. Calls have been made for trialists to make recruitment performance publicly available. This article presents our experience of recruiting to a pilot RCT of cardiac rehabilitation for patients with bowel cancer with an embedded process evaluation.

Methods

Recruitment took place at three UK hospitals. Recruitment figures were based on the following: i) estimated number of patient admissions, ii) number of patients likely to meet inclusion criteria from clinician input and iii) recruitment rates in previous studies. The following recruitment procedure was used:

Nurse assessed patients for eligibility.

Patients signed a screening form indicating interest in and agreement to be approached by a researcher about the study.

An appointment was made at which the patient signed a consent form and was randomised to the intervention or control group.

Information about all patients considered for the study and subsequently included or excluded at each stage of the recruitment process and reasons given were recorded.

Results

There were variations in the time taken to award Research Management approval to run the study at the three sites (45–359 days). Sixty-two percent of the original recruitment estimate was reached. The main reason for under-recruitment was due to over-estimation of the number of patient admissions; other reasons were i) not assessing all patients for eligibility, ii) not completing a screening form for eligible patients and iii) patients who signed a screening form being lost to the study before consenting and randomisation.

Conclusions

Pilot trials should not simply aim to improve recruitment estimates but should also identify factors likely to influence recruitment performance in a future trial and inform the development of that trial’s recruitment strategies. Pilot trials are a crucial part of RCT design. Nevertheless, pilot trials are likely to be small scale, involving only a small number of sites, and contextual differences between sites are likely to impact recruitment performance in any future trial. This means that ongoing monitoring and evaluation in trials are likely to be required.

Trial registration

ISRCTN63510637; UKCRN id 14092.

Anticoagulants for acute ischaemic stroke

BACKGROUND

Most ischaemic strokes are caused by a blood clot blocking an artery in the brain. Clot prevention with anticoagulants might improve outcomes if bleeding risks are low. This is an update of a Cochrane review first published in 1995, with recent updates in 2004 and 2008.

OBJECTIVES

To assess the effectiveness and safety of early anticoagulation (within the first 14 days of onset) in people with acute presumed or confirmed ischaemic stroke.

SEARCH METHODS

We searched the Cochrane Stroke Group Trials Register (June 2014), the Cochrane Central Register of Controlled Trials (CENTRAL), the Cochrane Database of Systematic Reviews (CDSR), the Database of Reviews of Effects (DARE) and the Health Technology Assessment Database (HTA) (The Cochrane Library 2014 Issue 6), MEDLINE (2008 to June 2014) and EMBASE (2008 to June 2014). In addition, we searched ongoing trials registries and reference lists of relevant papers. For previous versions of this review, we searched the register of the Antithrombotic Trialists' (ATT) Collaboration, consulted MedStrategy (1995), and contacted relevant drug companies.

SELECTION CRITERIA

Randomised trials comparing early anticoagulant therapy (started within two weeks of stroke onset) with control in people with acute presumed or confirmed ischaemic stroke.

DATA COLLECTION AND ANALYSIS

Two review authors independently selected trials for inclusion, assessed trial quality, and extracted the data.

MAIN RESULTS

We included 24 trials involving 23,748 participants. The quality of the trials varied considerably. The anticoagulants tested were standard unfractionated heparin, low-molecular-weight heparins, heparinoids, oral anticoagulants, and thrombin inhibitors. Over 90% of the evidence relates to the effects of anticoagulant therapy initiated within the first 48 hours of onset. Based on 11 trials (22,776 participants) there was no evidence that anticoagulant therapy started within the first 14 days of stroke onset reduced the odds of death from all causes (odds ratio (OR) 1.05; 95% confidence interval (CI) 0.98 to 1.12) at the end of follow-up. Similarly, based on eight trials (22,125 participants), there was no evidence that early anticoagulation reduced the odds of being dead or dependent at the end of follow-up (OR 0.99; 95% CI 0.93 to 1.04). Although early anticoagulant therapy was associated with fewer recurrent ischaemic strokes (OR 0.76; 95% CI 0.65 to 0.88), it was also associated with an increase in symptomatic intracranial haemorrhages (OR 2.55; 95% CI 1.95 to 3.33). Similarly, early anticoagulation reduced the frequency of symptomatic pulmonary emboli (OR 0.60; 95% CI 0.44 to 0.81), but this benefit was offset by an increase in extracranial haemorrhages (OR 2.99; 95% CI 2.24 to 3.99).

AUTHORS' CONCLUSIONS

Since the last version of the review, no new relevant studies have been published and so there is no additional information to change the conclusions. Early anticoagulant therapy is not associated with net short- or long-term benefit in people with acute ischaemic stroke. Treatment with anticoagulants reduced recurrent stroke, deep vein thrombosis and pulmonary embolism, but increased bleeding risk. The data do not support the routine use of any of the currently available anticoagulants in acute ischaemic stroke.

In search of justification for the unpredictability paradox

A 2011 Cochrane Review found that adequately randomized trials sometimes revealed larger, sometimes smaller, and often similar effect sizes to inadequately randomized trials. However, they found no average statistically significant difference in effect sizes between the two study types. Yet instead of concluding that adequate randomization had no effect the review authors postulated the “unpredictability paradox”, which states that randomized and non-randomized studies differ, but in an unpredictable direction. However, stipulating the unpredictability paradox is problematic for several reasons: 1) it makes the authors’ conclusion that adequate randomization makes a difference unfalsifiable—if it turned out that adequately randomized trials had significantly different average results from inadequately randomized trials the authors could have pooled the results and concluded that adequate randomization protected against bias; 2) it leaves other authors of reviews with similar results confused about whether or not to pool results (and hence which conclusions to draw); 3) it discourages researchers from investigating the conditions under which adequate randomization over- or under-exaggerates apparent treatment benefits; and 4) it could obscure the relative importance of allocation concealment and blinding which may be more important than adequate randomization.

Evaluating causal relationships between urban built environment characteristics and obesity: a methodological review of observational studies

Background

Existing reviews identify numerous studies of the relationship between urban built environment characteristics and obesity. These reviews do not generally distinguish between cross-sectional observational studies using single equation analytical techniques and other studies that may support more robust causal inferences. More advanced analytical techniques, including the use of instrumental variables and regression discontinuity designs, can help mitigate biases that arise from differences in observable and unobservable characteristics between intervention and control groups, and may represent a realistic alternative to scarcely-used randomised experiments. This review sought first to identify, and second to compare the results of analyses from, studies using more advanced analytical techniques or study designs.

Methods

In March 2013, studies of the relationship between urban built environment characteristics and obesity were identified that incorporated (i) more advanced analytical techniques specified in recent UK Medical Research Council guidance on evaluating natural experiments, or (ii) other relevant methodological approaches including randomised experiments, structural equation modelling or fixed effects panel data analysis.

Results

Two randomised experimental studies and twelve observational studies were identified. Within-study comparisons of results, where authors had undertaken at least two analyses using different techniques, indicated that effect sizes were often critically affected by the method employed, and did not support the commonly held view that cross-sectional, single equation analyses systematically overestimate the strength of association.

Conclusions

Overall, the use of more advanced methods of analysis does not appear necessarily to undermine the observed strength of association between urban built environment characteristics and obesity when compared to more commonly-used cross-sectional, single equation analyses. Given observed differences in the results of studies using different techniques, further consideration should be given to how evidence gathered from studies using different analytical approaches is appraised, compared and aggregated in evidence synthesis.

Recruitment difficulties in obstetric trials: a case study and review

BACKGROUND

The CONSORT statement calls for complete data on flow of participants, including all losses and exclusions. Incomplete reporting of flow into trials versus flow through trials is not uncommon. Where complete data exist in obstetric trials, poor recruitment seems a recurring theme.

AIMS

To explore difficulties in recruitment and differences between assessed-but-not-recruited and included women to improve future trial participation, using a case study of a recently published randomised trial of outpatient Foley catheter versus inpatient PGE2 gel for cervical ripening.

MATERIALS & METHODS

The assessed-but-not-recruited population of an obstetric trial (ACTRN:12609000420246) was prospectively studied for reasons for noninclusion, demographic data and pregnancy outcome. Women assessed-but-not-recruited due to declined consent or obstetrician declined participation were compared to included women. Main outcome measures included demographic and outcome differences associated with trial participation.

RESULTS

Of 468 assessed participants, 220 (47%) were not eligible by exclusion criteria (potential 'trial factor' recruitment difficulties), 147 (31%) declined consent (n = 100, 'participant factor') or their obstetrician declined participation (n = 47, 'clinician factor') and 101 (22%) were included. Declining women were more likely than participants to be parous (24 vs 10%, P < 0.05), induced for nonmedical reasons (18 vs 4%, P < 0.001), privately admitted (31 vs 3%, P < 0.001) and have longer inpatient stay (4.9 vs 4.2 days, P < 0.05).

CONCLUSION

The high assessed-but-not-recruited rate highlights important issues with external validity and feasibility when conducting obstetric trials, including recruitment difficulties related to participant, clinician and trial factors. Assessed: recruited ratios and demographic and outcome differences need consideration in planning and interpretation of randomised trials.

Accuracy of the Berger-Exner test for detecting third-order selection bias in randomised controlled trials: a simulation-based investigation

BACKGROUND

Randomised controlled trials (RCT) are highly influential upon medical decisions. Thus RCTs must not distort the truth. One threat to internal trial validity is the correct prediction of future allocations (selection bias). The Berger-Exner test detects such bias but has not been widely utilized in practice. One reason for this non-utilisation may be a lack of information regarding its test accuracy. The objective of this study is to assess the accuracy of the Berger-Exner test on the basis of relevant simulations for RCTs with dichotomous outcomes.

METHODS

Simulated RCTs with various parameter settings were generated, using R software, and subjected to bias-free and selection bias scenarios. The effect size inflation due to bias was quantified. The test was applied in both scenarios and the pooled sensitivity and specificity, with 95% confidence intervals for alpha levels of 1%, 5%, and 20%, were computed. Summary ROC curves were generated and the relationships of parameters with test accuracy were explored.

RESULTS

An effect size inflation of 71% - 99% was established. Test sensitivity was 1.00 (95% CI: 0.99 - 1.00) for alpha level 1%, 5%, and 20%; test specificity was 0.94 (95% CI: 0.93 - 0.96); 0.82 (95% CI: 0.80 - 0.84), and 0.56 (95% CI: 0.54 - 0.58) for alpha 1%, 5%, and 20%, respectively. Test accuracy was best with the maximal procedure used with a maximum tolerated imbalance (MTI) = 2 as the randomisation method at alpha 1%.

CONCLUSIONS

The results of this simulation study suggest that the Berger-Exner test is generally accurate for identifying third-order selection bias.

A systematic review of the cost-effectiveness of nurse practitioners and clinical nurse specialists: what is the quality of the evidence?

Background. Improved quality of care and control of healthcare costs are important factors influencing decisions to implement nurse practitioner (NP) and clinical nurse specialist (CNS) roles. Objective. To assess the quality of randomized controlled trials (RCTs) evaluating NP and CNS cost-effectiveness (defined broadly to also include studies measuring health resource utilization). Design. Systematic review of RCTs of NP and CNS cost-effectiveness reported between 1980 and July 2012. Results. 4,397 unique records were reviewed. We included 43 RCTs in six groupings, NP-outpatient (n = 11), NP-transition (n = 5), NP-inpatient (n = 2), CNS-outpatient (n = 11), CNS-transition (n = 13), and CNS-inpatient (n = 1). Internal validity was assessed using the Cochrane risk of bias tool; 18 (42%) studies were at low, 17 (39%) were at moderate, and eight (19%) at high risk of bias. Few studies included detailed descriptions of the education, experience, or role of the NPs or CNSs, affecting external validity. Conclusions. We identified 43 RCTs evaluating the cost-effectiveness of NPs and CNSs using criteria that meet current definitions of the roles. Almost half the RCTs were at low risk of bias. Incomplete reporting of study methods and lack of details about NP or CNS education, experience, and role create challenges in consolidating the evidence of the cost-effectiveness of these roles.

The need for randomization in animal trials: an overview of systematic reviews

Background and Objectives

Randomization, allocation concealment, and blind outcome assessment have been shown to reduce bias in human studies. Authors from the Collaborative Approach to Meta Analysis and Review of Animal Data from Experimental Studies (CAMARADES) collaboration recently found that these features protect against bias in animal stroke studies. We extended the scope the work from CAMARADES to include investigations of treatments for any condition.

Methods

We conducted an overview of systematic reviews. We searched Medline and Embase for systematic reviews of animal studies testing any intervention (against any control) and we included any disease area and outcome. We included reviews comparing randomized versus not randomized (but otherwise controlled), concealed versus unconcealed treatment allocation, or blinded versus unblinded outcome assessment.

Results

Thirty-one systematic reviews met our inclusion criteria: 20 investigated treatments for experimental stroke, 4 reviews investigated treatments for spinal cord diseases, while 1 review each investigated treatments for bone cancer, intracerebral hemorrhage, glioma, multiple sclerosis, Parkinson's disease, and treatments used in emergency medicine. In our sample 29% of studies reported randomization, 15% of studies reported allocation concealment, and 35% of studies reported blinded outcome assessment. We pooled the results in a meta-analysis, and in our primary analysis found that failure to randomize significantly increased effect sizes, whereas allocation concealment and blinding did not. In our secondary analyses we found that randomization, allocation concealment, and blinding reduced effect sizes, especially where outcomes were subjective.

Conclusions

Our study demonstrates the need for randomization, allocation concealment, and blind outcome assessment in animal research across a wide range of outcomes and disease areas. Since human studies are often justified based on results from animal studies, our results suggest that unduly biased animal studies should not be allowed to constitute part of the rationale for human trials.

Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials

BACKGROUND

Researchers and organizations often use evidence from randomized controlled trials (RCTs) to determine the efficacy of a treatment or intervention under ideal conditions. Studies of observational designs are often used to measure the effectiveness of an intervention in 'real world' scenarios. Numerous study designs and modifications of existing designs, including both randomized and observational, are used for comparative effectiveness research in an attempt to give an unbiased estimate of whether one treatment is more effective or safer than another for a particular population.A systematic analysis of study design features, risk of bias, parameter interpretation, and effect size for all types of randomized and non-experimental observational studies is needed to identify specific differences in design types and potential biases. This review summarizes the results of methodological reviews that compare the outcomes of observational studies with randomized trials addressing the same question, as well as methodological reviews that compare the outcomes of different types of observational studies.

OBJECTIVES

To assess the impact of study design (including RCTs versus observational study designs) on the effect measures estimated.To explore methodological variables that might explain any differences identified.To identify gaps in the existing research comparing study designs.

SEARCH METHODS

We searched seven electronic databases, from January 1990 to December 2013.Along with MeSH terms and relevant keywords, we used the sensitivity-specificity balanced version of a validated strategy to identify reviews in PubMed, augmented with one term ("review" in article titles) so that it better targeted narrative reviews. No language restrictions were applied.

SELECTION CRITERIA

We examined systematic reviews that were designed as methodological reviews to compare quantitative effect size estimates measuring efficacy or effectiveness of interventions tested in trials with those tested in observational studies. Comparisons included RCTs versus observational studies (including retrospective cohorts, prospective cohorts, case-control designs, and cross-sectional designs). Reviews were not eligible if they compared randomized trials with other studies that had used some form of concurrent allocation.

DATA COLLECTION AND ANALYSIS

In general, outcome measures included relative risks or rate ratios (RR), odds ratios (OR), hazard ratios (HR). Using results from observational studies as the reference group, we examined the published estimates to see whether there was a relative larger or smaller effect in the ratio of odds ratios (ROR).Within each identified review, if an estimate comparing results from observational studies with RCTs was not provided, we pooled the estimates for observational studies and RCTs. Then, we estimated the ratio of ratios (risk ratio or odds ratio) for each identified review using observational studies as the reference category. Across all reviews, we synthesized these ratios to get a pooled ROR comparing results from RCTs with results from observational studies.

MAIN RESULTS

Our initial search yielded 4406 unique references. Fifteen reviews met our inclusion criteria; 14 of which were included in the quantitative analysis.The included reviews analyzed data from 1583 meta-analyses that covered 228 different medical conditions. The mean number of included studies per paper was 178 (range 19 to 530).Eleven (73%) reviews had low risk of bias for explicit criteria for study selection, nine (60%) were low risk of bias for investigators' agreement for study selection, five (33%) included a complete sample of studies, seven (47%) assessed the risk of bias of their included studies,Seven (47%) reviews controlled for methodological differences between studies,Eight (53%) reviews controlled for heterogeneity among studies, nine (60%) analyzed similar outcome measures, and four (27%) were judged to be at low risk of reporting bias.Our primary quantitative analysis, including 14 reviews, showed that the pooled ROR comparing effects from RCTs with effects from observational studies was 1.08 (95% confidence interval (CI) 0.96 to 1.22). Of 14 reviews included in this analysis, 11 (79%) found no significant difference between observational studies and RCTs. One review suggested observational studies had larger effects of interest, and two reviews suggested observational studies had smaller effects of interest.Similar to the effect across all included reviews, effects from reviews comparing RCTs with cohort studies had a pooled ROR of 1.04 (95% CI 0.89 to 1.21), with substantial heterogeneity (I(2) = 68%). Three reviews compared effects of RCTs and case-control designs (pooled ROR: 1.11 (95% CI 0.91 to 1.35)).No significant difference in point estimates across heterogeneity, pharmacological intervention, or propensity score adjustment subgroups were noted. No reviews had compared RCTs with observational studies that used two of the most common causal inference methods, instrumental variables and marginal structural models.

AUTHORS' CONCLUSIONS

Our results across all reviews (pooled ROR 1.08) are very similar to results reported by similarly conducted reviews. As such, we have reached similar conclusions; on average, there is little evidence for significant effect estimate differences between observational studies and RCTs, regardless of specific observational study design, heterogeneity, or inclusion of studies of pharmacological interventions. Factors other than study design per se need to be considered when exploring reasons for a lack of agreement between results of RCTs and observational studies. Our results underscore that it is important for review authors to consider not only study design, but the level of heterogeneity in meta-analyses of RCTs or observational studies. A better understanding of how these factors influence study effects might yield estimates reflective of true effectiveness.

Doctors record higher blood pressures than nurses: systematic review and meta-analysis

BACKGROUND

The magnitude of the 'white coat effect', the alerting rise in blood pressure, is greater for doctors than nurses. This could bias interpretation of studies on nurse-led care in hypertension, and risks overestimating or overtreating high blood pressure by doctors in clinical practice.

AIM

To quantify differences between blood pressure measurements made by doctors and nurses.

DESIGN AND SETTING

Systematic review and meta-analysis using searches of MEDLINE, CENTRAL, CINAHL, Embase, journal collections, and conference abstracts.

METHOD

Studies in adults reporting mean blood pressures measured by doctors and nurses at the same visit were selected, and mean blood pressures extracted, by two reviewers. Study risk of bias was assessed using modified Cochrane criteria. Outcomes were pooled across studies using random effects meta-analysis.

RESULTS

In total, 15 studies (11 hypertensive; four mixed hypertensive and normotensive populations) were included from 1899 unique citations. Compared with doctors' measurements, nurse-measured blood pressures were lower (weighted mean differences: systolic -7.0 [95% confidence interval {CI} = -4.7 to -9.2] mmHg, diastolic -3.8 [95% CI = -2.2 to -5.4] mmHg). For studies at low risk of bias, differences were lower: systolic -4.6 (95% CI = -1.9 to -7.3) mmHg; diastolic -1.7 (95% CI = -0.1 to -3.2) mmHg. White coat hypertension was diagnosed more frequently based on doctors' than on nurses' readings: relative risk 1.6 (95% CI =1.2 to 2.1).

CONCLUSIONS

The white coat effect is smaller for blood pressure measurements made by nurses than by doctors. This systematic difference has implications for hypertension diagnosis and management. Caution is required in pooling data from studies using both nurse- and doctor-measured blood pressures.

Oral antiplatelet therapy for acute ischaemic stroke

BACKGROUND

In people with acute ischaemic stroke, platelets become activated and can cause blood clots to form and block an artery in the brain, resulting in damage to part of the brain. Such damage gives rise to the symptoms of stroke. Antiplatelet therapy might reduce the volume of brain damaged by ischaemia and also reduce the risk of early recurrent ischaemic stroke, thereby reducing the risk of early death and improving long-term outcomes in survivors. However, antiplatelet therapy might also increase the risk of fatal or disabling intracranial haemorrhage.

OBJECTIVES

To assess the efficacy and safety of immediate oral antiplatelet therapy (that is started as soon as possible and no later than two weeks after stroke onset) in people with acute presumed ischaemic stroke.

SEARCH METHODS

We searched the Cochrane Stroke Group Trials Register (last searched 16 October 2013), the Cochrane Central Register of Controlled Trials (CENTRAL) (The Cochrane Library Issue 4, 2013), MEDLINE (June 1998 to May 2013), and EMBASE (June 1998 to May 2013). In 1998, for a previous version of this review, we searched the register of the Antiplatelet Trialists' Collaboration, MedStrategy and contacted relevant drug companies.

SELECTION CRITERIA

Randomised trials comparing oral antiplatelet therapy (started within 14 days of the stroke) with control in people with definite or presumed ischaemic stroke.

DATA COLLECTION AND ANALYSIS

Two review authors independently applied the inclusion criteria and assessed trial quality. For the included trials, they extracted and cross-checked the data.

MAIN RESULTS

We included eight trials involving 41,483 participants. No new trials have been added since the last update.Two trials testing aspirin 160 mg to 300 mg once daily, started within 48 hours of onset, contributed 98% of the data. The risk of bias was low. The maximum follow-up was six months. With treatment, there was a significant decrease in death or dependency at the end of follow-up (odds ratio (OR) 0.95, 95% confidence interval (CI) 0.91 to 0.99). For every 1000 people treated with aspirin, 13 people would avoid death or dependency (number needed to treat 79). Antiplatelet therapy was associated with a small but definite excess of symptomatic intracranial haemorrhages, but this small hazard was significantly outnumbered by the benefit, the reduction in recurrent ischaemic stroke and pulmonary embolus.

AUTHORS' CONCLUSIONS

Antiplatelet therapy with aspirin 160 mg to 300 mg daily, given orally (or by nasogastric tube or per rectum in people who cannot swallow) and started within 48 hours of onset of presumed ischaemic stroke, reduced the risk of early recurrent ischaemic stroke without a major risk of early haemorrhagic complications; long-term outcomes were improved.

Using multiple types of studies in systematic reviews of health care interventions--a systematic review

BACKGROUND

A systematic review may evaluate different aspects of a health care intervention. To accommodate the evaluation of various research questions, the inclusion of more than one study design may be necessary. One aim of this study is to find and describe articles on methodological issues concerning the incorporation of multiple types of study designs in systematic reviews on health care interventions. Another aim is to evaluate methods studies that have assessed whether reported effects differ by study types.

METHODS AND FINDINGS

We searched PubMed, the Cochrane Database of Systematic Reviews, and the Cochrane Methodology Register on 31 March 2012 and identified 42 articles that reported on the integration of single or multiple study designs in systematic reviews. We summarized the contents of the articles qualitatively and assessed theoretical and empirical evidence. We found that many examples of reviews incorporating multiple types of studies exist and that every study design can serve a specific purpose. The clinical questions of a systematic review determine the types of design that are necessary or sufficient to provide the best possible answers. In a second independent search, we identified 49 studies, 31 systematic reviews and 18 trials that compared the effect sizes between randomized and nonrandomized controlled trials, which were statistically different in 35%, and not different in 53%. Twelve percent of studies reported both, different and non-different effect sizes.

CONCLUSIONS

Different study designs addressing the same question yielded varying results, with differences in about half of all examples. The risk of presenting uncertain results without knowing for sure the direction and magnitude of the effect holds true for both nonrandomized and randomized controlled trials. The integration of multiple study designs in systematic reviews is required if patients should be informed on the many facets of patient relevant issues of health care interventions.

Meeting the challenges of recruitment to multicentre, community-based, lifestyle-change trials: a case study of the BeWEL trial

Background

Recruiting participants to multicentre, community-based trials is a challenge. This case study describes how this challenge was met for the BeWEL trial, which evaluated the impact of a diet and physical activity intervention on body weight in people who had had pre-cancerous bowel polyps.

Methods

The BeWEL trial was a community-based trial, involving centres linked to the Scottish National Health Service (NHS) colorectal cancer screening programme. BeWEL had a recruitment target of 316 and its primary recruitment route was the colonoscopy clinics of the Scottish Bowel Screening Programme.

Results

BeWEL exceeded its recruitment target but needed a 6-month no-cost extension from the funder to achieve this. The major causes of delay were lower consent rates (49% as opposed to 70% estimated from earlier work), the time taken for NHS research and development department approvals and the inclusion of two additional sites to increase recruitment, for which there were substantial bureaucratic delays. A range of specific interventions to increase recruitment, for example, telephone reminders and a shorter participant information leaflet, helped to increase the proportion of eligible individuals consenting and being randomized.

Conclusions

Recruitment to multicentre trials is a challenge but can be successfully achieved with a committed team. In a UK context, NHS research and development approval can be a substantial source of delay. Investigators should be cautious when estimating consent rates. If consent rates are less than expected, qualitative analysis might be beneficial, to try and identify the reason. Finally, investigators should select trial sites on the basis of a formal assessment of a site’s past performance and the likelihood of success in the trial being planned.

Trial registration

Current Controlled Trials ISRCTN53033856

Valproic acid, valproate and divalproex in the maintenance treatment of bipolar disorder

BACKGROUND

Bipolar disorder is a recurrent illness that is amongst the top 30 causes of disability worldwide and is associated with significant healthcare costs. In the past, emphasis was placed solely on the treatment of acute episodes of bipolar disorder; recently, the importance of episode prevention and of minimisation of iatrogenicity has been recognised. For many years, lithium was the only mood stabiliser in common use, and it remains an agent of first choice in the preventative treatment of bipolar disorder. However, an estimated 20% to 40% of patients may not respond adequately to lithium. Valproate is an anticonvulsant drug that has been shown to be effective in acute mania and is frequently used in maintenance treatment of bipolar disorder. When the acceptability of long-term treatment is considered, together with efficacy, the adverse event profile of a medication is also important. This is an update of a Cochrane review first published in 2001 and last updated in 2009.

OBJECTIVES

1. To determine the efficacy of valproate continuation and maintenance treatment:a) in preventing or attenuating manic, depressive and mixed episodes of bipolar disorder;b) in preventing or attenuating episodes of bipolar disorder in patients with rapid cycling disorder; and; c) in improving patients' general health and social functioning, as measured by global clinical impression, employment and marital stability.2. To review the acceptability to patients of long-term valproate treatment, as measured by numbers of dropouts and reasons for dropping out, by compliance and by reference to patients' expressed views regarding treatment.3. To investigate the adverse effects of valproate treatment (including general prevalence of side effects) and overall mortality rates.

SEARCH METHODS

Search of the Cochrane Register of Controlled Trials and the Cochrane Depression, Anxiety and Neurosis Group Register (CCDANCTR) (to January 2013), which includes relevant randomised controlled trials from the following bibliographic databases: The Cochrane Library (all years), EMBASE, (1974 to date), MEDLINE (1950 to date) and PsycINFO (1967 to date). No language restrictions were applied. Reference lists of relevant papers and previous systematic reviews were handsearched. Pharmaceutical companies marketing valproate and experts in this field were contacted for supplemental data.

SELECTION CRITERIA

Randomised controlled trials allocating participants with bipolar disorder to long-term treatment with valproate or any other mood stabiliser, or antipsychotic drugs, or placebo. Maintenance treatment was defined as treatment instituted specifically or mainly to prevent further episodes of illness.

DATA COLLECTION AND ANALYSIS

Three review authors independently extracted data. A double-entry procedure was employed by two review authors. Information extracted included study characteristics, participant characteristics, intervention details and outcome measures in terms of efficacy, acceptability and tolerability. For dichotomous data, risk ratios were calculated with 95% confidence intervals (CIs). For statistically significant results, we calculated the number needed to treat for an additional beneficial outcome (NNTB) and the number needed to treat for an additional harmful outcome (NNTH). For continuous data, mean differences (MDs) or standardised mean differences (SMDs) were calculated along with 95% CIs. MDs were used when the same scale was used to measure an outcome; SMDs were employed when different scales were used to measure the same outcome. The primary analysis used a fixed-effect model. Binary outcomes were calculated on a strict intention-to-treat (ITT) basis; dropouts were included in this analysis. When data were missing and the method of "last observation carried forward" (LOCF) had been used to do an ITT analysis, then the LOCF data were used.

MAIN RESULTS

Six randomised controlled trials (overall 876 participants) lasting 6 to 24 months were included. Two studies (overall 312 participants) compared valproate with placebo, four studies (overall 618 participants) valproate with lithium, one study (overall 23 participants) valproate with olanzapine and one study (overall 220 participants) valproate with the combination of valproate plus lithium. In terms of study quality, most studies reported the methods used to generate random sequence; however, only one study reported enough details on allocation concealment. Four of six included studies described their design as "double blind", but only two trials reported full details about blinding. Valproate was more effective than placebo in preventing study withdrawal due to any mood episode (RR 0.68, 95% CI 0.49 to 0.93; NNTB 8), but no difference in efficacy was found between valproate and lithium (RR 1.02, 95% CI 0.87 to 1.20). Valproate was associated with fewer participants dropping out of treatment for any cause when compared with placebo or lithium (RR 0.82, 95% CI 0.71 to 0.95 and RR 0.87, 95% CI 0.77 to 0.98, respectively). However, combination therapy with lithium plus valproate was more likely to prevent relapse than was monotherapy with valproate (RR 0.78, 95% CI 0.63 to 0.96). Significant differences in adverse event frequencies were found, and lithium was associated with more frequent diarrhoea, polyuria, increased thirst and enuresis, whereas valproate was associated with increased sedation and infection.

AUTHORS' CONCLUSIONS

Limited evidence supports the efficacy of valproate in the long-term treatment of bipolar disorder. Clinicians and patients should consider acceptability and tolerability profile when choosing between lithium and valproate-their combination or other agents-as long-term treatment for bipolar disorder.

Moxibustion in the management of irritable bowel syndrome: systematic review and meta-analysis

BACKGROUND

Irritable bowel syndrome (IBS) is a common functional gastrointestinal disorder. Many patients suffer from IBS that can be difficult to treat, thus complementary therapies which may be effective and have a lower likelihood of adverse effects are being sought.This systematic review and meta-analysis aimed at critically evaluating the current evidence on moxibustion for improving global symptoms of IBS.

METHODS

We searched Medline, EMBASE, the Cochrane Central Register of Controlled Trials, AMED, CINAHL, and CNKI databases for randomised controlled trials (RCTs) of moxibustion comparing with sham moxibustion, pharmacological medications, and other active treatments in patients with IBS. Trials should report global symptom improvement as an outcome measure. Risk of bias for each RCT was assessed according to criteria by the Cochrane Collaboration, and the dichotomous data were pooled according to the control intervention to obtain a risk ratio (RR) of global symptom improvement after moxibustion, with 95% confidence intervals (CI).

RESULTS

A total of 20 RCTs were eligible for inclusion (n = 1625). The risk of bias was generally high. Compared with pharmacological medications, moxibustion significantly alleviated overall IBS symptoms but there was a moderate inconsistency among studies (7 RCTs, RR 1.33, 95% CI [1.15, 1.55], I² = 46%). Moxibustion combined with acupuncture was more effective than pharmacological therapy but a moderate inconsistency among studies was found (4 RCTs, RR 1.24, 95% CI [1.09, 1.41], I² = 36%). When moxibustion was added to pharmacological medications or herbal medicine, no additive benefit of moxibustion was shown compared with pharmacological medications or herbal medicine alone. One small sham-controlled trial found no difference between moxibustion and sham control in symptom severity (mean difference 0.35, 95% CI [-0.77, 1.47]). Moxibustion appears to be associated with few adverse events but the evidence is limited due to poor reporting.

CONCLUSIONS

This systematic review and meta-analysis suggests that moxibustion may provide benefit to IBS patients although the risk of bias in the included studies is relatively high. Future studies are necessary to confirm whether this finding is reproducible in carefully-designed and conducted trials and to firmly establish the place of moxibustion in current practice.

Inspiratory muscle training for asthma

BACKGROUND

In some people with asthma, expiratory airflow limitation, premature closure of small airways, activity of inspiratory muscles at the end of expiration and reduced pulmonary compliance may lead to lung hyperinflation. With the increase in lung volume, chest wall geometry is modified, shortening the inspiratory muscles and leaving them at a sub-optimal position in their length-tension relationship. Thus, the capacity of these muscles to generate tension is reduced. An increase in cross-sectional area of the inspiratory muscles caused by hypertrophy could offset the functional weakening induced by hyperinflation. Previous studies have shown that inspiratory muscle training promotes diaphragm hypertrophy in healthy people and patients with chronic heart failure, and increases the proportion of type I fibres and the size of type II fibres of the external intercostal muscles in patients with chronic obstructive pulmonary disease. However, its effects on clinical outcomes in patients with asthma are unclear.

OBJECTIVES

To evaluate the efficacy of inspiratory muscle training with either an external resistive device or threshold loading in people with asthma.

SEARCH METHODS

We searched the Cochrane Airways Group Specialised Register of trials, Cochrane Central Register of Controlled Trials (CENTRAL), ClinicalTrials.gov and reference lists of included studies. The latest search was performed in November 2012.

SELECTION CRITERIA

We included randomised controlled trials that involved the use of an external inspiratory muscle training device versus a control (sham or no inspiratory training device) in people with stable asthma.

DATA COLLECTION AND ANALYSIS

We used standard methodological procedures expected by The Cochrane Collaboration.

MAIN RESULTS

We included five studies involving 113 adults. Participants in four studies had mild to moderate asthma and the fifth study included participants independent of their asthma severity. There were substantial differences between the studies, including the training protocol, duration of training sessions (10 to 30 minutes) and duration of the intervention (3 to 25 weeks). Three clinical trials were produced by the same research group. Risk of bias in the included studies was difficult to ascertain accurately due to poor reporting of methods.The included studies showed a statistically significant increase in inspiratory muscle strength, measured by maximal inspiratory pressure (PImax) (mean difference (MD) 13.34 cmH2O, 95% CI 4.70 to 21.98, 4 studies, 84 participants, low quality evidence). Our other primary outcome, exacerbations requiring a course of oral or inhaled corticosteroids or emergency department visits, was not reported. For the secondary outcomes, results from one trial showed no statistically significant difference between the inspiratory muscle training group and the control group for maximal expiratory pressure, peak expiratory flow rate, forced expiratory volume in one second, forced vital capacity, sensation of dyspnoea and use of beta2-agonist. There were no studies describing inspiratory muscle endurance, hospital admissions or days off work or school.

AUTHORS' CONCLUSIONS

There is no conclusive evidence in this review to support or refute inspiratory muscle training for asthma. The evidence was limited by the small number of trials with few participants together with the risk of bias. More well conducted randomised controlled trials are needed. Future trials should investigate the following outcomes: lung function, exacerbation rate, asthma symptoms, hospital admissions, use of medications and days off work or school. Inspiratory muscle training should also be assessed in people with more severe asthma and conducted in children with asthma.