The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed

A Systematic Review and Appraisal of Clinical Practice Guidelines for Musculoskeletal Soft Tissue Injuries and Conditions

BACKGROUND

Clinical practice guidelines (CPGs) are published by several sports medicine institutions. A systematic evaluation can help identify the highest quality CPGs for clinical use and identify any deficiencies that remain.

PURPOSE

To identify and appraise CPGs relevant to clinical sports medicine professionals.

STUDY DESIGN

Systematic review.

METHODS

Predetermined selection criteria were utilized by 2 reviewers who independently identified published CPGs before January 1, 2014. CPGs were excluded if they focused on injured workers, radiological criteria, medical pathology, or the axial skeleton (back/neck). The remaining guidelines were scored by 6 reviewers with different clinical backgrounds using the Appraisal of Guidelines for Research and Evaluation II (AGREE II). Scores lower than 50% indicated deficiency. Scores were also stratified by the publishing institution and anatomic location and compared using Kruskal-Wallis tests. The Spearman correlation coefficient was used to assess the range of interobserver agreement between the evaluators.

RESULTS

Seventeen CPGs met the inclusion criteria. The majority of guidelines pertained to the knee, ankle, or shoulder. Interobserver agreement was strong ( r = 0.548-0.740), and mean total scores between nonsurgical (107.8) and surgical evaluators (109.3) were not statistically different. Overall guideline quality was variable but not deficient for 16 of 17 guidelines (>50%), except regarding clinical "applicability" and "editorial independence." No difference was found between CPGs of the knee, shoulder, foot/ankle, or chronic conditions. However, CPG publishing institutions had significantly different scores; the American Academy of Orthopaedic Surgeons (AAOS) guidelines scored significantly higher (141.4) than the total mean score (108.0).

CONCLUSION

The overall quality of sports medicine CPGs was variable but generally not deficient, except regarding applicability and editorial independence. Bias through poor editorial independence is a concern. To improve future guideline quality, authors should pay particular attention to these areas and use existing highest quality guidelines, or the AGREE II instrument, as templates. CPGs dedicated to anatomic areas other than the knee, ankle, and shoulder are needed.

Hematology journals do not sufficiently adhere to reporting guidelines: a systematic review

Essentials Reporting guidelines and trial/review registration aim to limit bias in research. We systematically reviewed hematology journals to examine the use of these policies. Forty-eight percent of journals made no use of these policies. Improving the use of reporting guidelines will improve research for all stakeholders.

SUMMARY

Background Reporting guidelines and trial/review registration policies have been instituted in order to minimize bias and improve research practices. Objective The objective of this study was to investigate the policies of hematology journals concerning reporting guideline adoption and trial/review registration. Methods We performed a web-based data abstraction from the Instructions for Authors of 67 hematology journals catalogued in the Expanded Science Citation Index of the 2014 Journal Citation Reports to identify whether each journal required, recommended or made no mention of the following reporting guidelines: EQUATOR, ICMJE, CONSORT, MOOSE, QUOROM, PRISMA, STARD, STROBE, ARRIVE and CARE. We also extracted whether journals required or recommended trial or systematic review registration. We e-mailed editors three times to determine which types of studies their journal accepts. Results Forty-eight per cent (32/67) of hematology journals do not adhere to any reporting guidelines. For responding journals, the QUOROM statement, MOOSE, CARE and PROSPERO were the least often mentioned, whereas the ICMJE guidelines, CONSORT statement and general trial registration were most often mentioned. Discussion Reporting guidelines are infrequently required or recommended by hematology journals. Furthermore, few require clinical trial or systematic review database registration. A higher rate of adherence to reporting guidelines can prevent bias from entering the literature. Participation from all stakeholders, including authors and journal editors, to improve reporting guideline and policy practices is required.

Worldwide trends in volume and quality of published protocols of randomized controlled trials

Introduction

Publishing protocols of randomized controlled trials (RCT) facilitates a more detailed description of study rational, design, and related ethical and safety issues, which should promote transparency. Little is known about how the practice of publishing protocols developed over time. Therefore, this study describes the worldwide trends in volume and methodological quality of published RCT protocols.

Methods

A systematic search was performed in PubMed and EMBASE, identifying RCT protocols published over a decade from 1 September 2001. Data were extracted on quality characteristics of RCT protocols. The primary outcome, methodological quality, was assessed by individual methodological characteristics (adequate generation of allocation, concealment of allocation and intention-to-treat analysis). A comparison was made by publication period (First, September 2001- December 2004; Second, January 2005-May 2008; Third, June 2008-September 2011), geographical region and medical specialty.

Results

The number of published RCT protocols increased from 69 in the first, to 390 in the third period (p<0.0001). Internal medicine and paediatrics were the most common specialty topics. Whereas most published RCT protocols in the first period originated from North America (n = 30, 44%), in the second and third period this was Europe (respectively, n = 65, 47% and n = 190, 48%, p = 0.02). Quality of RCT protocols was higher in Europe and Australasia, compared to North America (OR = 0.63, CI = 0.40–0.99, p = 0.04). Adequate generation of allocation improved with time (44%, 58%, 67%, p = 0.001), as did concealment of allocation (38%, 53%, 55%, p = 0.03). Surgical protocols had the highest quality among the three specialty topics used in this study (OR = 1.94, CI = 1.09–3.45, p = 0.02).

Conclusion

Publishing RCT protocols has become popular, with a five-fold increase in the past decade. The quality of published RCT protocols also improved, although variation between geographical regions and across medical specialties was seen. This emphasizes the importance of international standards of comprehensive training in RCT methodology.

Reporting of Randomized Controlled Trials in Cleft Lip and Palate: A 10-Year Review

Objectives

Reviews of the quality of reporting of randomized controlled trials (RCTs) have recently been conducted in different surgical specialties. In this review of RCTs relating to cleft lip, cleft palate, and cleft lip and palate (CL/P), we investigate the quality of reporting against the Consolidated Standards of Reporting Trials (CONSORT) checklist.

Design

A systematic review of CL/P RCTs published from 2004 to 2013, with the included articles scored against the CONSORT checklist.

Patients, Participants

The literature search identified 174 articles. Studies were selected for participants with CL/P who were involved in an RCT with prospective data collection and reported in a full journal article. A total of 6352 participants were included from 65 CUP RCTs during the study period.

Main Outcome Measures

The methodological quality of RCTs was assessed using the CONSORT checklist and Jadad scale.

Results

The mean CONSORT score was 15.8, and the mean Jadad score was 3.3. There was a significant positive correlation between the CONSORT and Jadad score ( P < .0001, ρ = .47). The only significant correlation showed that with an increasing number of authors, both the CONSORT and the Jadad score increased.

Conclusion

This analysis has shown that that there are deficiencies in the transparent reporting of factors such as randomization implementation, blinding, and participant flow. Interventions, outcomes, and the interpretation of results are well presented. We would recommend that RCTs are conceived and undertaken using the CONSORT checklist and reported in a clear and reproducible manner.

Simple randomization did not protect against bias in smaller trials

OBJECTIVES

By removing systematic differences across treatment groups, simple randomization is assumed to protect against bias. However, random differences may remain if the sample size is insufficiently large. We sought to determine the minimal sample size required to eliminate random differences, thereby allowing an unbiased estimation of the treatment effect.

STUDY DESIGN AND SETTING

We reanalyzed two published multicenter, large, and simple trials: the International Stroke Trial (IST) and the Coronary Artery Bypass Grafting (CABG) Off- or On-Pump Revascularization Study (CORONARY). We reiterated 1,000 times the analysis originally reported by the investigators in random samples of varying size. We measured the covariates balance across the treatment arms. We estimated the effect of aspirin and heparin on death or dependency at 30 days after stroke (IST), and the effect of off-pump CABG on a composite primary outcome of death, nonfatal stroke, nonfatal myocardial infarction, or new renal failure requiring dialysis at 30 days (CORONARY). In addition, we conducted a series of Monte Carlo simulations of randomized trials to supplement these analyses.

RESULTS

Randomization removes random differences between treatment groups when including at least 1,000 participants, thereby resulting in minimal bias in effects estimation. Later, substantial bias is observed. In a short review, we show such an enrollment is achieved in 41.5% of phase 3 trials published in the highest impact medical journals.

CONCLUSIONS

Conclusions drawn from completely randomized trials enrolling a few participants may not be reliable. In these circumstances, alternatives such as minimization or blocking should be considered for allocating the treatment.

Reporting Characteristics and Quality of Systematic Reviews of Acupuncture Analgesia

BACKGROUND

Evaluating the clinical efficacy of acupuncture analgesia with systematic reviews (SRs) has attracted wide interest.

OBJECTIVE

To collect a sample of published SRs on acupuncture analgesia in PubMed and examine them in terms of reporting characteristics and quality.

METHODS

A search in PubMed was performed in January 2016. All SRs on acupuncture analgesia were included. To assess the quality of the SRs, AMSTAR tool and PRISMA Statements were used.

RESULTS

One hundred and nine SRs were included in our analysis, the yearly number of publications ranging from 1 in 1997 to 15 in 2015. Only 17% of these publications were Cochrane Systematic Reviews, and 94% were published in Science Citation Index journals. The United Kingdom was the country with the higher number of publications. Low back pain, headache, cancer pain, and labor pain were the most reported diseases or phenotypes. Nearly 73% of these SRs conducted a meta-analysis, 58% revealed positive results, 53% used RevMan software to analyze data, and 44% used the Cochrane Risk of Bias Tool for quality assessment. Only a few SRs assessed the likelihood of publication bias, reported details about the protocol and the registration information, and performed additional analyses.

CONCLUSIONS

The quantity and the quality of SRs regarding acupuncture analgesia have been promoted in recent years. More effort should be expended on the assessment of publication bias, the provision of detailed information about the protocol and the registration process, and the implementation of additional analyses to improve the validity of the SRs.

Test-treatment RCTs are susceptible to bias: a review of the methodological quality of randomized trials that evaluate diagnostic tests

Background

There is a growing recognition for the need to expand our evidence base for the clinical effectiveness of diagnostic tests. Many international bodies are calling for diagnostic randomized controlled trials to provide the most rigorous evidence of impact to patient health. Although these so-called test-treatment RCTs are very challenging to undertake due to their methodological complexity, they have not been subjected to a systematic appraisal of their methodological quality. The extent to which these trials may be producing biased results therefore remains unknown. We set out to address this issue by conducting a methodological review of published test-treatment trials to determine how often they implement adequate methods to limit bias and safeguard the validity of results.

Methods

We ascertained all test-treatment RCTs published 2004–2007, indexed in CENTRAL, including RCTs which randomized patients to diagnostic tests and measured patient outcomes after treatment. Tests used for screening, monitoring or prognosis were excluded. We assessed adequacy of sequence generation, allocation concealment and intention-to-treat, appropriateness of primary analyses, blinding and reporting of power calculations, and extracted study characteristics including the primary outcome.

Results

One hundred three trials compared 105 control with 119 experimental interventions, and reported 150 primary outcomes. Randomization and allocation concealment were adequate in 57 and 37% of trials. Blinding was uncommon (patients 5%, clinicians 4%, outcome assessors 21%), as was an adequate intention-to-treat analysis (29%). Overall 101 of 103 trials (98%) were at risk of bias, as judged using standard Cochrane criteria.

Conclusion

Test-treatment trials are particularly susceptible to attrition and inadequate primary analyses, lack of blinding and under-powering. These weaknesses pose much greater methodological and practical challenges to conducting reliable RCT evaluations of test-treatment strategies than standard treatment interventions. We suggest a cautious approach that first examines whether a test-treatment intervention can accommodate the methodological safeguards necessary to minimize bias, and highlight that test-treatment RCTs require different methods to ensure reliability than standard treatment trials.

Please see the companion paper to this article: http://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-016-0286-0.

Contemporary methodological issues in drug development: the functionality of blinding is poorly studied

Blinding is an important methodological principle in drug development. This article reviews the functionality of blinding within the context of bias minimizing structures of randomized clinical trials. Blinding is not well understood and not well studied. The risk of bias domain 'blinding' is commonly assumed to be 'low-risk' whenever readers come across randomized clinical trials labeled as 'blinded'. However, empirical evidence has shown that this assumption cannot be taken for granted. The functionality of blinding techniques in preventing bias requires further studies.

Timing of publication of abstracts of randomized controlled trials presented in congresses: The example of the European League against Rheumatism meeting

INTRODUCTION

The EUropean League Against Rheumatism (EULAR) congress is an important meeting to present abstracts of research in rheumatology. However, information in these abstracts is insufficient and must be followed and developed in a published article. This study aims to assess to what extent abstracts presented in the EULAR Meeting are published in journals.

METHODS

We identified all abstracts of randomized controlled trials accepted for presentation at the EULAR annual meeting from 2002 to 2011. Using a standardized research, we used Pubmed to find the corresponding publication to each congress abstract. We assessed the cumulative percentage of published abstracts over time and compared the number of included patients, the outcome and the moment of outcome assessment between abstracts presented in congress and in published article.

RESULTS

A total of 1839 abstracts were analyzed. The cumulative probability of abstracts being published in journals was 16.9% (95% CI: 15.2% to 18.6%) at 12 months, 34.3% (95% CI: 32.2% to 36.5%) at 24 months, and 46.5% (95% CI: 44.2% to 48.8%) at 36 months. When evaluable, a difference between the congress and published abstracts was found in 29.0% (247/852). The most frequent difference was a different number of included patients (196/247: 79.4%).

CONCLUSION

More than half of the abstracts of randomized trials were not published three years after they were presented in EULAR meeting. Thirty percent of abstracts were different between their presented and published version.

Reporting of adverse events and statistical details of efficacy estimates in randomized clinical trials of pain in temporomandibular disorders: Analgesic, Anesthetic, and Addiction Clinical Trial Translations, Innovations, Opportunities, and Networks systematic review

BACKGROUND

Statistical methods and adverse events (that is, harms) data affect the accuracy of conclusions about the risk-to-benefit ratio of treatments for temporomandibular disorders (TMDs). The authors reviewed the quality of reporting in TMD clinical trials to highlight practices that are in need of improvement.

TYPES OF STUDIES REVIEWED

The authors included articles published between 1969 and May 31, 2013, in which the investigators reported randomized clinical trials of TMD treatments with pain as a principal outcome variable. Investigators in trials of nonpharmacologic and noninvasive treatments were required to at least mask the participants and assessors; all others were required to be double masked.

RESULTS

Ninety articles qualified for this review: 39 published between 1971 and 2005 (older articles) and 51 published between 2006 and 2013 (newer articles). Specification of primary outcome analyses, methods to accommodate missing data, and adverse event collection methods and rates were generally poor. In some cases, there was apparent improvement from the older to the newer cohort; however, reporting of these methodological details remained inadequate even in the newer articles.

PRACTICAL IMPLICATIONS

This review is designed to alert authors, reviewers, editors, and readers of TMD clinical trials to these issues and improve reporting quality in the future.

The quality of surgical versus non-surgical randomized controlled trials

Background

Randomized controlled trials (RCTs) of surgical interventions are often more difficult to conduct, and their quality may lag behind other trials.

Objectives

To compare quality domains in trials of surgical interventions, to a previously reported control sample of general medical trials from December 2006.

Data sources

MEDLINE, EMBASE and CENTRAL were searched in May 2009.

Study eligibility

English language, full text RCTs, performed on humans that compared a surgical intervention to any other intervention.

Study appraisal and synthesis

Records were ordered according to their date of publication. The most recently published trials were assessed first, until the required sample of 400 trials was obtained. The search and data collection were piloted by three authors, and was thereafter collected by one author. Data was checked.

Results

Although most quality domains were poorly reported, surgical trials compared favourably to general medical trials. Surgical trials were 24% more likely to have an adequate method of random sequence generation, and 71% more likely to have an adequate method of allocation concealment. However, blinding was 40% less likely to be adequate in surgical trials, and sources of funding were 33% less likely to be reported.

Limitations

A single author collected most data. Data was checked and this resulted in few changes.

Conclusions

Reporting of most quality domains in surgical intervention trials was better than general medical trials. Blinding was less likely due to the difficulty in conducting sham surgical trials, and reporting of sources of support should improve with adherence to reporting guidelines.

The quality of reporting of randomized controlled trials of electroacupuncture for stroke

Background

Electroacupuncture (EA), as an extension technique of acupuncture based on traditional acupuncture combined with modern electrotherapy, is commonly used for stroke in clinical treatment and researches. However, there is still a lack of enough evidence to recommend the routine use of EA for stroke. This study is aimed at evaluating the quality of reporting of randomized controlled trials (RCTs) on EA for stroke.

Methods

RCTs on EA for stroke were evaluated by using CONSORT guidelines and STRICTA guidelines. Microsoft Excel 2010 and the R software were used for descriptive statistics analyses.

Results

Seventy studies involving 5468 stroke patients were identified. The CONSORT scores ranged from 16.2 to 67.6% and STRICTA scores from 29.4 to 82.4%. The central items in CONSORT as eligibility criterion, sample size calculation, primary outcome, method of randomization sequence generation, allocation concealment, implementation of randomization, description of blinding, and detailed statistical methods were reported in 100, 6, 68, 37, 14, 10, 16, and 97% of trials, respectively. The reporting of items in STRICTA as acupuncture rationale was 1a (91%), 1b (86%) and 1c 0%; needling details 2a (33%), 2b (97%), 2c (29%), 2d (64%), 2e (100%), 2f (55%) and 2 g (66%); treatment regimen 3a (69%) and 3b (100%); other components of treatment 4a (86%) and 4b (13%); practitioner background item 5 (16%); control intervention(s) 6a (93%) and 6b (10%).

Conclusions

The quality of reporting of RCTs on EA for stroke was generally moderate. The reporting quality needs further improvement.

The quality of reporting in cluster randomised crossover trials: proposal for reporting items and an assessment of reporting quality

BACKGROUND

The cluster randomised crossover (CRXO) design is gaining popularity in trial settings where individual randomisation or parallel group cluster randomisation is not feasible or practical. Our aim is to stimulate discussion on the content of a reporting guideline for CRXO trials and to assess the reporting quality of published CRXO trials.

METHODS

We undertook a systematic review of CRXO trials. Searches of MEDLINE, EMBASE, and CINAHL Plus as well as citation searches of CRXO methodological articles were conducted to December 2014. Reporting quality was assessed against both modified items from 2010 CONSORT and 2012 cluster trials extension and other proposed quality measures.

RESULTS

Of the 3425 records identified through database searching, 83 trials met the inclusion criteria. Trials were infrequently identified as "cluster randomis(z)ed crossover" in title (n = 7, 8%) or abstract (n = 21, 25%), and a rationale for the design was infrequently provided (n = 20, 24%). Design parameters such as the number of clusters and number of periods were well reported. Discussion of carryover took place in only 17 trials (20%). Sample size methods were only reported in 58% (n = 48) of trials. A range of approaches were used to report baseline characteristics. The analysis method was not adequately reported in 23% (n = 19) of trials. The observed within-cluster within-period intracluster correlation and within-cluster between-period intracluster correlation for the primary outcome data were not reported in any trial. The potential for selection, performance, and detection bias could be evaluated in 30%, 81%, and 70% of trials, respectively.

CONCLUSIONS

There is a clear need to improve the quality of reporting in CRXO trials. Given the unique features of a CRXO trial, it is important to develop a CONSORT extension. Consensus amongst trialists on the content of such a guideline is essential.

Impact of a web-based tool (WebCONSORT) to improve the reporting of randomised trials: results of a randomised controlled trial

BACKGROUND

The CONSORT Statement is an evidence-informed guideline for reporting randomised controlled trials. A number of extensions have been developed that specify additional information to report for more complex trials. The aim of this study was to evaluate the impact of using a simple web-based tool (WebCONSORT, which incorporates a number of different CONSORT extensions) on the completeness of reporting of randomised trials published in biomedical publications.

METHODS

We conducted a parallel group randomised trial. Journals which endorsed the CONSORT Statement (i.e. referred to it in the Instruction to Authors) but do not actively implement it (i.e. require authors to submit a completed CONSORT checklist) were invited to participate. Authors of randomised trials were requested by the editor to use the web-based tool at the manuscript revision stage. Authors registering to use the tool were randomised (centralised computer generated) to WebCONSORT or control. In the WebCONSORT group, they had access to a tool allowing them to combine the different CONSORT extensions relevant to their trial and generate a customised checklist and flow diagram that they must submit to the editor. In the control group, authors had only access to a CONSORT flow diagram generator. Authors, journal editors, and outcome assessors were blinded to the allocation. The primary outcome was the proportion of CONSORT items (main and extensions) reported in each article post revision.

RESULTS

A total of 46 journals actively recruited authors into the trial (25 March 2013 to 22 September 2015); 324 author manuscripts were randomised (WebCONSORT n = 166; control n = 158), of which 197 were reports of randomised trials (n = 94; n = 103). Over a third (39%; n = 127) of registered manuscripts were excluded from the analysis, mainly because the reported study was not a randomised trial. Of those included in the analysis, the most common CONSORT extensions selected were non-pharmacologic (n = 43; n = 50), pragmatic (n = 20; n = 16) and cluster (n = 10; n = 9). In a quarter of manuscripts, authors either wrongly selected an extension or failed to select the right extension when registering their manuscript on the WebCONSORT study site. Overall, there was no important difference in the overall mean score between WebCONSORT (mean score 0.51) and control (0.47) in the proportion of CONSORT and CONSORT extension items reported pertaining to a given study (mean difference, 0.04; 95% CI -0.02 to 0.10).

CONCLUSIONS

This study failed to show a beneficial effect of a customised web-based CONSORT checklist to help authors prepare more complete trial reports. However, the exclusion of a large number of inappropriately registered manuscripts meant we had less precision than anticipated to detect a difference. Better education is needed, earlier in the publication process, for both authors and journal editorial staff on when and how to implement CONSORT and, in particular, CONSORT-related extensions.

TRIAL REGISTRATION

ClinicalTrials.gov: NCT01891448 [registered 24 May 2013].

Do emergency medicine journals promote trial registration and adherence to reporting guidelines? A survey of "Instructions for Authors"

Background

The aim of this study was to evaluate the current state of two publication practices, reporting guidelines requirements and clinical trial registration requirements, by analyzing the “Instructions for Authors” of emergency medicine journals.

Methods

We performed a web-based data abstraction from the “Instructions for Authors” of the 27 Emergency Medicine journals catalogued in the Expanded Science Citation Index of the 2014 Journal Citation Reports and Google Scholar Metrics h5-index to identify whether each journal required, recommended, or made no mention of the following reporting guidelines: EQUATOR Network, ICMJE, ARRIVE, CARE, CONSORT, STARD, TRIPOD, CHEERS, MOOSE, STROBE, COREQ, SRQR, SQUIRE, PRISMA-P, SPIRIT, PRISMA, and QUOROM. We also extracted whether journals required or recommended trial registration. Authors were blinded to one another’s ratings until completion of the data validation. Cross-tabulations and descriptive statistics were calculated using IBM SPSS 22.

Results

Of the 27 emergency medicine journals, 11 (11/27, 40.7%) did not mention a single guideline within their “Instructions for Authors,” while the remaining 16 (16/27, 59.3%) mentioned one or more guidelines. The QUOROM statement and SRQR were not mentioned by any journals whereas the ICMJE guidelines (18/27, 66.7%) and CONSORT statement (15/27, 55.6%) were mentioned most often. Of the 27 emergency medicine journals, 15 (15/27, 55.6%) did not mention trial or review registration, while the remaining 12 (12/27, 44.4%) at least mentioned one of the two. Trial registration through ClinicalTrials.gov was mentioned by seven (7/27, 25.9%) journals while the WHO registry was mentioned by four (4/27, 14.8%). Twelve (12/27, 44.4%) journals mentioned trial registration through any registry platform.

Discussion

The aim of this study was to evaluate the current state of two publication practices, reporting guidelines requirements and clinical trial registration requirements, by analyzing the “Instructions for Authors” of emergency medicine journals. In this study, there was not a single reporting guideline mentioned in more than half of the journals. This undermines efforts of other journals to improve the completeness and transparency of research reporting.

Conclusions

Reporting guidelines are infrequently required or recommended by emergency medicine journals. Furthermore, few require clinical trial registration. These two mechanisms may limit bias and should be considered for adoption by journal editors in emergency medicine.

Trial registration

UMIN000022486

Electronic supplementary material

The online version of this article (doi:10.1186/s13049-016-0331-3) contains supplementary material, which is available to authorized users.

Trial Designs and Characteristics in Laser Studies in Dermatology: A Systematic Review

BACKGROUND

Lasers are increasingly used in Dermatology. Clinical studies are among the best means to assess their efficacy.

OBJECTIVE

To obtain an overview of therapeutic laser studies in the field of Dermatology, the authors conducted a systematic review to describe the types of study published over the past 25 years, and their reporting in the abstracts.

METHODS

The authors selected interventional studies that evaluated laser treatment on human subjects with skin diseases. Reviews or single-case reports were excluded. Journal characteristics, study design, and trial characteristics were recorded and analyzed according to trends over time and journal impact factor ratings.

RESULTS

Of the 681 studies included, 57% were comparative, of which 46% were randomized, controlled studies. Trial duration was clearly mentioned in 63% of the abstracts. Trial duration was generally under 6 months (78%). Some parameters were rarely mentioned as follows: the number of patients lost to follow-up (only 12%), blinding (21%), and a clearly stated main outcome (36%). Over the 25 years, there was a significant trend toward an increased proportion of randomized, controlled trials, from 15% between 1990 and 2000 to 41% between 2010 and 2015 (p < .001).

CONCLUSION

This systematic review highlights areas for improvement in laser studies in Dermatology.

Methodological reporting quality of randomized controlled trials: A survey of seven core journals of orthopaedics from Mainland China over 5 years following the CONSORT statement

BACKGROUND

In recent years, the number of randomized controlled trials (RCTs) in the field of orthopaedics is increasing in Mainland China. However, randomized controlled trials (RCTs) are inclined to bias if they lack methodological quality. Therefore, we performed a survey of RCT to assess: (1) What about the quality of RCTs in the field of orthopedics in Mainland China? (2) Whether there is difference between the core journals of the Chinese department of orthopedics and Orthopaedics Traumatology Surgery & Research (OTSR).

MATERIAL AND METHODS

This research aimed to evaluate the methodological reporting quality according to the CONSORT statement of randomized controlled trials (RCTs) in seven key orthopaedic journals published in Mainland China over 5 years from 2010 to 2014. All of the articles were hand researched on Chongqing VIP database between 2010 and 2014. Studies were considered eligible if the words "random", "randomly", "randomization", "randomized" were employed to describe the allocation way. Trials including animals, cadavers, trials published as abstracts and case report, trials dealing with subgroups analysis, or trials without the outcomes were excluded. In addition, eight articles selected from Orthopaedics Traumatology Surgery & Research (OTSR) between 2010 and 2014 were included in this study for comparison. The identified RCTs are analyzed using a modified version of the Consolidated Standards of Reporting Trials (CONSORT), including the sample size calculation, allocation sequence generation, allocation concealment, blinding and handling of dropouts.

RESULTS

A total of 222 RCTs were identified in seven core orthopaedic journals. No trials reported adequate sample size calculation, 74 (33.4%) reported adequate allocation generation, 8 (3.7%) trials reported adequate allocation concealment, 18 (8.1%) trials reported adequate blinding and 16 (7.2%) trials reported handling of dropouts. In OTSR, 1 (12.5%) trial reported adequate sample size calculation, 4 (50.0%) reported adequate allocation generation, 1 (12.5%) trials reported adequate allocation concealment, 2 (25.0%) trials reported adequate blinding and 5 (62.5%) trials reported handling of dropouts. There were statistical differences as for sample size calculation and handling of dropouts between papers from Mainland China and OTSR (P<0.05).

CONCLUSION

The findings of this study show that the methodological reporting quality of RCTs in seven core orthopaedic journals from the Mainland China is far from satisfaction and it needs to further improve to keep up with the standards of the CONSORT statement.

LEVEL OF EVIDENCE

Level III case control.

Outcomes Definitions and Statistical Tests in Oncology Studies: A Systematic Review of the Reporting Consistency

BACKGROUND

Quality of reporting for Randomized Clinical Trials (RCTs) in oncology was analyzed in several systematic reviews, but, in this setting, there is paucity of data for the outcomes definitions and consistency of reporting for statistical tests in RCTs and Observational Studies (OBS). The objective of this review was to describe those two reporting aspects, for OBS and RCTs in oncology.

METHODS

From a list of 19 medical journals, three were retained for analysis, after a random selection: British Medical Journal (BMJ), Annals of Oncology (AoO) and British Journal of Cancer (BJC). All original articles published between March 2009 and March 2014 were screened. Only studies whose main outcome was accompanied by a corresponding statistical test were included in the analysis. Studies based on censored data were excluded. Primary outcome was to assess quality of reporting for description of primary outcome measure in RCTs and of variables of interest in OBS. A logistic regression was performed to identify covariates of studies potentially associated with concordance of tests between Methods and Results parts.

RESULTS

826 studies were included in the review, and 698 were OBS. Variables were described in Methods section for all OBS studies and primary endpoint was clearly detailed in Methods section for 109 RCTs (85.2%). 295 OBS (42.2%) and 43 RCTs (33.6%) had perfect agreement for reported statistical test between Methods and Results parts. In multivariable analysis, variable "number of included patients in study" was associated with test consistency: aOR (adjusted Odds Ratio) for third group compared to first group was equal to: aOR Grp3 = 0.52 [0.31-0.89] (P value = 0.009).

CONCLUSION

Variables in OBS and primary endpoint in RCTs are reported and described with a high frequency. However, statistical tests consistency between methods and Results sections of OBS is not always noted. Therefore, we encourage authors and peer reviewers to verify consistency of statistical tests in oncology studies.

Wounds research for patient benefit: a 5-year programme of research

Background

Complex wounds are those that heal by secondary intention and include lower-limb ulcers, pressure ulcers and some surgical wounds. The care of people with complex wounds is costly, with care mainly being delivered by community nurses. There is a lack of current, high-quality data regarding the numbers and types of people affected, care received and outcomes achieved.

Objectives

To (1) assess how high-quality data about complex wounds can be captured effectively for use in both service planning and research while ensuring integration with current clinical data collection systems and minimal impact on staff time; (2) investigate whether or not a clinical register of people with complex wounds could give valid estimates of treatment effects, thus reducing dependence on large-scale randomised controlled trials (RCTs); (3) identify the most important research questions and outcomes for people with complex wounds from the perspectives of patients, carers and health-care professionals; (4) evaluate the potential contributions to decision-making of individual patient data meta-analysis and mixed treatment comparison meta-analysis; and (5) complete and update systematic reviews in topic areas of high priority.

Methods

To meet objectives 1 and 2 we conducted a prevalence survey and developed and piloted a longitudinal disease register. A consultative, deliberative method and in-depth interviews were undertaken to address objective 3. To address objectives 4 and 5 we conducted systematic reviews including mixed treatment comparison meta-analysis.

Results

From the prevalence survey we estimated the point prevalence of all complex wounds to be 1.47 per 1000 people (95% confidence interval 1.38 to 1.56 per 1000 people). Pressure ulcers and venous leg ulcers were the most common type of complex wound. A total of 195 people with a complex wound were recruited to a complex wounds register pilot. We established the feasibility of correctly identifying, extracting and transferring routine NHS data into the register; however, participant recruitment, data collection and tracking individual wounds in people with multiple wounds were challenging. Most patients and health professionals regarded healing of the wound as the primary treatment goal. Patients were greatly troubled by the social consequences of having a complex wound. Complex wounds are frequently a consequence of, and are themselves, a long-term condition but treatment is usually focused on healing the wound. Consultative, deliberative research agenda setting on pressure ulcer prevention and treatment with patients, carers and clinicians yielded 960 treatment uncertainties and a top 12 list of research priorities. Of 167 RCTs of complex wound treatments in a systematic review of study quality, 41% did not specify a primary outcome and the overall quality of the conduct and reporting of the research was poor. Mixed-treatment comparison meta-analysis in areas of high priority identified that matrix hydrocolloid dressings had the highest probability (70%) of being the most effective dressing for diabetic foot ulcers, whereas a hyaluronan fleece dressing had the highest probability (35%) of being the most effective dressing for venous ulcers; however, the quality of this evidence was low and uncertainty is high.

Conclusions

Complex wounds are common and costly with a poor evidence base for many frequent clinical decisions. There is little routine clinical data collection in community nursing. A prospective complex wounds register has the potential to both assist clinical decision-making and provide important research evidence but would be challenging to implement without investment in information technology in NHS community services. Future work should focus on developing insights into typical wound healing trajectories, identifying factors that are prognostic for healing and assessing the cost-effectiveness of selected wound treatments.

Funding

The National Institute for Health Research Programme Grants for Applied Research programme.

Association between bibliometric parameters, reporting and methodological quality of randomised controlled trials in vascular and endovascular surgery

Purpose We aimed to investigate association between bibliometric parameters, reporting and methodological quality of vascular and endovascular surgery randomised controlled trials. Methods The most recent 75 and oldest 75 randomised controlled trials published in leading journals over a 10-year period were identified. The reporting quality was analysed using the CONSORT statement, and methodological quality with the Intercollegiate Guidelines Network checklist. We used exploratory univariate and multivariable linear regression analysis to investigate associations. Findings Bibliometric parameters such as type of journal, study design reported in title, number of pages; external funding, industry sponsoring and number of citations are associated with reporting quality. Moreover, parameters such as type of journal, subject area and study design reported in title are associated with methodological quality. Conclusions The bibliometric parameters of randomised controlled trials may be independent predictors for their reporting and methodological quality. Moreover, the reporting quality of randomised controlled trials is associated with their methodological quality and vice versa.

Why Most Clinical Research Is Not Useful

John Ioannidis argues that problem base, context placement, information gain, pragmatism, patient centeredness, value for money, feasibility, and transparency define useful clinical research. He suggests most clinical research is not useful and reform is overdue.

A systematic review of randomized controlled trials and reviews in the management of ventral hernias

BACKGROUND

The literature supporting ventral hernia management is growing; however, it is unclear whether the quality of work is improving. We hypothesize that the quality of clinical ventral hernia research has improved over the past 2.5 decades.

METHODS

A review of MEDLINE, Scopus, and Cochrane databases was conducted for all ventral hernia studies from January 1, 1980 to May 1, 2015. Relevant abstracts were assigned a level according to the Oxford Center for Evidence-Based Medicine. Reviews, and meta-analyses were graded using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) Checklist and randomized controlled trials (RCTs) using the Consolidated Standards of Reporting Trials (CONSORT) checklist. Studies that did not fulfill at least 70% of the elements for the PRISMA (19/27) or CONSORT (26/37) checklists were considered to contain substantial methodological flaws.

RESULTS

Of 12,431 citations, 1336 met criteria for quality evaluation. Level 1 studies were sparse (n = 104, 7.8%), and most were level 2 or 3 (n = 463, 34.7%) or 4 (n = 769, 57.6%). Of the level 1 studies, 37 (35.6%) were RCTs, 61(58.7%) were reviews and/or meta-analyses, and 6 (5.8%) were consensus statements. Most RCTs and reviews and/or meta-analyses contained substantial methodological flaws (75.7%, 75.8%). Critical areas of weakness in RCTs were explaining losses and exclusions after randomization and/or allocation and reporting determination of sample size. For reviews and/or meta-analyses, areas of weakness were presenting an electronic search strategy and providing an assessment of risk of bias before pooling data. Linear regressions of PRISMA and CONSORT scores demonstrated improvement over time (PRISMA slope 0.95, R(2) = 0.24; CONSORT slope 0.34, R(2) = 0.08).

CONCLUSIONS

Although the quality of literature guiding ventral hernia management has improved over time, there is room for improvement.

Methodological quality of antimalarial randomized controlled trials during pregnancy and its impact on the risk of low birth weight

AIM

To describe biases in antimalarial randomized controlled trials (RCTs) during pregnancy and their influence on antimalarial drug efficacy to reduce the risk of low birth weight (LBW).

METHODS

RCT characteristics and results were retrieved from a previous systematic review on the efficacy of antimalarials. The Cochrane risk of bias assessment was used to investigate source of biases in each RCT. The quality of RCT reporting published after the introduction of the CONSORT statement in medical literature in 1996 were compared to those published before 1996. A meta-regression analysis was performed to examine the impact of bias on the efficacy of antimalarials to reduce LBW after controlling for the time period prior to 1996.

RESULTS

Twenty out of 25 RCTs (80%) had a high risk of bias. The proportion of RCTs having a low risk of bias was higher in manuscripts published after the introduction of CONSORT compared to those published before 1996 for sequence generation (P = 0.04) and allocation concealment (P = 0.04). Heterogeneity between RCTs was associated with an overestimation of the efficacy of antimalarial drugs in reducing LBW in RCTs with inadequate methods for randomization, allocation concealment or not being free of other bias.

CONCLUSION

Antimalarial RCTs during pregnancy are poorly reported but may be improved by using the CONSORT statement. After taking into account the time period before 1996, we found that biases had an impact on the efficacy of antimalarials to reduce the risk of LBW.

Does pharmacist-supervised intervention through pharmaceutical care program influence direct healthcare cost burden of newly diagnosed diabetics in a tertiary care teaching hospital in Nepal: a non-clinical randomised controlled trial approach

BACKGROUND

Cost is a vital component for people with chronic diseases as treatment is expected to be long or even lifelong in some diseases. Pharmacist contributions in decreasing the healthcare cost burden of chronic patients are not well described due to lack of sufficient evidences worldwide. In developing countries like Nepal, the estimation of direct healthcare cost burden among newly diagnosed diabetics is still a challenge for healthcare professionals, and pharmacist role in patient care is still theoretical and practically non-existent. This study reports the impact of pharmacist-supervised intervention through pharmaceutical care program on direct healthcare costs burden of newly diagnosed diabetics in Nepal through a non-clinical randomised controlled trial approach.

METHODS

An interventional, pre-post non-clinical randomised controlled study was conducted among randomly distributed 162 [control (n = 54), test 1 (n = 54) and test 2 (n = 54) groups] newly diagnosed diabetics by a consecutive sampling method for 18 months. Direct healthcare costs (direct medical and non-medical costs) from patients perspective was estimated by 'bottom up' approach to identify their out-of-pocket expenses (1USD = NPR 73.38) before and after intervention at the baseline, 3, 6, 9 and 12 months follow-ups. Test groups' patients were nourished with pharmaceutical care intervention while control group patients only received care from physician/nurses. Non-parametric tests i.e. Friedman test, Mann-Whitney U test and Wilcoxon signed rank test were used to find the differences in direct healthcare costs among the groups before and after the intervention (p ≤ 0.05).

RESULTS

Friedman test identified significant differences in direct healthcare cost of test 1 (p < 0.001) and test 2 (p < 0.001) groups patients. However, Mann-Whitney U test justified significant differences in direct healthcare cost between control group and test 1 group, and test 2 group patients at 6-months (p = 0.009, p = 0.010 respectively), 9-months (p = 0.005, p = 0.001 respectively) and 12-months (p < 0.001, p < 0.001 respectively).

CONCLUSION

Pharmacist supervised intervention through pharmaceutical care program significantly decreased direct healthcare costs of diabetics in test groups compared to control group and hence describes pharmacist's contribution in minimizing direct healthcare cost burden of patients.

Covariate adjustments in randomized controlled trials increased study power and reduced biasedness of effect size estimation

OBJECTIVES

This study aims to show that under several assumptions, in randomized controlled trials (RCTs), unadjusted, crude analysis will underestimate the Cohen's d effect size of the treatment, and an unbiased estimate of effect size can be obtained only by adjusting for all predictors of the outcome.

STUDY DESIGN AND SETTING

Four simulations were performed to examine the effects of adjustment on the estimated effect size of the treatment and power of the analysis. In addition, we analyzed data from the Advanced Cognitive Training for Independent and Vital Elderly (ACTIVE) study (older adults aged 65-94), an RCT with three treatment arms and one control arm.

RESULTS

We showed that (1) the number of unadjusted covariates was associated with the effect size of the treatment; (2) the biasedness of effect size estimation was minimized if all covariates were adjusted for; (3) the power of the statistical analysis slightly decreased with the number of adjusted noise variables; and (4) exhaustively searching the covariates and noise variables adjusted for can lead to exaggeration of the true effect size. Analysis of the ACTIVE study data showed that the effect sizes adjusting for covariates of all three treatments were 7.39-24.70% larger than their unadjusted counterparts, whereas the effect size would be elevated by at most 57.92% by exhaustively searching the variables adjusted for.

CONCLUSION

All covariates of the outcome in RCTs should be adjusted for, and if the effect of a particular variable on the outcome is unknown, adjustment will do more good than harm.

Professional medical writing support and the quality of randomised controlled trial reporting: a cross-sectional study

OBJECTIVES

Authors may choose to work with professional medical writers when writing up their research for publication. We examined the relationship between medical writing support and the quality and timeliness of reporting of the results of randomised controlled trials (RCTs).

DESIGN

Cross-sectional study.

STUDY SAMPLE

Primary reports of RCTs published in BioMed Central journals from 2000 to 16 July 2014, subdivided into those with medical writing support (n=110) and those without medical writing support (n=123).

MAIN OUTCOME MEASURES

Proportion of items that were completely reported from a predefined subset of the Consolidated Standards of Reporting Trials (CONSORT) checklist (12 items known to be commonly poorly reported), overall acceptance time (from manuscript submission to editorial acceptance) and quality of written English as assessed by peer reviewers. The effect of funding source and publication year was examined.

RESULTS

The number of articles that completely reported at least 50% of the CONSORT items assessed was higher for those with declared medical writing support (39.1% (43/110 articles); 95% CI 29.9% to 48.9%) than for those without (21.1% (26/123 articles); 95% CI 14.3% to 29.4%). Articles with declared medical writing support were more likely than articles without such support to have acceptable written English (81.1% (43/53 articles); 95% CI 67.6% to 90.1% vs 47.9% (23/48 articles); 95% CI 33.5% to 62.7%). The median time of overall acceptance was longer for articles with declared medical writing support than for those without (167 days (IQR 114.5-231 days) vs 136 days (IQR 77-193 days)).

CONCLUSIONS

In this sample of open-access journals, declared professional medical writing support was associated with more complete reporting of clinical trial results and higher quality of written English. Medical writing support may play an important role in raising the quality of clinical trial reporting.

Issues affecting the level of prosthetics research evidence: Secondary analysis of a systematic review

BACKGROUND

Systematic reviews of scientific literature are valuable sources of synthesized knowledge. Systematic review results may also be used to inform readers about challenges inherent to an area of research, guide future research efforts, and facilitate improvements in evidence quality.

OBJECTIVES

To identify methodological issues that affected the overall level of scientific evidence reported in a contemporary systematic review and to offer suggestions for enhancing publications' contribution to the overall evidence.

STUDY DESIGN

Secondary analysis of a systematic review.

METHODS

Publications included in a systematic review related to microprocessor-controlled prosthetic knees were analyzed with respect to established methodological quality criteria. Common issues were identified and discussed.

RESULTS

Internal validity was commonly affected by variable comparison conditions, limited justification of accommodation time, potential fatigue and learning effects, lack of blinding, small sample sizes, limited evidence of measurement reliability, subject attrition, and limited descriptions of selection criteria. Similarly, external validity was affected by limited descriptions of the study sample, indeterminate representativeness, and suboptimal description of the interventions.

CONCLUSION

Results suggest that efforts to address methodological limitations, educate evidence consumers, and improve research reporting are needed to advance the quality and use of evidence in the field of prosthetics.

CLINICAL RELEVANCE

Critical analysis of the strengths and limitations of publications included in a systematic review can inform evidence consumers and contributors about challenges inherent to a field of research. Results of this analysis suggest that efforts to address identified limitations are needed to enhance the overall level of prosthetics evidence.

The current practice of handling and reporting missing outcome data in eight widely used PROMs in RCT publications: a review of the current literature

Purpose

Patient-reported outcome measures (PROMs) are designed to assess patients’ perceived health states or health-related quality of life. However, PROMs are susceptible to missing data, which can affect the validity of conclusions from randomised controlled trials (RCTs). This review aims to assess current practice in the handling, analysis and reporting of missing PROMs outcome data in RCTs compared to contemporary methodology and guidance.

Methods

This structured review of the literature includes RCTs with a minimum of 50 participants per arm. Studies using the EQ-5D-3L, EORTC QLQ-C30, SF-12 and SF-36 were included if published in 2013; those using the less commonly implemented HUI, OHS, OKS and PDQ were included if published between 2009 and 2013.

Results

The review included 237 records (4–76 per relevant PROM). Complete case analysis and single imputation were commonly used in 33 and 15 % of publications, respectively. Multiple imputation was reported for 9 % of the PROMs reviewed. The majority of publications (93 %) failed to describe the assumed missing data mechanism, while low numbers of papers reported methods to minimise missing data (23 %), performed sensitivity analyses (22 %) or discussed the potential influence of missing data on results (16 %).

Conclusions

Considerable discrepancy exists between approved methodology and current practice in handling, analysis and reporting of missing PROMs outcome data in RCTs. Greater awareness is needed for the potential biases introduced by inappropriate handling of missing data, as well as the importance of sensitivity analysis and clear reporting to enable appropriate assessments of treatment effects and conclusions from RCTs.

Electronic supplementary material

The online version of this article (doi:10.1007/s11136-015-1206-1) contains supplementary material, which is available to authorized users.

The quality of reporting methods and results of cost-effectiveness analyses in Spain: a methodological systematic review

BACKGROUND

Cost-effectiveness analysis has been recognized as an important tool to determine the efficiency of healthcare interventions and services. There is a need for evaluating the reporting of methods and results of cost-effectiveness analyses and establishing their validity. We describe and examine reporting characteristics of methods and results of cost-effectiveness analyses conducted in Spain during more than two decades.

METHODS

A methodological systematic review was conducted with the information obtained through an updated literature review in PubMed and complementary databases (e.g. Scopus, ISI Web of Science, National Health Service Economic Evaluation Database (NHS EED) and Health Technology Assessment (HTA) databases from Centre for Reviews and Dissemination (CRD), Índice Médico Español (IME) Índice Bibliográfico Español en Ciencias de la Salud (IBECS)). We identified cost-effectiveness analyses conducted in Spain that used quality-adjusted life years (QALYs) as outcome measures (period 1989-December 2014). Two reviewers independently extracted the data from each paper. The data were analysed descriptively.

RESULTS

In total, 223 studies were included. Very few studies (10; 4.5 %) reported working from a protocol. Most studies (200; 89.7 %) were simulation models and included a median of 1000 patients. Only 105 (47.1 %) studies presented an adequate description of the characteristics of the target population. Most study interventions were categorized as therapeutic (189; 84.8 %) and nearly half (111; 49.8 %) considered an active alternative as the comparator. Effectiveness of data was derived from a single study in 87 (39.0 %) reports, and only few (40; 17.9 %) used evidence synthesis-based estimates. Few studies (42; 18.8 %) reported a full description of methods for QALY calculation. The majority of the studies (147; 65.9 %) reported that the study intervention produced "more costs and more QALYs" than the comparator. Most studies (200; 89.7 %) reported favourable conclusions. Main funding source was the private for-profit sector (135; 60.5 %). Conflicts of interest were not disclosed in 88 (39.5 %) studies.

CONCLUSIONS

This methodological review reflects that reporting of several important aspects of methods and results are frequently missing in published cost-effectiveness analyses. Without full and transparent reporting of how studies were designed and conducted, it is difficult to assess the validity of study findings and conclusions.

Machine learning to assist risk-of-bias assessments in systematic reviews

Background: Risk-of-bias assessments are now a standard component of systematic reviews. At present, reviewers need to manually identify relevant parts of research articles for a set of methodological elements that affect the risk of bias, in order to make a risk-of-bias judgement for each of these elements. We investigate the use of text mining methods to automate risk-of-bias assessments in systematic reviews. We aim to identify relevant sentences within the text of included articles, to rank articles by risk of bias and to reduce the number of risk-of-bias assessments that the reviewers need to perform by hand.

Methods: We use supervised machine learning to train two types of models, for each of the three risk-of-bias properties of sequence generation, allocation concealment and blinding. The first model predicts whether a sentence in a research article contains relevant information. The second model predicts a risk-of-bias value for each research article. We use logistic regression, where each independent variable is the frequency of a word in a sentence or article, respectively.

Results: We found that sentences can be successfully ranked by relevance with area under the receiver operating characteristic (ROC) curve (AUC) > 0.98. Articles can be ranked by risk of bias with AUC > 0.72. We estimate that more than 33% of articles can be assessed by just one reviewer, where two reviewers are normally required.

Conclusions: We show that text mining can be used to assist risk-of-bias assessments.

SPIRIT 2013 Statement: defining standard protocol items for clinical trials

The protocol of a clinical trial serves as the foundation for study planning, conduct, reporting, and appraisal. However, trial protocols and existing protocol guidelines vary greatly in content and quality. This article describes the systematic development and scope of SPIRIT (Standard Protocol Items: Recommendations for Interventional Trials) 2013, a guideline for the minimum content of a clinical trial protocol. The 33-item SPIRIT checklist applies to protocols for all clinical trials and focuses on content rather than format. The checklist recommends a full description of what is planned; it does not prescribe how to design or conduct a trial. By providing guidance for key content, the SPIRIT recommendations aim to facilitate the drafting of high-quality protocols. Adherence to SPIRIT would also enhance the transparency and completeness of trial protocols for the benefit of investigators, trial participants, patients, sponsors, funders, research ethics committees or institutional review boards, peer reviewers, journals, trial registries, policymakers, regulators, and other key stakeholders.

Impact of an online writing aid tool for writing a randomized trial report: the COBWEB (Consort-based WEB tool) randomized controlled trial

BACKGROUND

Incomplete reporting is a frequent waste in research. Our aim was to evaluate the impact of a writing aid tool (WAT) based on the CONSORT statement and its extension for non-pharmacologic treatments on the completeness of reporting of randomized controlled trials (RCTs).

METHODS

We performed a 'split-manuscript' RCT with blinded outcome assessment. Participants were masters and doctoral students in public health. They were asked to write, over a 4-hour period, the methods section of a manuscript based on a real RCT protocol, with a different protocol provided to each participant. Methods sections were divided into six different domains: 'trial design', 'randomization', 'blinding', 'participants', 'interventions', and 'outcomes'. Participants had to draft all six domains with access to the WAT for a random three of six domains. The random sequence was computer-generated and concealed. For each domain, the WAT comprised reminders of the corresponding CONSORT item(s), bullet points detailing all the key elements to be reported, and examples of good reporting. The control intervention consisted of no reminders. The primary outcome was the mean global score for completeness of reporting (scale 0-10) for all domains written with or without the WAT.

RESULTS

Forty-one participants wrote 41 different manuscripts of RCT methods sections, corresponding to 246 domains (six for each of the 41 protocols). All domains were analyzed. For the primary outcome, the mean (SD) global score for completeness of reporting was higher with than without use of the WAT: 7.1 (1.2) versus 5.0 (1.6), with a mean (95 % CI) difference 2.1 (1.5-2.7; P <0.01). Completeness of reporting was significantly higher with the WAT for all domains except for blinding and outcomes.

CONCLUSION

Use of the WAT could improve the completeness of manuscripts reporting the results of RCTs.

TRIAL REGISTRATION

Clinicaltrials.gov ( http://clinicaltrials.gov NCT02127567 , registration date first received April 29, 2014).

Some reflections on the evolution of meta-analysis

I reflect on the evolution of meta-analysis in clinical research over 35 years since I first encountered the key ideas in the late 1970s.

Are licensed canine parvovirus (CPV2 and CPV2b) vaccines able to elicit protection against CPV2c subtype in puppies?: A systematic review of controlled clinical trials

Severe gastroenteritis caused by canine parvovirus type 2 (CPV2) is a serious life-threatening disease in puppies less than 4-months of age. The emergence of new variants has provoked some concern about the cross-protection elicited by licensed canine parvovirus modified-live type 2 (CPV2) and type 2b (CPV2b) vaccines against the most recent subtype CPV2c. A systematic review was carried out to assess the efficacy of commercial vaccines. We conducted a literature search of Pub Med/MEDLINE from January 1990 to May 2014. This was supplemented by hand-searching of related citations and searches in Google/Google Scholar. Controlled clinical trials in which vaccinated puppies were challenged with CPV2c virus were evaluated. Reporting of outcome measures and results for vaccine efficacy were critically appraised through a variety of clinical signs, serological tests, virus shedding and the ability to overcome maternally derived antibodies (MDA) titres. Six controlled clinical trials were included in the review. In most cases, the results of the selected studies reported benefits in terms of clinical signs, serological tests and virus shedding. However, MDA interference was not considered or evaluated in 5 of the selected trials. No accurate definitions of baseline healthy status and/or clinical outcomes were provided. Methods of randomization, allocation concealment and blinding were usually poorly reported. As a result of the limited number of included studies matching the inclusion criteria, the small sample sizes, short follow-up and the methodological limitations observed, it was not possible to reach a final conclusion regarding the cross-protection of licensed CPV2 and CPV2b vaccines against the subtype 2c in puppies. Further and specifically designed trials are required in order to elucidate whether cross-protection is acquired from licensed CPV vaccines.

The Effectiveness of Conservative Management for Acute Whiplash Associated Disorder (WAD) II: A Systematic Review and Meta-Analysis of Randomised Controlled Trials

OBJECTIVE

To evaluate the effectiveness of conservative management (except drug therapy) for acute Whiplash Associated Disorder (WAD) II.

DESIGN

Systematic review and meta-analysis of Randomised Controlled Trials (RCTs) using a pre-defined protocol. Two independent reviewers searched information sources, decided eligibility of studies, and assessed risk of bias (RoB) of included trials. Data were extracted by one reviewer and checked by the other. A third reviewer mediated any disagreements throughout. Qualitative trial and RoB data were summarised descriptively. Quantitative syntheses were conducted across trials for comparable interventions, outcome measures and assessment points. Meta-analyses compared effect sizes with random effects, using STATA version 12.

DATA SOURCES

PEDro, Medline, Embase, AMED, CINAHL, PsycINFO, and Cochrane Library with manual searching in key journals, reference lists, British National Bibliography for Report Literature, Center for International Rehabilitation Research Information & Exchange, and National Technical Information Service were searched from inception to 15th April 2015. Active researchers in the field were contacted to determine relevant studies.

ELIGIBILITY CRITERIA FOR SELECTING STUDIES

RCTs evaluating acute (<4 weeks) WADII, any conservative intervention, with outcome measures important to the International Classification of Function, Disability and Health.

RESULTS

Fifteen RCTs all assessed as high RoB (n=1676 participants) across 9 countries were included. Meta-analyses enabled 4 intervention comparisons: conservative versus standard/control, active versus passive, behavioural versus standard/control, and early versus late. Conservative intervention was more effective for pain reduction at 6 months (95%CI: -20.14 to -3.38) and 1-3 years (-25.44 to -3.19), and improvement in cervical mobility in the horizontal plane at <3 months (0.43 to 5.60) compared with standard/control intervention. Active intervention was effective for pain alleviation at 6 months (-17.19 to -3.23) and 1-3 years (-26.39 to -10.08) compared with passive intervention. Behavioural intervention was more effective than standard/control intervention for pain reduction at 6 months (-15.37 to -1.55), and improvement in cervical movement in the coronal (0.93 to 4.38) and horizontal planes at 3-6 months (0.43 to 5.46). For early (<4 days) versus late (>10 days) interventions, there were no statistically significant differences in all outcome measures between interventions at any time.

CONCLUSIONS

Conservative and active interventions may be useful for pain reduction in patients with acute WADII. Additionally, cervical horizontal mobility could be improved by conservative intervention. The employment of a behavioural intervention (e.g. act-as-usual, education and self-care including regularly exercise) could have benefits for pain reduction and improvement in cervical movement in the coronal and horizontal planes. The evidence was evaluated as low/very low level according to the Grading of Recommendations Assessment, Development and Evaluation system.

If the results of an article are noteworthy, read the entire article; do not rely on the abstract alone

Clinicians typically update their knowledge by reading articles on the Internet. Easy access to the articles' abstracts and a lack of time to access other information sources creates a risk that therapeutic or diagnostic decisions will be made after reading just the abstracts. Occasionally, however, the abstracts of articles from clinical trials that have not obtained statistically significant differences in the primary study endpoint have reported other positive results, for example, of a secondary endpoint or a subgroup analysis. The article, however, correctly reports all results, including those of the primary endpoint. In the abstract, the safety information of the experimental treatment is usually deficient. The whole article should be read if a clinical decision is to be made.

Beautiful small: Misleading large randomized controlled trials? The example of colloids for volume resuscitation

In anesthesia and intensive care, treatment benefits that were claimed on the basis of small or modest-sized trials have repeatedly failed to be confirmed in large randomized controlled trials. A well-designed small trial in a homogeneous patient population with high event rates could yield conclusive results; however, patient populations in anesthesia and intensive care are typically heterogeneous because of comorbidities. The size of the anticipated effects of therapeutic interventions is generally low in relation to relevant endpoints. For regulatory purposes, trials are required to demonstrate efficacy in clinically important endpoints, and therefore must be large because clinically important study endpoints such as death, sepsis, or pneumonia are dichotomous and infrequently occur. The rarer endpoint events occur in the study population; that is, the lower the signal-to-noise ratio, the larger the trials must be to prevent random events from being overemphasized. In addition to trial design, sample size determination on the basis of event rates, clinically meaningful risk ratio reductions and actual patient numbers studied are among the most important characteristics when interpreting study results. Trial size is a critical determinant of generalizability of study results to larger or general patient populations. Typical characteristics of small single-center studies responsible for their known fragility include low variability of outcome measures for surrogate parameters and selective publication and reporting. For anesthesiology and intensive care medicine, findings in volume resuscitation research on intravenous infusion of colloids exemplify this, since both the safety of albumin infusion and the adverse effects of the artificial colloid hydroxyethyl starch have been confirmed only in large-sized trials.