Reporting randomised clinical trials of analgesics after traumatic or orthopaedic surgery is inadequate: a systematic review

Harms reporting in randomized controlled trials underpinning the American Academy of Orthopaedic Surgeons clinical practice guidelines for glenohumeral osteoarthritis

BACKGROUND

Glenohumeral osteoarthritis is one of the most common causes of shoulder pain. As such, the American Academy of Orthopaedic Surgeons (AAOS) has developed clinical practice guidelines (CPGs) to address the management of glenohumeral osteoarthritis. These CPG recommendations stem from the findings of randomized controlled trials (RCTs), which have been shown to influence clinical decision making and health policy. Therefore, it is essential that trial outcomes, including harms data (ie, adverse events), are adequately reported. We intend to evaluate the reporting quality of harms-related data in orthopedic literature specifically relating to AAOS CPG recommendations on the management of glenohumeral osteoarthritis.

METHODS

We adhered to the Preferred Reporting Items for Systematic Reviews (PRISMA) as well as guidance for reporting meta-research. The AAOS CPGs for glenohumeral osteoarthritis were obtained from orthoguidelines.org, and 2 authors independently screened the guidelines for the RCTs referenced. A total of 14 studies were identified. Data were extracted from the 14 included studies independently by the same 2 authors. Adherence to the Consolidated Standards of Reporting Trials (CONSORT) Extension for Harms Checklist was assessed using an 18-item scoring chart, with 1 point being awarded for meeting a checklist item and 0 points being awarded for not meeting a checklist item. Descriptive statistics, such as frequencies, percentages, and 95% confidence intervals were used to summarize RCT adherence to the CONSORT checklist.

RESULTS

The average score among the studies included was 7.36/18 items (39% adherence). No study adhered to all criteria, with the highest-performing study meeting 11 of 18 items (58%) and the lowest meeting 3 of 18 items (16%). A positive correlation between checklist score and year of publication was observed, with studies published more recently receiving a higher score on the CONSORT checklist (P < .05). Studies that disclosed funding information received a higher score than those that did not (P < .05), but there was no significant difference when the different funding sources were compared. Finally, double-blinded studies scored higher on the checklist than those with lower levels of blinding (single or no blinding, P < .05).

CONCLUSION

Adverse events are poorly reported amongst RCTs cited as supporting evidence for AAOS Management of Glenohumeral Osteoarthritis CPGs, evidenced by a CONSORT checklist compliance rate of only 41% in this study. We recommend the development of an updated checklist with information that makes it easier for authors to recognize, evaluate, and report on harms data. Additionally, we encourage authors to include information about adverse events or negative outcomes in the abstract.

Conducting Trial-Based Economic Evaluations Using R: A Tutorial

Trial-based economic evaluations are increasingly being conducted to support healthcare decision-making. When analysing trial-based economic evaluation data, different methodological challenges may be encountered, including (i) missing data, (ii) correlated costs and effects, (iii) baseline imbalances and (iv) skewness of costs and/or effects. Despite the broad range of methods available to account for these methodological challenges in effectiveness studies, they may not always be directly applicable in trial-based economic evaluations where costs and effects are analysed jointly, and more than one methodological challenge typically needs to be addressed simultaneously. The use of inappropriate methods can bias results and conclusions regarding the cost-effectiveness of healthcare interventions. Eventually, such low-quality evidence can hamper healthcare decision-making, which may in turn result in a waste of already scarce healthcare resources. Therefore, this tutorial aims to provide step-by-step guidance on how to combine appropriate statistical methods for handling the abovementioned methodological challenges using a ready-to-use R script. The theoretical background of the described methods is provided, and their application is illustrated using a simulated trial-based economic evaluation.

Analysis of the Evidence Underpinning the American Academy of Orthopedic Surgeons Knee Osteoarthritis Clinical Practice Guidelines

CONTEXT

Clinical practice guidelines (CPGs) are vital to establishing a standardized and evidence-based approach in medicine. These guidelines rely on the use of methodologically sound clinical trials, and the subsequent reporting of their methodology.

OBJECTIVE

To evaluate the completeness of randomized controlled trials (RCTs) underpinning CPGs published by the American Academy of Orthopedic Surgeons (AAOS) for management of osteoarthritis of the knee.

DATA SOURCES

We searched the most recent AAOS CPGs for surgical and nonsurgical management of osteoarthritis of the knee for RCTs. To estimate the necessary sample size, we performed a power analysis using OpenEpi 3.0 (openepi.com).

STUDY SELECTION

Two authors independently screened the reference sections of the included CPGs. Included studies met the definition of an RCT, were retrievable in the English language, and were cited in at least one of the included CPGs.

STUDY DESIGN

Meta-Analysis.

LEVEL OF EVIDENCE

Level 1a.

DATA EXTRACTION

We performed double-blind screening and extraction of RCTs included in the AAOS CPGs. We evaluated each RCT for adherence to the Consolidated Standards of Reporting Trials (CONSORT) 2010 checklist. A multiple regression analysis was conducted to assess CONSORT adherence against characteristics of included studies (ie, type of intervention, funding source, etc).

RESULTS

Our study included 179 RCTs. The overall adherence was 68.5% with significant differences between those published before and since the development of the 2010 CONSORT guidelines (P = 0.02). We found that RCTs receiving funding from industry/private sources as well as studies that included a conflict of interest statement showed more completeness than RCTs that reported receiving no funding (P < 0.01).

CONCLUSION

We found suboptimal CONSORT adherence for RCTs cited in AAOS CGPs for management of osteoarthritis of the knee. Therefore, the CPGs are likely supported by outdated evidence and lack of high-quality reporting. It is important that evidence used to guide clinical decision making be of the highest quality in order to optimize patient outcomes. In order for clinicians to confer the greatest benefits to their patients, CPGs should provide the totality of evidence and emphasize emerging high-quality RCTs to ensure up-to-date, evidence-based clinical decision-making.

Network meta-analysis: What is its purpose in Orthopaedic literature?

Systematic reviews, of level-I primary literature, are the gold standard for the formation of Clinical Practice Guidelines in Orthopaedic Surgery. When systematic reviews have multiple groups of data, meta-analyses can be conducted to analyse the direct comparison of the data points (pairwise meta-analysis). Over recent years, statisticians have created a new statistical model called network meta-analyses that can be applied to systematic reviews. network meta-analyses allow for comparison of different treatment outcomes that may or may not have been directly assessed through level-I primary studies. network meta-analyses are appearing more and more in Orthopaedic Surgery literature; therefore, in this article, we discuss what a Network Meta-analysis is and its application in Orthopaedics.

The statistical approach in trial-based economic evaluations matters: get your statistics together!

Background

Baseline imbalances, skewed costs, the correlation between costs and effects, and missing data are statistical challenges that are often not adequately accounted for in the analysis of cost-effectiveness data. This study aims to illustrate the impact of accounting for these statistical challenges in trial-based economic evaluations.

Methods

Data from two trial-based economic evaluations, the REALISE and HypoAware studies, were used. In total, 14 full cost-effectiveness analyses were performed per study, in which the four statistical challenges in trial-based economic evaluations were taken into account step-by-step. Statistical approaches were compared in terms of the resulting cost and effect differences, ICERs, and probabilities of cost-effectiveness.

Results

In the REALISE study and HypoAware study, the ICER ranged from 636,744€/QALY and 90,989€/QALY when ignoring all statistical challenges to − 7502€/QALY and 46,592€/QALY when accounting for all statistical challenges, respectively. The probabilities of the intervention being cost-effective at 0€/ QALY gained were 0.67 and 0.59 when ignoring all statistical challenges, and 0.54 and 0.27 when all of the statistical challenges were taken into account for the REALISE study and HypoAware study, respectively.

Conclusions

Not accounting for baseline imbalances, skewed costs, correlated costs and effects, and missing data in trial-based economic evaluations may notably impact results. Therefore, when conducting trial-based economic evaluations, it is important to align the statistical approach with the identified statistical challenges in cost-effectiveness data. To facilitate researchers in handling statistical challenges in trial-based economic evaluations, software code is provided.

Supplementary Information

The online version contains supplementary material available at 10.1186/s12913-021-06513-1.

Intervention Reporting of Published Trials Is Insufficient in Orthopaedic Surgery Journals: Application of the Template for Intervention Description and Replication Checklist

Purpose

Outcomes of randomized controlled trials (RCTs) have been shown to influence clinical decision making. Thus, the quality and reliability of these outcomes are essential for both patients and medical care providers. To date, no study has assessed the quality of intervention reporting of RCTs in orthopaedics. The aim of this study was to evaluate the quality of intervention reporting of published RCTs in the field of orthopaedics using the Template for Intervention Description and Replication (TIDieR) checklist.

Methods

In this cross-sectional analysis, we applied the TIDieR checklist to assess the quality of intervention reporting in orthopaedic RCTs. Additionally, we evaluated the TIDieR checklist’s influence on intervention reporting by comparing overall adherence to checklist items in trials published before the TIDieR checklist’s release versus trials published after its release. Finally, we assessed whether certain factors were associated with the quality of intervention reporting.

Results

From a random sample of 300 publications in orthopaedic journals, 175 parallel-arm and cluster RCTs were identified. The overall rate of adherence to TIDieR items was 58.4%. Only 31.4% of orthopaedic RCTs adhered to at least 6 of the 12 TIDieR checklist items, whereas 0% adhered to all 12 items. We found no significant improvement in the quality of intervention reporting in studies published after the TIDieR checklist’s release compared with studies published before its release (P = .97). Additionally, preregistered trials were associated with more complete intervention reporting.

Conclusions

Our results suggest suboptimal reporting of orthopaedic RCT interventions. In addition, the TIDieR checklist’s intended effect—to better the quality of RCT intervention reporting—appears to have fallen short of its goal.

Clinical Relevance

Because outcomes of RCTs are used to guide clinical decision making, it is essential that orthopaedic surgeons and clinical practice guideline panels are equipped with high-quality published research. Increasing the accuracy of intervention reporting may lead to more accurate clinical application. Thus, adoption of more stringent reporting of trial interventions by researchers, authors, and journal editors may improve the quality of orthopaedic research, as well as improve patient outcomes.

Colloidal Silicon Dioxide in Tablet form (Carbowhite) Efficacy in Patients with Acute Diarrhea: Results of Randomized, Double-Blind, Placebo-Controlled, Multi-Center Study

The acute diarrhea is a wide-spread disease. The prescription of enterosorbents is appropriate as a primary measure for the treatment of the acute diarrhea for effective prevention of the fluid and electrolyte loss, as well as method for symptom relief of the attack of the disease. Aim of the study - the antidiarrheal efficacy and safety study of high-dispersion silicon dioxide enterosorbent in tablet dosage form in patients with acute diarrhea. This was randomized, double-blind, placebo-controlled, 4-center study. Acute diarrhea was defined as three and more episodes of watery stool per day either during 48 hours or less before study entry in the patients having normal stool recently. It has been postulated that symptoms and signs of acute diarrhea have to be caused by direct infection of the gastrointestinal tract and did not associated with moderate-to-severe systemic states. 144 patients with established acute diarrhea were randomized into treatment group (enterosorbent "Carbowhite", n = 120) or placebo group. Date collection including severity diarrhea, systemic symptoms was performed at baseline and daily during 7 days. Stool examination and serological assay were performed at baseline. The primary end points were declared as time to complete recovery from acute diarrhea. It has been found that the use of the siliceous enterosorbent ("Carbowhite") allowed to reduce (p < 0.001) the treatment period averagely for 0.9 days (95% confidence interval 0.5-1.2 days) in comparison with placebo. Data of safety monitoring has revealed that both patient groups had negative stool culture, while initiation of antibiotic treatment was run more frequently in placebo group (8.3%) compared to investigational product group (4.1%, P = 0.044). The siliceous enterosorbent "Carbowhite" was well tolerated and reduced the recovery time of the acute episode of the diarrhea in the clinically significant form.

The reporting of progression criteria in protocols of pilot trials designed to assess the feasibility of main trials is insufficient: a meta-epidemiological study

Introduction

Pilot and feasibility trials are conducted to determine feasibility or to collect information that would inform the design of a larger definitive trial. Clear progression criteria are required to determine if a definitive or main trial is feasible and how it should be designed. We sought to determine how often progression criteria are reported and the associated factors.

Methods

We conducted a methodological review of protocols for pilot randomised trials published in three journals that publish research protocols (BMJ Open, Trials, Pilot and Feasibility Studies), using a PubMed search (2013–2017). We extracted bibliometric information including the country in which the study was conducted, source of funding, type of intervention, use of a primary feasibility outcome, sample size reporting, and justification. We used generalised linear models to determine the factors associated with reporting progression criteria.

Results

Our search retrieved 276 articles, of which 49 were not eligible. We included 227 articles. Overall, 45/227 (19.8%; 95% confidence interval [CI] 14.8–25.6) reported progression criteria. Protocols published in more recent years were significantly associated with higher odds of reporting progression criteria (adjusted odds ratio [aOR] 1.40; 95% CI 1.03–1.92; p = 0.034). Pilot trials from Europe (aOR 0.19; 95% CI 0.08–0.48; p < 0.001) and the rest of the world (aOR 0.05; 95% CI 0.01–0.18; p < 0.003) compared to North America were significantly associated with lower odds of reporting progression criteria. Journal, source of funding, sample size, intervention type, and having a primary outcome related to feasibility were not significantly associated with reporting progression criteria.

Conclusion

Progression criteria are not often explicitly stated in protocols of pilot trials leaving room for varied interpretation of findings. The development of formal guidance for progression criteria in protocols of pilot trials is warranted.

A methodological survey on reporting of pilot and feasibility trials for physiotherapy interventions: a study protocol

INTRODUCTION

Pilot and feasibility trials aim to test whether a full trial can be conducted or if any procedures must be changed for the full trial. Pilot trials must be reported in a transparent, accurate and complete way. In this report, we present a protocol for a methodological survey with the following aims: (1) to determine the percentage of physiotherapy trial reports which claim to be pilot or feasibility trials that evaluate feasibility, (2) to determine the aspect of feasibility evaluated in the primary objectives of the pilot or feasibility trials, (3) to describe the completeness of reporting of abstracts and full articles of pilot or feasibility trials using the Consolidated Standards of Reporting Trials (CONSORT) extension to randomised pilot and feasibility trials and (4) to investigate factors associated with completeness of reporting of pilot or feasibility trials.

METHODS AND ANALYSIS

Reports of randomised controlled trials indexed in the Physiotherapy Evidence Database (PEDro) that claim to be pilot or feasibility trials and published in 2011-2017 will be included. Two independent reviewers will confirm eligibility and classify the aspect of feasibility being evaluated in the objectives of the included pilot or feasibility trials. Completeness of reporting of both the abstract and the full article will be evaluated using the CONSORT extension to randomised pilot and feasibility trials. The primary analysis will be a descriptive analysis about the reporting quality of abstracts and full texts of pilot and feasibility trials. We will use generalised estimating equation analysis to explore factors associated with completeness of reporting.

ETHICS AND DISSEMINATION

The results of this study will be disseminated by presentation at conferences and will be submitted for publication in a peer-reviewed journal. Ethical approval is not necessary for this study.

Does the medical literature remain inadequately described despite having reporting guidelines for 21 years? - A systematic review of reviews: an update

PURPOSE

Reporting guidelines (eg, Consolidated Standards of Reporting Trials [CONSORT] statement) are intended to improve reporting standards and enhance the transparency and reproducibility of research findings. Despite accessibility of such guidelines, researchers are not required to adhere to them. Our goal was to determine the current status of reporting quality in the medical literature and examine whether adherence of reporting guidelines has improved since the inception of reporting guidelines.

MATERIALS AND METHODS

Eight reporting guidelines, such as CONSORT, Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), STrengthening the Reporting of OBservational studies in Epidemiology (STROBE), Quality of Reporting of Meta-analysis (QUOROM), STAndards for Reporting of Diagnostic accuracy (STARD), Animal Research: Reporting In Vivo Experiments (ARRIVE), Consolidated Health Economic Evaluation Reporting Standards (CHEERS), and Meta-analysis of Observational Studies in Epidemiology (MOOSE) were examined. Our inclusion criteria included reviews published between January 1996 to September 2016 which investigated the adherence to reporting guidelines in the literature that addressed clinical trials, systematic reviews, observational studies, meta-analysis, diagnostic accuracy, economic evaluations, and preclinical animal studies that were in English. All reviews were found on Web of Science, Excerpta Medical Database (EMBASE), MEDLINE, and Cumulative Index to Nursing and Allied Health Literature (CINAHL).

RESULTS

Among the general searching of 26,819 studies by using the designed searching method, 124 studies were included post screening. We found that 87.9% of the included studies reported suboptimal adherence to reporting guidelines. Factors associated with poor adherence included non-pharmacological interventions, year of publication, and trials concluding with significant results. Improved adherence was associated with better study designs such as allocation concealment, random sequence, large sample sizes, adequately powered studies, multiple authorships, and being published in journals endorsing guidelines.

CONCLUSION

We conclude that the level of adherence to reporting guidelines remains suboptimal. Endorsement of reporting guidelines by journals is important and recommended.

Levodopa-Carbidopa Intestinal Gel in Parkinson's Disease: A Systematic Review and Meta-Analysis

Background: Levodopa has been widely used and regarded as the most effective therapy for Parkinson's disease (PD), but long-term treatment with oral levodopa may result in motor fluctuations and involuntary movements (dyskinesias). There is evidence to suggest that Continuous infusion of levodopa-carbidopa intestinal gel (LCIG) can effectively manage motor and non-motor complications in PD, but clinical studies investigating this have yielded inconsistent results. This systematic review and meta-analysis was performed to examine the efficacy and safety of LCIG for patients with PD.

Methods: A systematic search was conducted to retrieve published data in the EMBASE, PubMed, and the Cochrane Library up to March 2018. Both efficiency and safety of LCIG were analyzed using pooled standardized mean differences (SMDs) or odds ratio (ORs) with 95% confidence interval (CIs).

Results: Eight trials with 384 PD patients were included in the present study. Compared with the control group, LCIG significantly decreased off-time (SMD, −1.19; 95% CI, −2.25 to −0.12; p = 0.003) and increased on-time without troublesome dyskinesia (SMD, 0.55; 95% CI, 0.20 to 0.90; p = 0.002). However, no significant difference of LCIG was found in on-time with troublesome dyskinesia. There were no significant differences in UPDRS, Hoehn & Yahr and PDQ-39 scores. Besides, no significant differences in the drop-out and adverse effects.

Conclusions: Continuous delivery of LCIG may offer a promising option for PD patients. More randomized double-blind controlled studies with large sample sizes were needed to further confirm the efficacy and safety of LCIG for PD patients.

Systematic review of acute physically active learning and classroom movement breaks on children's physical activity, cognition, academic performance and classroom behaviour: understanding critical design features

Objective

To examine the impact of acute classroom movement break (CMB) and physically active learning (PAL) interventions on physical activity (PA), cognition, academic performance and classroom behaviour.

Design

Systematic review.

Data sources

PubMed, EBSCO, Academic Search Complete, Education Resources Information Center, PsycINFO, SPORTDiscus, SCOPUS and Web of Science.

Eligibility criteria for selecting studies

Studies investigating school-based acute bouts of CMB or PAL on (PA), cognition, academic performance and classroom behaviour. The Downs and Black checklist assessed risk of bias.

Results

Ten PAL and eight CMB studies were identified from 2929 potentially relevant articles. Risk of bias scores ranged from 33% to 64.3%. Variation in study designs drove specific, but differing, outcomes. Three studies assessed PA using objective measures. Interventions replaced sedentary time with either light PA or moderate-to-vigorous PA dependent on design characteristics (mode, duration and intensity). Only one study factored individual PA outcomes into analyses. Classroom behaviour improved after longer moderate-to-vigorous (>10 min), or shorter more intense (5 min), CMB/PAL bouts (9 out of 11 interventions). There was no support for enhanced cognition or academic performance due to limited repeated studies.

Conclusion

Low-to-medium quality designs predominate in investigations of the acute impacts of CMB and PAL on PA, cognition, academic performance and classroom behaviour. Variable quality in experimental designs, outcome measures and intervention characteristics impact outcomes making conclusions problematic. CMB and PAL increased PA and enhanced time on task. To improve confidence in study outcomes, future investigations should combine examples of good practice observed in current studies.

PROSPERO registration number

CRD42017070981.

An Evaluation of Reporting Guidelines and Clinical Trial Registry Requirements Among Orthopaedic Surgery Journals

BACKGROUND

The responsibility for ensuring that studies are adequately reported is primarily that of those conducting the study; however, journal policies may influence how thoroughly authors choose to report their research. The use of reporting guidelines and prospective trial registration are promising avenues for ensuring that published studies adhere to the highest methodological standards. The purpose of this study is to evaluate orthopaedic surgery journal policies regarding reporting guidelines and trial registration, and to evaluate the effects that these policies have on adherence to reporting.

METHODS

We conducted a cross-sectional survey of journal policies and "Instructions for Authors" to determine the journals' policies and guidance regarding use of reporting guidelines and study registration. We also examined whether trials published in journals referencing CONSORT (Consolidated Standards of Reporting Trials) had higher rates of compliance with publishing a CONSORT flow diagram and whether journals with trial registration policies were more likely to contain registered trials than journals without these requirements.

RESULTS

Of the 21 orthopaedic surgery journals, 6 (29%) did not mention a single guideline, and clinical trial registration was required by 11 (52%) orthopaedic surgery journals and recommended by 2 (10%). Of the 21 general medical journals, 3 (14%) did not mention a single guideline, and trial registration was required by 13 (62%) general medical journals and recommended by 5 (24%) others. Furthermore, journals that referenced CONSORT were more likely to publish trials with a CONSORT flow diagram. Journals with trial registration policies were more likely to publish registered trials.

CONCLUSIONS

Reporting guidelines and trial registration are suboptimally required or recommended by orthopaedic surgery journals. These 2 mechanisms may improve methodology and quality, and should be considered for adoption by journal editors in orthopaedic surgery.

CLINICAL RELEVANCE

Because orthopaedic surgeons rely on high-quality research to direct patient care, measures must be taken to ensure that published research is of the highest quality. The use of reporting guidelines and prospective clinical trial registration may improve the quality of orthopaedic research, thereby improving patient care.

The framing of research questions using the PICOT format in randomized controlled trials of venous ulcer disease is suboptimal: A systematic survey

Despite several publications on venous ulcers, there is still a lack of evidence from randomized controlled trials (RCTs) to support certain treatments for patients with this disorder. Well-designed research questions using the PICOT (Population; Intervention; Comparator; Outcome; Time-frame) format in RCTs can improve the quality of research. The objectives of this study were to assess how the PICOT format is used to frame research questions in RCTs published on venous ulcer disease and to determine the factors associated with better adherence to the PICOT format. We conducted a systematic survey of RCTs on venous ulcers published in the PubMed database between January 2009 and May 2016. All RCTs published in English addressing therapeutic interventions for venous ulcer disease in human subjects were included. We examined whether the five elements of the PICOT format were used in formulating the research question and scored them between 0 and 5. The primary outcome of this systematic survey was the percentage of studies that adequately reported all five PICOT elements. Eighty-five (85) RCTs were included with median PICOT score of 3 (IQR = 1.5). Four elements of PICOT were present in 28 reports (32.9%) and only 2 RCTS (2.3%) reported all the PICOT elements. Population and intervention were often appropriately described, in (70/85) 82.4% and (83/85) 97.6% of the studies, respectively; however, comparison intervention and outcome were presented in only (53/85) 62.3% and (48/85) 56.5% of studies, respectively. Very few RCTs (7.1%; 6/85) reported the study time frame. No journal or RCT characteristics were found to be significantly associated with better reporting. Use of the PICOT format to frame research questions in RCTs published on venous ulcers is suboptimal, and our study reinforces the importance of framing a good research question to improve the design of trials and quality of evidence in venous ulcer disease.

Safety, Tolerability and Efficacy of Drugs for Treating Behavioural Insomnia in Children with Attention-Deficit/Hyperactivity Disorder: A Systematic Review with Methodological Quality Assessment

OBJECTIVE

A large proportion of paediatric patients with attention-deficit/hyperactivity disorder (ADHD) have associated sleep problems which not only affect the child's wellbeing but also impact family functioning. Management of sleep problems is consequently an important aspect of overall ADHD management in paediatric patients. Although some drugs are being used off-label for the management of paediatric insomnia, there is scant clinical evidence supporting their use. Our aim was to identify and assess the quality of published studies reporting the safety, tolerability and efficacy of drugs used for treating behavioural insomnia in children with ADHD.

METHODS

After an initial screen to determine which drugs were most commonly used, we conducted a systematic review of English-language publications from searches of PubMed, EMBASE, PsycINFO and two trial register databases to February 2017, using keywords 'clonidine', 'melatonin', 'zolpidem', 'eszopiclone', 'L-theanine', 'guanfacine', 'ADHD', 'sleep disorder' and 'children'. For quality assessment of included studies, we used the CONSORT checklist for randomised control trials (RCTs) and the Downs and Black checklist for non-RCTs.

RESULTS

Twelve studies were included. Two case series for clonidine, two RCTs and four observational studies for melatonin and one RCT each for zolpidem, eszopiclone, L-theanine and guanfacine. Of the 12 included studies, only one on eszopiclone scored excellent for quality. The quality of the rest of the studies varied from moderate to low. For clonidine, melatonin and L-theanine, improvements in sleep-onset latency and total sleep duration were reported; however, zolpidem, eszopiclone and guanfacine failed to show any improvement when compared with placebo. Clonidine, melatonin, L-theanine, eszopiclone and guanfacine were well tolerated with mild to moderate adverse events; zolpidem was associated with neuropsychiatric adverse effects.

CONCLUSION

There is generally poor evidence for prescribing drugs for behavioural insomnia in children with ADHD. Further controlled studies are warranted.

Framing of research question using the PICOT format in randomised controlled trials of venous ulcer disease: a protocol for a systematic survey of the literature

INTRODUCTION

Although venous ulcers have a great social and economic impact, there is a lack of evidence from randomised controlled trials (RCTs) to support appropriate management for this disease. Framing the research question using the Population; Intervention; Comparator; Outcome; Time frame (PICOT) format in RCTs can improve the quality of the research design.

OBJECTIVES

To evaluate how the PICOT format is used to frame a research question in reports of RCTs of venous ulcer disease and to determine the factors associated with better adherence to the PICOT format in framing the research question.

METHODS AND ANALYSES

We will conduct a systematic survey of RCTs on venous ulcers published in the National Institute of Health, PubMed database between January 2009 and May 2016. We will include all RCTs addressing therapeutic intervention for venous ulcer disease involving human subjects, and published in the English language. The selection process will be carried out in duplicate by two independent investigators. First, titles and abstracts will be screened, then full-text articles. We will examine whether the five elements of the PICOT format are used in formulating the research question and give a score between 0 and 5. The primary outcome will be the proportion of studies that have adequately reported all five PICOT elements.

DISSEMINATION

This will be the first survey to assess how the PICOT format is used to frame research questions on the management of venous ulcers in reports of RCTs. On completion, this review will be submitted to a peer-reviewed biomedical journal for publication and the findings will also be presented at scientific conferences.

Evaluation of Quality of Lower Limb Arthroplasty Observational Studies Using the Assessment of Quality in Lower Limb Arthroplasty (AQUILA) Checklist

This study used the assessment of quality in lower limb arthroplasty (AQUILA) checklist to assess the quality of lower limb arthroplasty observational studies. Among 132 studies the mean reporting quality score was 5.4 (SD=1.2) out of 8 possible points. Most studies adequately reported reasons for revisions (98%) and prosthesis brand and fixation (95%) in sufficient detail. Only 3% of studies adequately reported the number of patients unwilling to participate, 15% stated a clear primary research question or hypothesis, 11% reported a worst-case analysis or competing risk analysis for endpoints, and 42% reported more than 5% of patients were lost to follow-up. There is significant room for improvement in the reporting and methodology of lower limb arthroplasty observational studies.

LEVEL OF EVIDENCE

Level III.

A systematic review and analysis of factors associated with methodological quality in laparoscopic randomized controlled trials

Several methods for assessment of methodological quality in randomized controlled trials (RCTs) have been developed during the past few years. Factors associated with quality in laparoscopic surgery have not been defined till date. The aim of this study was to investigate the relationship between bibliometric and the methodological quality of laparoscopic RCTs. The PubMed search engine was queried to identify RCTs on minimally invasive surgery published in 2012 in the 10 highest impact factor surgery journals and the 5 highest impact factor laparoscopic journals. Eligible studies were blindly assessed by two independent investigators using the Scottish Intercollegiate Guidelines Network (SIGN) tool for RCTs. Univariate and multivariate analyses were performed to identify potential associations with methodological quality. A total of 114 relevant RCTs were identified. More than half of the trials were of high or acceptable quality. Half of the reports provided information on comparative demo graphic data and only 21% performed intention-to-treat analysis. RCTs with sample size of at least 60 patients presented higher methodological quality (p = 0.025). Upon multiple regression, reporting on preoperative care and the experience level of surgeons were independent factors of quality.

How to do random allocation (randomization)

Purpose

To explain the concept and procedure of random allocation as used in a randomized controlled study.

Methods

We explain the general concept of random allocation and demonstrate how to perform the procedure easily and how to report it in a paper.

The use of probiotic strains in caries prevention: a systematic review

This paper aims to provide a systematic review of the caries-prevention effect of probiotics in human. The hypothesis was that the administration of probiotic strains might play a role in caries lesion prevention and in the control of caries-related risk factors. The main relevant databases (Medline, Embase) were searched. Quality of the Randomized Clinical Trials (RCTs) was classified using the “Consolidated Standards of Reporting Trials” (CONSORT) checklist and the Impact Factor (IF) value of each journal was recorded. Sixty-six papers were identified, and 23 fulfilled the inclusion criteria. Only three studies had caries lesion development as outcome, all the others reported caries risk factors as interim evaluation. Using the CONSORT Score, the papers were coded as 4 excellent, 9 good and 10 poor. The mean IF value recorded was 1.438. Probiotics may play a role as antagonistic agent on mutans streptococci (MS), acidogenic/aciduric bacteria that contributes to the caries process. In two-thirds of the selected papers, probiotics have demonstrated the capacity to reduce MS counts in saliva and/or plaque in short-term. The effect of probiotics on the development of caries lesion seems encouraging, but to date, RCTs on this topic are insufficient to provide scientific clinical evidence.

Risk of bias of randomized controlled trials published in orthopaedic journals

Background

The purpose of this study was to assess the quality of methodology in orthopaedics-related randomized controlled trials (RCTs) published from January 2006 to December 2010 in the top orthopaedic journals based on impact scores from the Thompson ISI citation reports (2010).

Methods

Journals included American Journal of Sports Medicine; Journal of Orthopaedic Research; Journal of Bone and Joint Surgery, American; Spine Journal; and Osteoarthritis and Cartilage. Each RCT was assessed on ten criteria (randomization method, allocation sequence concealment, participant blinding, outcome assessor blinding, outcome measurement, interventionist training, withdrawals, intent to treat analyses, clustering, and baseline characteristics) as having empirical evidence for biasing treatment effect estimates when not performed properly.

Results

A total of 232 RCTs met our inclusion criteria. The proportion of RCTs in published journals fell from 6% in 2006 to 4% in 2010. Forty-nine percent of the criteria were fulfilled across these journals, with 42% of the criteria not being amendable to assessment due to inadequate reporting. The results of our regression revealed that a more recent publication year was significantly associated with more fulfilled criteria (β = 0.171; CI = −0.00 to 0.342; p = 0.051).

Conclusion

In summary, very few studies met all ten criteria. Thus, many of these studies likely have biased estimates of treatment effects. In addition, these journals had poor reporting of important methodological aspects.

A systematic scoping review of adherence to reporting guidelines in health care literature

Background

Reporting guidelines have been available for the past 17 years since the inception of the Consolidated Standards of Reporting Trials statement in 1996. These guidelines were developed to improve the quality of reporting of studies in medical literature. Despite the widespread availability of these guidelines, the quality of reporting of medical literature remained suboptimal. In this study, we assess the current adherence practice to reporting guidelines; determine key factors associated with better adherence to these guidelines; and provide recommendations to enhance adherence to reporting guidelines for future studies.

Methods

We undertook a systematic scoping review of systematic reviews of adherence to reporting guidelines across different clinical areas and study designs. We searched four electronic databases (Cumulative Index to Nursing and Allied Health Literature, Web of Science, Embase, and Medline) from January 1996 to September 2012. Studies were included if they addressed adherence to one of the following guidelines: Consolidated Standards of Reporting Trials (CONSORT), Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), Quality of Reporting of Meta-analysis (QUOROM), Transparent Reporting of Evaluations with Nonrandomized Designs (TREND), Meta-analysis Of Observational Studies in Epidemiology (MOOSE) and Strengthening the Reporting of Observational Studies in Epidemiology (STROBE). A protocol for this study was devised. A literature search, data extraction, and quality assessment were performed independently by two authors in duplicate. This study reporting follows the PRISMA guidelines.

Results

Our search retrieved 5159 titles, of which 50 were eligible. Overall, 86.0% of studies reported suboptimal levels of adherence to reporting guidelines. Factors associated with better adherence included journal impact factor and endorsement of guidelines, publication date, funding source, multisite studies, pharmacological interventions and larger studies.

Conclusion

Reporting guidelines in the clinical literature are important to improve the standards of reporting of clinical studies; however, adherence to these guidelines remains suboptimal. Action is therefore needed to enhance the adherence to these standards. Strategies to enhance adherence include journal editorial policies endorsing these guidelines.

Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals

BACKGROUND

An overwhelming body of evidence stating that the completeness of reporting of randomised controlled trials (RCTs) is not optimal has accrued over time. In the mid-1990s, in response to these concerns, an international group of clinical trialists, statisticians, epidemiologists, and biomedical journal editors developed the CONsolidated Standards Of Reporting Trials (CONSORT) Statement. The CONSORT Statement, most recently updated in March 2010, is an evidence-based minimum set of recommendations including a checklist and flow diagram for reporting RCTs and is intended to facilitate the complete and transparent reporting of trials and aid their critical appraisal and interpretation. In 2006, a systematic review of eight studies evaluating the "effectiveness of CONSORT in improving reporting quality in journals" was published.

OBJECTIVES

To update the earlier systematic review assessing whether journal endorsement of the 1996 and 2001 CONSORT checklists influences the completeness of reporting of RCTs published in medical journals.

SEARCH METHODS

We conducted electronic searches, known item searching, and reference list scans to identify reports of evaluations assessing the completeness of reporting of RCTs. The electronic search strategy was developed in MEDLINE and tailored to EMBASE. We searched the Cochrane Methodology Register and the Cochrane Database of Systematic Reviews using the Wiley interface. We searched the Science Citation Index, Social Science Citation Index, and Arts and Humanities Citation Index through the ISI Web of Knowledge interface. We conducted all searches to identify reports published between January 2005 and March 2010, inclusive.

SELECTION CRITERIA

In addition to studies identified in the original systematic review on this topic, comparative studies evaluating the completeness of reporting of RCTs in any of the following comparison groups were eligible for inclusion in this review: 1) Completeness of reporting of RCTs published in journals that have and have not endorsed the CONSORT Statement; 2) Completeness of reporting of RCTs published in CONSORT-endorsing journals before and after endorsement; or 3) Completeness of reporting of RCTs before and after the publication of the CONSORT Statement (1996 or 2001). We used a broad definition of CONSORT endorsement that includes any of the following: (a) requirement or recommendation in journal's 'Instructions to Authors' to follow CONSORT guidelines; (b) journal editorial statement endorsing the CONSORT Statement; or (c) editorial requirement for authors to submit a CONSORT checklist and/or flow diagram with their manuscript. We contacted authors of evaluations reporting data that could be included in any comparison group(s), but not presented as such in the published report and asked them to provide additional data in order to determine eligibility of their evaluation. Evaluations were not excluded due to language of publication or validity assessment.

DATA COLLECTION AND ANALYSIS

We completed screening and data extraction using standardised electronic forms, where conflicts, reasons for exclusion, and level of agreement were all automatically and centrally managed in web-based management software, DistillerSR(®). One of two authors extracted general characteristics of included evaluations and all data were verified by a second author. Data describing completeness of reporting were extracted by one author using a pre-specified form; a 10% random sample of evaluations was verified by a second author. Any discrepancies were discussed by both authors; we made no modifications to the extracted data. Validity assessments of included evaluations were conducted by one author and independently verified by one of three authors. We resolved all conflicts by consensus.For each comparison we collected data on 27 outcomes: 22 items of the CONSORT 2001 checklist, plus four items relating to the reporting of blinding, and one item of aggregate CONSORT scores. Where reported, we extracted and qualitatively synthesised data on the methodological quality of RCTs, by scale or score.

MAIN RESULTS

Fifty-three publications reporting 50 evaluations were included. The total number of RCTs assessed within evaluations was 16,604 (median per evaluation 123 (interquartile range (IQR) 77 to 226) published in a median of six (IQR 3 to 26) journals. Characteristics of the included RCT populations were variable, resulting in heterogeneity between included evaluations. Validity assessments of included studies resulted in largely unclear judgements. The included evaluations are not RCTs and less than 8% (4/53) of the evaluations reported adjusting for potential confounding factors.   Twenty-five of 27 outcomes assessing completeness of reporting in RCTs appeared to favour CONSORT-endorsing journals over non-endorsers, of which five were statistically significant. 'Allocation concealment' resulted in the largest effect, with risk ratio (RR) 1.81 (99% confidence interval (CI) 1.25 to 2.61), suggesting that 81% more RCTs published in CONSORT-endorsing journals adequately describe allocation concealment compared to those published in non-endorsing journals. Allocation concealment was reported adequately in 45% (393/876) of RCTs in CONSORT-endorsing journals and in 22% (329/1520) of RCTs in non-endorsing journals. Other outcomes with results that were significant include: scientific rationale and background in the 'Introduction' (RR 1.07, 99% CI 1.01 to 1.14); 'sample size' (RR 1.61, 99% CI 1.13 to 2.29); method used for 'sequence generation' (RR 1.59, 99% CI 1.38 to 1.84); and an aggregate score over reported CONSORT items, 'total sum score' (standardised mean difference (SMD) 0.68 (99% CI 0.38 to 0.98)).

AUTHORS' CONCLUSIONS

Evidence has accumulated to suggest that the reporting of RCTs remains sub-optimal. This review updates a previous systematic review of eight evaluations. The findings of this review are similar to those from the original review and demonstrate that, despite the general inadequacies of reporting of RCTs, journal endorsement of the CONSORT Statement may beneficially influence the completeness of reporting of trials published in medical journals. Future prospective studies are needed to explore the influence of the CONSORT Statement dependent on the extent of editorial policies to ensure adherence to CONSORT guidance.

The quality of randomized controlled trial reporting in spine literature

STUDY DESIGN

Retrospective literature review.

OBJECTIVE

To assess the quality of randomized controlled trial reporting in spine surgery.

SUMMARY OF BACKGROUND DATA

The use of the Consolidated Standards of Reporting Trials (CONSORT) criteria in clinical trials aims to improve adherence to a set of generally accepted practices in the reporting of randomized controlled trials. Randomized controlled trials in spine surgery are important sources of evidence-based practice, but the quality of their execution and reporting have not been reported adequately.

METHODS

All randomized controlled trials published in three selected dedicated spine journals from 2008 were reviewed with respect to the 40 criteria derived from CONSORT descriptors; 10 criteria applying to the abstract, and 30 to the body of the article. Each article was scored by three reviewers in group format for each criterion and was assigned a score. Reviewers always came to a consensus before assigning a score.

RESULTS

A total of 32 RCTs met the inclusion criteria for this review. The average modified CONSORT score was 65%. Disclosing certain criteria was associated with a higher overall score, including method of generation of random sequence (P < 0.000071), allocation concealment (P < 0.00014), inclusion of a flow chart or description of patient numbers at different stages (P < 0.00034), and identification of outcome measures (P < 0.00064).

CONCLUSION

Conclusions drawn from current randomized controlled trials in the spine literature may thus be difficult to interpret. Greater effort must be put into compliance with these guidelines to improve data quality.

How useful are meta-analyses in orthopedic trauma?

BACKGROUND

Meta-analysis has become accepted as a methodically rigorous research tool, and as a result, many meta-analyses have been undertaken in orthopedic trauma, the implication being that their conclusions have improved surgeons' knowledge and facilitated improved clinical care. There have been criticisms of the methodology used in a number of meta-analyses; however, there has been no previous study of the clinical usefulness of their conclusions.

METHODS

A study of 60 orthopedic trauma meta-analyses published during or after 2000 was undertaken. We divided the conclusions of the meta-analyses into four types depending on their innovation and potential for improving clinical care. We also separated the meta-analyses according to whether they were undertaken by the Cochrane Collaboration or by independent surgeons.

RESULTS

Only 4 (6.7%) of the meta-analyses contained new, clinically useful, information, and 22 (36.7%) had no conclusions at all. A further 25 (41.7%) contained conclusions that were already in the standard orthopedic trauma literature. Approximately 70% of the Cochrane Collaborations had no conclusions.

CONCLUSIONS

We found considerable variability in the clinical usefulness of orthopedic trauma meta-analyses, and we question the clinical usefulness of this type of research.

Improvement of quality of reporting in randomised controlled trials to prevent hypotension after spinal anaesthesia for caesarean section

Hypotension is a frequent complication of spinal anaesthesia for caesarean section and can threaten the well-being of the unborn child. Numerous randomised controlled trials (RCTs) dealt with measures to prevent hypotension. The aim of this study was to determine the reporting quality of RCTs using the Consolidated Standards of Reporting Trials (CONSORT) statement since low quality can lend false credibility to a study and overestimate the effect of an intervention. We performed a systematic literature search in PubMed to identify relevant RCTs in a pre-CONSORT period (1990-1994) and a post-CONSORT period (2004-2008). A comparative evaluation was done between the two periods, and the trials were assessed for compliance with each of the 22 CONSORT items. A total of 37 RCTs was identified. The CONSORT score increased significantly (p < 0.05) from 66.7% (±12.5%) in the pre-CONSORT period to 87.4% (±6.9%) in the post-CONSORT period. A statistically significant improvement was found for eight items, including randomization, blinding and intention-to-treat analysis. The CONSORT score in the post-CONSORT era was fairly good, also in comparison to other medical fields. In the post-CONSORT era, reporting of important items improved, in particular in the domains that are crucial to avoid bias and to improve internal validity. Use of CONSORT should be encouraged in order to keep or even improve the reporting quality.