The quality of randomized controlled trial reporting in spine literature

What is the quality of reporting in randomized controlled trials in spinal conditions

Purpose

Substandard quality across published randomized controlled trials (RCTs) is a major concern. Imperfect reporting has the potential to distort the evidence landscape and waste valuable health-care resources. In this study, we aim to assess the current quality of reporting in the field of spine using a modified version of the Consolidated Standards of Reporting Trials (CONSORT) checklist.

Materials and Methods

A list of published RCTs in the field of spine disease from January 1, 2013, to December 31, 2020, was built. Two reviewers scored the published RCTs against a modified CONSORT checklist. The mean adjusted CONSORT scores for each study, reporting category, and checklist item were calculated.

Results

The mean and median scores across all of the RCTs were 0.72 and 0.74 out of 1.00, respectively. The spectrum of scores was wide, ranging from 0.45 to 0.94. The reporting categories with the lowest score included randomization, blinding, and abstract. The items which were most under-reported included allocation sequence generation, type of randomization used, full trial protocol details, and abstract methodology. The inter-rater reliability between our reviewers was substantial (κ = 0.7, κ = 0.71).

Conclusion

Our findings correlate with only a moderate level of compliance to the CONSORT criteria on the quality of reporting for RCTs in spinal conditions. This is in line with previous reports on compliance, both within and outside the field of spinal conditions. Further continued and sustained efforts are still required to enhance the quality and consistency of RCT reporting, ultimately reducing health-care resource wastage and improving patient safety.

Is there resource wastage in the research for spinal diseases? An observational analysis of discontinuation and non-publication in randomised controlled trials

Introduction

The scale of waste in research funding systems is large and detrimental to research capacity. Both incompleteness and non-publication of Randomised Controlled Trials (RCTs) have been increasingly reported in the literature. This is a serious consequence as RCTs demand monumental amounts of healthcare resources leading to wastage. Most importantly, both under-reporting and non-publication can distort the evidence landscape and obscure rationale behind clinical decisions.

Research question

We, therefore, aimed at conducting the first systematic assessment of registered trial discontinuation and non-publication in the field of spinal disorders.

Material and methods

A list of RCTs was obtained from the U.S National Library of Medicine ClinicalTrials.gov database from January 1st, 2013, to December 31st, 2020. Two independent authors excluded all non-RCTs, trials unrelated to spinal diseases, and trials that are in or before the recruitment phase. We extracted the progress status, sources of funding, the number of centres, type of intervention, principal investigator's department affiliation, publication status, location, the reason for discontinuation, publication date, and subtopics.

Results

112 trials were included in the study. 25 (22%) trials were discontinued early, with slow recruitment being the major reason (38%). Only 56 (50%) of the trials were published in peer-reviewed journals. The publication rate amongst discontinued trials was significantly lower compared to completed trials (P ​< ​0·001). The trial discontinuation rate was much higher in trials registered in the United States (US) compared to other countries (P ​= ​0·009). Industry-sponsored studies had 11 trials (23·4%) that were discontinued whilst there was 20% of non-industry-sponsored studies that were unfinished. Only 20% of the trials were compliant with the FDA reporting requirements over the study period.

Discussion and conclusion

Nearly a quarter of all trials in spinal disorders were discontinued. Half of the trials were unpublished. There was over a third of trials that were completed but not published. These rates remain worrisome from an ethical and financial perspective. Both under-reporting and non-publication adversely affect efforts in evidence synthesis and can compromise clinical guideline development.

•

Nearly a quarter of all trials in spinal disorders were discontinued early.

•

Only half of the trials were published in peer-reviewed journals.

•

Over a third of the trials were completed but not published.

•

The rates of trial discontinuation and non-publication are worrisome from an ethical and financial perspective.

•

Both under-reporting and non-publication adversely affect efforts in evidence synthesis and can compromise clinical guideline development.

Quality of reporting of systematic reviews and meta-analyses of surgical randomized clinical trials

Background

Well designed and conducted systematic reviews are essential to clinical practice. Surgical intervention is more complex than medical intervention when considering special items related to procedures. There has been no cross‐sectional study of the reporting quality of systematic reviews of surgical randomized trials focused on special items relating to surgical interventions.

Methods

A cross‐sectional survey of systematic reviews of surgical randomized trials published in 2007 and 2017 was undertaken via a PubMed search. Quality of reporting was assessed by the PRISMA checklist, with intervention details containing 27 items. Univariable and multivariable linear regression was used to explore factors in the checklist as indicators of reporting quality.

Results

A total of 204 systematic reviews were identified. The median score for the PRISMA checklist was 22 (i.q.r. 20–24), and systematic reviews published in 2017 had a significantly higher median score than those from 2007 (22 (i.q.r. 21–24) versus 20 (17–22); P < 0·001). Among the 27 items, 15 were reported adequately and three were reported poorly (in less than 50 per cent of reports). The proportion of other items reported ranged from 54·4 to 77·9 per cent. In multivariable analysis, systematic reviews published in 2017 (coefficient 0·59, 95 per cent c.i. 0·50 to 0·69) and Cochrane reviews (coefficient 0·67, 0·55 to 0·81) were associated with better reporting.

Conclusion

The quality of reporting of systematic reviews of surgical randomized trials has improved in the past 10 years. Some information relating to specific surgical interventions is, however, still reported poorly.

Neurosurgical Randomized Controlled Trials-Distance Travelled

BACKGROUND

The evidence base for many neurosurgical procedures has been limited. We performed a comprehensive and systematic analysis of study design, quality of reporting, and trial results of neurosurgical randomized controlled trials (RCTs).

OBJECTIVE

To systematically assess the design and quality characteristics of neurosurgical RCTs.

METHODS

From January 1961 to June 2016, RCTs with >5 patients assessing any 1 neurosurgical procedure against another procedure, nonsurgical treatment, or no treatment were retrieved from MEDLINE, Scopus, and Cochrane Library.

RESULTS

The median sample size in the 401 eligible RCTs was 73 patients with a mean patient age of 49.6. Only 111 trials (27.1%) described allocation concealment, 140 (34.6%) provided power calculations, and 117 (28.9%) were adequately powered. Significant efficacy or trend for efficacy was claimed in 226 reports (56.4%), no difference between the procedures was found in 166 trials (41.4%), and significant harm was reported in 9 trials (2.2%). Trials with a larger sample size were more likely to report randomization mode, specify allocation concealment, and power calculations (all P < .001). Government funding was associated with better specification of power calculations (P = .008) and of allocation concealment (P = .026), while industry funding was associated with reporting significant efficacy (P = .02). Reporting of funding, specification of randomization mode and primary outcomes, and mention of power calculations improved significantly (all, P < .05) over time.

CONCLUSION

Several aspects of the design and reporting of RCTs on neurosurgical procedures have improved over time. Better powered and accurately reported trials are needed in neurosurgery to deliver evidence-based care and achieve optimal outcomes.

An Evaluation of Reporting Guidelines and Clinical Trial Registry Requirements Among Orthopaedic Surgery Journals

BACKGROUND

The responsibility for ensuring that studies are adequately reported is primarily that of those conducting the study; however, journal policies may influence how thoroughly authors choose to report their research. The use of reporting guidelines and prospective trial registration are promising avenues for ensuring that published studies adhere to the highest methodological standards. The purpose of this study is to evaluate orthopaedic surgery journal policies regarding reporting guidelines and trial registration, and to evaluate the effects that these policies have on adherence to reporting.

METHODS

We conducted a cross-sectional survey of journal policies and "Instructions for Authors" to determine the journals' policies and guidance regarding use of reporting guidelines and study registration. We also examined whether trials published in journals referencing CONSORT (Consolidated Standards of Reporting Trials) had higher rates of compliance with publishing a CONSORT flow diagram and whether journals with trial registration policies were more likely to contain registered trials than journals without these requirements.

RESULTS

Of the 21 orthopaedic surgery journals, 6 (29%) did not mention a single guideline, and clinical trial registration was required by 11 (52%) orthopaedic surgery journals and recommended by 2 (10%). Of the 21 general medical journals, 3 (14%) did not mention a single guideline, and trial registration was required by 13 (62%) general medical journals and recommended by 5 (24%) others. Furthermore, journals that referenced CONSORT were more likely to publish trials with a CONSORT flow diagram. Journals with trial registration policies were more likely to publish registered trials.

CONCLUSIONS

Reporting guidelines and trial registration are suboptimally required or recommended by orthopaedic surgery journals. These 2 mechanisms may improve methodology and quality, and should be considered for adoption by journal editors in orthopaedic surgery.

CLINICAL RELEVANCE

Because orthopaedic surgeons rely on high-quality research to direct patient care, measures must be taken to ensure that published research is of the highest quality. The use of reporting guidelines and prospective clinical trial registration may improve the quality of orthopaedic research, thereby improving patient care.

Interpretation of CIs in clinical trials with non-significant results: systematic review and recommendations

OBJECTIVES

Interpretation of CIs in randomised clinical trials (RCTs) with treatment effects that are not statistically significant can distinguish between results that are 'negative' (the data are not consistent with a clinically meaningful treatment effect) or 'inconclusive' (the data remain consistent with the possibility of a clinically meaningful treatment effect). This interpretation is important to ensure that potentially beneficial treatments are not prematurely abandoned in future research or clinical practice based on invalid conclusions.

DESIGN

Systematic review of RCT reports published in 2014 in Annals of Internal Medicine, New England Journal of Medicine, JAMA, JAMA Internal Medicine and The Lancet (n=247).

RESULTS

85 of 99 articles with statistically non-significant results reported CIs for the treatment effect. Only 17 of those 99 articles interpreted the CI. Of the 22 articles in which CIs indicated an inconclusive result, only four acknowledged that the study could not rule out a clinically meaningful treatment effect.

CONCLUSIONS

Interpretation of CIs is important but occurs infrequently in study reports of trials with treatment effects that are not statistically significant. Increased author interpretation of CIs could improve application of RCT results. Reporting recommendations are provided.

The quality of randomised controlled trials involving surgery from the hand to the elbow

Aims

Our purpose was to determine the quality of current randomised controlled trials (RCTs) in hand surgery using standardised metrics.

Materials and Methods

Based on five-year mean impact factors, we selected the six journals that routinely publish studies of upper extremity surgery. Using a journal-specific search query, 62 RCTs met our inclusion criteria. Then three blinded reviewers used the Jadad and revised Coleman Methodology Score (RCMS) to assess the quality of the manuscripts.

Results

Based on the Jadad scale, 28 studies were of high quality and 34 were of low quality. Methodological deficiencies in poorly scoring trials included the absence of rate of enrolment, no power analysis, no description of withdrawal or dropout, and a failure to use validated outcomes assessments with an independent investigator.

Conclusion

A large number of RCTs in hand, wrist, and elbow surgery were of suboptimal quality when judged against the RCMS and Jadad scales. Even with a high level of evidence, study design and execution of RCTs should be critically assessed. Methodological deficiencies may introduce bias and lead to statistically underpowered studies. Cite this article: Bone Joint J 2017;99-B:94–9.

Quality of reporting in surgical randomized clinical trials

BACKGROUND

RCTs testing surgical interventions can change clinical practice. The adequate reporting of surgical trials is an important issue.

METHODS

A cross-sectional survey was undertaken by searching PubMed for two-arm parallel randomized trials assessing surgical interventions published in 2003 or 2013. Quality of reporting was evaluated against the CONSORT checklist containing 29 items (standard CONSORT plus CONSORT Extension for Trials Assessing Non-Pharmacological Treatments). Univariable and multivariable linear regression was undertaken to explore factors associated with quality of reporting measured with CONSORT scores.

RESULTS

Some 120 trials were identified and included. The mean(s.d.) CONSORT score was 12·7(4·0). Trials published in 2013 achieved a higher CONSORT score than those published in 2003 (mean 14·5(3·8) versus 10·8(3·4) respectively; P < 0·001). The extent to which these trials met the requirement for CONSORT reporting differed substantially across items: four of 29 items were reported adequately across trials, and seven were reported adequately in less than 20 per cent of trials. Less than 40 per cent of the trials described the additional items related to surgical interventions and care providers (such as nursing care and anaesthetic management). In multivariable regression analysis, trials published in 2013 (coefficient 3·05, 95 per cent c.i. 1·89 to 4·20) and multicentre studies (coefficient 2·08, 0·79 to 3·37) were associated with statistically higher quality of reporting.

CONCLUSION

The quality of reporting in surgical trials has improved in the past decade. Overall quality, however, remains suboptimal, particularly in relation to details regarding surgical interventions and management.

An assessment of the quality of rotator cuff randomized controlled trials: utilizing the Jadad score and CONSORT criteria

BACKGROUND

The AAOS's Clinical Practice Guideline on "Optimizing Care of Rotator Cuff Problems" suggested a lack of high-quality data. Our purpose is to quantify the quality of randomized controlled trials of rotator cuff disorders via the Jadad score, and to apply the 2010 Consolidated Standards of Reporting Trials CONSORT Criteria to determine factors associated with high Jadad scores and areas for improvement.

METHODS

A systematic review using PRISMA guidelines was performed. Utilizing an iterative search strategy of the top 6 impact factor orthopaedic journals from 2001 to 2011, all randomized controlled studies involving rotator cuff disorders were identified and scored in a systematic, blinded fashion. Each study received a Jadad score. Adherence to CONSORT criteria was quantified and linked to the Jadad score via linear regression. Common deficiencies were described.

RESULTS

A total of 129 manuscripts were identified; 54 met inclusion criteria: total patients n = 4099; mean patients per article = 76; range, 16-660. The mean Jadad score was 3.0. Sixty-six percent (35/53) of studies were high quality (high quality: >3). Among these, the majority (63%, 22/35) were nonoperative trials. Adherence to CONSORT Criteria was associated with higher Jadad scores (R(2) = 0.3). The most common deficient CONSORT Criteria were: trial design descriptions (66%; 36/54 studies), descriptions of randomization type (65%; 35/54), and power analysis (46%; 25/54).

CONCLUSION

The majority of randomized controlled trials of rotator cuff pathology are high-quality studies based on the Jadad score. Adherence to CONSORT criteria is linked to high-quality scores. Future studies should use full CONSORT Criteria.

Standards for reporting randomized controlled trials in medical informatics: a systematic review of CONSORT adherence in RCTs on clinical decision support

INTRODUCTION

The Consolidated Standards for Reporting Trials (CONSORT) were published to standardize reporting and improve the quality of clinical trials. The objective of this study is to assess CONSORT adherence in randomized clinical trials (RCT) of disease specific clinical decision support (CDS).

METHODS

A systematic search was conducted of the Medline, EMBASE, and Cochrane databases. RCTs on CDS were assessed against CONSORT guidelines and the Jadad score.

RESULT

32 of 3784 papers identified in the primary search were included in the final review. 181 702 patients and 7315 physicians participated in the selected trials. Most trials were performed in primary care (22), including 897 general practitioner offices. RCTs assessing CDS for asthma (4), diabetes (4), and hyperlipidemia (3) were the most common. Thirteen CDS systems (40%) were implemented in electronic medical records, and 14 (43%) provided automatic alerts. CONSORT and Jadad scores were generally low; the mean CONSORT score was 30.75 (95% CI 27.0 to 34.5), median score 32, range 21-38. Fourteen trials (43%) did not clearly define the study objective, and 11 studies (34%) did not include a sample size calculation. Outcome measures were adequately identified and defined in 23 (71%) trials; adverse events or side effects were not reported in 20 trials (62%). Thirteen trials (40%) were of superior quality according to the Jadad score (≥3 points). Six trials (18%) reported on long-term implementation of CDS.

CONCLUSION

The overall quality of reporting RCTs was low. There is a need to develop standards for reporting RCTs in medical informatics.