Improving the quality of reporting randomized controlled trials in cardiothoracic surgery: the way forward

Adherence to the CONSORT statement and extension for nonpharmacological treatments in randomized controlled trials of bariatric surgery: A systematic survey

Reporting is critical for establishing the value of randomized controlled trials (RCTs). This study evaluated the adherence of bariatric surgery RCT reporting to the CONsolidated Standards Of Reporting Trials (CONSORT) statement 2010 and its 2017 extension for non-pharmacologic treatments (NPT extension). We identified all RCTs comparing bariatric surgery with conservational therapy or alternative bariatric surgery up to June 30, 2020. Reporting quality was assessed using criteria developed from the CONSORT statement and the NPT extension and scored as a percentage. The factors associated with reporting quality were explored by univariate and multivariate analysis. In total, 102 RCTs of bariatric surgery were included. The median scores according to the CONSORT statement and NPT extension were 63.3 and 26.8 of a maximum possible 100, respectively. Two-thirds of NPT extension items were reported in less than 25% of the RCTs. The median score improved over time for the CONSORT statement but not the NPT extension. A higher CONSORT score was associated with publication in core clinical journals, protocol registration, and funding. No factors associated with the NPT extension score were identified. Substantial efforts are warranted from authors, journals, registration platforms, and funders to overcome the flaws in the reporting of bariatric surgery RCTs.

Intervention reporting of clinical trials published in high-impact cardiology journals: effect of the TIDieR checklist and guide

BACKGROUND

Randomised controlled trials (RCTs) provide the highest-level of evidence among primary research in cardiovascular medicine. Yet, even the best trial may be less useful if it fails to provide an accurate means of reproducibility. Unfortunately, discrepancies in the standards of trial reporting have been persistent in previous trials. The Template for Intervention Description and Replication (TIDieR) checklist aims to improve research efficacy by setting standards for quality intervention reporting and reproducibility. The goal of this study was to assess adherence to the TIDieR checklist among RCTs published in cardiovascular health journals. We also compared the quality of intervention reporting before and after the publication of TIDieR.

METHODS

This cross-sectional, methodological study analysed 101 trials published within high-impact cardiology journals. Our primary objective was to assess overall adherence to the TIDieR checklist. Our secondary objective was to use an interrupted time-series analysis to determine if intervention reporting increased following the publication of TIDieR in March 2014. Additionally, we used generalised estimating equations to identify trial characteristics associated with intervention reporting.

RESULTS

Trials in our sample reported 8.6/12 TIDieR checklist items, on average. The most under-reported items were those for describing the expertise of the interventionists and for describing the location of the intervention.

CONCLUSION

Improved outcome reporting and intervention reproducibility among RCTs are greatly needed in cardiovascular medicine. Clinicians and researchers should advocate for the ethical publication of complete, translatable and replicable clinical research results.

Challenges to randomized trials in adult and congenital cardiac and thoracic surgery

Randomized trials in surgery face additional challenges compared to those in medicine. Some of the challenges are intrinsic to the nature of the field (such as issues with blinding, learning curve and surgeons experience and difficulties in defining the appropriate timing for comparative trials). Other issues are due to the surgical culture, the attitude of surgeons toward randomized trials and the lack of support by professional and national bodies. In this review a group with experience in trials in congenital and adult cardiac and thoracic surgery discusses the key issues with surgical trials and suggest potential solutions.

Assessing the Quality of Abstracts in Randomized Controlled Trials Published in High Impact Cardiovascular Journals

BACKGROUND

In the busy world of cardiovascular medicine, abstracts may be the only part of a publication that clinicians read. Therefore, it is critical for abstracts to accurately reflect article content. The extended CONSORT (Consolidated Standards of Reporting Trials) Statement for Abstracts was developed to ensure high abstract quality. However, it is unknown how often adherence to CONSORT guidelines occurs among cardiovascular journals.

METHODS AND RESULTS

We searched MEDLINE for randomized controlled trials published in 3 major cardiovascular journals ( Circulation, Journal of the American College of Cardiology, and European Heart Journal) from 2011 to 2017. Post hoc, interim, and cost-effective analyses of randomized controlled trials were excluded. Two independent investigators extracted the data using a prespecified data collection form and a third investigator adjudicated the data. The primary outcome was frequency of subcategory adherence to CONSORT guidelines. A total of 478 abstracts were included in the analysis. Approximately half of the abstracts (53%; 255/478; 95% CI, 49%-57%) identified the article as randomized in the title. All abstracts detailed the interventions for both study groups (100%) and 81% (95% CI, 78%-85%) reported trial registration. Methodological quality reporting was relatively low: 9% (45/478; 95% CI, 6%-12%) described participant eligibility criteria with settings for data collection, 43% (204/478; 95% CI, 39%-47%) reported details of blinding, and <1% (4/478; 95% CI, 0%-2%) reported allocation concealment. Approximately 60% (301/478; 95% CI, 59%-67%) of the included abstracts provided primary outcome results while 55% (262/478; 95% CI, 51%-60%) reported harms or adverse effects.

CONCLUSIONS

There is a high prevalence of nonadherence to CONSORT guidelines among leading cardiovascular journals. Efforts by editors, authors, and reviewers should be made to increase adherence and promote transparent and unbiased presentation of study results.

Randomized Clinical Trials in Pediatric Dentistry: Application of Evidence-Based Dentistry through the CONSORT Statement

In order to include appropriate informed decisions on dental therapeutic or preventive procedures in children, Pediatric Dentists should apply the fundamentals of "Evidence-Based Dentistry" (EBD). This oral health approach assists clinicians in understanding and applying the most relevant research published on evidence in the clinical setting when treating their patients. One of the crucial steps of EBD is to critically appraise and use the primary articles about therapy or prevention, namely, Randomized Clinical Trials (RCT), the study design that best addresses the questions related with these clinical areas. The aim of the present paper was to provide the basic concepts and an example of how to critically read and understand articles on RCT studies in Pediatric Dentistry employing the CONSORT statement, a process that involves assessing the reliability of results, risk of bias (internal validity), and applicability of reported clinical findings (external validity).

Scoping review on interventions to improve adherence to reporting guidelines in health research

OBJECTIVES

The goal of this study is to identify, analyse and classify interventions to improve adherence to reporting guidelines in order to obtain a wide picture of how the problem of enhancing the completeness of reporting of biomedical literature has been tackled so far.

DESIGN

Scoping review.

SEARCH STRATEGY

We searched the MEDLINE, EMBASE and Cochrane Library databases and conducted a grey literature search for (1) studies evaluating interventions to improve adherence to reporting guidelines in health research and (2) other types of references describing interventions that have been performed or suggested but never evaluated. The characteristics and effect of the evaluated interventions were analysed. Moreover, we explored the rationale of the interventions identified and determined the existing gaps in research on the evaluation of interventions to improve adherence to reporting guidelines.

RESULTS

109 references containing 31 interventions (11 evaluated) were included. These were grouped into five categories: (1) training on the use of reporting guidelines, (2) improving understanding, (3) encouraging adherence, (4) checking adherence and providing feedback, and (5) involvement of experts. Additionally, we identified lack of evaluated interventions (1) on training on the use of reporting guidelines and improving their understanding, (2) at early stages of research and (3) after the final acceptance of the manuscript.

CONCLUSIONS

This scoping review identified a wide range of strategies to improve adherence to reporting guidelines that can be taken by different stakeholders. Additional research is needed to assess the effectiveness of many of these interventions.

Quality of reporting in randomized controlled trials of therapeutic cardiovascular medical devices

BACKGROUND

Therapeutic medical devices play an important role in the treatment of cardiovascular diseases. The reliability of the randomized controlled trial, which is the best design for assessing treatment effects, largely depends on the information found in published reports. Limited information regarding the quality of reporting about therapeutic medical devices in trials was provided.

METHOD

A cross-sectional study was conducted to assess the reporting quality of randomized controlled trials that tested the effects of therapeutic cardiovascular medical devices. The quality of reporting was assessed against a modified Consolidated Standards of Reporting Trials checklist, including 47 items from the Consolidated Standards of Reporting Trials statement and Consolidated Standards of Reporting Trials extension. We also examined the specific items regarding medical devices. Univariable and multivariable linear regressions were undertaken to explore potential factors associated with Consolidated Standards of Reporting Trials scores.

RESULT

Some 115 randomized controlled trials were identified. The mean (standard deviation) Consolidated Standards of Reporting Trials score was 20.5 (5.0). The extent of compliance with the Consolidated Standards of Reporting Trials reporting guideline differed substantially across items: 5 of the 47 items were reported adequately across trials (more than 90%), and 10 were reported adequately in less than 5% of trials. Less than 50% of the trials reported additional items related to the medical device. Multivariable regression analysis showed that trials published in general journals (coefficient 7.44, 95% confidence interval [CI]: 5.50-9.38), with larger sample sizes (coefficient 2.30, 95% CI: 0.76-3.83), and multiple-center studies (coefficient 3.14, 95% CI: 1.27-5.01) were associated with a higher quality of reporting.

CONCLUSION

The overall reporting quality in randomized controlled trials of therapeutic medical device trials is suboptimal, particularly in terms of items regarding surgeons and hospitals. We suggest that the existing Consolidated Standards of Reporting Trials and extension should be modified to be more applicable to therapeutic medical devices.

The Current Landscape of Atrial Fibrillation and Atrial Flutter Clinical Trials: A Report of 348 Studies Registered With ClinicalTrials.gov

OBJECTIVES

This analysis sought to systematically characterize trial-level patterns in atrial fibrillation/atrial flutter (AF/AFL) by using the ClinicalTrials.gov database.

BACKGROUND

Despite an abundance of clinical trials in this field, there is a lack of high-level evidence guiding management of AF/AFL.

METHODS

We queried all closed, phase II to IV interventional trials registered in the ClinicalTrials.gov database through October 2016 that enrolled patients known to have AF/AFL. Published trials were evaluated for methodological quality, using the 3-item Jadad scale (range: 0 to 5, where 5 = highest quality).

RESULTS

The initial search yielded 465 uniquely registered studies, of which 348 directly studied AF/AFL. Of those studies, 173 (50%) were published, enrolling a median of 190 patients from a median of 15 sites. The volume of published trials increased over time (7% prior to 2008 vs. 41% from 2014 to 2016; p < 0.001 for trend). Of the completed trials, 29% remain unpublished. Industry sources accounted for most funding (54%). Recurrence of AF/AFL was the most common endpoint (45%), whereas rates of primary clinical endpoints were low (13%). The mean Jadad score of published trials of pharmacological approaches (n = 112) was 4.0 ± 1.4. Of the 61 AF/AFL trials involving ablation or device therapies, 69% were randomized, 28% were single-arm studies, and patient, proceduralist, and event-ascertainment blinding was used in 16%, 4%, and 44%, respectively.

CONCLUSIONS

Contemporary trials of AF/AFL are often multicenter and modest in size. The primary study endpoint is commonly recurrence of arrhythmia, even in high-quality and late-phase trials. Although methodological quality is high in trials of pharmacologic approaches, trials of AF/AFL ablation and device therapies variably employ randomization and blinding.

An Evaluation of Reporting Guidelines and Clinical Trial Registry Requirements Among Orthopaedic Surgery Journals

BACKGROUND

The responsibility for ensuring that studies are adequately reported is primarily that of those conducting the study; however, journal policies may influence how thoroughly authors choose to report their research. The use of reporting guidelines and prospective trial registration are promising avenues for ensuring that published studies adhere to the highest methodological standards. The purpose of this study is to evaluate orthopaedic surgery journal policies regarding reporting guidelines and trial registration, and to evaluate the effects that these policies have on adherence to reporting.

METHODS

We conducted a cross-sectional survey of journal policies and "Instructions for Authors" to determine the journals' policies and guidance regarding use of reporting guidelines and study registration. We also examined whether trials published in journals referencing CONSORT (Consolidated Standards of Reporting Trials) had higher rates of compliance with publishing a CONSORT flow diagram and whether journals with trial registration policies were more likely to contain registered trials than journals without these requirements.

RESULTS

Of the 21 orthopaedic surgery journals, 6 (29%) did not mention a single guideline, and clinical trial registration was required by 11 (52%) orthopaedic surgery journals and recommended by 2 (10%). Of the 21 general medical journals, 3 (14%) did not mention a single guideline, and trial registration was required by 13 (62%) general medical journals and recommended by 5 (24%) others. Furthermore, journals that referenced CONSORT were more likely to publish trials with a CONSORT flow diagram. Journals with trial registration policies were more likely to publish registered trials.

CONCLUSIONS

Reporting guidelines and trial registration are suboptimally required or recommended by orthopaedic surgery journals. These 2 mechanisms may improve methodology and quality, and should be considered for adoption by journal editors in orthopaedic surgery.

CLINICAL RELEVANCE

Because orthopaedic surgeons rely on high-quality research to direct patient care, measures must be taken to ensure that published research is of the highest quality. The use of reporting guidelines and prospective clinical trial registration may improve the quality of orthopaedic research, thereby improving patient care.

The quality of randomised controlled trials involving surgery from the hand to the elbow

Aims

Our purpose was to determine the quality of current randomised controlled trials (RCTs) in hand surgery using standardised metrics.

Materials and Methods

Based on five-year mean impact factors, we selected the six journals that routinely publish studies of upper extremity surgery. Using a journal-specific search query, 62 RCTs met our inclusion criteria. Then three blinded reviewers used the Jadad and revised Coleman Methodology Score (RCMS) to assess the quality of the manuscripts.

Results

Based on the Jadad scale, 28 studies were of high quality and 34 were of low quality. Methodological deficiencies in poorly scoring trials included the absence of rate of enrolment, no power analysis, no description of withdrawal or dropout, and a failure to use validated outcomes assessments with an independent investigator.

Conclusion

A large number of RCTs in hand, wrist, and elbow surgery were of suboptimal quality when judged against the RCMS and Jadad scales. Even with a high level of evidence, study design and execution of RCTs should be critically assessed. Methodological deficiencies may introduce bias and lead to statistically underpowered studies. Cite this article: Bone Joint J 2017;99-B:94–9.

Association between bibliometric parameters, reporting and methodological quality of randomised controlled trials in vascular and endovascular surgery

Purpose We aimed to investigate association between bibliometric parameters, reporting and methodological quality of vascular and endovascular surgery randomised controlled trials. Methods The most recent 75 and oldest 75 randomised controlled trials published in leading journals over a 10-year period were identified. The reporting quality was analysed using the CONSORT statement, and methodological quality with the Intercollegiate Guidelines Network checklist. We used exploratory univariate and multivariable linear regression analysis to investigate associations. Findings Bibliometric parameters such as type of journal, study design reported in title, number of pages; external funding, industry sponsoring and number of citations are associated with reporting quality. Moreover, parameters such as type of journal, subject area and study design reported in title are associated with methodological quality. Conclusions The bibliometric parameters of randomised controlled trials may be independent predictors for their reporting and methodological quality. Moreover, the reporting quality of randomised controlled trials is associated with their methodological quality and vice versa.

Lessons learned on the path to a healthier brain: dispelling the myths and challenging the hypotheses

Neurologic dysfunction remains the most significant complication associated with cardiopulmonary bypass (CPB). The insidious change of cognitive decline has been perceived as a key factor that has contributed to the shift to percutaneous intervention for coronary disease. Current neuropsychologic testing provides the most sensitive means of demonstrating clinically relevant cerebral damage of this nature. Through extensive experience in randomized clinical trials of over 900 patients undergoing CPB, our team has addressed several key hypotheses related to the embolic/ischemic nature of cerebral injury in cardiac surgery, using this testing. In the first temperature study, patients randomized to hypothermia with passive re-warming had a lower incidence of neurocognitive deficit when compared with those patients who were actively re-warmed to 37°. In order to clarify the role of the hypothermia as opposed to the re-warming process, a second temperature study was completed. In the hypothermic group, patients were cooled and maintained at 34° with no active re-warming whereas, in the normothermic group, the patients were kept at 37° throughout the perioperative period. No difference in neurocognitive outcome in the two groups was seen, implying that the benefit seen in the first temperature study was related not to the hypothermia, but rather to the absence of active re-warming. In the cardiotomy study, patients were randomized to either a control group in which their cardiotomy blood was returned unprocessed, or a treatment group in which this blood was sequestered and processed with centrifugal washing and fat filtration. No significant difference in neurocognitive outcome was found in these two groups. On the other hand, there was a significant increase in bleeding and transfusion requirements in the treatment group. Many of our daily practices in CPB management are based upon assumptions from observational studies without sound reference to evidence-based medicine. Our recent studies have challenged our assumptions related to ischemia and embolic events during CPB. They have also confirmed that, when high standards in trial design are applied, the results can have universal implications in terms of our practice. Perfusion (2007) 22, 153—160.

Quality of reporting of randomised controlled trials in chiropractic using the CONSORT checklist

BACKGROUND

Reviews indicate that the quality of reporting of randomised controlled trials (RCTs) in the medical literature is less than optimal, poor to moderate, and require improving. However, the reporting quality of chiropractic RCTs is unknown. As a result, the aim of this study was to assess the reporting quality of chiropractic RCTs and identify factors associated with better reporting quality. We hypothesized that quality of reporting of RCTs was influenced by industry funding, positive findings, larger sample sizes, latter year of publication and publication in non-chiropractic journals.

METHODS

RCTs published between 2005 and 2014 were sourced from clinical trial registers, PubMed and the Cochrane Reviews. RCTs were included if they involved high-velocity, low-amplitude (HVLA) spinal and/or extremity manipulation and were conducted by a chiropractor or within a chiropractic department. Data extraction, and reviews were conducted by all authors independently. Disagreements were resolved by consensus.

OUTCOMES

a 39-point overall quality of reporting score checklist was developed based on the CONSORT 2010 and CONSORT for Non-Pharmacological Treatments statements. Four key methodological items, based on allocation concealment, blinding of participants and assessors, and use of intention-to-treat analysis (ITT) were also investigated.

RESULTS

Thirty-five RCTs were included. The overall quality of reporting score ranged between 10 and 33 (median score 26.0; IQR = 8.00). Allocation concealment, blinding of participants and assessors and ITT analysis were reported in 31 (87 %), 16 (46 %), 25 (71 %) and 21 (60 %) of the 35 RCTs respectively. Items most underreported were from the CONSORT for Non-Pharmacological Treatments statement. Multivariate regression analysis, revealed that year of publication (t32 = 5.17, p = 0.000, 95 % CI: 0.76, 1.76), and sample size (t32 = 3.01, p = 0.005, 95 % CI: 1.36, 7.02), were the only two factors associated with reporting quality.

CONCLUSION

The overall quality of reporting RCTs in chiropractic ranged from poor to excellent, improving between 2005 and 2014. This study suggests that quality of reporting, was influenced by year of publication and sample size but not journal type, funding source or outcome positivity. Reporting of some key methodological items and uptake of items from the CONSORT Extension for Non-Pharmacological Treatments items was suboptimal. Future recommendations were made.

Risk of bias and magnitude of effect in orthodontic randomized controlled trials: a meta-epidemiological review

AIM

To assess the risk of bias (RoB) in a subset of randomized controlled trials (RCTs) published in orthodontic journals using the Cochrane RoB tool and to identify associations between domain RoB assessment and treatment effect estimates.

MATERIALS AND METHODS

Fifty consecutive issues of four major orthodontic journals were electronically searched to identify RCTs. The quality of the included studies was assessed using the Cochrane RoB tool, which involves seven domains rated as 'low', 'unclear' or 'high': random sequence generation, allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, and selective outcome reporting, and other threats to internal validity. Estimates and confidence intervals (CIs) were recorded or calculated where possible for binary and continuous outcome measures. Meta-regression models were employed to assess the impact of RoB per domain on the magnitude of treatment effect.

RESULTS

One hundred and one eligible studies involving 128 pair-wise comparisons were retrieved. Blinding of outcome assessors and incomplete outcome data were frequently judged as 'high' for RoB both for studies with binary and continuous outcome (42.9 and 48.8 per cent, respectively). For binary outcomes, high RoB regarding random sequence generation [odds ratio (OR): 5.97, 95% CI: 2.03, 17.63, P-value: 0.002] and incomplete outcome data (OR: 4.07, 95% CI: 1.03, 16.15, P-value: 0.05) were more likely to provide exaggerated effect estimates.

CONCLUSIONS

There is a need for improved clinical trial methodology and reporting, in order to avoid inflated associations and erroneous conclusions.

Quality of reporting clinical trials published in five leading Sri Lankan medical journals

OBJECTIVES

To assess the quality of reporting of clinical trials published in five leading Sri Lankan medical journals.

METHODS

Six medical journals were hand searched for clinical trials published from 1982 to 2011. Eligible criteria were all randomized and non-randomized clinical trials conducted in humans. A checklist was developed based on CONSORT and TREND recommendations. Each study was independently evaluated by two reviewers. Outcome measures were presence of checklist items in published reports.

RESULTS

Fifteen randomized and 24 non-randomized parallel group trials from 724 studies in five journals met the inclusion criteria. Out of 39 trials, 39 (97%) clearly described the objectives, 16 (41%) defined the periods of recruitment, 7 (18%) reported how sample size was determined, 10 (25.6%) reported the methods to enhance the quality of measurements, 20 (51%) reported baseline demographic and clinical characteristics of each group, 4 (10%) showed flow diagrams, 23 (69%) reported statistical methods used to compare groups for primary outcomes, 21 (54%) reported effect size, 4 (10%) reported its precision, and 20 (51%) interpreted the results in the context of current evidence, and 5 (13%) described the generalizability of the findings. Of the 15 randomized trials, only one (7%) reported sequence generation, 5 (33%) allocation concealment, 9 (60%) reported blinding status of participants or investigators, and 2 (13%) reported intention to treat analysis.

CONCLUSIONS

Reporting of several essential recommendations remained suboptimal. Education and training of trial methods and awareness of the CONSORT and TREND statements and more attention to the quality of reporting may improve matters.

Reporting quality of abstracts of randomized controlled trials published in dental specialty journals

OBJECTIVES

A widespread assessment of the reporting of RCT abstracts published in dental journals is lacking. Our aim was to investigate the quality of reporting of abstracts published in leading dental specialty journals using, as a guide, the CONSORT for abstracts checklist.

METHODS

Electronic and supplementary hand searching were undertaken to identify RCTs published in seven dental specialty journals. The quality of abstract reporting was evaluated using a modified checklist based on the CONSORT for abstracts checklist. Descriptive statistics followed by univariate and multivariate analyses were conducted.

RESULTS

228 RCT abstracts were identified. Reporting of interventions, objectives and conclusions within abstracts were adequate. Inadequately reported items included: title, participants, outcomes, random number generation, numbers randomized and effect size estimate. Randomization restrictions, allocation concealment, blinding, numbers analyzed, confidence intervals, intention-to-treat analysis, harms, registration and funding were rarely described.

CONCLUSIONS

The mean overall reporting quality score was suboptimal at 62.5% (95% CI: 61.9, 63.0). Significantly better abstract reporting was noted in certain specialty journals and in multicenter trials.

Do authors report surgical expertise in open spine surgery related randomized controlled trials? A systematic review on quality of reporting

STUDY DESIGN

A systematic review of published trials in orthopedic spine literature.

OBJECTIVE

To determine the quality of reporting in open spine surgery randomized controlled trials (RCTs) between 2005 and 2010 with special focus on the reporting of surgical skill or expertise.

SUMMARY OF BACKGROUND DATA

In technically demanding procedures such as spine surgery, a surgeon's skill and expertise is expected to play an important role in the outcome of the procedure. To appraise the reported treatment effect of spine surgery related RCTs adequately, any potential skill or experience bias must be reported.

METHODS

MEDLINE, the Cochrane Library, and EMBASE were systematically searched for open spine surgery RCTs published between January 1, 2005, and December 31, 2010. Percutaneous techniques were excluded. The quality of reporting of all eligible studies was determined using the checklist to evaluate a report of a nonpharmacological trial. The reporting of surgeons' skill and experience was scored additionally. Subsequently, all authors were surveyed to determine if any information on methodological safeguards was omitted from their reports. All data were analyzed in 2-year time frames.

RESULTS

Ninety-nine RCTs were included. Ten studies (10%) described surgical skill or experience, mostly as a description of the learning curve. The majority of publications were unclear about "concealment of treatment allocation" (77%), "blinding of participants" (68%), "blinding of outcome assessors" (77%), and "adhering to the intention-to-treat principle" (67%). Of the 99 surveys, we received 22 (22%) completed questionnaires. In these questionnaires, information about essential methodological safeguards was often available, although not reported in the primary publication.

CONCLUSION

This study shows that in open spine surgery RCTs information on skill and experience is scarcely reported. Authors often fail to report essential methodological safeguards. These studies may therefore be prone to expertise bias.

Mislabeling controlled clinical trials (CCTs) as "randomized clinical trials (RCTs)" in dental specialty journals

OBJECTIVES

This study aimed to investigate whether studies published in dental journals with the highest impact factor, representing the 5 major dental specialties and titled as randomized clinical trials (RCTs) are truly RCTs. A second objective was to explore the association of journal type and other publication characteristics on correct classification.

METHODS

The American Journal of Orthodontics and Dentofacial Orthopedics (AJODO), the British Journal of Oral and Maxillofacial Surgery (BJOMS), the International Journal of Prosthodontics (IJP), the Journal of Clinical Periodontology (JCP), and the Journal of Endodontics (JOE) were hand searched for clinical trials labeled in the title as randomized from 1979 to July 2011. The data were analyzed using descriptive statistics, and univariable and multivariable examination of statistical associations via ordinal logistic regression modeling (proportional odds model).

RESULTS

A total of 222 trials were identified. From the included trials, 88 (39.64%) were considered to be RCTs, 107 (48.20%) were considered to be of unclear status, and 27 (12.16%) were not considered as RCTs. In the multivariable analysis among the included variables, journal type, involvement of statistician, year of publication, multicenter trial, and number of authors were significant predictors of correctly classifying a study as an RCT versus no RCT and of unclear status.

CONCLUSIONS

This study indicates the need for clear and accurate reporting of clinical trials and the need for educating investigators on RCT methodology.

Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals

BACKGROUND

An overwhelming body of evidence stating that the completeness of reporting of randomised controlled trials (RCTs) is not optimal has accrued over time. In the mid-1990s, in response to these concerns, an international group of clinical trialists, statisticians, epidemiologists, and biomedical journal editors developed the CONsolidated Standards Of Reporting Trials (CONSORT) Statement. The CONSORT Statement, most recently updated in March 2010, is an evidence-based minimum set of recommendations including a checklist and flow diagram for reporting RCTs and is intended to facilitate the complete and transparent reporting of trials and aid their critical appraisal and interpretation. In 2006, a systematic review of eight studies evaluating the "effectiveness of CONSORT in improving reporting quality in journals" was published.

OBJECTIVES

To update the earlier systematic review assessing whether journal endorsement of the 1996 and 2001 CONSORT checklists influences the completeness of reporting of RCTs published in medical journals.

SEARCH METHODS

We conducted electronic searches, known item searching, and reference list scans to identify reports of evaluations assessing the completeness of reporting of RCTs. The electronic search strategy was developed in MEDLINE and tailored to EMBASE. We searched the Cochrane Methodology Register and the Cochrane Database of Systematic Reviews using the Wiley interface. We searched the Science Citation Index, Social Science Citation Index, and Arts and Humanities Citation Index through the ISI Web of Knowledge interface. We conducted all searches to identify reports published between January 2005 and March 2010, inclusive.

SELECTION CRITERIA

In addition to studies identified in the original systematic review on this topic, comparative studies evaluating the completeness of reporting of RCTs in any of the following comparison groups were eligible for inclusion in this review: 1) Completeness of reporting of RCTs published in journals that have and have not endorsed the CONSORT Statement; 2) Completeness of reporting of RCTs published in CONSORT-endorsing journals before and after endorsement; or 3) Completeness of reporting of RCTs before and after the publication of the CONSORT Statement (1996 or 2001). We used a broad definition of CONSORT endorsement that includes any of the following: (a) requirement or recommendation in journal's 'Instructions to Authors' to follow CONSORT guidelines; (b) journal editorial statement endorsing the CONSORT Statement; or (c) editorial requirement for authors to submit a CONSORT checklist and/or flow diagram with their manuscript. We contacted authors of evaluations reporting data that could be included in any comparison group(s), but not presented as such in the published report and asked them to provide additional data in order to determine eligibility of their evaluation. Evaluations were not excluded due to language of publication or validity assessment.

DATA COLLECTION AND ANALYSIS

We completed screening and data extraction using standardised electronic forms, where conflicts, reasons for exclusion, and level of agreement were all automatically and centrally managed in web-based management software, DistillerSR(®). One of two authors extracted general characteristics of included evaluations and all data were verified by a second author. Data describing completeness of reporting were extracted by one author using a pre-specified form; a 10% random sample of evaluations was verified by a second author. Any discrepancies were discussed by both authors; we made no modifications to the extracted data. Validity assessments of included evaluations were conducted by one author and independently verified by one of three authors. We resolved all conflicts by consensus.For each comparison we collected data on 27 outcomes: 22 items of the CONSORT 2001 checklist, plus four items relating to the reporting of blinding, and one item of aggregate CONSORT scores. Where reported, we extracted and qualitatively synthesised data on the methodological quality of RCTs, by scale or score.

MAIN RESULTS

Fifty-three publications reporting 50 evaluations were included. The total number of RCTs assessed within evaluations was 16,604 (median per evaluation 123 (interquartile range (IQR) 77 to 226) published in a median of six (IQR 3 to 26) journals. Characteristics of the included RCT populations were variable, resulting in heterogeneity between included evaluations. Validity assessments of included studies resulted in largely unclear judgements. The included evaluations are not RCTs and less than 8% (4/53) of the evaluations reported adjusting for potential confounding factors.   Twenty-five of 27 outcomes assessing completeness of reporting in RCTs appeared to favour CONSORT-endorsing journals over non-endorsers, of which five were statistically significant. 'Allocation concealment' resulted in the largest effect, with risk ratio (RR) 1.81 (99% confidence interval (CI) 1.25 to 2.61), suggesting that 81% more RCTs published in CONSORT-endorsing journals adequately describe allocation concealment compared to those published in non-endorsing journals. Allocation concealment was reported adequately in 45% (393/876) of RCTs in CONSORT-endorsing journals and in 22% (329/1520) of RCTs in non-endorsing journals. Other outcomes with results that were significant include: scientific rationale and background in the 'Introduction' (RR 1.07, 99% CI 1.01 to 1.14); 'sample size' (RR 1.61, 99% CI 1.13 to 2.29); method used for 'sequence generation' (RR 1.59, 99% CI 1.38 to 1.84); and an aggregate score over reported CONSORT items, 'total sum score' (standardised mean difference (SMD) 0.68 (99% CI 0.38 to 0.98)).

AUTHORS' CONCLUSIONS

Evidence has accumulated to suggest that the reporting of RCTs remains sub-optimal. This review updates a previous systematic review of eight evaluations. The findings of this review are similar to those from the original review and demonstrate that, despite the general inadequacies of reporting of RCTs, journal endorsement of the CONSORT Statement may beneficially influence the completeness of reporting of trials published in medical journals. Future prospective studies are needed to explore the influence of the CONSORT Statement dependent on the extent of editorial policies to ensure adherence to CONSORT guidance.

Adherence of trials of operative intervention to the CONSORT statement extension for non-pharmacological treatments: a comparative before and after study

INTRODUCTION

Use of the Consolidated Standards of Reporting Trials (CONSORT) statement has been shown to improve the reporting of randomised controlled trials and it is endorsed by leading surgical journals. The CONSORT statement for non-pharmacological treatment (CONSORT-NPT) provides specific items to aid in the reporting of trials of operative intervention. This study compares the reporting practice of trials of operative intervention published in time periods before and after publication of the CONSORT-NPT statement.

METHODS

A 30-point checklist containing the salient CONSORT-NPT items was designed and the adherence of trials meeting the inclusion criteria determined independently by two authors.

RESULTS

There was a significant improvement of 3.95 points in the mean CONSORT-NPT score from 2004 to 2010 (95% confidence interval: 3.61-4.29, p<0.001). This related specifically to items present in the original CONSORT statement rather than to CONSORT-NPT items, which remained poorly reported in 2010. The mean CONSORT-NPT score was 17.5 (standard deviation [SD]: 4.5) for trials published in CONSORT endorsing journals compared with 15.6 (SD: 4.0) for those that did not mention endorsement of the CONSORT statement although this was not a significant difference (p=0.064).

CONCLUSIONS

Although there has been a significant improvement in the reporting of trials of operative intervention published in the surgical literature since 2004, items specific to the CONSORT-NPT extension remain underreported. Improved awareness of this important addition to the CONSORT statement throughout the surgical community and its endorsement by surgical journals will help to improve the reporting practice of trials of operative intervention.

Methodological and ethical quality of randomized controlled clinical trials in gastrointestinal surgery

BACKGROUND

The randomized controlled trial (RCT) is the gold standard tool used to evaluate therapeutic interventions. Methodological and ethical aspects should be adequately reported to enable readers to make informed and justified judgments regarding the validity of a trial and the treatment effectiveness.

OBJECTIVE

The aim of this study was to evaluate the methodological and ethical qualities of randomized clinical trials in gastrointestinal surgery and to assess the relationship between these two qualities.

STUDY DESIGN

All of the articles chosen for review reported on phase III randomized controlled gastrointestinal surgical trials were published in 12 international journals during 2006 and 2007. The eligible studies were identified, selected, and then evaluated based on a broad set of predetermined criteria. The methodological quality was evaluated using the Jadad scale, and the ethical quality was evaluated using the Berdeu score.

RESULTS

The mean Jadad score was 9.7 ± 1.78. The methodological quality was insufficient in 64 RCTs (37.4 %; Jadad score <9). The mean Berdeu score was 0.36 ± 0.08. The journal impact factor, number of randomized patients, and number of centers correlated with the outcome of the Jadad score, and the journal impact factor, industry funding, and year in which the trial began correlated with the outcome of the Berdeu score. Informed consent from patients was not obtained in 7 % (n = 12) of the RCTs, and research ethics committee approval was not mentioned in 14.6 % (n = 25) of the RCTs.

CONCLUSIONS

The reporting of gastrointestinal surgery RCTs is less than optimal. In our study, the trials of higher methodological quality were more likely to provide information about their ethical aspects. These results suggest the need for more attention to be paid to the conduct of clinical research and the reporting of ethical aspects. The appropriation of the ethical rules by surgeons involved in human clinical trials could improve the methodology and reporting of RCTs in gastrointestinal surgery.

What's in a title? An assessment of whether randomized controlled trial in a title means that it is one

INTRODUCTION

In this study, we aimed to investigate whether studies published in orthodontic journals and titled as randomized clinical trials are truly randomized clinical trials. A second objective was to explore the association of journal type and other publication characteristics on correct classification.

METHODS

American Journal of Orthodontics and Dentofacial Orthopedics, European Journal of Orthodontics, Angle Orthodontist, Journal of Orthodontics, Orthodontics and Craniofacial Research, World Journal of Orthodontics, Australian Orthodontic Journal, and Journal of Orofacial Orthopedics were hand searched for clinical trials labeled in the title as randomized from 1979 to July 2011. The data were analyzed by using descriptive statistics, and univariable and multivariable examinations of statistical associations via ordinal logistic regression modeling (proportional odds model).

RESULTS

One hundred twelve trials were identified. Of the included trials, 33 (29.5%) were randomized clinical trials, 52 (46.4%) had an unclear status, and 27 (24.1%) were not randomized clinical trials. In the multivariable analysis among the included journal types, year of publication, number of authors, multicenter trial, and involvement of statistician were significant predictors of correctly classifying a study as a randomized clinical trial vs unclear and not a randomized clinical trial.

CONCLUSIONS

From 112 clinical trials in the orthodontic literature labeled as randomized clinical trials, only 29.5% were identified as randomized clinical trials based on clear descriptions of appropriate random number generation and allocation concealment. The type of journal, involvement of a statistician, multicenter trials, greater numbers of authors, and publication year were associated with correct clinical trial classification. This study indicates the need of clear and accurate reporting of clinical trials and the need for educating investigators on randomized clinical trial methodology.

Publication guidelines need widespread adoption

OBJECTIVE

During the past two decades teams of researchers and editors have developed a variety of publishing guidelines to improve the quality of published research reports. Journals and editorial groups have adopted many of these guidelines. Whereas some guidelines are widely used, others have yet to be generally applied, thwarting attainment of consistent reporting among published research reports. The aim of this study is to describe the development and adoption of general publication guidelines for various study designs, provide examples of guidelines adapted for specific topics, and recommend next steps.

STUDY DESIGN AND SETTING

We reviewed generic guidelines for reporting research results and surveyed their use in PubMed and Science Citation Index.

RESULTS

Existing guidelines cover a broad spectrum of research designs, but there are still gaps in topics and use. Appropriate next steps include increasing use of available guidelines and their adoption among journals, educating peer reviewers on their use, and incorporating guideline use into the curriculum of medical, nursing, and public health schools.

CONCLUSION

Wider adoption of existing guidelines should result in research that is increasingly reported in a standardized, consistent manner.

Standards for reporting randomized controlled trials in neurosurgery

OBJECT

The Consolidated Standards for Reporting of Trials (CONSORT) criteria were published in 1996 to standardize the reporting and improve the quality of clinical trials. Despite having been endorsed by major medical journals and shown to improve the quality of reported trials, neurosurgical journals have yet to formally adopt these reporting criteria. The purpose of this study is to evaluate the quality and reporting of randomized controlled trials (RCTs) in neurosurgery and the factors that may affect the quality of reported trials.

METHODS

The authors evaluated all neurosurgical RCTs published in 2006 and 2007 in the principal neurosurgical journals (Journal of Neurosurgery; Neurosurgery; Surgical Neurology; Journal of Neurology, Neurosurgery, and Psychiatry; and Acta Neurochirurgica) and in 3 leading general medical journals (Journal of the American Medical Association, Lancet, and the New England Journal of Medicine). Randomized controlled trials that addressed operative decision making or the treatment of neurosurgical patients were included in this analysis. The RCT quality was evaluated using the Jadad score and the CONSORT checklist.

RESULTS

In 2006 and 2007, 27 RCTs relevant to intracranial neurosurgery were reported. Of these trials, only 59% had a Jadad score ≥ 3. The 3 major medical journals all endorsed the CONSORT guidelines, while none of the neurosurgical journals have adopted these guidelines. Randomized controlled trials published in the 3 major medical journals had a significantly higher mean CONSORT score (mean 41, range 39-44) compared with those published in neurosurgical journals (mean 26.4, range 17-38; p < 0.0001). Jadad scores were also significantly higher for the major medical journals (mean 3.42, range 2-5) than neurosurgical journals (mean 2.45, range 1-5; p = 0.05).

CONCLUSIONS

Despite the growing volume of RCTs in neurosurgery, the quality of reporting of these trials remains suboptimal, especially in the neurosurgical journals. Improved awareness of the CONSORT guidelines by journal editors, reviewers, and authors of these papers could improve the methodology and reporting of RCTs in neurosurgery.

An assessment of quality characteristics of randomised control trials published in dental journals

OBJECTIVES

The purpose of this study was to investigate the quality of reporting of randomised clinical trials (RCTs) published in dental specialty journals.

METHODS

The journals possessing the highest impact factor (2008 data) in the six major dental specialties were included in the study. The contents of the 24 most recent issues of each journal were hand-searched and research articles identified as randomised controlled trials (RCTs) were selected. Quality evaluation was performed using the modified Consolidated Standards of Reporting Trials (CONSORT) statement checklist. The data were analysed using descriptive statistics followed by univariate and multivariate examination of statistical associations (alpha=0.05).

RESULTS

Ninety-five RCTs were identified with generally suboptimal scores on quality reporting on key CONSORT areas. Significant differences were found among journals with the Journal of Clinical Periodontology achieving the highest score, followed by the American Journal of Orthodontics and Dentofacial Orthopedics. There was a positive association between quality score and number of authors, involvement of statistician/epidemiologist, and multicentre trials.

CONCLUSIONS

The quality scores of RCTs in major dental journals are considered suboptimal in key CONSORT areas. This receives critical importance considering that improved quality of RCTs is a fundamental prerequisite for improved dental care.

Quality of reporting in randomized trials published in high-quality surgical journals

BACKGROUND

Randomized controlled trials (RCTs) in surgery can provide valuable evidence of the efficacy of interventions if they are well-designed, appropriately executed, and adequately reported. Adequate reporting of methodology in surgical RCTs is known to be poor, and adverse-event reporting in surgical research is inconsistent. The Consolidated Standards of Reporting Trials (CONSORT) statement is a framework to help authors report their findings in a transparent manner. Extensions to the CONSORT statement have been published recently to address deficiencies in adverse-event reporting and in reporting of specific criteria related to nonpharmacologic treatments. The aim of this study was to assess the quality of reporting of trial methodology and adverse events in a sample of general surgical RCTs published in high-quality surgical journals using the criteria specified in the CONSORT statements.

STUDY DESIGN

We used impact factor to identify the top three ranked surgical journals in 2004. We then obtained information on all RCTs published in these journals in the 2005 calendar year. We assessed quality of reporting using Jadad score, compared the quality of RCTs from CONSORT-endorsing journals with nonendorsers, and assessed the number of RCTs adequately reporting key generic methodologic, adverse-event-related, and specific nonpharmacologic criteria.

RESULTS

Of 42 RCTs analyzed, only 40% (17 of 42) had a Jadad score > or = 3. There was no significant difference in the number of high-quality RCTs published in CONSORT-endorsing journals compared with nonendorsers (p = 0.3). The median percentage of RCTs adequately reporting generic methodologic, adverse-event-related, and specific nonpharmacologic criteria was 32.5%, 17%, and 36.5%, respectively.

CONCLUSIONS

Quality of reporting of generic methodologic, adverse-event-related, and specific nonpharmacologic criteria in surgical RCTs is poor. Increased attention to quality of reporting of surgical RCTs is required if studies are to meet published criteria.

Quality of reporting of randomized controlled trials in general endocrinology literature

CONTEXT

The reporting quality of randomized controlled trials (RCTs) is poor in general medicine and several areas of specialization but unknown in endocrinology.

OBJECTIVE

Our aim was to assess the reporting quality of RCTs in general endocrinology. A secondary objective was to identify predictors for better reporting quality.

DESIGN AND SETTING

We systematically reviewed RCTs published in three general endocrinology journals between January 2005 and December 2006.

PARTICIPANTS

We included parallel-design RCTs that addressed a question of treatment or prevention. Article selection and data abstraction were conducted by two reviewers independently, and disagreements were resolved by consensus.

MAIN OUTCOMES

There were two main outcomes: 1) a 15-point overall reporting quality score (OQS) based on the Consolidated Standards for Reporting Trials (CONSORT); and 2) a 3-point key score, based on allocation concealment, blinding, and use of intention-to-treat analysis.

RESULTS

Eighty nine RCTs were included. The median OQS was 10 (interquartile range = 2). Allocation concealment, blinding, and analysis by intention to treat were reported in 10, 20, and 16 of the 89 RCTs, respectively. A multivariable regression analysis showed that complete industrial funding [incidence rate ratio (IRR) = 1.014; 95% confidence interval (CI), 1.010-1.018], journal of publication (IRR = 1.068; 95% CI, 1.007-1.132), and sample size (IRR = 1.048; 95% CI, 1.026-1.070) were significantly associated with a slightly better OQS.

CONCLUSIONS

The quality of RCT reporting in general endocrine literature is suboptimal. We discuss our results, highlight the areas where improvements are needed, and provide some recommendations.

How good is the orthopaedic literature?

Randomized trials constitute approximately 3% of the orthopaedic literature Concerns regarding quality of the orthopaedic literature stem from a widespread notion that the overall quality of the surgical literature is in need of improvement. Limitations in surgical research arises primarily from two pervasive issues: 1) A reliance on low levels of evidence to advance surgical knowledge, and 2) Poor reporting quality among the high level surgical evidence that is available. The scarcity of randomized trials may be largely attributable to several unique challenges which make them difficult to conduct. We present characteristics of the orthopaedic literature and address the challenges of conducting randomized trials in surgery.

Trends in the methodological quality of published randomized controlled trials on antibacterial agents

WHAT IS ALREADY KNOWN ABOUT THIS SUBJECT

Randomized controlled trials (RCTs) are believed to be one of the best methods of clinical research because they can minimize systematic errors of various types. Temporal trends in the various aspects of RCTs have been studied in several medical fields (e.g. nephrology, hepatology, oncology). However, there is lack of data regarding the trends in the methodological quality of RCTs focusing on antimicrobial agents.

WHAT THIS STUDY ADDS

Several important methodological aspects of RCTs on antibacterial agents, such as description of randomization, double blinding, description of the blinding and allocation concealment, have not improved during the last 30 years. AIM To investigate the trends of the methodological quality of randomized controlled trials (RCTs) of antimicrobial agents published during the last 30 years.

METHODS

We randomly selected from the Cochrane Central Register of Controlled Trials database 70 RCTs of antibacterial agents that were published during a 30-year study period (1975-2005); specifically, we randomly selected 10 RCTs published during each of the following years: 1975, 1980, 1985, 1990, 1995, 2000 and 2005. In each of the selected RCTs, we searched for information on various methodological aspects and graded the methodological quality of the RCTs to evaluate trends for possible improvement.

RESULTS

No improvement was noted in most of the analysed methodological aspects of the RCTs during the 30-year study period. Description of randomization, double blinding, description of the blinding, and allocation concealment were rather scarce among the evaluated RCTs, without observing a trend for improvement during the study period. We noted improvement in reporting power of the study calculations, baseline data as well as in reporting the presence or not of statistical significance and the statistical cut-off of significance. In only 1/70 RCTs were all 13 of the examined methodological quality aspects met and in one more RCT 12 of them were met.

CONCLUSIONS

We did not observe considerable improvement in the quality of the reporting and methodology of RCTs on antibacterial agents during the last 30 years. The methodological quality aspects that need most improvement are those that help safeguard against various types of biases.

Scales to assess the quality of randomized controlled trials: a systematic review

BACKGROUND AND PURPOSE

The methodological quality of randomized controlled trials (RCTs) is commonly evaluated in order to assess the risk of biased estimates of treatment effects. The purpose of this systematic review was to identify scales used to evaluate the methodological quality of RCTs in health care research and summarize the content, construction, development, and psychometric properties of these scales.

METHODS

Extensive electronic database searches, along with a manual search, were performed.

RESULTS

One hundred five relevant studies were identified. They accounted for 21 scales and their modifications. The majority of scales had not been rigorously developed or tested for validity and reliability. The Jadad Scale presented the best validity and reliability evidence; however, its validity for physical therapy trials has not been supported.

DISCUSSION AND CONCLUSION

Many scales are used to evaluate the methodological quality of RCTs, but most of these scales have not been adequately developed and have not been adequately tested for validity and reliability. A valid and reliable scale for the assessment of the methodological quality of physical therapy trials needs to be developed.

Checklists to improve the quality of the orthopaedic literature

Several checklists have been developed in an effort to help journals and researchers improve the quality of reporting in research. The CONSORT statement and the CLEAR NPT evaluate randomized trials. The MOOSE and QUOROM checklists evaluate meta-analyses. The STROBE checklists assists readers in evaluating observational studies and the STARD checklist was developed for diagnostic test evaluation. The checklists presented here provide an invaluable source of guidance to authors, journal editors and readers who are seeking to prepare and evaluate reports. As evidence-based medicine continues to establish itself as the new paradigm by which medicine is practiced, the need for good reporting for all research designs must also become commonplace as opposed to the exception.

Standards of reporting of randomized controlled trials in general surgery: can we do better?

OBJECTIVE

To evaluate the quality of reporting of surgical randomized controlled trials published in surgical and general medical journals using Jadad score, allocation concealment, and adherence to CONSORT guidelines and to identify factors associated with good quality.

SUMMARY BACKGROUND DATA

Randomized controlled trials (RCTs) provide the best evidence about the relative effectiveness of different interventions. Improper methodology and reporting of RCTs can lead to erroneous conclusions about treatment effects, which may mislead decision-making in health care at all levels.

METHODS

Information was obtained on RCTs published in 6 general surgical and 4 general medical journals in the year 2003. The quality of reporting of RCTs was assessed under masked conditions using allocation concealment, Jadad score, and a CONSORT checklist devised for the purpose.

RESULTS

Of the 69 RCTs analyzed, only 37.7% had a Jadad score of > or =3, and only 13% of the trials clearly explained allocation concealment. The modified CONSORT score of surgical trials reported in medical journals was significantly higher than those reported in surgical journals (Mann-Whitney U test, P < 0.001). Overall, the modified CONSORT score was higher in studies with higher author numbers (P = 0.03), multicenter studies (P = 0.002), and studies with a declared funding source (P = 0.022).

CONCLUSION

The overall quality of reporting of surgical RCTs was suboptimal. There is a need for improving awareness of the CONSORT statement among authors, reviewers, and editors of surgical journals and better quality control measures for trial reporting and methodology.