Assessment of adherence to the CONSORT statement for quality of reports on randomized controlled trial abstracts from four high-impact general medical journals

The reporting quality and spin of randomized controlled trials of endometriosis pain: Methodological study based on CONSORT extension on abstracts

OBJECTIVE

To assess the reporting quality of published RCT abstracts regarding patients with endometriosis pelvic pain and investigate the prevalence and characteristics of spin in these abstracts.

METHODS

PubMed and Scopus were searched for RCT abstracts addressing endometriosis pelvic pain published from January 1st, 2010 to December 1st, 2023.The reporting quality of RCT abstracts was assessed using the CONSORT statement for abstracts. Additionally, spin was evaluated in the results and conclusions section of the abstracts, defined as the misleading reporting of study findings to emphasize the perceived benefits of an intervention or to confound readers from statistically non-significant results. Assessing factors affecting the reporting quality and spin existence, linear and logistic regression was used, respectively.

RESULTS

A total of 47 RCT abstracts were included. Out of 16 checklist items, only three items including objective, intervention and conclusions were sufficiently reported in the most abstracts (more than 95%), and none of the abstracts presented precise data as required by the CONSORT-A guidelines. In the reporting quality of material and method section, trial design, type of randomization, the generation of random allocation sequences, the allocation concealment and blinding were most items identified that were suboptimal. The total score for the quality varied between 5 and 15 (mean: 9.59, SD: 3.03, median: 9, IQR: 5). Word count (beta = 0.015, p-value = 0.005) and publishing in open-accessed journals (beta = 2.023, p-value = 0.023) were the significant factors that affecting the reporting quality. Evaluating spin within each included paper, we found that 18 (51.43%) papers had statistically non-significant results. From these studies, 12 (66.66%) had spin in both results and conclusion sections. Furthermore, the spin intensity increased during 2010-2023 and 38.29% of abstracts had spin in both results and conclusion sections.

CONCLUSION

Overall poor adherence to CONSORT-A was observed, with spin detected in several RCTs featuring non-significant primary endpoints in obstetrics and gynecology literature.

The efficacy and safety of soluble guanylate cyclase modulation in patients with heart failure: a comprehensive meta-analysis of randomized controlled trials

Soluble guanylate cyclase (sGC) modulation has been scrutinized in several disease states including heart failure (HF). Recently, it was shown that an sGC modulator improved HF-related hospitalization significantly, though, there was no benefit related to mortality. Herein, a comprehensive meta-analysis of randomized controlled trials (RCTs) for sGC modulation in HF patients was provided in agreement with the PRISMA statement. A total of 10 RCTs yielding 12 papers were included. There were 7526 patients with heart failure of each phenotype, 4253 in the sGC modulator group and 3273 in the placebo group. Use of sGC modulators in HF patients yielded no significant difference in the risk of all-cause mortality compared to placebo (RR = 0.97, 95% CI 0.88-1.08, p = 0.62). The use of sGC modulators was associated with a trend toward a considerable but non-significant increase in the incidence of SAEs (RR = 1.10, 95% CI 0.99-1.22, p = 0.07), as well as an increased incidence of hypotension and anemia. There was an overall neutral effect of sGC modulation on NT-proBNP levels, 6MWD and mortality, at a cost of slight increase in hypotension and anemia. Of note, the improvement in EQ-5D-based quality of life was significant. Hence, the benefit seems to be driven by distinctive domains of quality of life.

Quality of Reporting Randomized Controlled Trials Published in Three of the Most Citable Periodontal Journals from 2018 to 2022

This study aimed to evaluate the reporting quality of randomized clinical trials (RCTs) in periodontology. Three leading periodontology journals, the Journal of Periodontology (JOP), the Journal of Clinical Periodontology (JOCP), and the Journal of Periodontal Research (JOPR), were selected for this investigation. The RCTs were identified by manually searching for human trial articles published in these three journals. Two authors independently conducted the literature search, and a pre-piloted extraction sheet was used to screen the potential RCTs. The CONSORT checklist guidelines were employed to calculate the score value. Intra-examiner reliability was assessed by scoring a random sample of 10% of the papers in a second round conducted by the first examiner three months after the initial data collection. A search of abstracts published over a five-year period yielded 176 articles that reported RCTs, accounting for 11.7% of all articles published in the three journals. The highest number of RCTs was published in 2020, and more than half of the included RCTs (51%) originated from Europe. Many of the analyzed RCTs inadequately reported almost half of the items on the CONSORT checklist. Furthermore, univariate analysis revealed significant associations between certain factors and the overall CONSORT score, such as publication in JOP (p = 0.048), publication year of 2019 (p = 0.041) and 2021 (p = 0.042), first author from North America (p = 0.016), and RCTs with more than six authors (p = 0.042). Clinical trial research in periodontics has made significant progress in the past five years. However, there is room for improvement in adhering to the CONSORT guidelines.

Abstracts of randomized controlled trials in pediatric dentistry: reporting quality and spin

BACKGROUND

Abstracts provide readers a concise and readily accessible information of the trials. However, poor reporting quality and spin (misrepresentation of research findings) can lead to an overestimation in trial validity. This methodological study aimed to assess the reporting quality and spin among randomized controlled trial (RCT) abstracts in pediatric dentistry.

METHODS

We hand-searched RCTs in five leading pediatric dental journals between 2015 and 2021. Reporting quality in each abstract was assessed using the original 16-item CONSORT for abstracts checklist. Linear regression analyses were performed to identify factors associated with reporting quality. We evaluated the presence and characteristics of spin only in abstracts of parallel-group RCTs with nonsignificant primary outcomes according to pre-determined spin strategies.

RESULTS

One hundred eighty-two abstracts were included in reporting quality evaluation. The mean overall quality score was 4.57 (SD, 0.103; 95% CI, 4.36-4.77; score range, 1-10). Only interventions, objective, and conclusions were adequately reported. Use of flow diagram (P < 0.001) was the only significant factor of higher reporting quality. Of the 51 RCT abstracts included for spin analysis, spin was identified in 40 abstracts (78.4%), among which 23 abstracts (45.1%) had spin in the Results section and 39 in the Conclusions Sect. (76.5%).

CONCLUSIONS

The reporting quality of RCT abstracts in pediatric dentistry is suboptimal and the prevalence of spin is high. Joint efforts are needed to improve reporting quality and minimize spin.

Methodology reporting improved over time in 176,469 randomized controlled trials

OBJECTIVES

To describe randomized controlled trial (RCT) methodology reporting over time.

STUDY DESIGN AND SETTING

We used a deep learning-based sentence classification model based on the Consolidated Standards of Reporting Trials (CONSORT) statement, considered minimum requirements for reporting RCTs. We included 176,469 RCT reports published between 1966 and 2018. We analyzed the reporting trends over 5-year time periods, grouping trials from 1966 to 1990 in a single stratum. We also explored the effect of journal impact factor (JIF) and medical discipline.

RESULTS

Population, Intervention, Comparator, Outcome (PICO) items were commonly reported during each period, and reporting increased over time (e.g., interventions: 79.1% during 1966-1990 to 87.5% during 2010-2018). Reporting of some methods information has increased, although there is room for improvement (e.g., sequence generation: 10.8-41.8%). Some items are reported infrequently (e.g., allocation concealment: 5.1-19.3%). The number of items reported and JIF are weakly correlated (Pearson's r (162,702) = 0.16, P < 0.001). The differences in the proportion of items reported between disciplines are small (<10%).

CONCLUSION

Our analysis provides large-scale quantitative support for the hypothesis that RCT methodology reporting has improved over time. Extending these models to all CONSORT items could facilitate compliance checking during manuscript authoring and peer review, and support metaresearch.

Assessment of the Adherence of Radiologists in Reporting the Ovarian Cysts to the 2010 Society of Radiologists in Ultrasound Guidelines

Background

The objective of this study was to determine the adherence of radiologists to the guideline of the Society of Radiologists in Ultrasound (SRU)-2010 for the follow-up of ovarian cysts in patients during 2015-2016.

Methods

The patients' data, referring for transvaginal and pelvic ultrasonography, suffering from ovarian cyst were assessed in terms of menopause status, cyst size, and type, as well as follow-ups recommended by radiologist to assess the adherence of reports to SRU-2010.

Results

Three hundred and sixty-four sonography reports were investigated. Seventy-seven percent of the reports had adhered to SRU-2010, 9.9% and 9.1% had under/overmanagement, and 4.1% was incomplete. 94.2% and 5.8% of cases were in pre/postmenopause status, respectively. The highest adherence belonged to cysts in size <1 cm, 1-3 cm, 5-7 cm. The highest adherence, over/undermanagement, and incomplete reports belonged to corpus luteum, hemorrhagic, dermoid cysts, and nodules without flow. The adherence of sonography reports to SRU-2010 for accidental ovarian cysts was 76.9%.

Conclusion

The tendency for overmanagement of simple cysts in premenopausal women and the tendency for undermanagement in simple cysts and in postmenopausal women were higher, respectively. It is expected that more training of the guideline to radiologists will lead to the reduction of unnecessary follow-up, which in turn leads to reduced patient's anxiety and cost of treatment.

Reporting quality of abstracts from randomised controlled trials published in leading critical care nursing journals: a methodological quality review

OBJECTIVES

To evaluate the methodological quality of randomised controlled trial (RCT) abstracts in leading critical care nursing journals against the Consolidated Standards of Reporting Trials-Abstracts (CONSORT-A) checklist and to identify variables related with abstract reporting quality.

DESIGN

Descriptive methodological quality review.

DATA SOURCE

We searched the PubMed database and the websites of each included journal.

ELIGIBILITY CRITERIA

We included RCT abstracts published between 2011 and 2021 in the first 11 Scopus-ranking (2021) critical care nursing journals that reported the results of RCTs in English that referred to the care of adult patients with acute/critical illness or their relatives and/or caregivers and conducted in ICUs.

DATA EXTRACTION AND SYNTHESIS

Two independent investigators extracted the data using a prespecified 17-item checklist directly derived from CONSORT-A. For the primary outcome, each item was evaluated whether it was adequately reported or not and descriptive statistics were reported. An overall score was calculated by summing the results of all items and multivariate linear regression was conducted to detect potential predictors.

RESULTS

Seventy-eight RCT abstracts were included in this review. The items with the highest CONSORT-A adherence were authors, objective, conclusion, participants, interventions and outcome. The randomisation item had the lowest CONSORT-A adherence, followed by trial registration, funding source, harms or side effects, recruitment, blinding and outcome results. The average CONSORT-A score was 8.5±1.5 points (of the maximum 17.5 points). Multivariate analysis indicated that the categorised word score and publication date were positively associated with the overall CONSORT-A score, while the first author's country being in Asia was negatively associated with the overall CONSORT-A score.

CONCLUSIONS

The 78 RCT abstracts showed poor overall adherence to CONSORT-A. The results indicate that the methodological reporting quality of RCT abstracts in critical care nursing journals requires improvement to facilitate assessment of the applicability and relevance of the results reported.

'Spin' among abstracts of randomised controlled trials in sleep medicine: A research-on-research study

STUDY OBJECTIVES

'Spin', using reporting strategies to distort study results, can mislead readers of medical research. This study aimed to evaluate the prevalence and characteristics of 'spin' among randomised controlled trial (RCT) abstracts published in sleep medicine journals, and to identify factors associated with its presence and severity.

METHODS

The search for RCTs published between 2010 and 2020 were conducted in seven reputable journals of sleep medicine. Abstracts of RCTs with statistically nonsignificant primary outcomes were included and analysed for 'spin', according to pre-determined 'spin' strategies. Chi-square tests or logistic regression analyses were performed to detect the association between characteristics of included abstracts and the presence and severity of 'spin'.

RESULTS

A total of 114 RCT abstracts were included in this study, of which 89 (78.1%) were identified as having at least one type of 'spin' strategy. Sixty-six abstracts (57.9%) had 'spin' in the Results section, 82 (71.9%) abstracts presented with 'spin' in the Conclusions section. The presence of 'spin' varied significantly among RCTs based on the different categories of research area (P=0.047) and the statistician involvement (P=0.045). Furthermore, research area (P=0.019) and funding status (P=0.033) were significant factors associated with the severity of 'spin'.

CONCLUSIONS

The prevalence of 'spin' is high among RCT abstracts in sleep medicine. This raise the need for researchers, editors and other stakeholders to be aware of the issue of 'spin' and make joint efforts to eliminate it in future publications.

Reporting quality was suboptimal in a systematic review of randomized controlled trials with adaptive designs

OBJECTIVE

The study was conducted to evaluate the reporting quality of randomized controlled trials (RCTs) that use an adaptive design (AD) based on the 2020 AD Consolidated Standards for Reporting Trials 2010 extension (ACE) guidelines and identify factors associated with better reporting quality.

STUDY DESIGN AND SETTING

PubMed, Embase, Cochrane, Web of Science, and Google Scholar were searched until November 1, 2022. Multivariable linear regression analysis was performed to investigate potential predictors.

RESULTS

In total, 109 RCTs were included in our study. The mean compliance rate for the ACE checklist items was 69.75% ± 16.02. Key methodological items including allocation concealment and its implementations were poorly reported. There was also suboptimal reporting of checklist items related to the conduct of interim analyzes. Multivariable regression analysis showed better reporting quality with trial registration, nonindustry affiliation (first author), a sample size of >100, general medical journal type, publication date (≥2020), funding, and disclosure of the number of interim analyzes.

CONCLUSION

Our study showed that RCTs with AD had suboptimal reporting of 2020 ACE checklist items, particularly AD-specific items. Following the development of ACE guidelines, stricter adherence to the ACE guideline is necessary to improve their reporting quality. Pre-ACE and post-ACE adherence comparisons can be conducted in the future.

Abstracts of randomized controlled trials in the field of dentofacial trauma: Reporting quality and spin

BACKGROUND/AIM

Abstracts of randomized controlled trials (RCTs) provide a summary of the entire trial report. Their transparent, detailed, and accurate reporting is essential for clinical decision-making and evidence-based dental practice. The aim of this study was to assess the reporting quality and prevalence of spin in abstracts of RCTs in the field of dentofacial trauma.

MATERIALS AND METHODS

The PubMed database was searched to identify RCT reports published between 2017 and 2021 in the field of dentofacial trauma. The reporting quality of abstracts was assessed according to the 16-item CONSORT for Abstracts checklist and measured with an overall quality score (OQS, range: 0-16). Linear regression analyses were used to identify factors associated with reporting quality. Among the included RCTs, parallel-group RCTs with non-significant primary outcomes were selected for spin assessment using predefined spin strategies.

RESULTS

One hundred and twelve eligible abstracts were identified and included. The mean OQS was 4.51 (SD, 1.35; 95% CI, 4.26-4.76). Abstracts with more than 250 words (p = .004) and a structured format (p = .032) had significantly better reporting quality. Of the 30 abstracts that were eligible for spin assessment, spin was identified in 23 (76.7%). Among these, spin was observed in the Conclusions sections of 22 abstracts (73.3%) and the Results sections of 9 abstracts (30.0%).

CONCLUSIONS

Among RCT abstracts in the field of dentofacial trauma, the reporting quality was sub-optimal and the prevalence of spin was relatively high. Strict adherence to the CONSORT for Abstracts guidelines is needed to ensure complete and transparent reporting. Relevant stakeholders need to make concerted efforts to avoid spin.

The reporting standards of randomised controlled trials in leading medical journals between 2019 and 2020: a systematic review

Randomised controlled trials (RCTs) are the gold standard study design used to evaluate the safety and effectiveness of healthcare interventions. The reporting quality of RCTs is of fundamental importance for readers to appropriately analyse and understand the design and results of studies which are often labelled as practice changing papers. The aim of this article is to assess the reporting standards of a representative sample of randomised controlled trials (RCTs) published between 2019 and 2020 in four of the highest impact factor general medical journals. A systematic review of the electronic database Medline was conducted. Eligible RCTs included those published in the New England Journal of Medicine, Lancet, Journal of the American Medical Association, and British Medical Journal between January 1, 2019, and June 9, 2020. The study protocol was registered on medRxiv ( https://doi.org/10.1101/2020.07.06.20147074 ). Of a total eligible sample of 497 studies, 50 full-text RCTs were reviewed against the CONSORT 2010 statement and relevant extensions where necessary. The mean adherence to the CONSORT checklist was 90% (SD 9%). There were specific items on the CONSORT checklist which had recurring suboptimal adherence, including in title (item 1a, 70% adherence), randomisation (items 9 and 10, 56% and 30% adherence) and outcomes and estimation (item 17b, 62% adherence). Amongst a sample of RCTs published in four of the highest impact factor general medical journals, there was good overall adherence to the CONSORT 2010 statement. However there remains significant room for improvement in areas such as description of allocation concealment and implementation of randomisation.

Reporting quality for abstracts of randomised trials on child and adolescent depression prevention: a meta-epidemiological study on adherence to CONSORT for abstracts

OBJECTIVES

This study aimed to investigate adherence to Consolidated Standards of Reporting Trials (CONSORT) for abstracts in reports of randomised trials on child and adolescent depression prevention. Secondary objective was to examine factors associated with overall reporting quality.

DESIGN

Meta-epidemiological study.

DATA SOURCES

We searched MEDLINE, EMBASE, PsycINFO, PsycArticles and CENTRAL.

ELIGIBILITY CRITERIA

Trials were eligible if the sample consisted of children and adolescents under 18 years with or without an increased risk for depression or subthreshold depression. We included reports published from 1 January 2003 to 8 August 2020 on randomised controlled trials (RCTs) and cluster randomised trials (CRTs) assessing universal, selective and indicated interventions aiming to prevent the onset of depression or reducing depressive symptoms.

DATA EXTRACTION AND SYNTHESIS

As the primary outcome measure, we assessed for each trial abstract whether information recommended by CONSORT was adequately reported, inadequately reported or not reported. Moreover, we calculated a summative score of overall reporting quality and analysed associations with trial and journal characteristics.

RESULTS

We identified 169 eligible studies, 103 (61%) RCTs and 66 (39%) CRTs. Adequate reporting varied considerably across CONSORT items: while 9 out of 10 abstracts adequately reported the study objective, no abstract adequately provided information on blinding. Important adverse events or side effects were only adequately reported in one out of 169 abstracts. Summative scores for the abstracts' overall reporting quality ranged from 17% to 83%, with a median of 40%. Scores were associated with the number of authors, abstract word count, journal impact factor, year of publication and abstract structure.

CONCLUSIONS

Reporting quality for abstracts of trials on child and adolescent depression prevention is suboptimal. To help health professionals make informed judgements, efforts for improving adherence to reporting guidelines for abstracts are needed.

Clinical observation of Long chiropractic treatment on patients with neurogenic cervical spondylosis: Study protocol for a randomized controlled trial

Neurogenic cervical spondylosis is the most common type of cervical spondylosis, accounting for approximately 60% percent of the incidence of cervical spondylosis. Cervical spine Long manipulation and sling exercise training (SET) have obtained good therapeutic results in clinical rehabilitation. The aim of this study was to evaluate the effect of Long manipulation combined with SET on neurogenic cervical spondylosis. In this assessor-blind, randomized controlled trial, 90 eligible patients will be randomized into a combination treatment group (Long manipulation combined with SET), a Long manipulation group and a conventional massage group. The visual analogue score, the Neck Disability Index score, and muscle fatigue in the bilateral upper oblique and Musculus sternocleidomastoideus, using mean power frequency and median frequency from the surface electromyography frequency domain index, will be assessed before and after the intervention at 0 and 4 weeks, respectively.Trial registration: Registered in the Chinese Clinical Trial Registration Center with the number ChiCTR2100054978. Registered December 30, 2021.

Blinding and Its Quality in Clinical Trials Conducted on Patients with Breast Cancer: A Systematic Review

Background

Blinding is one of the critical criteria of clinical trials that prevents probable bias. Judgment regarding results of an intervention significantly depends on the quality of such studies, one of which is blinding. This study aimed to investigate blinding and its quality in clinical trials in patients with breast cancer.

Materials and Methods

A systematic review was conducted on the online databases of PubMed, ScienceDirect and ProQuest using keywords, MeSH terms and grey literature. Articles were screened by predefined inclusion and exclusion criteria. They were evaluated based on the checklists introduced by Cochrane database.

Results

From 22519 articles obtained at the initial stage, 20 articles remained after screening for the inclusion and exclusion criteria. Fifteen articles had used single, five: double and none had used triple or quadruple blinding. Seventeen studies had described the details of blinding. Of the 15 single blind articles, the blinded subjects were patients in five, patients and research assistants in three, research assistants in five studies, and two had not given any details.

Conclusions

The majority of researchers had used the single blind method, though using double, triple or quadruple blinding increases the trustworthiness of results and increases the quality of clinical trials. The details of blinding should be explained to other researchers and for a better understanding of the method if it is to be repeated. Thereafter, nurses can apply new interventions and earn their patients' trust and help those with breast cancer by relieving them of their disease symptoms and its treatment complications.

Evaluation of quality of abstracts of randomized controlled trials on procedural sedation in children after publication of CONSORT guidelines for abstracts: A systematic review

The extension of the Consolidated Standards of Reporting Trials (CONSORT) statement provides guidelines for abstracts of randomized controlled trials (RCTs). This study was done to assess the reporting quality of abstracts of RCTs, on procedural sedation in children and identify factors associated with better quality. A PubMed search was conducted from inception of database till July 2017 to identify RCTs on procedural sedation in children. Search terms used were (procedural [All Fields] AND sedation [All Fields]) AND ("child" [MeSH Terms] OR "child" [All Fields] OR "children" [All Fields]) were included in the analysis, while primary RCTs, published in the English language unstructured abstracts, secondary analysis of primary RCTs and studies not exclusively on children we excluded. Our search strategy initially yielded 582 abstracts. Out of these, 535 abstracts were excluded. 47 articles were included in the final analysis. We extracted basic information and data on CONSORT items from abstracts. Each abstract was assessed using a 16-item composite abstract score (CAS) based on the CONSORT guidelines. This abstract quality was further explored by Method Score and by Result Score. Regression analysis was conducted to analyze factors associated with reporting quality. In majority of the abstracts, only objectives and conclusion were adequately reported. Inadequately reported items in >90% of abstracts included randomization, trial status, registration & funding. There was no significant difference in the CAS of abstracts (mean ± SD) published in & before 2008 (12.63 ± 4.0), to those published after 2009 (12.48 ± 4.23). Similarly, there was no significant difference in Result Score and Method Score of the abstracts. After the publication of 'CONSORT for abstracts' guideline, the quality of abstracts of RCTs on procedural sedation has shown suboptimal improvement. We suggest stricter adherence to guidelines by editors and reviewers. A checklist for adherence to CONSORT guidelines could be introduced during submission for the same.

Evaluation of reporting quality of randomized controlled trials in patients with COVID-19 using the CONSORT statement

OBJECTIVE

To evaluate the reporting quality of randomized controlled trials (RCTs) regarding patients with COVID-19 and analyse the influence factors.

METHODS

PubMed, Embase, Web of Science and the Cochrane Library databases were searched to collect RCTs regarding patients with COVID-19. The retrieval time was from the inception to December 1, 2020. The CONSORT 2010 statement was used to evaluate the overall reporting quality of these RCTs.

RESULTS

53 RCTs were included. The study showed that the average reporting rate for 37 items in CONSORT checklist was 53.85% with mean overall adherence score of 13.02±3.546 (ranged: 7 to 22). The multivariate linear regression analysis showed the overall adherence score to the CONSORT guideline was associated with journal impact factor (P = 0.006), and endorsement of CONSORT statement (P = 0.014).

CONCLUSION

Although many RCTs of COVID-19 have been published in different journals, the overall reporting quality of these articles was suboptimal, it can not provide valid evidence for clinical decision-making and systematic reviews. Therefore, more journals should endorse the CONSORT statement, authors should strictly follow the relevant provisions of the CONSORT guideline when reporting articles. Future RCTs should particularly focus on improvement of detailed reporting in allocation concealment, blinding and estimation of sample size.

Assessment of reporting quality in randomised controlled clinical trial abstracts of dental implantology published from 2014 to 2016

OBJECTIVES

Access to full texts of randomised controlled clinical trials (RCTs) is often limited, so brief summaries of studies play a pivotal role. In 2008, a checklist was provided to ensure the transparency and completeness of abstracts. The aim of this investigation was to estimate adherence to the reporting guidelines of the Consolidated Standards of Reporting Trials (CONSORT) criteria for abstracts (CONSORT-A) in RCT publications.

PRIMARY ENDPOINT

Assessment according to the percentage of compliance with the 16 CONSORT-A criteria per study.

MATERIALS AND METHODS

This study is based on a full survey (212 RCT abstracts in dental implantology, PubMed search, publication period 2014-2016, 45 journals, median impact factor: 2.328). In addition to merely documenting 'adherence' to criteria, the authors also assessed the 'complete implementation' of the requested information where possible. The collection of data was performed independently by two dentists, and a final consensus was reached. The primary endpoint was evaluated by medians and quartiles. Additionally, a Poisson regression was conducted to detect influencing factors.

RESULTS

A median of 50% (Q1-Q3: 44%-63%) was documented for the 16 criteria listed in the CONSORT-A statement. Nine of the 16 criteria were considered in fewer than 50% of the abstracts. 'Correct implementation' was achieved for a median of 43% (Q1-Q3: 31%-50%) of the criteria. An additional application of Poisson regression revealed that the number of words used had a locally significant impact on the number of reported CONSORT criteria for abstracts (incidence rate ratio 1.001, 95% CI 1.001 to 1.002).

CONCLUSION

Transparent and complete reporting in abstracts appears problematic. A limited word count seems to result in a reduction in necessary information. As current scientific knowledge is often not readily available in the form of publications, abstracts constitute the primary basis for decision making in clinical practice and research. This is why journals should refrain from limiting the number of words too strictly in order to facilitate comprehensive reporting in abstracts.

Intervention reporting of clinical trials published in high-impact cardiology journals: effect of the TIDieR checklist and guide

BACKGROUND

Randomised controlled trials (RCTs) provide the highest-level of evidence among primary research in cardiovascular medicine. Yet, even the best trial may be less useful if it fails to provide an accurate means of reproducibility. Unfortunately, discrepancies in the standards of trial reporting have been persistent in previous trials. The Template for Intervention Description and Replication (TIDieR) checklist aims to improve research efficacy by setting standards for quality intervention reporting and reproducibility. The goal of this study was to assess adherence to the TIDieR checklist among RCTs published in cardiovascular health journals. We also compared the quality of intervention reporting before and after the publication of TIDieR.

METHODS

This cross-sectional, methodological study analysed 101 trials published within high-impact cardiology journals. Our primary objective was to assess overall adherence to the TIDieR checklist. Our secondary objective was to use an interrupted time-series analysis to determine if intervention reporting increased following the publication of TIDieR in March 2014. Additionally, we used generalised estimating equations to identify trial characteristics associated with intervention reporting.

RESULTS

Trials in our sample reported 8.6/12 TIDieR checklist items, on average. The most under-reported items were those for describing the expertise of the interventionists and for describing the location of the intervention.

CONCLUSION

Improved outcome reporting and intervention reproducibility among RCTs are greatly needed in cardiovascular medicine. Clinicians and researchers should advocate for the ethical publication of complete, translatable and replicable clinical research results.

Intervention Reporting of Published Trials Is Insufficient in Orthopaedic Surgery Journals: Application of the Template for Intervention Description and Replication Checklist

PURPOSE

Outcomes of randomized controlled trials (RCTs) have been shown to influence clinical decision making. Thus, the quality and reliability of these outcomes are essential for both patients and medical care providers. To date, no study has assessed the quality of intervention reporting of RCTs in orthopaedics. The aim of this study was to evaluate the quality of intervention reporting of published RCTs in the field of orthopaedics using the Template for Intervention Description and Replication (TIDieR) checklist.

METHODS

In this cross-sectional analysis, we applied the TIDieR checklist to assess the quality of intervention reporting in orthopaedic RCTs. Additionally, we evaluated the TIDieR checklist's influence on intervention reporting by comparing overall adherence to checklist items in trials published before the TIDieR checklist's release versus trials published after its release. Finally, we assessed whether certain factors were associated with the quality of intervention reporting.

RESULTS

From a random sample of 300 publications in orthopaedic journals, 175 parallel-arm and cluster RCTs were identified. The overall rate of adherence to TIDieR items was 58.4%. Only 31.4% of orthopaedic RCTs adhered to at least 6 of the 12 TIDieR checklist items, whereas 0% adhered to all 12 items. We found no significant improvement in the quality of intervention reporting in studies published after the TIDieR checklist's release compared with studies published before its release (P = .97). Additionally, preregistered trials were associated with more complete intervention reporting.

CONCLUSIONS

Our results suggest suboptimal reporting of orthopaedic RCT interventions. In addition, the TIDieR checklist's intended effect-to better the quality of RCT intervention reporting-appears to have fallen short of its goal.

CLINICAL RELEVANCE

Because outcomes of RCTs are used to guide clinical decision making, it is essential that orthopaedic surgeons and clinical practice guideline panels are equipped with high-quality published research. Increasing the accuracy of intervention reporting may lead to more accurate clinical application. Thus, adoption of more stringent reporting of trial interventions by researchers, authors, and journal editors may improve the quality of orthopaedic research, as well as improve patient outcomes.

Evaluation of the Completeness of Interventions Reported in Published Randomized Controlled Trials in Plastic Surgery: A Systematic Review

BACKGROUND

With the increasing number of randomized control trials being conducted and published in plastic surgery, complete reporting of trial information is critical for readers to properly evaluate a trial's methodology and arrive at appropriate conclusions about its merits and applicability to patients. The Template for Intervention Description and Replication (TIDieR) checklist was introduced to address the limited guidance for reporting trial interventions.

OBJECTIVES

The authors applied the TIDieR checklist to evaluate the completeness of intervention reporting of randomized control trials in plastic surgery, compare the quality of intervention reporting before and after the guideline was published, and evaluate characteristics associated with TIDieR compliance.

METHODS

A PubMed search identified 1 cohort published prior to the release of TIDieR and another published after its release. From the final sample, the TIDieR checklist was applied to intervention descriptions, and relevant study characteristics were extracted in a duplicate, blinded manner.

RESULTS

In total, 130 trials were included for analysis. The mean TIDieR score was 6.4 of 12. Five items were reported 90% of the time, and 4 items were reported less than 10% of the time. We found that TIDieR publication did not affect intervention reporting (P = 0.22).

CONCLUSIONS

Our study identified areas in which intervention reporting could be improved. The extent of TIDieR adoption by trialists appears to be limited, and greater efforts are needed to disseminate this reporting guideline if widespread uptake is to be expected. Alternately, it may be beneficial to incorporate TIDieR into the more widely recognized Consolidated Standards of Reporting Trials statement.

Do not make clinical decisions based on abstracts of healthcare research: A systematic review

OBJECTIVE

To summarize the reporting quality of healthcare abstracts and inconsistencies between abstracts and full texts.

STUDY DESIGN AND SETTING

This systematic review included overviews of randomized controlled trials (RCTs) and systematic reviews (SRs) that summarized data of healthcare abstracts on reporting of abstracts and consistency of abstracts with the full text. Searches were performed in PubMed, CENTRAL, Cochrane Library and EMBASE databases from 1900 to February 2019. Two authors screened the overviews and extracted the data. All analyses were descriptive and divided into two main groups: abstracts' reporting quality and abstracts' consistency with the full text. Abstracts were considered poorly reported and inconsistent with the full text if more than 5% of abstracts' information was not fully reported or not consistent with the full text.

RESULTS

27 overviews analyzing 5,194 RCTs and 866 SRs were retrieved for reporting quality of abstracts. A total of 22 overviews analyzing 2,025 RCTs and 551 SRs were included for consistency of abstracts with the full text. Abstracts across all healthcare areas presented poor reporting quality and were inconsistent with the full texts, with results and conclusions as the most inconsistent sections.

CONCLUSION

Abstracts of healthcare RCTs and SRs have shown a large room for improvement in reporting quality and consistency with the full text. Authors, journal editors and reviewers need to give the highest priority to this matter.

Completeness of Intervention Reporting of Clinical Trials Published in Highly Ranked Obesity Journals

OBJECTIVE

Randomized controlled trials (RCTs) play a crucial role in the research and advancement of medical treatment. A cross-sectional study design was utilized to analyze the completeness of intervention reporting using the Template for Intervention Description and Replication (TIDieR) checklist and to evaluate factors associated with intervention reporting. A comparison of the completeness of intervention reporting before and after the publication of TIDieR was sought.

METHODS

PubMed was searched for RCTs in the top 10 obesity journals per the Google h5-index. After excluding non-RCTs, 300 articles were randomly sampled. After assessing each publication for eligibility, two authors (SLR and DT) extracted data related to intervention reporting from records in an independent, masked fashion. Data were then verified and analyzed.

RESULTS

The analysis revealed that the quality of intervention reporting is quite variable. Overall, no statistically significant difference in the quality of intervention reporting before and after the release of TIDieR guidelines was found. In general, obesity research has good intervention reporting in areas such as the mode of delivery, material lists for intervention, and procedure lists. However, four main areas in which obesity researchers can improve reporting quality were determined. These include providing the expertise and background of intervention providers and providing statements regarding the assessment of fidelity of the intervention.

CONCLUSIONS

Urgent intervention is warranted to improve the quality of research reporting in obesity research, which is a fundamental component of obesity management. This will likely require a unified approach from researchers, journals, and funding sources.

Are published randomized clinical trials abstracts on periodontics reported adequately?

Introduction

Randomized clinical trials (RCTs) remain the golden standard in biomedical research, which makes their reporting to a high quality essential to control RCTs’ internal validity. The purpose of this study was to evaluate the quality of abstract reporting of RCTs published in periodontic journals and their compliance with the CONSORT guidelines.

Methods

A hand search was undertaken to identify RCTs published in three periodontic journals [1] Journal of periodontology (JOP) [2], the Journal of periodontal research (JOPR) and [3] the Journal of clinical periodontology (JOCP) from 2015 to 2018.The completeness of abstract reporting was evaluated with a modified CONSORT for abstracts statement checklist.

Results

Abstracts of 177 randomized controlled trials were identified and assessed. The distribution of published reports was in the Journal of periodontology (JOP), (42%) the Journal of periodontal research (JOPR) (7%) and the Journal of clinical periodontology (JOCP) (51%). The mean overall reporting quality score was 49.0%(95% CI: 47.7–50.2%). Most of the abstracts (91–100%) clearly reported and described the study design as randomized in the RCTs' title and recruitment status, as well as study interventions, objective(s), outcome(s) and conclusions. There was insufficient description and reporting of the authors’ contact details, trial design, method of randomization, blinding, number of analyzed participants per group, harms, trial registration and source of funding.

Conclusions

The quality of reporting of abstracts of randomized controlled trials in periodontic journals is suboptimal. In view of the current guidelines of reporting RCTs abstracts, efforts should be made to better reporting.

Development and Validation of a Natural Language Processing Tool to Generate the CONSORT Reporting Checklist for Randomized Clinical Trials

IMPORTANCE

Adherence to the Consolidated Standards of Reporting Trials (CONSORT) for randomized clinical trials is associated with improvingquality because inadequate reporting in randomized clinical trials may complicate the interpretation and the application of findings to clinical care.

OBJECTIVE

To evaluate an automated reporting checklist generation tool that uses natural language processing (NLP), called CONSORT-NLP.

DESIGN, SETTING, AND PARTICIPANTS

This study used published journal articles as training, testing, and validation sets to develop, refine, and evaluate the CONSORT-NLP tool. Articles reporting randomized clinical trials were selected from 25 high-impact-factor journals under the following categories: (1) general and internal medicine, (2) oncology, and (3) cardiac and cardiovascular systems.

MAIN OUTCOMES AND MEASURES

For an evaluation of the performance of this tool, an accuracy metric defined as the number of correct assessments divided by all assessments was calculated.

RESULTS

The CONSORT-NLP tool uses the widely used Portable Document Format as an input file. Of the 37 CONSORT reporting items, 34 (92%) were included in the tool. Of these 34 reporting items, 30 were fully implemented; 28 (93%) of the fully implemented CONSORT reporting items had an accuracy of more than 90% for the validation set. The remaining 2 (7%) had an accuracy between 80% and 90% for the validation set. Two to 5 articles were selected from each of these journals for a total of 158 articles to establish a training set of 111 articles to train CONSORT-NLP for CONSORT reporting items, a testing set of 25 articles to refine CONSORT-NLP, and a validation set of 22 articles to assess the performance of CONSORT-NLP. The CONSORT-NLP tool used the Portable Document Format of the articles as input files. A CONSORT-NLP graphical user interface was built using Java in 2019. The time required to complete the CONSORT checklist manually vs using the CONSORT-NLP tool was compared for 30 articles. Two case studies for randomized clinical trials are provided as an illustration for the CONSORT-NLP tool. For the 30 articles investigated, CONSORT-NLP required a mean (SD) 23.0 (4.1) seconds, whereas the manual reviewer required a mean (SD) 11.9 (2.2), 22.6 (4.6), and 57.6 (7.1) minutes, for 3 reviewers, respectively.

CONCLUSIONS AND RELEVANCE

The CONSORT-NLP tool is designed to assist in the reporting of randomized clinical trials. Potential users of CONSORT-NLP include clinicians, researchers, and scientists who plan to publish a randomized trial study in a peer-reviewed journal. The use of CONSORT-NLP may help them save substantial time when generating the CONSORT checklist. This tool may also be useful for manuscript reviewers and journal editors who review these articles.

Quality of adverse event reporting in phase III randomized controlled trials of breast and colorectal cancer: A systematic review

BACKGROUND

Clinical trial reports often emphasize efficacy over harms, leading to misinterpretation of the risk-to-benefit ratio of new therapies. Clear and sufficiently detailed reporting of methods and results is especially important in the abstracts of trial reports, as readers often base their assessment of a trial on such information. In this study, we evaluated the quality of adverse event (AE) reporting and abstract quality in phase III randomized controlled trials (RCTs) of systemic therapies in breast and colorectal cancer.

METHODS

Medline, EMBASE, Cochrane Database of RCTs, and Cochrane Database of Systematic Reviews were searched from November 2005 to September 2018. Phase III RCTs evaluating systemic therapies in breast or colorectal cancer were included. Each article was independently reviewed by two investigators using a standardized data extraction form based on guidelines developed by the Consolidated Standards of Reporting Trials (CONSORT) group. Descriptive statistics, bivariate analysis, and multivariable linear regression were used to analyze data. All statistical tests were two-sided.

RESULTS

Of 166 RCTs identified, 99.4% reported harms in the manuscript body, and 59.6% reported harms in the abstract. Reporting was restricted to severe harms in 15.6% of RCTs. Statistical comparison of AE rates went unreported in 59.0% of studies. Information regarding AEs leading to dose reductions, treatment discontinuations, or study withdrawals went unreported in 59.3%, 18.7%, and 86.8% of studies, respectively. Recently published RCTs (P = .009) and those sponsored at least partially by for-profit companies (P = .003) had higher abstract quality scores.

CONCLUSIONS

Breast and colorectal cancer phase III RCTs inadequately report CONSORT-compliant AE data. Improved guideline adherence and abstract reporting is required to properly weigh benefits and harms of new oncologic therapies.

SYSTEMATIC REVIEW REGISTRATION NUMBER

CRD42019140673.

Completeness of intervention reporting in neurology randomized controlled trials: a retrospective, cross-sectional study

AIM

Incomplete reporting of interventions in randomized controlled trials (RCTs) may hinder the replicability of studies and thus lead to adverse clinical outcomes for patients. Currently, little is known about the completeness of intervention reporting in neurology clinical trials. This study's primary objective was to determine the completeness of intervention reporting in neurology RCTs, as measured by the Template for Intervention Description and Replication (TIDieR) checklist. The secondary objectives of this study were to compare the completeness of intervention reporting before and after TIDieR publication and to evaluate factors associated with intervention reporting.

METHODS

We conducted a cross-sectional, pilot-tested, double-blind investigation of 141 neurology RCTs to determine the completeness of intervention reporting in neurology.

RESULTS

Overall, the average number of TIDieR checklist items that reached completion per study was 7.4 out of 12 (61.7%). We found no increase in intervention reporting following the publication of TIDieR (P = 0.35). Our generalized estimating equation analysis found that poorer TIDieR adherence was associated with trials lacking a trial registration (P = 0.03).

CONCLUSION

Our findings showed that on average, authors reported only 61.7% of items on the TIDieR checklist. These findings have significant implications, because incomplete reporting may hinder the replicability of studies and thus negatively affect clinical outcomes for patients.

Assessing the Quality of Abstracts in Randomized Controlled Trials Published in High Impact Cardiovascular Journals

BACKGROUND

In the busy world of cardiovascular medicine, abstracts may be the only part of a publication that clinicians read. Therefore, it is critical for abstracts to accurately reflect article content. The extended CONSORT (Consolidated Standards of Reporting Trials) Statement for Abstracts was developed to ensure high abstract quality. However, it is unknown how often adherence to CONSORT guidelines occurs among cardiovascular journals.

METHODS AND RESULTS

We searched MEDLINE for randomized controlled trials published in 3 major cardiovascular journals ( Circulation, Journal of the American College of Cardiology, and European Heart Journal) from 2011 to 2017. Post hoc, interim, and cost-effective analyses of randomized controlled trials were excluded. Two independent investigators extracted the data using a prespecified data collection form and a third investigator adjudicated the data. The primary outcome was frequency of subcategory adherence to CONSORT guidelines. A total of 478 abstracts were included in the analysis. Approximately half of the abstracts (53%; 255/478; 95% CI, 49%-57%) identified the article as randomized in the title. All abstracts detailed the interventions for both study groups (100%) and 81% (95% CI, 78%-85%) reported trial registration. Methodological quality reporting was relatively low: 9% (45/478; 95% CI, 6%-12%) described participant eligibility criteria with settings for data collection, 43% (204/478; 95% CI, 39%-47%) reported details of blinding, and <1% (4/478; 95% CI, 0%-2%) reported allocation concealment. Approximately 60% (301/478; 95% CI, 59%-67%) of the included abstracts provided primary outcome results while 55% (262/478; 95% CI, 51%-60%) reported harms or adverse effects.

CONCLUSIONS

There is a high prevalence of nonadherence to CONSORT guidelines among leading cardiovascular journals. Efforts by editors, authors, and reviewers should be made to increase adherence and promote transparent and unbiased presentation of study results.

Evaluation of intervention reporting in published emergency medicine clinical trials

OBJECTIVE

The extent of intervention reporting in emergency medicine journals remains unclear. The primary objective is to assess overall completion of the Template for Intervention Description and Replication (TIDieR) checklist described in emergency medicine randomized clinical trials (RCTs). The secondary outcomes were to (1) compare reporting before and after TIDieR publication; (2) evaluate factors associated with intervention reporting.

METHODS

Our cross-sectional study used Google Scholar's metrics to identify seven emergency medicine journals; of which, we randomly sampled 300 articles. Using two PubMed searches, we extracted 150 RCTs before and after publications of TIDieR. Two investigators independently extracted data. The primary analysis to measure overall completion included descriptive statistics for each checklist item. Our secondary analysis used an interrupted time series analysis and generalized estimating equations to determine the effect of TIDieR publication on intervention reporting.

RESULTS

Our initial search yielded 635 articles; from which, we randomly sampled 300 articles. We excluded 67 articles, leaving 233 for analysis. The mean number of TIDieR items reported was 5.4 (standard deviation = 1.18). Of the 233 trials, 42.9% provided information about materials, 67% provided intervention procedures, and 99.1% provided intervention delivery. The least reported items were intervention modifications (2.6%), intervention adherence assessment methods (3.4%), and intervention adherence assessment outcomes (2.2%).

CONCLUSIONS

The completeness of intervention reporting is suboptimal in emergency medicine journals, necessitating improvement. The current state of adherence could be improved through the combined efforts of journal editors, major editorial organizations, and authors.

Abstracts of published randomised controlled trials in Endodontics: Reporting quality and spin

AIMS

To assess the reporting quality of recently published randomised controlled trial (RCT) abstracts in Endodontics, to investigate factors associated with reporting quality, and to evaluate the existence and characteristics of spin. Spin refers to reporting strategies that distort study results and misguide readers.

METHODOLOGY

The PubMed database was searched to identify abstracts of RCTs in the field of Endodontics published during 2017 to 2018. Two authors assessed the reporting quality of each included abstract using the original 16-item CONSORT for Abstracts checklist, with the overall quality score (OQS, range: 0 to 16) being the primary outcome measure. For each individual item, a score of '1' was given if it was described adequately, and '0' if the description was inadequate. Linear regression analyses were conducted to identify factors associated with reporting quality. For the evaluation of spin, two authors selected parallel-group RCTs with a nonsignificant primary outcome from the included abstracts, and evaluated independently the existence and characteristics of spin among these abstracts.

RESULTS

A total of 162 abstracts were included for assessment of reporting, for which the mean OQS was 3.97 (SD, 1.30; 95 % CI, 3.77 to 4.17). According to multivariable analysis, origin from Europe (P=0.001) and reporting of the exact P value (P=0.020) were significantly associated with better reporting. Forty abstracts with statistically nonsignificant results for their primary outcome were included for spin evaluation, among which 34 (85.0%) had at least one type of spin. Thirty-two abstracts (94.1%) had spin in their conclusions section, and six abstracts (17.6%) had spin in the results section.

CONCLUSIONS

The reporting quality of RCT abstracts in Endodontics needs to be improved. The occurrence rate of spin in the sample of abstracts of RCTs in the field of Endodontics was high. Relevant stakeholders are recommended to be familiar with the CONSORT for Abstracts guideline and develop active strategies to ensure its implementation.

Biomedical conferences’ author instructions rarely mention guidelines for reporting abstracts of trials and systematic reviews

Aim: To analyze whether instructions for authors of biomedical conference abstracts mention guidelines for writing randomized controlled trial and systematic review abstracts and to evaluate reasons for their absence from instructions. Materials & methods: We analyzed instructions for authors of biomedical conferences advertized in 2019 and assessed whether they mentioned Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Abstracts and Consolidated Standards of Reporting Trials for Abstracts guidelines. We surveyed contact persons from abstract/publication committees of selected conferences to analyze why relevant guidelines were missing. Results: Instructions for abstracts were available for 819 conferences. Only two (0.2%) had reporting instructions for randomized controlled trial/systematic review authors. Almost half of the contacted conference organizers whose response we received were not aware of Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Abstracts and Consolidated Standards of Reporting Trials for Abstracts guidelines. Conclusion: Conference organizers do not require and are not familiar enough with reporting guidelines.

Mapping the nomenclature, methodology, and reporting of studies that review methods: a pilot methodological review

BACKGROUND

A relatively novel method of appraisal, methodological reviews (MRs) are used to synthesize information on the methods used in health research. There are currently no guidelines available to inform the reporting of MRs.

OBJECTIVES

This pilot review aimed to determine the feasibility of a full review and the need for reporting guidance for methodological reviews.

METHODS

Search strategy: We conducted a search of PubMed, restricted to 2017 to include the most recently published studies, using different search terms often used to describe methodological reviews: "literature survey" OR "meta-epidemiologic* review" OR "meta-epidemiologic* survey" OR "methodologic* review" OR "methodologic* survey" OR "systematic survey."Data extraction: Study characteristics including country, nomenclature, number of included studies, search strategy, a priori protocol use, and sampling methods were extracted in duplicate and summarized.Outcomes: Primary feasibility outcomes were the sensitivity and specificity of the search terms (criteria for success of feasibility set at sensitivity and specificity of ≥ 70%).Analysis: The estimates are reported as a point estimate (95% confidence interval).

RESULTS

Two hundred thirty-six articles were retrieved and 31 were included in the final analysis. The most accurate search term was "meta-epidemiological" (sensitivity [Sn] 48.39; 95% CI 31.97-65.16; specificity [Sp] 97.56; 94.42-98.95). The majority of studies were published by authors from Canada (n = 12, 38.7%), and Japan and USA (n = 4, 12.9% each). The median (interquartile range [IQR]) number of included studies in the MRs was 77 (13-1127). Reporting of a search strategy was done in most studies (n = 23, 74.2%). The use of a pre-published protocol (n = 7, 22.6%) or a justifiable sampling method (n = 5, 16.1%) occurred rarely.

CONCLUSIONS

Using the MR nomenclature identified, it is feasible to build a comprehensive search strategy and conduct a full review. Given the variation in reporting practices and nomenclature attributed to MRs, there is a need for guidance on standardized and transparent reporting of MRs. Future guideline development would likely include stakeholders from Canada, USA, and Japan.

Global mapping of randomised trials related articles published in high-impact-factor medical journals: a cross-sectional analysis

Background

Randomised controlled trials (RCTs) provide the most reliable information to inform clinical practice and patient care. We aimed to map global clinical research publication activity through RCT-related articles in high-impact-factor medical journals over the past five decades.

Methods

We conducted a cross-sectional analysis of articles published in the highest ranked medical journals with an impact factor > 10 (according to Journal Citation Reports published in 2017). We searched PubMed/MEDLINE (from inception to December 31, 2017) for all RCT-related articles (e.g. primary RCTs, secondary analyses and methodology papers) published in high-impact-factor medical journals. For each included article, raw metadata were abstracted from the Web of Science. A process of standardization was conducted to unify the different terms and grammatical variants and to remove typographical, transcription and/or indexing errors. Descriptive analyses were conducted (including the number of articles, citations, most prolific authors, countries, journals, funding sources and keywords). Network analyses of collaborations between countries and co-words are presented.

Results

We included 39,305 articles (for the period 1965–2017) published in forty journals. The Lancet (n = 3593; 9.1%), the Journal of Clinical Oncology (n = 3343; 8.5%) and The New England Journal of Medicine (n = 3275 articles; 8.3%) published the largest number of RCTs. A total of 154 countries were involved in the production of articles. The global productivity ranking was led by the United States (n = 18,393 articles), followed by the United Kingdom (n = 8028 articles), Canada (n = 4548 articles) and Germany (n = 4415 articles). Seventeen authors who had published 100 or more articles were identified; the most prolific authors were affiliated with Duke University (United States), Harvard University (United States) and McMaster University (Canada). The main funding institutions were the National Institutes of Health (United States), Hoffmann-La Roche (Switzerland), Pfizer (United States), Merck Sharp & Dohme (United States) and Novartis (Switzerland). The 100 most cited RCTs were published in nine journals, led by The New England Journal of Medicine (n = 78 articles), The Lancet (n = 9 articles) and JAMA (n = 7 articles). These landmark contributions focused on novel methodological approaches (e.g. the “Bland-Altman method”) and trials on the management of chronic conditions (e.g. diabetes control, hormone replacement therapy in postmenopausal women, multiple therapies for diverse cancers, cardiovascular therapies such as lipid-lowering statins, antihypertensive medications, and antiplatelet and antithrombotic therapy).

Conclusions

Our analysis identified authors, countries, funding institutions, landmark contributions and high-impact-factor medical journals publishing RCTs. Over the last 50 years, publication production in leading medical journals has increased, with Western countries leading in research but with low- and middle-income countries showing very limited representation.

Completeness of reporting in abstracts of randomized controlled trials in subscription and open access journals: cross-sectional study

Background

Open access (OA) journals are becoming a publication standard for health research, but it is not clear how they differ from traditional subscription journals in the quality of research reporting. We assessed the completeness of results reporting in abstracts of randomized controlled trials (RCTs) published in these journals.

Methods

We used the Consolidated Standards of Reporting Trials Checklist for Abstracts (CONSORT-A) to assess the completeness of reporting in abstracts of parallel-design RCTs published in subscription journals (n = 149; New England Journal of Medicine, Journal of the American Medical Association, Annals of Internal Medicine, and Lancet) and OA journals (n = 119; BioMedCentral series, PLoS journals) in 2016 and 2017.

Results

Abstracts in subscription journals completely reported 79% (95% confidence interval [CI], 77–81%) of 16 CONSORT-A items, compared with 65% (95% CI, 63–67%) of these items in abstracts from OA journals (P < 0.001, chi-square test). The median number of completely reported CONSORT-A items was 13 (95% CI, 12–13) in subscription journal articles and 11 (95% CI, 10–11) in OA journal articles. Subscription journal articles had significantly more complete reporting than OA journal articles for nine CONSORT-A items and did not differ in reporting for items trial design, outcome, randomization, blinding (masking), recruitment, and conclusions. OA journals were better than subscription journals in reporting randomized study design in the title.

Conclusion

Abstracts of randomized controlled trials published in subscription medical journals have greater completeness of reporting than abstracts published in OA journals. OA journals should take appropriate measures to ensure that published articles contain adequate detail to facilitate understanding and quality appraisal of research reports about RCTs.

Accuracy in detecting inadequate research reporting by early career peer reviewers using an online CONSORT-based peer-review tool (COBPeer) versus the usual peer-review process: a cross-sectional diagnostic study

BACKGROUND

The peer review process has been questioned as it may fail to allow the publication of high-quality articles. This study aimed to evaluate the accuracy in identifying inadequate reporting in RCT reports by early career researchers (ECRs) using an online CONSORT-based peer-review tool (COBPeer) versus the usual peer-review process.

METHODS

We performed a cross-sectional diagnostic study of 119 manuscripts, from BMC series medical journals, BMJ, BMJ Open, and Annals of Emergency Medicine reporting the results of two-arm parallel-group RCTs. One hundred and nineteen ECRs who had never reviewed an RCT manuscript were recruited from December 2017 to January 2018. Each ECR assessed one manuscript. To assess accuracy in identifying inadequate reporting, we used two tests: (1) ECRs assessing a manuscript using the COBPeer tool (after completing an online training module) and (2) the usual peer-review process. The reference standard was the assessment of the manuscript by two systematic reviewers. Inadequate reporting was defined as incomplete reporting or a switch in primary outcome and considered nine domains: the eight most important CONSORT domains and a switch in primary outcome(s). The primary outcome was the mean number of domains accurately classified (scale from 0 to 9).

RESULTS

The mean (SD) number of domains (0 to 9) accurately classified per manuscript was 6.39 (1.49) for ECRs using COBPeer versus 5.03 (1.84) for the journal's usual peer-review process, with a mean difference [95% CI] of 1.36 [0.88-1.84] (p < 0.001). Concerning secondary outcomes, the sensitivity of ECRs using COBPeer versus the usual peer-review process in detecting incompletely reported CONSORT items was 86% [95% CI 82-89] versus 20% [16-24] and in identifying a switch in primary outcome 61% [44-77] versus 11% [3-26]. The specificity of ECRs using COBPeer versus the usual process to detect incompletely reported CONSORT domains was 61% [57-65] versus 77% [74-81] and to identify a switch in primary outcome 77% [67-86] versus 98% [92-100].

CONCLUSIONS

Trained ECRs using the COBPeer tool were more likely to detect inadequate reporting in RCTs than the usual peer review processes used by journals. Implementing a two-step peer-review process could help improve the quality of reporting.

TRIAL REGISTRATION

Clinical.Trials.gov NCT03119376 (Registered April, 18, 2017).

Quality of reporting in abstracts of RCTs published in emergency medicine journals: a systematic survey of the literature suggests we can do better

OBJECTIVE

We investigated the association between the publication of the Consolidated Standards of Reporting Trials extension for abstracts (CONSORT-EA) and other variables of interest on the quality of reporting of abstracts of randomised controlled trials (RCTs) published in emergency medicine (EM) journals.

METHODS

We performed a survey of the literature, comparing the quality of reporting before (2005-2007) with after (2014-2015) the publication of the dedicated CONSORT-EA in 2008. The quality of reporting was measured as the sum of items of the CONSORT-EA checklist reported in each abstract, ranging from 0 to 15. The main explanatory variable was the period of publication: pre-CONSORT-EA versus post-CONSORT-EA public. Other explanatory variables were journal's endorsement of the CONSORT statement, number of centres participating in the study, study's sample size, type of intervention, significance of results, source of funding and study setting. We analysed the data using generalised estimation equations, performing a univariate and a multivariable analysis.

RESULTS

We retrieved 844 articles, and randomly selected 60 per period for review, after stratifying for journal. The mean (SD) number of items reported was 6.4 (1.9) in the period before and 6.9 (1.8) in the period after the publication of the CONSORT-EA, with an adjusted mean difference (aMD) of 0.47 (95% CI -0.13 to 1.06). Abstracts of trials of pharmacological interventions had a significantly larger mean number of reported items than those of trials of non-pharmacological interventions (aMD 1.59; 95% CI 0.94 to 2.24).

CONCLUSIONS

The quality of reporting in abstracts of RCTs published in EM journals is low and was not significantly impacted by the publication of a dedicated CONSORT-EA.

Evaluation of reporting of CONSORT flow diagrams in randomized controlled trials in a national and international pharmacology journal

OBJECTIVE

To evaluate and compare the reporting of consolidated standard of reporting trial (CONSORT) flow diagrams in randomized controlled trials (RCTs) published in a national and international pharmacology journal.

METHODS

RCTs in an international pharmacology journal, European Journal of Clinical Pharmacology (EJCP), and a national journal, Indian Journal of Pharmacology (IJP), published from January 2014 to July 2016 were evaluated for reporting and completeness of CONSORT flow diagrams. A total of 138 articles (EJCP = 90; IJP = 48) were analyzed and compared.

RESULTS

Of 138 RCTs analyzed, 90 were from EJCP and 48 were from IJP. 76.6% (69/90) articles from EJCP and 37.5% (18/48) articles from IJP had reported the CONSORT flow diagram. Of these, 95.5% (66/69) had assessed for eligibility in EJCP and 88.8% (16/18) had reported the same in IJP. The number of participants excluded was reported in 86.9% (60/69) flow diagrams in EJCP and 83.3% (15/18) in IJP. 82.6% (57/69) flow diagrams in EJCP and 77.7% (14/18) in IJP had mentioned the details of randomization. Allocation of intervention was reported in 91.3% (63/69) flow diagrams in EJCP and 88.8% (16/18) in IJP. 60.8% (42/69) flow diagrams in EJCP and 44.4% (08/18) in IJP had mentioned the details of follow-up of participants. 95.5% (66/69) flow diagrams in EJCP and 94.4% (17/18) in IJP had mentioned details about analysis.

CONCLUSION

In spite of both the journals, IJP and EJCP endorsing the CONSORT statement, a significant difference in the reporting of CONSORT flow diagrams in RCTs can be noted. The quality of reporting can be improved by stringent publication guidelines by the editors.

Randomized Controlled Trials on Intracerebral Hemorrhage: A Cross Sectional Retrospective Analysis of CONSORT Item Adherence

Object: Intracranial hemorrhage (ICH) is the second most common cause of stroke but still there is little consolidated knowledge about the optimal treatment strategies (e.g., the benefit of surgical evacuation). We evaluated the current randomized controlled trials (RCTs) on primary ICH (01.2013–03.2017) according to their fulfillment of the CONSORT statement's criteria (published in 2010) –as a marker of transparency and quality of study planning and realization.

Methods: A Pubmed and a Cochrane database (including clinicaltrials.gov) search was carried out (01.2014–3.2017, respectively 01.2013–12.2013). Abstracts were screened for inclusion. Eligible full text manuscripts were assessed for the implementation of the CONSORT criteria. Citation frequencies and impact factors of the journals were related to ratio of CONSORT criteria fulfillment. Further, the risk of bias according to the Risk of bias tool 2 (RoB 2) was assessed.

Results: Overall 3097 abstracts were screened for inclusion; 39 studies were suitable for final analysis. A mean fulfillment ratio of 51% (±28%) was found. A high correlation between impact factor and adherence to CONSORT criteria was shown (r = 0.7664; p < 0.0001). Citation frequency per year was related to ratio of CONSORT item fulfillment (r = 0.6747; p < 0.0001) and to the impact factor of the publishing journal (r = 0.7310; p < 0.0001). Of note, the items 10 (randomization: implementation) and 21 (generalizability) showed particularly high rates of non-fulfillment (87 and 85%). The majority of studies (95%) complied with item 2b (specific objectives or hypotheses), but strikingly objectives were mostly described vaguely. Other essential criteria such as sample size determination, definition of outcome parameters, and participant flow were only fulfilled weakly (51, 54, and 39%).

Conclusions: Over 20 years after its inception there is still weak adherence to the CONSORT statement. As a consequence, conclusions are hampered by inadequate planning and/or reporting. Particularly with respect to pathologies as ICH lacking clear, evidence-based guidelines adherence to the CONSORT statement might improve research quality in order to define valuable treatment strategies.

Use of publication procedures to improve research integrity by addiction journals

BACKGROUND AND AIMS

The credibility crisis evident in many academic disciplines has led peer-reviewed journals to implement procedures to reduce use of flexible data analysis practices and selective reporting of results. This exploratory study examined the adoption of six of these procedures by addiction journals.

METHODS

Thirty-eight high-impact addiction journals were identified using the 2018 Clarivate Analytics Journal Citation Report for 2017 ranks. The online instructions for authors were reviewed for references to six publication procedures: conflict of interest disclosure, reporting guidelines, clinical trial registration, registration of other study designs, data-sharing and registered reports. The webpages of the Center for Open Science and Consolidated Standards of Reporting Trials (CONSORT) were also reviewed for data pertaining to registered reports and reporting guidelines, respectively.

RESULTS

The range of procedures adopted by the addiction journals was 0-5, with a mean of 2.66. Conflict-of-interest disclosure was required by all but one journal. Encouraging data-sharing was the next most commonly required procedure. Fewer than half the journals recommended specific reporting guidelines or required registration of clinical trials, and only four required procedures to pre-specify hypotheses and analytical methods.

CONCLUSIONS

While many addiction journals have adopted publication procedures to improve research integrity, these can be limited by their voluntary nature and monitoring difficulties. More stringent requirements that lock researchers into specific hypotheses and analyses have not been widely adopted.

The quality of reporting RCT abstracts in four major orthodontics journals for the period 2012-2017

INTRODUCTION

Clear reporting of the abstracts of randomised controlled trials (RCTs) facilitates the assessment and identification of such trials.

AIM

To assess whether authors in the orthodontic field of research currently report RCT abstracts adequately, as defined by the Consolidated Standards of Reporting Trials (CONSORT) statement.

DESIGN

An observational retrospective study.

METHODS

Electronic searches with supplementary hand searching were undertaken to identify RCTs published in (1) American Journal of Orthodontics and Dentofacial Orthopedics (AJO-DO), (2) Angle Orthodontist (AO), (3) European Journal of Orthodontics (EJO) and (4) Journal of Orthodontics (JO) for the period from January 2012 to December 2017. The completeness of the abstract reporting was evaluated using a modified CONSORT for abstract statement checklist.

RESULTS

A total of 3678 articles were retrieved, but only 224 RCT abstracts were identified and assessed. A high volume of RCTs were published with either the AO (39%) or AJO-DO (32%); the majority of the RCT abstracts (93.6%) were structured. The mean overall abstract reporting quality score was 69.1% (95% confidence interval = 67.5-70.7). In relation to individual quality items, the majority of the RCT abstracts (range = 96-100%) demonstrated clear reporting of the author/contact details, trial design, participants, interventions, objectives, outcomes, number of participants randomised to each group, recruitment, results and conclusions. However, reporting of the title, trial registration, funding and number of analysed participants were only moderately adequate and reporting of the assessment of blinding and adverse events were the least-reported items in the identified abstracts.

CONCLUSIONS

As several CONSORT reporting items were poorly reported, it is the responsibility of authors, referees and editors alike to ensure that the CONSORT guidelines are followed.

Abstracts of low back pain trials are poorly reported, contain spin of information and are inconsistent with the full text: An overview study

OBJECTIVE

To investigate trials abstracts evaluating treatments for low back pain with regards to completeness of reporting, spin (i.e., interpretation of study results that overemphasizes the beneficial effects of the intervention), and inconsistencies in data with the full text.

DATA SOURCES

The search was performed on Physiotherapy Evidence Database (PEDro) in February 2016.

STUDY SELECTION

This is an overview study of a random sample of 200 low back pain trials published between 2010 and 2015. The languages of publication were restricted to English, Spanish and Portuguese.

DATA EXTRACTION

Completeness of reporting was assessed using the CONSORT for Abstracts checklist (CONSORT-A). Spin was assessed using a SPIN-checklist. Consistency between abstract and full text were assessed by applying the assessment tools to both the abstract and full text of each trial and calculating inconsistencies in the summary score (paired t test) and agreement in the classification of each item (Kappa statistics). Methodological quality was analyzed using the total PEDro score.

DATA SYNTHESIS

The mean number of fully reported items for abstracts using the CONSORT-A was 5.1 (SD 2.4) out of 15 points and the mean number of items with spin was 4.9 (SD 2.6) out of 7 points. Abstract and full text scores were statistically inconsistent (P=0.01). There was slight to moderate agreement between items of the CONSORT-A in the abstracts and full text (mean Kappa 0.20 SD 0.13) and fair to moderate agreement for items of the SPIN-checklist (mean Kappa 0.47 SD 0.09).

CONCLUSIONS

The abstracts were incomplete, with spin and inconsistent with the full text. We advise health care professionals to avoid making clinical decisions based solely upon abstracts. Journal editors, reviewers and authors are jointly responsible for improving abstracts, which could be guided by amended editorial policies.

Tools used to assess the quality of peer review reports: a methodological systematic review

Background

A strong need exists for a validated tool that clearly defines peer review report quality in biomedical research, as it will allow evaluating interventions aimed at improving the peer review process in well-performed trials. We aim to identify and describe existing tools for assessing the quality of peer review reports in biomedical research.

Methods

We conducted a methodological systematic review by searching PubMed, EMBASE (via Ovid) and The Cochrane Methodology Register (via The Cochrane Library) as well as Google® for all reports in English describing a tool for assessing the quality of a peer review report in biomedical research. Data extraction was performed in duplicate using a standardized data extraction form. We extracted information on the structure, development and validation of each tool. We also identified quality components across tools using a systematic multi-step approach and we investigated quality domain similarities among tools by performing hierarchical, complete-linkage clustering analysis.

Results

We identified a total number of 24 tools: 23 scales and 1 checklist. Six tools consisted of a single item and 18 had several items ranging from 4 to 26. None of the tools reported a definition of ‘quality’. Only 1 tool described the scale development and 10 provided measures of validity and reliability. Five tools were used as an outcome in a randomized controlled trial (RCT). Moreover, we classified the quality components of the 18 tools with more than one item into 9 main quality domains and 11 subdomains. The tools contained from two to seven quality domains. Some domains and subdomains were considered in most tools such as the detailed/thorough (11/18) nature of reviewer’s comments. Others were rarely considered, such as whether or not the reviewer made comments on the statistical methods (1/18).

Conclusion

Several tools are available to assess the quality of peer review reports; however, the development and validation process is questionable and the concepts evaluated by these tools vary widely. The results from this study and from further investigations will inform the development of a new tool for assessing the quality of peer review reports in biomedical research.

Electronic supplementary material

The online version of this article (10.1186/s12874-019-0688-x) contains supplementary material, which is available to authorized users.

A Systems Approach to Understanding and Improving Research Integrity

Concern about the integrity of empirical research has arisen in recent years in the light of studies showing the vast majority of publications in academic journals report positive results, many of these results are false and cannot be replicated, and many positive results are the product of data dredging and the application of flexible data analysis practices coupled with selective reporting. While a number of potential solutions have been proposed, the effects of these are poorly understood and empirical evaluation of each would take many years. We propose that methods from the systems sciences be used to assess the effects, both positive and negative, of proposed solutions to the problem of declining research integrity such as study registration, Registered Reports, and open access to methods and data. In order to illustrate the potential application of systems science methods to the study of research integrity, we describe three broad types of models: one built on the characteristics of specific academic disciplines; one a diffusion of research norms model conceptualizing researchers as susceptible, "infected" and recovered; and one conceptualizing publications as a product produced by an industry comprised of academics who respond to incentives and disincentives.

Quality Assessment of Randomized Clinical Trials Reporting in Endodontic Journals: An Observational Study from 2012 to 2017

INTRODUCTION

Reports on randomized clinical trials (RCTs) are of critical importance because readers of research often do not access the full text. This study aimed to assess the reporting quality of RCTs in 2 leading endodontic journals.

METHODS

Issues of 2 endodontic journals, the Journal of Endodontics and the International Journal of Endodontics, dated from 2012 to 2017 were hand searched to identify RCT reports. A 37-item checklist based on the Consolidated Standards of Reporting Trials statement was used to examine the completeness of RCT reporting.

RESULTS

One hundred nine RCT reports were included in this study. The majority were published in the Journal of Endodontics (82%). The mean overall reporting quality score was 65.0% (95% confidence interval, 77.3-66.5). Most (80%-100%) RCTs clearly reported the author/contact details, trial design, participant characteristics, number of participants, and recruitment status as well as the study's intervention(s), objective(s), outcome(s), and conclusions. Conversely, only 56 of the 109 articles (51%) satisfactorily reported all 5 items related to the randomization method. Registration of reviews was not reported in any of the included abstracts. Most of the studies included in this analysis did not report their RCT registration (22%), funding (35%), or protocols (23%).

CONCLUSIONS

The results of this study suggest that the reporting quality of RCTs in endodontic journals requires further improvement. Better reporting of RCTs is particularly important for ensuring the reliability of research findings and ultimately promoting the practice of evidence-based dentistry. Optimal RCT reporting should be encouraged, preferably by complying with the Consolidated Standards of Reporting Trials guidelines.

The efficacy and safety of soluble guanylate cyclase stimulators in patients with heart failure: A systematic review and meta-analysis

BACKGROUND

Several randomized controlled trials (RCTs) have been investigated the benefits of soluble guanylate cyclase (sGC) stimulators in the treatment of heart failure, but a comprehensive evaluation is lacking. We performed a meta-analysis to evaluate the efficacy and safety of oral sGC stimulators (vericiguat and riociguat) in patients with heart failure.

METHODS

Studies were searched and screened in PubMed, Embase, and Cochrane Library. Eligible RCTs were included that reported mortality, the change of EuroQol Group 5-Dmensional Self-report Questionnaire (EQ-5D) US index, N-terminal pro-B-type natriuretic peptide (NT-proBNP), or serious adverse events (SAEs). Relative risk or weight mean difference (WMD) was estimated using fixed effect model or random effect model. Analysis of sensitivity and publication bias was conducted.

RESULTS

Five trials with a total of 1200 patients were included. sGC stimulators had no impact on the mortality (1.25; 95% confidence interval 0.50-3.11) and significantly improved EQ-5D US index (0.04; 95% confidence interval 0.020-0.05). Furthermore, in comparison with control group, NT-proBNP was statistically decreased in riociguat group (-0.78; 95% confidence interval -1.01 to -0.47), but not in vericiguat group (0.04, 95% confidence interval -0.18 to 0.25). There were not obverse differences in SAEs between sGC stimulators and control groups (0.90; 95% confidence interval 0.72-1.12).

CONCLUSION

Our meta-analysis suggests that sGC stimulators could improve the quality of life in patients with heart failure with good tolerance and safety, but their long-term benefits need to be observed in the future. sGC stimulators are likely to be promising add-on strategies for the treatment of heart failure.

The degree of adherence to CONSORT reporting guidelines for the abstracts of randomised clinical trials published in anaesthesia journals: A cross-sectional study of reporting adherence in 2010 and 2016

BACKGROUND

Abstracts are intended to be concise summaries of the entire randomised clinical trial (RCT). Despite their importance, few studies have examined the reporting quality of abstracts in the anaesthesiology literature.

OBJECTIVES

To examine the quality of RCT abstract reporting according to the CONSORT for Abstracts guidelines and determine whether recommended items omitted from the abstract were present in the body of the article.

DESIGN

A cross-sectional study of RCTs.

SETTING

This study was performed at the University of Western Ontario and University Hospital, London Health Sciences Centre.

PARTICIPANTS

All RCTs meeting inclusion criteria that were published in 2010 or 2016 in six general anaesthesiology journals (Anaesthesia, Anesthesia & Analgesia, Anesthesiology, British Journal of Anaesthesia, Canadian Journal of Anesthesia and European Journal of Anaesthesiology).

MAIN OUTCOME MEASURES

The 16 checklist items from the CONSORT for Abstracts statement were used to create a convenience score as a proxy for RCT abstract reporting quality, with each criterion measured as being reported in abstract, not reported in abstract but reported in full-text article, or not reported in abstract or full-text article.

RESULTS

Of the 395 RCTs identified, 219 were published in 2010 and 176 were published in 2016. Out of the maximum possible score of 16, the median abstract score increased from 4 points [interquartile range (IQR): 3 to 5] in 2010 to 6 points [IQR: 5 to 8] in 2016. Although most checklist items showed improvement from 2010 to 2016, around 75% of RCTs in 2016 met fewer than half of the 16 items with no RCTs reporting all 16 items in the abstract. A majority of the RCTs had the information present in the full-text. In 2016, only 71 out of 176 (40%) of RCTs reported outcomes conforming to the CONSORT guidelines (with an effect size and a confidence interval around the effect size) in the Abstract.

CONCLUSION

Abstracts for many anaesthesiology RCTs are incomplete selective summaries of the entire article.