The degree of adherence to CONSORT reporting guidelines for the abstracts of randomised clinical trials published in anaesthesia journals: A cross-sectional study of reporting adherence in 2010 and 2016

Alternatives to the P value: connotations of significance

The statistical significance of a clinical trial analysis result is determined by a mathematical calculation and probability based on null hypothesis significance testing. However, statistical significance does not always align with meaningful clinical effects; thus, assigning clinical relevance to statistical significance is unreasonable. A statistical result incorporating a clinically meaningful difference is a better approach to present statistical significance. Thus, the minimal clinically important difference (MCID), which requires integrating minimum clinically relevant changes from the early stages of research design, has been introduced. As a follow-up to the previous statistical round article on P values, confidence intervals, and effect sizes, in this article, we present hands-on examples of MCID and various effect sizes and discuss the terms statistical significance and clinical relevance, including cautions regarding their use.

Intraoperative pharmacologic opioid minimisation strategies and patient-centred outcomes after surgery: a scoping review

BACKGROUND

Postoperative patient-centred outcome measures are essential to capture the patient's experience after surgery. Although a large number of pharmacologic opioid minimisation strategies (i.e. opioid alternatives) are used for patients undergoing surgery, it remains unclear which strategies are most promising in terms of patient-centred outcome improvements. This scoping review had two main objectives: (1) to map and describe evidence from clinical trials assessing the patient-centred effectiveness of pharmacologic intraoperative opioid minimisation strategies in adult surgical patients, and (2) to identify promising pharmacologic opioid minimisation strategies.

METHODS

We searched MEDLINE, Embase, CENTRAL, Web of Science, and CINAHL databases from inception to February 2023. We included trials investigating the use of opioid minimisation strategies in adult surgical patients and reporting at least one patient-centred outcome. Study screening and data extraction were conducted independently by at least two reviewers.

RESULTS

Of 24,842 citations screened for eligibility, 2803 trials assessed the effectiveness of intraoperative opioid minimisation strategies. Of these, 457 trials (67,060 participants) met eligibility criteria, reporting at least one patient-centred outcome. In the 107 trials that included a patient-centred primary outcome, patient wellbeing was the most frequently used domain (55 trials). Based on aggregate findings, dexmedetomidine, systemic lidocaine, and COX-2 inhibitors were promising strategies, while paracetamol, ketamine, and gabapentinoids were less promising. Almost half of the trials (253 trials) did not report a protocol or registration number.

CONCLUSIONS

Researchers should prioritise and include patient-centred outcomes in the assessment of opioid minimisation strategy effectiveness. We identified three potentially promising pharmacologic intraoperative opioid minimisation strategies that should be further assessed through systematic reviews and multicentre trials. Findings from our scoping review may be influenced by selective outcome reporting bias.

STUDY REGISTRATION

OSF - https://osf.io/7kea3.

Evaluating the completeness of the reporting of abstracts since the publication of the CONSORT extension for abstracts: an evaluation of randomized controlled trial in ten nursing journals

BACKGROUND

As a practice-oriented discipline, strict adherence to reporting guidelines is particularly important in randomized controlled trial (RCT) abstracts of the nursing area. However, whether abstract reports after 2010 have complied with the Consolidated Standards of Reporting Trials for Abstracts (CONSORT-A) guideline is unclear. This study aimed to evaluate whether the publication of CONSORT-A has improved abstract reporting in nursing and explores the factors associated with better adherence to the guidelines.

METHODS

We searched the Web of Science for 200 RCTs randomly selected from ten nursing journals. We used a data extraction form based on CONSORT-A, including 16 items, to analyze the reporting adherence to the guidelines, and the reporting rate of each item and the total score for each abstract were used to indicate adherence and overall quality score (OQS, range 0-16). A comparison of the total mean score between the two periods was made, and affecting factors were analyzed.

RESULTS

In the studies we included, 48 abstracts were published pre-CONSORT-A whereas 152 post-CONSORT-A. The overall mean score for reporting adherence to 16 items was 7.41 ± 2.78 and 9.16 ± 2.76 for pre- and post-CONSORT-A, respectively (total score: 16). The most poorly reported items are "harms (0%)," "outcomes in method (8.5%)," "randomization (25%)," and "blinding (6.5%)." Items including the year of publication, impact factor, multiple center trial, word count, and structured abstract are significantly associated with higher adherence.

CONCLUSIONS

The adherence to abstract reporting in nursing literature has improved since the CONSORT-A era, but the overall completeness of RCT abstracts remained low. A joint effort by authors, editors, and journals is necessary to improve reporting quality of RCT abstracts.

Include or not to include conference abstracts in systematic reviews? Lessons learned from a large Cochrane network meta-analysis including 585 trials

BACKGROUND

Systematic reviews attempt to gather all available evidence. Controversy exists regarding effort and benefit of including study results presented at conferences only. We recently published a Cochrane network meta-analysis (NMA) including 585 randomized controlled trials comparing drugs for prevention of postoperative nausea and vomiting (PONV). Studies published as conference abstracts only were excluded. This study aimed to include all eligible studies published as abstracts only, assessing their added value regarding reporting quality and effect on the review's interpretation.

METHODS

Conference abstracts were searched in the review's excluded studies and conference proceedings of anaesthesiologic societies. We assessed their reporting quality regarding review's eligibility criteria, Cochrane 'risk of bias' assessment tool 1.0, and adherence to CONSORT (Consolidated Standards of Reporting Trials) for abstracts. Abstracts were included in sensitivity NMA, and impact on the NMA structure was investigated.

RESULTS

We identified 90 abstracts. A total of 14% (13/90) were eligible. A total of 86% (77/90) are awaiting classification due to insufficient reporting of review's eligibility criteria. In abstracts awaiting classification, sufficient information was missing on standardization of anaesthesia in 71% (55/77), age of participants in 56% (43/77), and outcome details in 46% (36/77). A total of 73% (66/90) of abstracts lacked sufficient information on 15/25 data extraction items. Reported study characteristics of abstracts were comparable to included studies of the review. A total of 62% (56/90) of abstract trials were assessed as overall high risk of bias due to poor reporting. Median adherence to CONSORT for abstracts was 24% (IQR, 18 to 29%). Six of the 13 eligible abstracts reported relevant outcome data in sufficient detail for NMA on seven outcomes of the Cochrane review. Inclusion of abstracts did not substantially change the network structure, network effect estimates, ranking of treatments, or the conclusion. Certainty of evidence for headache on palonosetron use was upgraded from very low to low.

CONCLUSIONS

Most conference abstracts on PONV were insufficiently reported regarding review's narrow inclusion criteria and could not be included in NMA. The resource-intensive search and evaluation of abstracts did not substantially extent the full-text evidence base of the review, given the few adequately reported abstracts. Conferences should oblige authors to adhere to CONSORT for abstracts.

Reporting quality for abstracts of randomised trials on child and adolescent depression prevention: a meta-epidemiological study on adherence to CONSORT for abstracts

OBJECTIVES

This study aimed to investigate adherence to Consolidated Standards of Reporting Trials (CONSORT) for abstracts in reports of randomised trials on child and adolescent depression prevention. Secondary objective was to examine factors associated with overall reporting quality.

DESIGN

Meta-epidemiological study.

DATA SOURCES

We searched MEDLINE, EMBASE, PsycINFO, PsycArticles and CENTRAL.

ELIGIBILITY CRITERIA

Trials were eligible if the sample consisted of children and adolescents under 18 years with or without an increased risk for depression or subthreshold depression. We included reports published from 1 January 2003 to 8 August 2020 on randomised controlled trials (RCTs) and cluster randomised trials (CRTs) assessing universal, selective and indicated interventions aiming to prevent the onset of depression or reducing depressive symptoms.

DATA EXTRACTION AND SYNTHESIS

As the primary outcome measure, we assessed for each trial abstract whether information recommended by CONSORT was adequately reported, inadequately reported or not reported. Moreover, we calculated a summative score of overall reporting quality and analysed associations with trial and journal characteristics.

RESULTS

We identified 169 eligible studies, 103 (61%) RCTs and 66 (39%) CRTs. Adequate reporting varied considerably across CONSORT items: while 9 out of 10 abstracts adequately reported the study objective, no abstract adequately provided information on blinding. Important adverse events or side effects were only adequately reported in one out of 169 abstracts. Summative scores for the abstracts' overall reporting quality ranged from 17% to 83%, with a median of 40%. Scores were associated with the number of authors, abstract word count, journal impact factor, year of publication and abstract structure.

CONCLUSIONS

Reporting quality for abstracts of trials on child and adolescent depression prevention is suboptimal. To help health professionals make informed judgements, efforts for improving adherence to reporting guidelines for abstracts are needed.

Did an introduction of CONSORT for abstracts guidelines improve reporting quality of randomised controlled trials' abstracts on Helicobacter pylori infection? Observational study

OBJECTIVE

To determine abstracts' adherence to the Consolidated Standards of Reporting Trials for Abstracts (CONSORT-A) statement and to explore the factors associated with reporting quality.

DESIGN

An observational study.

SETTING

Abstracts of randomised controlled trials published between 2010 and 2019, found searching the MEDLINE database.

PARTICIPANTS

A total of 451 abstracts of the clinical trials on Helicobacter pylori infections were included.

PRIMARY AND SECONDARY OUTCOME MEASURES

Abstracts' reporting quality was determined by assessing their adherence to 17-item CONSORT-A checklist, with overall score being calculated as the sum of items that were adequately reported for each abstract. Additional factors that might influence the reporting quality of the abstracts were analysed, with univariate and multivariate linear regression used to determine how those factors influenced the overall reporting quality.

RESULTS

Included abstracts had an overall median quality score of 8/17 (IQR 7-9). Large proportions of abstracts adequately reported interventions, participants, objectives, numbers randomised and conclusions (97.1, 99.3, 89.1. 94.7 and 98.4% of abstracts, respectively). Trial design, randomisation, blinding and funding were severely under-reported with only 8.0, 2.7, 11.0 and 2.0% of abstracts reporting each item. Overall quality scores for H. pylori abstracts were higher in association with CONSORT-A endorsement (B=5.698; 95% CI 1.781 to 9.615), pharmacological interventions (B=4.063; 95% CI 0.224 to 7.902), multicentre settings (B=5.057; 95% CI 2.370 to 7.743), higher numbers of participants (B=3.607; 95% CI 1.272 to 5.942), hospital settings (B=4.827; 95% CI 1.753 to 7.901) and longer abstracts (B=3.878; 95% CI 0.787 to 6.969 for abstracts with 251-300 words and B=7.404; 95% CI 3.930 to 10.878 for abstracts with more than 300 words).

CONCLUSIONS

The overall reporting quality of abstracts was inadequate. The endorsement of CONSORT-A guidelines by more journals might improve the standards of reporting.

Evaluation of reporting quality of abstracts of randomized controlled trials regarding patients with COVID-19 using the CONSORT statement for abstracts

OBJECTIVE

To evaluate the reporting quality of randomized controlled trial (RCT) abstracts regarding patients with coronavirus disease 2019 (COVID-19) and to analyze the factors influencing the quality.

METHODS

The PubMed, Embase, Web of Science, and Cochrane Library databases were searched to collect RCTs on patients with COVID-19. The retrieval time was from inception to December 1, 2020. The CONSORT statement for abstracts was used to evaluate the reporting quality of RCT abstracts.

RESULTS

A total of 53 RCT abstracts were included. The CONSORT statement for abstracts showed that the average reporting rate of all items was 50.2%. The items with a lower reporting quality were mainly the trial design and the details of randomization and blinding (<10%). The mean overall adherence score across all studies was 8.68 ± 2.69 (range 4-13.5). Multivariate linear regression analysis showed that the higher reporting scores were associated with higher journal impact factor (P < 0.01), international collaboration (P = 0.04), and structured abstract format (P < 0.01).

CONCLUSIONS

Although many RCTs on patients with COVID-19 have been published in different journals, the overall quality of reporting in the included RCT abstracts was suboptimal, thus diminishing their potential usefulness, and this may mislead clinical decision-making. In order to improve the reporting quality, it is necessary to promote and actively apply the CONSORT statement for abstracts.

Assessment of reporting quality in randomised controlled clinical trial abstracts of dental implantology published from 2014 to 2016

OBJECTIVES

Access to full texts of randomised controlled clinical trials (RCTs) is often limited, so brief summaries of studies play a pivotal role. In 2008, a checklist was provided to ensure the transparency and completeness of abstracts. The aim of this investigation was to estimate adherence to the reporting guidelines of the Consolidated Standards of Reporting Trials (CONSORT) criteria for abstracts (CONSORT-A) in RCT publications.

PRIMARY ENDPOINT

Assessment according to the percentage of compliance with the 16 CONSORT-A criteria per study.

MATERIALS AND METHODS

This study is based on a full survey (212 RCT abstracts in dental implantology, PubMed search, publication period 2014-2016, 45 journals, median impact factor: 2.328). In addition to merely documenting 'adherence' to criteria, the authors also assessed the 'complete implementation' of the requested information where possible. The collection of data was performed independently by two dentists, and a final consensus was reached. The primary endpoint was evaluated by medians and quartiles. Additionally, a Poisson regression was conducted to detect influencing factors.

RESULTS

A median of 50% (Q1-Q3: 44%-63%) was documented for the 16 criteria listed in the CONSORT-A statement. Nine of the 16 criteria were considered in fewer than 50% of the abstracts. 'Correct implementation' was achieved for a median of 43% (Q1-Q3: 31%-50%) of the criteria. An additional application of Poisson regression revealed that the number of words used had a locally significant impact on the number of reported CONSORT criteria for abstracts (incidence rate ratio 1.001, 95% CI 1.001 to 1.002).

CONCLUSION

Transparent and complete reporting in abstracts appears problematic. A limited word count seems to result in a reduction in necessary information. As current scientific knowledge is often not readily available in the form of publications, abstracts constitute the primary basis for decision making in clinical practice and research. This is why journals should refrain from limiting the number of words too strictly in order to facilitate comprehensive reporting in abstracts.

A systematic review of reporting quality for anaesthetic interventions in randomised controlled trials

Interventions from randomised controlled trials can only be replicated if they are reported in sufficient detail. The results of trials can only be confidently interpreted if the delivery of the intervention was systematic and the protocol adhered to. We systematically reviewed trials of anaesthetic interventions published in 12 journals from January 2016 to September 2019. We assessed the detail with which interventions were reported, using the Consolidated Standards of Reporting Trials statement for non-pharmacological treatments. We analysed 162 interventions reported by 78 trials in 18,675 participants. Detail sufficiently precise to replicate the intervention was reported for 111 (69%) interventions. Intervention standardisation was reported for 135 (83%) out of the 162 interventions, and protocol adherence was reported for 20 (12%) interventions. Sixty (77%) out of the 78 trials reported the administrative context in which interventions were delivered and 36 (46%) trials detailed the expertise of the practitioners. We conclude that bespoke reporting tools should be developed for anaesthetic interventions and interventions in other areas such as critical care.

Do not make clinical decisions based on abstracts of healthcare research: A systematic review

OBJECTIVE

To summarize the reporting quality of healthcare abstracts and inconsistencies between abstracts and full texts.

STUDY DESIGN AND SETTING

This systematic review included overviews of randomized controlled trials (RCTs) and systematic reviews (SRs) that summarized data of healthcare abstracts on reporting of abstracts and consistency of abstracts with the full text. Searches were performed in PubMed, CENTRAL, Cochrane Library and EMBASE databases from 1900 to February 2019. Two authors screened the overviews and extracted the data. All analyses were descriptive and divided into two main groups: abstracts' reporting quality and abstracts' consistency with the full text. Abstracts were considered poorly reported and inconsistent with the full text if more than 5% of abstracts' information was not fully reported or not consistent with the full text.

RESULTS

27 overviews analyzing 5,194 RCTs and 866 SRs were retrieved for reporting quality of abstracts. A total of 22 overviews analyzing 2,025 RCTs and 551 SRs were included for consistency of abstracts with the full text. Abstracts across all healthcare areas presented poor reporting quality and were inconsistent with the full texts, with results and conclusions as the most inconsistent sections.

CONCLUSION

Abstracts of healthcare RCTs and SRs have shown a large room for improvement in reporting quality and consistency with the full text. Authors, journal editors and reviewers need to give the highest priority to this matter.

Development and Validation of a Natural Language Processing Tool to Generate the CONSORT Reporting Checklist for Randomized Clinical Trials

IMPORTANCE

Adherence to the Consolidated Standards of Reporting Trials (CONSORT) for randomized clinical trials is associated with improvingquality because inadequate reporting in randomized clinical trials may complicate the interpretation and the application of findings to clinical care.

OBJECTIVE

To evaluate an automated reporting checklist generation tool that uses natural language processing (NLP), called CONSORT-NLP.

DESIGN, SETTING, AND PARTICIPANTS

This study used published journal articles as training, testing, and validation sets to develop, refine, and evaluate the CONSORT-NLP tool. Articles reporting randomized clinical trials were selected from 25 high-impact-factor journals under the following categories: (1) general and internal medicine, (2) oncology, and (3) cardiac and cardiovascular systems.

MAIN OUTCOMES AND MEASURES

For an evaluation of the performance of this tool, an accuracy metric defined as the number of correct assessments divided by all assessments was calculated.

RESULTS

The CONSORT-NLP tool uses the widely used Portable Document Format as an input file. Of the 37 CONSORT reporting items, 34 (92%) were included in the tool. Of these 34 reporting items, 30 were fully implemented; 28 (93%) of the fully implemented CONSORT reporting items had an accuracy of more than 90% for the validation set. The remaining 2 (7%) had an accuracy between 80% and 90% for the validation set. Two to 5 articles were selected from each of these journals for a total of 158 articles to establish a training set of 111 articles to train CONSORT-NLP for CONSORT reporting items, a testing set of 25 articles to refine CONSORT-NLP, and a validation set of 22 articles to assess the performance of CONSORT-NLP. The CONSORT-NLP tool used the Portable Document Format of the articles as input files. A CONSORT-NLP graphical user interface was built using Java in 2019. The time required to complete the CONSORT checklist manually vs using the CONSORT-NLP tool was compared for 30 articles. Two case studies for randomized clinical trials are provided as an illustration for the CONSORT-NLP tool. For the 30 articles investigated, CONSORT-NLP required a mean (SD) 23.0 (4.1) seconds, whereas the manual reviewer required a mean (SD) 11.9 (2.2), 22.6 (4.6), and 57.6 (7.1) minutes, for 3 reviewers, respectively.

CONCLUSIONS AND RELEVANCE

The CONSORT-NLP tool is designed to assist in the reporting of randomized clinical trials. Potential users of CONSORT-NLP include clinicians, researchers, and scientists who plan to publish a randomized trial study in a peer-reviewed journal. The use of CONSORT-NLP may help them save substantial time when generating the CONSORT checklist. This tool may also be useful for manuscript reviewers and journal editors who review these articles.

Biomedical conferences’ author instructions rarely mention guidelines for reporting abstracts of trials and systematic reviews

Aim: To analyze whether instructions for authors of biomedical conference abstracts mention guidelines for writing randomized controlled trial and systematic review abstracts and to evaluate reasons for their absence from instructions. Materials & methods: We analyzed instructions for authors of biomedical conferences advertized in 2019 and assessed whether they mentioned Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Abstracts and Consolidated Standards of Reporting Trials for Abstracts guidelines. We surveyed contact persons from abstract/publication committees of selected conferences to analyze why relevant guidelines were missing. Results: Instructions for abstracts were available for 819 conferences. Only two (0.2%) had reporting instructions for randomized controlled trial/systematic review authors. Almost half of the contacted conference organizers whose response we received were not aware of Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Abstracts and Consolidated Standards of Reporting Trials for Abstracts guidelines. Conclusion: Conference organizers do not require and are not familiar enough with reporting guidelines.

Protocol for a systematic review of reporting standards of anaesthetic interventions in randomised controlled trials

INTRODUCTION

There is significant variation in how anaesthesia is defined and reported in clinical research. This lack of standardisation complicates the interpretation of published evidence and planning of future clinical trials. This systematic review will assess the reporting of anaesthesia as an intervention in randomised controlled trials (RCT) against the Consolidated Standards of Reporting Trials for Non-Pharmacological Treatments (CONSORT-NPT) framework.

METHODS AND ANALYSIS

Online archives of the top six journals ranked by impact factor for anaesthesia and the top three general medicine and general surgery journals will be systematically hand searched over a 42-month time period to identify RCTs describing the use of anaesthetic interventions for any invasive procedure. All modes of anaesthesia and anaesthesia techniques will be included. All study data, including the type of anaesthetic intervention described, will be extracted in keeping with the CONSORT-NPT checklist. Descriptive statistics will be used to summarise general study details including types/modes of anaesthetic interventions, and reporting standards of the trials.

ETHICS AND DISSEMINATION

No ethical approval is required. The results will be used to inform a funding application to formally standardise general, local, regional anaesthesia and sedation for use in clinical research. The systematic review will be disseminated via peer-reviewed manuscript and conferences.

PROSPERO REGISTRATION NUMBER

CRD42019141670.