Better reporting of scientific studies: why it matters

Endorsements of five reporting guidelines for biomedical research by journals of prominent publishers

Biomedical research reporting guidelines provide a framework by which journal editors and the researchers who conduct studies can ensure that the reported research is both complete and transparent. With more than 16 different guidelines for the 11 major study types of medical and health research, authors need to be familiar with journal reporting standards. To assess the current endorsements of reporting guidelines for biomedical and health research, this study examined the instructions for authors (IFAs) of 559 biomedical journals by 11 prominent publishers that publish original research or systematic reviews/meta-analyses. Data from the above original sources were cleaned and restructured, and analyzed in a database and text miner. Each journal's instructions or information for authors were examined to code if any of five prominent reporting guidelines were mentioned and what form the guideline adherence demonstration took. Seventeen journals published the reporting guidelines. Four of the five reporting guidelines listed journals as endorsers. For journals with open peer review reports, a sample of journals and peer reviews was analyzed for mention of adherence to reporting guidelines. The endorsement of research guidelines by publishers and their associated journals is inconsistent for some publishers, with only a small number of journals endorsing relevant guidelines. Based on the analysis of open peer reviews, there is evidence that some reviewers check the adherence to the endorsed reporting guidelines. Currently, there is no universal endorsement of reporting guidelines by publishers nor ways of demonstrating adherence to guidelines. Journals may not directly inform authors of their guideline endorsements, making it more difficult for authors to adhere to endorsed guidelines. Suggestions derived from the findings are provided for authors, journals, and reporting guidelines to ensure increased adequate use of endorsed reporting guidelines.

Are reporting guidelines used in infectious diseases publications? An analysis of more than 1,000 articles

Objective

To assess whether 16 reporting guidelines of Enhancing the QUAlity and Transparency Of Health Research (EQUATOR) were used in infectious diseases research publications.

Design

This cross-sectional, audit-type study assessed articles published in five infectious diseases journals in 2019.

Methods

All articles were manually reviewed to assess if a reporting guideline was advisable and searched for the names and acronyms of 16 reporting guidelines. An "advisable use rate" was calculated.

Results

We reviewed 1,251 manuscripts across five infectious diseases journals. Guideline use was advisable for 973 (75%) articles. Reporting guidelines were used in 85 articles, 6.1% of total articles, and 8% (95% CI 6%-9%) of articles for which guidelines were advised. The advisable use rate ranged from 0.06 to 0.17 for any guideline, 0-0.08 for CONSORT, 0.53-1 for Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA), and 0-0.66 for Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD) : The TRIPOD statement. No trends were observed across the five journals.

Conclusions

The use of EQUATOR-related reporting guidelines is infrequent, despite journals and publishers promoting their usage. Whether this finding is attributable to knowledge, acceptance, or perceived usefulness of the guidelines still needs to be clarified.

Unusual Findings in Trials Evaluating Adjuncts to Scaling and Root Planing: Reporting Quality (Part 2)

INTRODUCTION

On the topic of adjuncts to scaling and root planing (SRP), numerous randomized clinical trials (RCTs) were published by a single group of authors and frequently reported unusually large effect sizes. A meta-analysis in part 1 of this project failed to explain the causes for these unusual findings. We assessed the reporting quality and trial registration discrepancies to examine the possibility of replicating the work of this research group as well as the overall rigor of the research methodology.

METHODS

This study was preregistered at the Open Science Framework (https://osf.io/4meyd/). The Scopus platform was utilized for the RCT search on SRP adjuncts in intrabony defects in patients with periodontitis as compared with SRP alone. The search analysis was limited from 2010 to 2017, and RCTs on SRP adjuncts published by a single research group were selected for screening and inclusion. RCT registration records were assessed for consistency.

RESULTS

Out of 92 studies that were retrieved from Scopus and PubMed, 32 were included for quality assessment per the CONSORT guidelines (Consolidated Standards of Reporting Trials). Results showed that all RCTs were characterized by a low reporting quality. The majority of CONSORT items scored <50%, including critical items (randomization, registration, and blinding). When registration records were compared with published RCTs, several discrepancies were found. The per-protocol follow-up duration was compared against the study's initiation and termination dates. Only 38% of the RCTs presented a follow-up period within the initiation and termination dates. The remaining RCTs showed inconsistent follow-up in comparison with the initiation and termination dates.

CONCLUSION

RCTs by this group were characterized by poor adherence to reporting quality guidelines. Crucial RCT elements, such as randomization, blinding, and primary outcomes, were not reported properly. RCT registration records revealed systematic inconsistencies when compared with RCT publication. Therefore, the unusually large effects reported by this group should be viewed with extreme caution.

KNOWLEDGE TRANSFER STATEMENT

The included randomized clinical trials were characterized by poor adherence to reporting quality guidelines, missing information about important trial items, and discrepancies between the reports and trial registrations. This quality assessment should guide clinical research and show clinicians that they should be cautious when applying evidence in their clinical practice.

Adherence to the Consolidated Standards of Reporting Trials (CONSORT) Guidelines for Reporting Randomized Controlled Trials Related to Mandibular Third Molars

PURPOSE

Randomized controlled trials (RCTs) are the backbone of evidence-based medicine. Despite the widespread acceptance of the Consolidated Standards of Reporting Trials (CONSORT), its use for reporting clinical trials in journals remains poor. The purpose of this study was to find out to what extent RCTs related to mandibular third molars have adhered to the CONSORT statement.

METHODS

This study was carried out during April 2020 to June 2020. PubMed was used to retrieve RCTs related to mandibular third molars conducted during the last 5 years. The search terms used were mandibular third molar OR lower third molar OR impacted mandibular third molar AND randomized controlled trial. Each article was assessed for adherence to the CONSORT statement.

RESULTS

Eighty unique articles were retrieved. The mean percentage adherence to the CONSORT statement noted was 60.26%. Of the 37 CONSORT items, only 4 items showed 100% adherence (2a, 2b, 4a, and 12a). The most underreported items were #10, 12b, 17a, 17b, 18, 19, 23 to 25. Of the 23 journals considered, 6 journals did not recommend CONSORT for reporting RCT in the "Instructions to Authors" guidelines.

CONCLUSIONS

Within the limits of the study, it is clear that the RCTs related to third molar do not show 100% adherence to the CONSORT checklist. The editorial policy, peer reviewers, and researchers should endorse the use of the CONSORT checklist for transparent reporting of the RCTs.

Implementation of interventions designed to promote healthy sleep and circadian rhythms in shiftworkers

Shiftwork is a significant risk factor for a host of negative health and safety outcomes, which have been at least partly attributed to disturbances of the circadian timing system. As a result, an entire sub-field of chronobiology has been devoted to developing and evaluating countermeasures for circadian misalignment, sleep disruption, fatigue, and other issues associated with shiftwork. Much of this research takes place under highly controlled laboratory conditions due to the necessity of accurately characterizing individual rhythms, both for intervention design and assessment of efficacy. Applied studies of interventions for shiftworkers are, by their nature, more complicated, often demonstrating less consistent findings. While this, in part, reflects execution under less rigorously controlled conditions, it may also stem from variability in implementation approaches. A systematic review of published studies (through May 2017) of interventions designed to enhance circadian health in shiftworkers was conducted to determine the frequency and quality of the assessment of implementation as well as barriers and enablers to implementation. A search of PubMed, PsychINFO, Web of Science, and CINAHL databases yielded a total of 5368 unique references. After a title and abstract screen, 323 proceeded to full-text review; 68 of those met final criteria for data extraction. Implementation was assessed to some degree in 60.3% of those 68 articles. Where it was assessed, the mean quality score on a scale from 1 to 5 (1 = very little, 3 = moderate, 5 = very in-depth) was 2.56. One or more enablers were identified in just 17 of the 68 studies (25.0%), and barriers in just 18 (26.5%). Implementation of these interventions is a critical but seldom-acknowledged component of their uptake and effectiveness, and we highly recommend that future shiftworker intervention research make an effort to incorporate formalized assessments of implementation and/or hybrid effectiveness-implementation approaches.

Postoperative follow-up time and justification in prospective hand surgery research: a systematic review

We systematically reviewed prospective studies for five hand procedures to analyse postoperative follow-up time, clinical or radiographic plateau, and whether the authors provide justification for times used. Demographic data, outcomes and mean follow-up were analysed. A total of 188 articles met our inclusion criteria. The mean postoperative follow-up time among these studies were carpal tunnel release, 21 months (range 1.5-111); cubital tunnel release, 27 months (2.5-46); open reduction and internal fixation for the distal radius fracture, 24 months (3-120); thumb carpometacarpal joint arthroplasty, 64 months (8.5-228); and flexor tendon repair, 25 months (3-59). Authors provided justification for follow-up intervals in 10% of these reports. We conclude that most prospective clinical studies in hand surgery do not properly justify follow-up length. Clinically unnecessary follow-up is costly without much benefit. In prospective research, we believe justified postoperative follow-up is essential, based on expected time to detect clinical plateau, capture complications and determine the need for secondary surgery.Level of evidence: III.

Reporting quality of systematic review abstracts in operative dentistry: An assessment using the PRISMA for Abstracts guidelines

OBJECTIVES

To assess and compare the reporting quality of systematic review (SR) abstracts in operative dentistry published before and after the release of Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Abstracts (PRISMA-A), and to identify factors associated with reporting quality.

METHODS

PubMed was searched for abstracts published during 2010-2012 (Pre-PRISMA period) and 2017-2019 (Post-PRISMA period). Reporting quality was assessed and scored using a modified 13-item PRSIMA-A checklist. Risk ratio (RR) was used to compare the adequate reporting rate of each item between the two periods. Univariable and multivariable linear regression analyses were performed to identify factors associated with reporting quality.

RESULTS

A total of 160 abstracts were included and assessed. Only four items ('objective', 'results of main outcomes', 'description of the effect' and 'interpretation') were adequately reported in most abstracts (>75 %). According to the multivariable analysis, greater word count (P = 0.001), being published in the Post-PRISMA period (P = 0.025) and geographic origin from Asia (P = 0.025) or South America (P = 0.015) were significantly associated with higher reporting quality.

CONCLUSIONS/CLINICAL SIGNIFICANCE

The reporting quality of SR abstracts in operative dentistry had improved significantly after the publication of PRISMA-A, but was still suboptimal. Researchers, reviewers and journal editors in operative dentistry need to be familiar with the PRISMA-A checklist, and make concerted efforts to improve the reporting of SR abstracts.

Abstracts of published randomised controlled trials in Endodontics: Reporting quality and spin

AIMS

To assess the reporting quality of recently published randomised controlled trial (RCT) abstracts in Endodontics, to investigate factors associated with reporting quality, and to evaluate the existence and characteristics of spin. Spin refers to reporting strategies that distort study results and misguide readers.

METHODOLOGY

The PubMed database was searched to identify abstracts of RCTs in the field of Endodontics published during 2017 to 2018. Two authors assessed the reporting quality of each included abstract using the original 16-item CONSORT for Abstracts checklist, with the overall quality score (OQS, range: 0 to 16) being the primary outcome measure. For each individual item, a score of '1' was given if it was described adequately, and '0' if the description was inadequate. Linear regression analyses were conducted to identify factors associated with reporting quality. For the evaluation of spin, two authors selected parallel-group RCTs with a nonsignificant primary outcome from the included abstracts, and evaluated independently the existence and characteristics of spin among these abstracts.

RESULTS

A total of 162 abstracts were included for assessment of reporting, for which the mean OQS was 3.97 (SD, 1.30; 95 % CI, 3.77 to 4.17). According to multivariable analysis, origin from Europe (P=0.001) and reporting of the exact P value (P=0.020) were significantly associated with better reporting. Forty abstracts with statistically nonsignificant results for their primary outcome were included for spin evaluation, among which 34 (85.0%) had at least one type of spin. Thirty-two abstracts (94.1%) had spin in their conclusions section, and six abstracts (17.6%) had spin in the results section.

CONCLUSIONS

The reporting quality of RCT abstracts in Endodontics needs to be improved. The occurrence rate of spin in the sample of abstracts of RCTs in the field of Endodontics was high. Relevant stakeholders are recommended to be familiar with the CONSORT for Abstracts guideline and develop active strategies to ensure its implementation.

Randomized controlled comparison of cross-sectional survey approaches to optimize follow-up completeness in clinical studies

Introduction

In outcome research, incomplete follow-up is a major, yet potentially correctable source of bias. Cross-sectional surveys may theoretically increase completeness of follow-up, but low response rates are reported typically. We investigated whether a pre-notification letter improved patient availability for follow-up phone interviews and thereby improved cross-sectional survey yield.

Methods

A consecutive series of vascular patients was randomly divided into a trial and a validation population. The trial population was then randomized 1:1 to one of two cross-sectional contact strategies: Strategy 1 consisted of direct contact attempts by up to 12 systematically timed phone calls, whereas Strategy 2 used a personalized pre-notification letter to arrange for scheduled phone call interviews. Response rates, average time and efforts needed per patient and overall survey duration were compared. Subsequently, trial findings were externally validated in the validation population.

Results

Of 728 consecutive patients, 370 were allocated to the trial population. Trial patients contacted by strategy 1 (n = 183) had a similar profile when compared to trial patients contacted by strategy 2 (n = 187). Follow-up periods following surgery (54.3 versus 53.6 months) and all-cause mortality rates (21.3% versus 18.7%) were comparable between the trial groups. Cross-sectional information on survival outcomes was almost complete after both contact strategies (99.5% versus 98.9%, P = 1.0). In 144/187 strategy 2 patients (77%) interviews were scheduled successfully necessitating significantly less contact attempts (median of 1.3 versus 2.3 per patient, P<0.0001). However, invested time per patient was similar between the groups (median of 10.1 versus 9.6 minutes), and survey strategy 1 completed earlier (median time to contact 4 versus 11 days, P<0.0001). Therefore, strategy 1 was validated in the validation population (n = 358): a low lost to follow-up rate below 1% (P = 1.0) was reconfirmed necessitating an average of 2.3 contact attempts per patient.

Conclusions

Both contact strategies were equally successful in contacting almost all patients cross-sectionally. If systematically timed, direct phone calls were less complicated to organize and faster completed. Given the low time and effort per patient, outcome studies should invest in systematic follow-up surveys to minimize attrition bias.

Rigor, Transparency, and Reporting Social Science Research: Why Guidelines Don't Have to Kill Your Story

Dissemination of research is the most challenging aspect of building the evidence base. Despite peer-review, evidence suggests that a substantial proportion of papers leave out details that are necessary to judge bias, consider replication, or initiate meta-analyses and systematic reviews. Reporting guidelines were created to ensure minimally adequate reporting of research and have become increasingly popular since the 1990s. There are over 200 guidelines for authors to assist in reporting a range of study methodologies. Although guidelines are freely available, they are underutilized and there is criticism regarding assumptions about methodologies targeted by guidelines. As journal editors lean into endorsements, social work authors may benefit from considering guidelines appropriate for their work. This paper explores pros and cons of guideline use by authors and journals and presents some suggestions for the field of social work, including assessment of whether profession-specific reporting guidelines are needed, and cautions regarding limitations.

Reporting quality of randomized controlled trial abstracts: survey of leading general dental journals

BACKGROUND

The authors conducted a study to assess the reporting quality of randomized controlled trial (RCT) abstracts published in leading general dental journals, investigate any improvement after the release of the Consolidated Standards of Reporting Trials (CONSORT) for Abstracts guidelines, and identify factors associated with better reporting quality.

METHODS

The authors searched PubMed for RCTs published in 10 leading general dental journals during the periods from 2005 to 2007 (pre-CONSORT period) and 2010 to 2012 (post-CONSORT period). The authors evaluated and scored the reporting quality of included abstracts by using the original 16-item CONSORT for Abstracts checklist. The authors used risk ratios and the t test to compare the adequate reporting rate of each item and the overall quality in the 2 periods. The authors used univariate and multivariate regressions to identify predictors of better reporting quality.

RESULTS

The authors included and evaluated 276 RCT abstracts. Investigators reported significantly more checklist items during the post-CONSORT period (mean [standard deviation {SD}], 4.53 [1.69]) than during the pre-CONSORT period (mean [SD], 3.87 [1.10]; mean difference, -0.66 [95% confidence interval, -0.99 to -0.33]; P < .001). Investigators reported 3 items-interventions, objective, and conclusions-adequately in most of the abstracts (> 80%). In contrast, the authors saw sufficient reporting of randomization, recruitment, outcome in the results section, and funding in none of the pre-CONSORT abstracts and less than 2% of the post-CONSORT abstracts. On the basis of the multivariate analysis, a higher impact factor (P < .001) and a publication date in the post-CONSORT period (P = .003) were associated significantly with higher reporting quality.

CONCLUSIONS

The reporting quality of RCT abstracts from leading general dental journals has improved significantly, but there is still room for improvement.

PRACTICAL IMPLICATIONS

Joint efforts by authors, reviewers, journal editors, and other stakeholders to improve the reporting of dental RCT abstracts are needed.

Completeness of Follow-Up Determines Validity of Study Findings: Results of a Prospective Repeated Measures Cohort Study

Background

Current reporting guidelines do not call for standardised declaration of follow-up completeness, although study validity depends on the representativeness of measured outcomes. The Follow-Up Index (FUI) describes follow-up completeness at a given study end date as ratio between the investigated and the potential follow-up period. The association between FUI and the accuracy of survival-estimates was investigated.

Methods

FUI and Kaplan-Meier estimates were calculated twice for 1207 consecutive patients undergoing aortic repair during an 11-year period: in a scenario A the population’s clinical routine follow-up data (available from a prospective registry) was analysed conventionally. For the control scenario B, an independent survey was completed at the predefined study end. To determine the relation between FUI and the accuracy of study findings, discrepancies between scenarios regarding FUI, follow-up duration and cumulative survival-estimates were evaluated using multivariate analyses.

Results

Scenario A noted 89 deaths (7.4%) during a mean considered follow-up of 30±28months. Scenario B, although analysing the same study period, detected 304 deaths (25.2%, P<0.001) as it scrutinized the complete follow-up period (49±32months). FUI (0.57±0.35 versus 1.00±0, P<0.001) and cumulative survival estimates (78.7% versus 50.7%, P<0.001) differed significantly between scenarios, suggesting that incomplete follow-up information led to underestimation of mortality. Degree of follow-up completeness (i.e. FUI-quartiles and FUI-intervals) correlated directly with accuracy of study findings: underestimation of long-term mortality increased almost linearly by 30% with every 0.1 drop in FUI (adjusted HR 1.30; 95%-CI 1.24;1.36, P<0.001).

Conclusion

Follow-up completeness is a pre-requisite for reliable outcome assessment and should be declared systematically. FUI represents a simple measure suited as reporting standard. Evidence lacking such information must be challenged as potentially flawed by selection bias.

Improving the transparency of prognosis research: the role of reporting, data sharing, registration, and protocols

George Peat and colleagues review and discuss current approaches to transparency and published debates and concerns about efforts to standardize prognosis research practice, and make five recommendations.

Please see later in the article for the Editors' Summary