Guidance in author instructions of hematology and oncology journals: A cross sectional and longitudinal study

The relationship between endorsing reporting guidelines or trial registration and the impact factor or total citations in surgical journals

BACKGROUND

A journal's impact factor (IF) and total citations are often used as indicators of its publication quality. Furthermore, journals that require authors to abide by reporting guidelines or conduct trial registration generally have a higher quality of reporting. In this study, we sought to explore the potential associations between the enforcement of reporting guidelines or trial registration and a surgical journal's IF or total citations in order to find new approaches and ideas to improve journal publication quality.

METHODS

We examined surgical journals from the 2018 Journal Citation Report's Expanded Scientific Citation Index to quantify the use of reporting guidelines or study registration. We reviewed the "instructions for authors" from each journal and used multivariable linear regression analysis to determine which guidelines were associated with the journal IF and total citations. The dependent variable was the logarithm base 10 of the IF in 2018 or the logarithm base 10 of total citations in 2018 (the results were presented as geometric means, specifically the ratio of the "endorsed group" results to "not endorsed group" results). The independent variable was one of the requirements (endorsed and not endorsed). Models adjust for the publication region, language, start year, publisher and journal size (only used to adjust total citations).

RESULTS

We included 188 surgical journals in our study. The results of multivariable linear regression analysis showed that journal IF was associated (P < 0.01) with the following requirements: randomized controlled trial (RCT) registration (geometric means ratio (GR) = 1.422, 95% CI [1.197-1.694]), Consolidated Standards of Reporting Trials (CONSORT) statement (1.318, [1.104-1.578]), Preferred Reporting Items for Systematic Reviews Meta-Analyses (PRISMA) statement (1.390, [1.148-1.683]), Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement (1.556, [1.262-1.919]), Standards for Reporting Diagnostic Accuracy (STARD) statement (1.585, [1.216-2.070]), and Meta-analysis of Observational Studies in Epidemiology (MOOSE) statement (2.113, [1.422-3.133]). We found associations between the endorsement of RCT registration (GR = 1.652, 95% CI [1.268-2.153]), CONSORT (1.570, [1.199-2.061]), PRISMA (1.698, [1.271-2.270]), STROBE (2.023, [1.476-2.773]), STARD (2.173, [1.452-3.243]), and MOOSE statements (2.249, [1.219-4.150]) and the number of total citations.

CONCLUSION

The presence of reporting guidelines and trial registration was associated with higher IF or more total citations in surgical journals. If more surgical journals incorporate these policies into their submission requirements, this may improve publication quality, thus increasing their IF and total citations.

Assessing journal author guidelines for systematic reviews and meta-analyses: findings from an institutional sample

OBJECTIVES

Systematic reviews and meta-analyses (SRs/MAs) are designed to be rigorous research methodologies that synthesize information and inform practice. An increase in their publication runs parallel to quality concerns and a movement toward standards to improve reporting and methodology. With the goal of informing the guidance librarians provide to SR/MA teams, this study assesses online journal author guidelines from an institutional sample to determine whether these author guidelines address SR/MA methodological quality.

METHODS

A Web of Science Core Collection (Clarivate) search identified SRs/MAs published in 2014-2019 by authors affiliated with a single institution. The AMSTAR 2 checklist was used to develop an assessment tool of closed questions specific to measures for SR/MA methodological quality in author guidelines, with questions added about author guidelines in general. Multiple reviewers completed the assessment.

RESULTS

The author guidelines of 141 journals were evaluated. Less than 20% addressed at least one of the assessed measures specific to SR/MA methodological quality. There was wide variation in author guidelines between journals from the same publisher apart from the American Medical Association, which consistently offered in-depth author guidelines. Normalized Eigenfactor and Article Influence Scores did not indicate author guideline breadth.

CONCLUSIONS

Most author guidelines in the institutional sample did not address SR/MA methodological quality. When consulting with teams embarking on SRs/MAs, librarians should not expect author guidelines to provide details about the requirements of the target journals. Librarians should advise teams to follow established SR/MA standards, contact journal staff, and review SRs/MAs previously published in the journal.

Requirements for Study Registration and Adherence to Reporting Guidelines in Surgery Journals: A Cross-Sectional Study

BACKGROUND

Reporting guidelines and study registration can minimize bias and improve the reporting quality of biomedical research, but may not be fully utilized. The objective of this study was to investigate the policies of surgery journals as for reporting guidelines and study registration and explore associated journal characteristic variables.

METHODS

Study samples were obtained from the Expanded Science Citation Index of the 2018 Journal Citation Reports (surgery category). The online guides for authors were browsed to identify which journals endorsed reporting guidelines and study registration. The predictors related to the endorsement were explored by using Chi-square test and multivariate logistic regression analysis, respectively.

RESULTS

One hundred and eighty-eight surgery journals were included in our study. One hundred and sixty-three journals (86.7%) endorsed reporting guidelines and 103 journals (54.8%) endorsed study registration. About reporting guidelines, ICMJE (International Committee of Medical Journal Editors) recommendations were the most frequently endorsed (n = 155, 82.4%) by journals, followed by CONSORT (Consolidated Standards of Reporting Trials) statement (n = 94, 50.0%). About study registration, randomized controlled trial registration was endorsed by 101 (53.7%) journals, whereas the systematic review registration was endorsed by only 9 journals (4.8%). The results of multivariate logistic regression analysis revealed that not North America, higher JCR (Journal Citation Reports) rank journals were more likely to endorse reporting guidelines and study registration.

CONCLUSIONS

Surgery journals frequently use reporting guidelines, but nearly half of journals did not require study registration. Implementing these two mechanisms can prevent bias, and their adoption should be strengthened by authors, reviewers and journal editors in surgery.

Compliance With Standards for STARD 2015 Reporting Recommendations in Pathology

OBJECTIVES

Lack of experimental reproducibility has led to growing interest in guidelines to enhance completeness and transparency in research reporting. This retrospective survey sought to determine compliance with Standards for Reporting of Diagnostic Accuracy Studies (STARD) 2015 statement in the recent pathology scientific literature.

METHODS

Two raters independently scored 171 pathology diagnostic accuracy studies for compliance with 34 STARD items and subcomponents. Overall adherence was calculated as a proportion after excluding nonapplicable items.

RESULTS

After excluding nonapplicable items, there was 50% overall adherence to STARD reporting recommendations. In total, 15.44 ± 3.59 items were reported per article (range, 4-28 out of maximum possible of 34). There was substantial heterogeneity in individual item reporting, with greater than 75% reporting in eight of 34 items and less than 25% reporting in 11 of 34 items. Less than 10% of articles reported hypotheses, subgroup analyses for confounding, sample size calculations, subject flow diagrams, study registrations, and links to full study protocols. Significantly more items were reported in articles from journals that endorsed STARD (16.14 vs 14.84, P = .0175).

CONCLUSIONS

These findings demonstrate incomplete reporting of essential items in pathology diagnostic accuracy studies. More vigorous enforcement of reporting checklists might improve adherence to minimum reporting standards.

Online survey about the STROBE statement highlighted diverging views about its content, purpose, and value

BACKGROUND AND OBJECTIVE

The endorsement rates of The STrengthening the Reporting of OBservational studies in Epidemiology (STROBE) Statement are low and little is known about authors' opinions about this reporting guideline. We conducted an online survey with observational study authors on attitude toward and experiences with the STROBE Statement with the aim of understanding how to effectively implement STROBE.

METHODS

A thematic analysis on the responses to an open-ended question was conducted using inductive coding. Two coders classified responses independently into themes using a codebook. The inter-rater agreement ranged from 87.7 to 99.9%.

RESULTS

15% (n = 150) of survey participants (n = 1,015) shared perceptions and insights on STROBE. We established four themes: 1) perceptions of the checklist, 2) academic confidence, 3) use in education and training, and 4) journal endorsement and use in peer review. Views were diverse and revealed multiple misunderstandings about the checklist's purpose and content, and lack of incentives for its use.

CONCLUSIONS

Better communication efforts are needed when disseminating STROBE and other reporting guidelines. These should focus on content, education for early career researchers, and encouragement of critical self-reflection on one's own work. In addition, results emphasized the need for better incentive and enforcement mechanisms.

Influential variables in the Journal Impact Factor of Dentistry journals

Objective

The aim of this contribution is to determine what variables influence the position, by quartiles of the impact factor, as a quality indicator of a journal in the field of Dentistry.

Methods

To this end, 24 journals included in Journal Citation Reports, 6 pertaining to each quartile were selected by a stratified sampling and then an ordinal regression model was estimated stepwise considering the journal impact factor quartile as response variable.

Results

The estimation procedure concluded that the average number of papers published yearly by a journal and the percentage of systematic reviews are the most significant variables to be considered, along with the factor representing the journal's degree of adherence to recommendations by the International Committee of Medical Journal Editors.

Conclusions/Clinical significance

Systematic reviews have significant effect on the Journal Impact Factor position of a journal as well as adherence to ICMJE recommendations, while papers publishing clinical trials bear no influence on this factor. Greater yearly average of published papers in a journal means a higher impact factor.

Biomedical conferences’ author instructions rarely mention guidelines for reporting abstracts of trials and systematic reviews

Aim: To analyze whether instructions for authors of biomedical conference abstracts mention guidelines for writing randomized controlled trial and systematic review abstracts and to evaluate reasons for their absence from instructions. Materials & methods: We analyzed instructions for authors of biomedical conferences advertized in 2019 and assessed whether they mentioned Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Abstracts and Consolidated Standards of Reporting Trials for Abstracts guidelines. We surveyed contact persons from abstract/publication committees of selected conferences to analyze why relevant guidelines were missing. Results: Instructions for abstracts were available for 819 conferences. Only two (0.2%) had reporting instructions for randomized controlled trial/systematic review authors. Almost half of the contacted conference organizers whose response we received were not aware of Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Abstracts and Consolidated Standards of Reporting Trials for Abstracts guidelines. Conclusion: Conference organizers do not require and are not familiar enough with reporting guidelines.

A cross-sectional bibliometric study showed suboptimal journal endorsement rates of STROBE and its extensions

OBJECTIVES

The STrengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement provides guidance on reporting observational studies. Many extensions have been created for specialized methods or fields. We determined endorsement prevalence and typology by journals in extension-related fields.

STUDY DESIGN AND SETTING

A published protocol defined search strategies to identify journals publishing observational studies (2007-2017) across seven fields relating to STROBE extensions. We extracted text regarding STROBE, seven STROBE extensions, reporting guidelines Consolidated Standards of Reporting Trials and Preferred Reporting Items for Systematic Reviews and Meta-Analyses, and transparent reporting documents/groups: International Committee of Medical Journal Editors, Committee on Publication Ethics (COPE), and the Enhancing the QUAlity and Transparency Of health Research (EQUATOR) networks. Relationships between endorsing STROBE, endorsing other guidelines, and journal impact factor were tested using chi square and Mann-Whitney tests.

RESULTS

Of 257 unique journals, 12 (5%) required STROBE on submission, 22 (9%) suggested use, 12 (5%) recommended a "relevant guideline," 72 (28%) mentioned it indirectly (via editorial policies or International Committee of Medical Journal Editors recommendations), and 139 (54%) did not mention STROBE. The relevant extension was required by 2 (<1%) journals; 4 (1%) suggested use. STROBE endorsement was not associated with journal impact indices but was with Consolidated Standards of Reporting Trials and Preferred Reporting Items for Systematic Reviews and Meta-Analyses endorsements.

CONCLUSION

Reporting guideline endorsement rates are low; information is vague and scattered. Unambiguous language is needed to improve adherence to reporting guidelines and increase the quality of reporting.

Quality of critical care clinical practice guidelines: Assessment with AGREE II instrument

STUDY OBJECTIVE

Clinical practice guidelines (CPGs) are cornerstones for the management of critically ill patients. Numerous CPGs have been generated in critical care medicine, but their qualities have never been systematically appraised. The aim of the present study was to systematically assess the quality of critical care CPGs.

DESIGN

A systematic electronic search was performed in PubMed and Scopus. All critical care CPGs were included for analysis.

SETTING

Not applicable.

PATIENTS

Not applicable.

INTERVENTION

None.

MEASUREMENTS

The Appraisal of guidelines for research & evaluation II (AGREE II) instrument was employed to appraise the quality. CPGs were assessed independently by three raters and intraclass correlation coefficient to represent the agreement among raters.

MAIN RESULTS

A total of 89 CPGs were included for quantitative analysis. The results showed that domain 1 (scope and purpose) had the highest scores (0.93, IQR: 0.89-0.98) and domain 2 (stakeholder involvement) had the lowest scores (0.37, IQR: 0.30-0.46). The overall score was 0.83 (IQR: 0.67-0.83). Publication year was not associated with scaled scores in each domain. Domain 2 (stakeholder involvement) was significantly associated with the number of societies (coefficient: 0.702, p = 0.033). Also, greater number of societies were associated with higher scaled scores of domain 3 (coefficient: 0.768, p = 0.027), 4 (coefficient: 0.730, p = 0.029) and 5 (coefficient: 0.995, p = 0.023).

CONCLUSIONS

The study showed that the reporting quality of critical care CPGs were suboptimal. The reporting quality varied across the six domains, with the highest quality in domain 1 and lowest quality in domain 2. Strenuous efforts need to be made to improve the reporting of critical care CPGs.

Evaluations of the uptake and impact of the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement and extensions: a scoping review

BACKGROUND

The PRISMA Statement is a reporting guideline designed to improve transparency of systematic reviews (SRs) and meta-analyses. Seven extensions to the PRISMA Statement have been published to address the reporting of different types or aspects of SRs, and another eight are in development. We performed a scoping review to map the research that has been conducted to evaluate the uptake and impact of the PRISMA Statement and extensions. We also synthesised studies evaluating how well SRs published after the PRISMA Statement was disseminated adhere to its recommendations.

METHODS

We searched for meta-research studies indexed in MEDLINE® from inception to 31 July 2017, which investigated some component of the PRISMA Statement or extensions (e.g. SR adherence to PRISMA, journal endorsement of PRISMA). One author screened all records and classified the types of evidence available in the studies. We pooled data on SR adherence to individual PRISMA items across all SRs in the included studies and across SRs published after 2009 (the year PRISMA was disseminated).

RESULTS

We included 100 meta-research studies. The most common type of evidence available was data on SR adherence to the PRISMA Statement, which has been evaluated in 57 studies that have assessed 6487 SRs. The pooled results of these studies suggest that reporting of many items in the PRISMA Statement is suboptimal, even in the 2382 SRs published after 2009 (where nine items were adhered to by fewer than 67% of SRs). Few meta-research studies have evaluated the adherence of SRs to the PRISMA extensions or strategies to increase adherence to the PRISMA Statement and extensions.

CONCLUSIONS

Many studies have evaluated how well SRs adhere to the PRISMA Statement, and the pooled result of these suggest that reporting of many items is suboptimal. An update of the PRISMA Statement, along with a toolkit of strategies to help journals endorse and implement the updated guideline, may improve the transparency of SRs.