Does open peer review improve citation count? Evidence from a propensity score matching analysis of PeerJ

The consistency of peer-reviewers: Assessment of separate parts of the manuscripts vs final recommendations

Delving into the review reports, this paper is aimed at analyzing reviewers` attitudes toward different sections of the manuscripts they review. The research focuses on the consistency of reviewers` evaluation through analysis of their assessment of separate parts of a paper, if it corresponds with the recommendations they made to the editors and whether a paper needs revision or should be accepted/rejected. It is assumed that the assessment of separate parts of a paper should be consistent with the final decision regarding the acceptance or rejection of a manuscript. Based on the analysis presented in this paper it can be concluded that the assessments of separate parts of articles in the evaluation sheets do not fully reflect the final recommendations of the reviewers. The results showed that the most correlated and therefore the most significant sections for the reviewers are the main text and the conclusions. The conditional probability analysis showed that the decision of reviewers, when number of points in the evaluation sheet is taken into consideration, is slightly unpredictable. No significant differences in the reviewers` recommendations based on gender or country of origin of the reviewers were found.

Which structure of academic articles do referees pay more attention to?: perspective of peer review and full-text of academic articles

Purpose

The purpose of this paper is to explore which structures of academic articles referees would pay more attention to, what specific content referees focus on, and whether the distribution of PRC is related to the citations.

Design/methodology/approach

Firstly, utilizing the feature words of section title and hierarchical attention network model (HAN) to identify the academic article structures. Secondly, analyzing the distribution of PRC in different structures according to the position information extracted by rules in PRC. Thirdly, analyzing the distribution of feature words of PRC extracted by the Chi-square test and TF-IDF in different structures. Finally, four correlation analysis methods are used to analyze whether the distribution of PRC in different structures is correlated to the citations.

Findings

The count of PRC distributed in Materials and Methods and Results section is significantly more than that in the structure of Introduction and Discussion, indicating that referees pay more attention to the Material and Methods and Results. The distribution of feature words of PRC in different structures is obviously different, which can reflect the content of referees' concern. There is no correlation between the distribution of PRC in different structures and the citations.

Research limitations/implications

Due to the differences in the way referees write peer review reports, the rules used to extract position information cannot cover all PRC.

Originality/value

The paper finds a pattern in the distribution of PRC in different academic article structures proving the long-term empirical understanding. It also provides insight into academic article writing: researchers should ensure the scientificity of methods and the reliability of results when writing academic article to obtain a high degree of recognition from referees.

“This article is interesting, however”: exploring the language use in the peer review comment of articles published in the BMJ

Purpose

The purpose of this paper is to reveal a symbol – “however” that authors are very interested in, but few research studies pay attention to the existing literature. The authors aim to further insight its function.

Design/methodology/approach

In this research, the authors selected 3,329 valid comments on articles published in the British Medical Journal (BMJ) from 2015 to 2020 as the research objects. The authors showed the length distribution of reviewers' comments. In what follows, the authors analyzed the general distribution of words in comments and reviewer comments’ position to understand reviewers' comments qualitatively in word dimension. Specially, the authors analyzed functions of “however” and “but”, words that authors are most concerned with. In addition, the authors also discussed some factors, which may be related to “however,” that reflect reviewers' praise through regression analysis.

Findings

The authors found that there are marked differences in the length of reviewers' comments under different review rounds. By mapping the reviewers' comments to different sections, the authors found that reviewers are deeply concerned to methods section. Adjectives and adverbs in comments written in different sections of the manuscripts also have different characteristics. The authors tried to interpret the turning function of “however” in scientific communication. Its frequency of use is related to reviewers' identities, specifically academic status. More precisely, junior researchers use “however” in praise more frequently than senior researchers do.

Research limitations/implications

The linguistic feature and function of “however” and “but” in the reviewers' comments of the rejected manuscripts may be different from accepted papers and also worth exploring. Regrettably, the authors cannot obtain the peer review comments of rejected manuscripts. This point may limit the conclusion of the investigation of this article.

Originality/value

Overall, the survey results revealed some language features of reviewers' comments, which could provide a basis of future endeavors for many reviewers in open peer review (OPR) field. Specially, the authors also put forward an interesting symbol to examine the review comments, “however”, for the first time.

Open Research Data and Open Peer Review: Perceptions of a Medical and Health Sciences Community in Greece

Recently significant initiatives have been launched for the dissemination of Open Access as part of the Open Science movement. Nevertheless, two other major pillars of Open Science such as Open Research Data (ORD) and Open Peer Review (OPR) are still in an early stage of development among the communities of researchers and stakeholders. The present study sought to unveil the perceptions of a medical and health sciences community about these issues. Through the investigation of researchers‘ attitudes, valuable conclusions can be drawn, especially in the field of medicine and health sciences, where an explosive growth of scientific publishing exists. A quantitative survey was conducted based on a structured questionnaire, with 179 valid responses. The participants in the survey agreed with the Open Peer Review principles. However, they ignored basic terms like FAIR (Findable, Accessible, Interoperable, and Reusable) and appeared incentivized to permit the exploitation of their data. Regarding Open Peer Review (OPR), participants expressed their agreement, implying their support for a trustworthy evaluation system. Conclusively, researchers need to receive proper training for both Open Research Data principles and Open Peer Review processes which combined with a reformed evaluation system will enable them to take full advantage of the opportunities that arise from the new scholarly publishing and communication landscape.