The manuscript reviewing process: Empirical research on review requests, review sequences, and decision rules in peer review

The consistency of peer-reviewers: Assessment of separate parts of the manuscripts vs final recommendations

Delving into the review reports, this paper is aimed at analyzing reviewers` attitudes toward different sections of the manuscripts they review. The research focuses on the consistency of reviewers` evaluation through analysis of their assessment of separate parts of a paper, if it corresponds with the recommendations they made to the editors and whether a paper needs revision or should be accepted/rejected. It is assumed that the assessment of separate parts of a paper should be consistent with the final decision regarding the acceptance or rejection of a manuscript. Based on the analysis presented in this paper it can be concluded that the assessments of separate parts of articles in the evaluation sheets do not fully reflect the final recommendations of the reviewers. The results showed that the most correlated and therefore the most significant sections for the reviewers are the main text and the conclusions. The conditional probability analysis showed that the decision of reviewers, when number of points in the evaluation sheet is taken into consideration, is slightly unpredictable. No significant differences in the reviewers` recommendations based on gender or country of origin of the reviewers were found.

Practical Tips of English Expressions for Non-Native English-Speaking Peer Reviewers

Peer review is an essential part in ensuring the quality of papers published in scientific journals. Good reviews are beneficial for both authors and journal editors by providing authors with the opportunity to improve their manuscripts and editors with valuable comments to aid themselves to make their decisions. Despite a plethora of research articles on general guidelines for peer reviewing, it is difficult to find papers on English expressions peer reviewers can use, in particular for non-native English-speaking peer reviewers. Therefore, this article provides “down-to-earth” guidance for non-native English-speaking reviewers to construct better quality reviews. To this end, I suggest useful English expressions to help peer reviewers, whose mother tongue is not English, enhance the quality of their reviews.

Everything You Need to Know About Peer Review - The Good, The Bad and The Ugly

Embarking on conducting peer reviews for academic journals can present a new and exciting challenge for early career researchers. This article offers succinct guidance about peer review: not only "what to do" (the Good) but also "what not to do" (the Bad) and "what to never do" (the Ugly). It outlines models of peer review and provides an overview of types of reviewer bias, including conflict of interest. More recent developments in journal peer review, such as author-suggested reviewers as well as manipulation of the peer review process are also discussed. A new position of Editorial Fellow at Heart, Lung and Circulation will provide aspiring researchers the opportunity for multi-faceted involvement with peer review at the Journal.

Gatekeepers of Reward: a Pilot Study on the Ethics of Editing and Competing Evaluations of Value

The reward infrastructure in science centres on publication, in which journal editors play a key role. Reward distribution hinges on value assessments performed by editors, who draw from plural value systems to judge manuscripts. This conceptual paper examines the numerous biases and other factors that affect editorial decisions. Hybrid and often conflicting value systems contribute to an infrastructure in which editors manage reward through editorial review, commissioned commentaries and reviews and weighing of peer review judgments. Taken together, these systems and processes push the editor into a role resembling censorship. Editors and authors both experience this phenomenon as an unintended side-effect of the reward infrastructure in science. To work towards a more constructive editor-author relationship, we propose a conversation, an exchange between editor and author in which value is collectively assessed (or constructed) as obligatory passage points in the publishing process are traversed. This paper contributes to the discourse on editorial practices by problematising editorial paradigms in a new way and suggesting solutions to entrenched problems.

The conceptual foundation of environmental decision support

Environmental decision support intends to use the best available scientific knowledge to help decision makers find and evaluate management alternatives. The goal of this process is to achieve the best fulfillment of societal objectives. This requires a careful analysis of (i) how scientific knowledge can be represented and quantified, (ii) how societal preferences can be described and elicited, and (iii) how these concepts can best be used to support communication with authorities, politicians, and the public in environmental management. The goal of this paper is to discuss key requirements for a conceptual framework to address these issues and to suggest how these can best be met. We argue that a combination of probability theory and scenario planning with multi-attribute utility theory fulfills these requirements, and discuss adaptations and extensions of these theories to improve their application for supporting environmental decision making. With respect to (i) we suggest the use of intersubjective probabilities, if required extended to imprecise probabilities, to describe the current state of scientific knowledge. To address (ii), we emphasize the importance of value functions, in addition to utilities, to support decisions under risk. We discuss the need for testing "non-standard" value aggregation techniques, the usefulness of flexibility of value functions regarding attribute data availability, the elicitation of value functions for sub-objectives from experts, and the consideration of uncertainty in value and utility elicitation. With respect to (iii), we outline a well-structured procedure for transparent environmental decision support that is based on a clear separation of scientific prediction and societal valuation. We illustrate aspects of the suggested methodology by its application to river management in general and with a small, didactical case study on spatial river rehabilitation prioritization.

The usefulness of peer review for selecting manuscripts for publication: a utility analysis taking as an example a high-impact journal

Background

High predictive validity – that is, a strong association between the outcome of peer review (usually, reviewers' ratings) and the scientific quality of a manuscript submitted to a journal (measured as citations of the later published paper) – does not as a rule suffice to demonstrate the usefulness of peer review for the selection of manuscripts. To assess usefulness, it is important to include in addition the base rate (proportion of submissions that are fundamentally suitable for publication) and the selection rate (the proportion of submissions accepted).

Methodology/Principal Findings

Taking the example of the high-impact journal Angewandte Chemie International Edition (AC-IE), we present a general approach for determining the usefulness of peer reviews for the selection of manuscripts for publication. The results of our study show that peer review is useful: 78% of the submissions accepted by AC-IE are correctly accepted for publication when the editor's decision is based on one review, 69% of the submissions are correctly accepted for publication when the editor's decision is based on two reviews, and 65% of the submissions are correctly accepted for publication when the editor's decision is based on three reviews.

Conclusions/Significance

The paper points out through what changes in the selection rate, base rate or validity coefficient a higher success rate (utility) in the AC-IE selection process could be achieved.