Hidden connections: Network effects on editorial decisions in four computer science journals

Does peer review improve the statistical content of manuscripts? A study on 27 467 submissions to four journals

Improving the methodological rigour and the quality of data analysis in manuscripts submitted to journals is key to ensure the validity of scientific claims. However, there is scant knowledge of how manuscripts change throughout the review process in academic journals. Here, we examined 27 467 manuscripts submitted to four journals from the Royal Society (2006-2017) and analysed the effect of peer review on the amount of statistical content of manuscripts, i.e. one of the most important aspects to assess the methodological rigour of manuscripts. We found that manuscripts with both initial low or high levels of statistical content increased their statistical content during peer review. The availability of guidelines on statistics in the review forms of journals was associated with an initial similarity of statistical content of manuscripts but did not have any relevant implications on manuscript change during peer review. We found that when reports were more concentrated on statistical content, there was a higher probability that these manuscripts were eventually rejected by editors.

Measuring the developmental function of peer review: a multi-dimensional, cross-disciplinary analysis of peer review reports from 740 academic journals

Reviewers do not only help editors to screen manuscripts for publication in academic journals; they also serve to increase the rigor and value of manuscripts by constructive feedback. However, measuring this developmental function of peer review is difficult as it requires fine-grained data on reports and journals without any optimal benchmark. To fill this gap, we adapted a recently proposed quality assessment tool and tested it on a sample of 1.3 million reports submitted to 740 Elsevier journals in 2018-2020. Results showed that the developmental standards of peer review are shared across areas of research, yet with remarkable differences. Reports submitted to social science and economics journals show the highest developmental standards. Reports from junior reviewers, women and reviewers from Western Europe are generally more developmental than those from senior, men and reviewers working in academic institutions outside Western regions. Our findings suggest that increasing the standards of peer review at journals requires effort to assess interventions and measure practices with context-specific and multi-dimensional frameworks.

Cumulative advantage and citation performance of repeat authors in scholarly journals

Cumulative advantage–commonly known as the Matthew Effect–influences academic output and careers. Given the challenge and uncertainty of gauging the quality of academic research, gatekeepers often possess incentives to prefer the work of established academics. Such preferences breach scientific norms of universalism and can stifle innovation. This article analyzes repeat authors within academic journals as a possible exemplar of the Matthew Effect. Using publication data for 347 economics journals from 1980–2017, as well as from three major generalist science journals, we analyze how articles written by repeat authors fare vis-à-vis less-experienced authors. Results show that articles written by repeat authors steadily decline in citation impact with each additional repeat authorship. Despite these declines, repeat authors also tend to garner more citations than debut authors. These contrasting results suggest both benefits and drawbacks associated with repeat authorships. Journals appear to respond to feedback from previous publications, as more-cited authors in a journal are more likely to be selected for repeat authorships. Institutional characteristics of journals also affect the likelihood of repeat authorship, as well as citation outcomes. Repeat authorships–particularly in leading academic journals–reflect innovative incentives and professional reward structures, while also influencing the intellectual content of science.

Introducing a bibliometric index based on factor analysis

This work applies a factor analysis with VARIMAX rotation to develop a bibliometric indicator, named the Weighted Factor Index, in order to derive a new classification for journals belonging to a certain category, alternative to the one provided by the Journal Impact Factor. For this, 16 metrics from three different databases (Web of Science, Scopus and SCImago Journal Rank) are considered. The Weighed Factor Index entails the advantage of incorporating and summarizing information from all the indicators; so as to test its performance, it was applied to rank journals belonging to the category Information Science & Library Science.

Journal titles and mission statements: Lexical structure, diversity, and readability in business, management and accounting research

There is an established research agenda on dissecting an article’s components and their association with a journal’s prestige. However, journals’ titles and their overview, aim and scope (i.e. journal’s mission statement – JMS(s)) have not been investigated with the same diligence. This study aims to conduct a comprehensive outlook of titles and JMSs’ lexical structure and identify significant differences between journals prestige and type of access and their JMS content in the field of business, management and accounting (BMA), considering the field’s experience in developing and applying mission statements. Titles and JMSs’ structural analysis reflected current and critical discussion in BMA: a predilection for counterintuitive findings and information and communication technology (ICT) tools. JMSs expressed primarily target customers and markets. JMSs from reputable journals showed a higher betweenness for key-terms related to rigorous features. In contrast, JMSs of lower reputable journals highlighted indexing attributes. The Wilcoxon rank-sum and the Kruskal–Wallis tests showed significant differences in the JMSs’ median diversity regarding the journal’s type of access and best quartiles.

Peer review and gender bias: A study on 145 scholarly journals

Scholarly journals are often blamed for a gender gap in publication rates, but it is unclear whether peer review and editorial processes contribute to it. This article examines gender bias in peer review with data for 145 journals in various fields of research, including about 1.7 million authors and 740,000 referees. We reconstructed three possible sources of bias, i.e., the editorial selection of referees, referee recommendations, and editorial decisions, and examined all their possible relationships. Results showed that manuscripts written by women as solo authors or coauthored by women were treated even more favorably by referees and editors. Although there were some differences between fields of research, our findings suggest that peer review and editorial processes do not penalize manuscripts by women. However, increasing gender diversity in editorial teams and referee pools could help journals inform potential authors about their attention to these factors and so stimulate participation by women.

Influential variables in the Journal Impact Factor of Dentistry journals

Objective

The aim of this contribution is to determine what variables influence the position, by quartiles of the impact factor, as a quality indicator of a journal in the field of Dentistry.

Methods

To this end, 24 journals included in Journal Citation Reports, 6 pertaining to each quartile were selected by a stratified sampling and then an ordinal regression model was estimated stepwise considering the journal impact factor quartile as response variable.

Results

The estimation procedure concluded that the average number of papers published yearly by a journal and the percentage of systematic reviews are the most significant variables to be considered, along with the factor representing the journal's degree of adherence to recommendations by the International Committee of Medical Journal Editors.

Conclusions/Clinical significance

Systematic reviews have significant effect on the Journal Impact Factor position of a journal as well as adherence to ICMJE recommendations, while papers publishing clinical trials bear no influence on this factor. Greater yearly average of published papers in a journal means a higher impact factor.

Editors Should Declare Conflicts of Interest

Editors have increasing pressure as scholarly publishing tries to shore up trust and reassure academics and the public that traditional peer review is robust, fail-safe, and corrective. Hidden conflicts of interest (COIs) may skew the fairness of the publishing process because they could allow the status of personal or professional relationships to positively influence the outcome of peer review or reduce the processing period of this process. Not all authors have such privileged relationships. In academic journals, editors usually have very specialized skills and are selected as agents of trust, entrusted with the responsibility of serving as quality control gate-keepers during peer review. In many cases, editors form extensive networks, either with other professionals, industry, academic bodies, journals, or publishers. Such networks and relationships may influence their decisions or even their subjectivity towards a set of submitting authors, paper, or institute, ultimately influencing the peer review process. These positions and relationships are not simply aspects of a curriculum, they are potential COIs. Thus, on the editorial board of all academic journals, editors should carry a COI statement that reflects their past history, as well as actual relationships and positions that they have, as these may influence their editorial functions.

The effect of publishing peer review reports on referee behavior in five scholarly journals

To increase transparency in science, some scholarly journals are publishing peer review reports. But it is unclear how this practice affects the peer review process. Here, we examine the effect of publishing peer review reports on referee behavior in five scholarly journals involved in a pilot study at Elsevier. By considering 9,220 submissions and 18,525 reviews from 2010 to 2017, we measured changes both before and during the pilot and found that publishing reports did not significantly compromise referees' willingness to review, recommendations, or turn-around times. Younger and non-academic scholars were more willing to accept to review and provided more positive and objective recommendations. Male referees tended to write more constructive reports during the pilot. Only 8.1% of referees agreed to reveal their identity in the published report. These findings suggest that open peer review does not compromise the process, at least when referees are able to protect their anonymity.

The peer review game: an agent-based model of scientists facing resource constraints and institutional pressures

This paper looks at peer review as a cooperation dilemma through a game-theory framework. We built an agent-based model to estimate how much the quality of peer review is influenced by different resource allocation strategies followed by scientists dealing with multiple tasks, i.e., publishing and reviewing. We assumed that scientists were sensitive to acceptance or rejection of their manuscripts and the fairness of peer review to which they were exposed before reviewing. We also assumed that they could be realistic or excessively over-confident about the quality of their manuscripts when reviewing. Furthermore, we assumed they could be sensitive to competitive pressures provided by the institutional context in which they were embedded. Results showed that the bias and quality of publications greatly depend on reviewer motivations but also that context pressures can have a negative effect. However, while excessive competition can be detrimental to minimising publication bias, a certain level of competition is instrumental to ensure the high quality of publication especially when scientists accept reviewing for reciprocity motives.