Peer-reviewing in Surgical Journals: Revolutionize or Perish?

Paying reviewers and regulating the number of papers may help fix the peer-review process

The exponential increase in the number of submissions, further accelerated by generative AI, and the decline in the availability of experts are burdening the peer review process. This has led to high unethical desk rejection rates, a growing appeal for the publication of unreviewed preprints, and a worrying proliferation of predatory journals. The idea of monetarily compensating peer reviewers has been around for many years; maybe, it is time to take it seriously as one way to save the peer review process. Here, I argue that paying reviewers, when done in a fair and transparent way, is a viable solution. Like the case of professional language editors, part-time or full-time professional reviewers, managed by universities or for-profit companies, can be an integral part of modern peer review. Being a professional reviewer could be financially attractive to retired senior researchers and to researchers who enjoy evaluating papers but are not motivated to do so for free. Moreover, not all produced research needs to go through peer review, and thus persuading researchers to limit submissions to their most novel and useful research could also help bring submission volumes to manageable levels. Overall, this paper reckons that the problem is not the peer review process per se but rather its function within an academic ecosystem dominated by an unhealthy culture of 'publish or perish'. Instead of reforming the peer review process, academia has to look for better science dissemination schemes that promote collaboration over competition, engagement over judgement, and research quality and sustainability over quantity.

The Peer Review System: A Journal Editor's 30-Year Perspective

The peer review system has become the standard by which scientific articles are refereed. Unfortunately, even from its beginnings in the mid-1800s it has been fraught with difficulties. Potential reviewers are volunteers who may be inundated with requests to review yet these reviews take considerable time and effort. There is little motivation to complete a review causing significant delays in the publication process. There may be biases unintentionally built into the system between reviewers, authors, editors, and journals. Attempts to overcome these biases by various blinding schemes have been met with limited success. Finally, the recent advent of Artificial Intelligence has the potential to completely upend the system, for good or bad.

A structured, journal-led peer-review mentoring program enhances peer review training

BACKGROUND

Peer review is essential to the advancement of knowledge. However, training on how to conduct peer review is limited, unorganized, and not well studied. Thus, we sought to determine if a structured mentored peer-review program improved peer review training as measured by multiple quantitative and qualitative assessments.

METHODS

This pre-post intervention study enrolled 55 mentees across 5 cohorts from 2020 to 2023. Each cohort completed pre-program evaluations, participated in 2 mentored reviews, and completed post-program evaluations over 6 months. Mentors and mentees completed pre-program demographic and review experience questionnaires. Outcome measures included (1) total and sub-scores on the modified Review Quality Index (mRQI) applied to the same pre-selected research manuscript reviewed by mentees both pre and post intervention, (2) mentee self-perceived comfort with and understanding of the review process using a custom questionnaire, and (3) mentor satisfaction surveys. Pre- and post-program measures were compared using the Wilcoxon signed-rank test.

RESULTS

Post-program total modified RQI score (median (IQR) = 31 (26.3-35.8)) was higher than pre-program total score (26.6 (19.7-29.7)) for the 42 mentees who completed both pre- and post-program reviews. Mentees reported improved perception of review (median (IQR) pre = 4 (3-4), post = 5 (4-5)) and editorial processes (pre = 3 (2-4), post = 4 (4-5)) as well as self-perceived confidence in completing an independent review of both scientific (median (IQR) pre = 2 (2-3), post = 4 (4-4)) and non-scientific (pre = 3 (2-4), post = 4 (4-5)) manuscripts following program participation. p < 0.0001 for all scores noted. Mentors reported high scores for enjoyment (median (range) 5/5 (3-5)) and interest in repeat participation (5/5 (2-5)).

CONCLUSIONS

A 6-month structured mentored-review program including 2 mentored reviews improves peer review training as measured by the modified RQI as well as participant self-perceived understanding of publication science with high mentor satisfaction.

Peer Review and Medical Journals

Journal editors depend on peer reviewers to make decisions about submitted manuscripts. These reviewers help evaluate the methods, the results, the discussion of the results, and the overall organization and presentation of the manuscript. In addition, reviewers can help identify important mistakes and possible misconduct. Editors frequently have difficulty obtaining enough peer reviews which are submitted in a timely manner. This increases the workload of editors and journal managers and potentially delays the publication of clinical and research studies. This commentary discusses of the importance of peer reviews and make suggestions which potentially can increase the participation of academic faculty and researchers in this important activity.

Peer review practices in academic medicine: how the example of orthopaedic surgery may help shift the paradigm?

PURPOSE

To establish the current peer-reviewed practices in the discipline of orthopaedic surgery and correlate these to the journal's impact factor. Unfortunately, this is not receiving much attention and a critical literature gap in various disciplines; thus, determining the current practices in the discipline of orthopaedic surgery could provide valid insight that may be potentially applicable to other academic medicine disciplines as well.

METHODS

Orthopaedic surgery journals belonging to the Journal Citation Reports were queried, and the following was extracted: impact factor (IF) and blinding practices: single (SBPR), double (DBPR), triple (TBPR), quadruple (QBPR), and open (OPR) blinding review process and possibility of author-suggested reviewer (ASR) and non-preferred reviewer (NPR) options.

RESULTS

Of the 82 journals, four were excluded as they allowed submission by invitation only. In the remaining, blinding was as follows: SBPR nine (11.5%), DBPR 52 (66.7%), TBPR two (2.6%), QBPR zero (0%), and OPR three (3.8%), and in 12 (15.4%), this was unclear. ASR and NPR options were offered by 34 (43.6%) and 27 (34.6%) journals respectively, whereas ASR was mandatory in eight (10.2%). No correlation between IF and any other parameter was found.

CONCLUSION

The rules of the "game" are unclear/not disclosed in a significant number of cases, and the SBPR system, along with the ASR (mandatory sometimes) and NPR, is still extensively used with questionable integrity and fairness. Several recommendations are provided to mitigate potentially compromising practices, along with future directions to address the scarcity of research in this critical aspect of science.

The Role of Publons in the Context of Open Peer Review

Publons was a peer reviewer rewards platform that aimed to recognize the contribution that academics made during peer review to a journal. For about 10 years of its existence, Publons became the most popular service among peer reviewers. Having gained traction and popularity, Publons was purchased in 2017 by Clarivate Analytics (now Clarivate), and many academics, journals and publishers invested time and effort to participate in Publons. Using Publons, various peer review-related experiments or pilot programs were initiated by some academic publishers regarding the introduction of open peer review into their journals’ editorial processes. In this paper, we examine pertinent literature related to Publons, and reflect on its benefits and flaws during its short-lived history. In mid-August 2022, Clarivate fused Publons into the Web of Science platform. Publons, as a brand peer review service, has now ceased to exist but some of the functionality remains in Web of Science while other aspects that used to be open and free at Publons are now paid-for services. We reflect on the effect of such experiments, which initially had bold and ambitious academic objectives to fortify peer review, on academics’ trust, especially when such projects become commercialized.

Brazilian authors don't cite Brazilian authors: Nothing has changed since 1994

Objective  To outline the profile of self-citations from Revista Brasileira de Ortopedia (Rev Bras Ortop) and citations of this journal in other medical orthopaedic journals with general or specific content in a knowledge area of the specialty.

Methods  This is an observational cross-sectional study of the frequency of self-citations and citations from Rev Bras Ortop in five other medical orthopaedic journals from different countries, all published in English. The last 15 articles published in 2020 in each of the six journals were analyzed. The references used in each of them were evaluated to identify the journal in which they were originally published. The frequency of distribution of the four main journals cited, their position, and the relative percentage to the total number of citations were observed and recorded in each of the six journals. The number of times that the Rev Bras Ortop was cited in each of the selected foreign journals was assessed using its absolute and relative frequencies.

Results  The total number of citations evaluated in this study was 2,527 (ranging from 386 to 486 per magazine). Rev Bras Ortop showed a low rate of self-citation (2.6%), being the sixth journal cited in the journal itself (10 out of a total of 386 references). Moreover, Rev Bras Ortop was not mentioned in any of the other five medical journals included in the study (absolute frequency 0, relative frequency 0).

Conclusion  Rev Bras Ortop has a low reference of itself, with a self-citation rate of 2.6% in the studied period, showing that the Brazilian orthopaedic surgeons do not mention the Brazilian orthopaedic surgeon who publishes in the journal. We suggest the elaboration and implementation of strong strategies to improve the journal's visibility in the world academic-scientific scenario. In addition, it is essential that Brazilian orthopaedic surgeons understand this reality and assist directly and effectively to change this scenario.