Group Peer Review: The Breakfast of Champions

The Inside Scoop: What We Learnt About Getting into Academic Publishing During Our Editorial Internship

The world of publication can seem intimidating and closed to the newcomer. How then does one even begin to get a foot in the door? In this paper, the authors draw from the literature and their recent lived experience as editorial interns to consider this challenge under the theme of access, and how it overlaps with the various components of academic publication. The main three components of the publication 'machine' are discussed in this article, authoring, reviewing, and editing. These are preceded by the first, and arguably foundational, interaction with academic journal publishing-reading. Without reading articles across different journals, and even in different disciplines, understanding the breadth of scholarship and its purpose is impossible. The subsequent components of authoring, reviewing, and editing, which are all enhanced by ongoing familiarity with current literature through further reading, are considered in further detail in the remainder of this article, with practical advice provided as to how to gain access and experience in each of these areas, for example, writing non-research article manuscripts, engaging in collaborative peer review, and applying for editorial opportunities (with perseverance) when the opportunity presents itself. Medical education publication can seem daunting and closed to entry-level academics. This article is written to dispel this view, and challenges the notion that the world of publication is reserved for experts only. On the contrary, newcomers to the field are essential for academic publications to retain relevance, dynamism, and innovation particularly in the face of the changing landscape of medical education.

A Collaborative Approach to Mentored Peer Reviews Sponsored by the Council of Residency Directors in Emergency Medicine

Introduction

Historically, there have been no systematic programs for teaching peer review, leaving trainees to learn by trial and error. Recently, a number of publications have advocated for programs where experienced reviewers mentor trainees to more efficiently acquire this knowledge.

Objective

Our goal was to develop an introductory learning experience that intentionally fosters peer-review skills.

Methods

The Council of Residency Directors in Emergency Medicine (CORD) offered education fellowship directors the opportunity to mentor their fellows by reviewing submitted manuscript(s) supplemented by educational material provided by their journal. Reviews were collaboratively created. The decision letter that was sent to manuscript authors was also sent to the mentees; it included all reviewers' and editor's comments, as feedback. In 2022, fellows received a post-experience survey regarding prior experiences and their perspectives of the mentored peer-review experience.

Results

From 2020-2022, participation grew from 14 to 30 education fellowships, providing 76 manuscript peer reviews. The 2022 survey-response rate of 87% (20/23) revealed that fellows were inexperienced in education scholarship prior to participation: 30% had authored an education paper, and 10% had performed peer review of an education manuscript. Overall, participants were enthusiastic about the program and anxious to participate the following year. In addition, participants identified a number of benefits of the mentored experience including improved understanding of the scholarship process; informing fellows' scholarly pursuits; improved conceptualization of concepts learned elsewhere in training; and learning through exposure to scholarship.

Conclusion

This program's early findings suggest that collaboration between academic societies and interested graduate medical education faculty has the potential to formalize the process of learning peer review, benefitting all involved stakeholders.

Co-creating scholarship through collaborative writing in health professions education: AMEE Guide No. 143

This AMEE guide provides a robust framework and practical strategies for health professions educators to enhance their writing skills and engage in successful scholarship within a collaborative writing team. Whether scholarly output involves peer-reviewed articles, book chapters, blogs and online posts, online educational resources, collaborative writing requires more than the usual core writing skills, it requires teamwork, leadership and followership, negotiation, and conflict resolution, mentoring and more. Whilst educators can attend workshops or courses to enhance their writing skills, there may be fewer opportunities to join a community of scholars and engage in successful collaborative writing. There is very little guidance on how to find, join, position oneself and contribute to a writing group. Once individuals join a group, further questions arise as to how to contribute, when and whom to ask for help, whether their contribution is significant, and how to move from the periphery to the centre of the group. The most important question of all is how to translate disparate ideas into a shared key message and articulate it clearly. In this guide, we describe the value of working within a collaborative writing group; reflect on principles that anchor the concept of writing as a team and guide team behaviours; suggest explicit strategies to overcome challenges and promote successful writing that contributes to and advances the field; and review challenges to starting, maintaining, and completing writing tasks. We approach writing through three lenses: that of the individual writer, the writing team, and the scholarly product, the ultimate goal being meaningful contributions to the field of Health Professions Education.

Peer-reviewing in Surgical Journals: Revolutionize or Perish?

The gold standard of safe-guarding the quality of published science is peer review. However, this long-standing system has not evolved in today's digital world, where there has been an explosion in the number of publications and surgical journals. A journal's quality depends not only on the quality of papers submitted but is reflected upon the quality of its peer review process. Over the past decade journals are experiencing a rapidly escalating "peer review crisis" with editors struggling in recruiting reliable reviewers who will provide their skilled work for free with ever-diminishing incentives within today's restricted time-constraints. The problem is complex and difficult to solve, but more urgent than ever. Time is valuable and academicians, researchers and clinicians are overburdened and already extremely busy publishing their own research along with their ever growing clinical and administrative duties. Fewer and fewer individuals volunteer to provide their skilled work for free which is expected. The current incentives to review do not have a big impact on one's career and therefore are not realistic effective countermeasures. As the limits of the system are constantly stretched, there will inevitably come a "point of no return" and Surgical Journals will be the ones to first take the hit as there is an overwhelming evidence of burnout in the surgical specialties and the Surgical community is almost 50% smaller than its Medical counterpart. This review identifies the potential causes of the peer-review crisis, outlines the incentives and drawbacks of being a reviewer, summarizes the currently established common practices of rewarding reviewers and the existing and potential solutions to the problem. The magnitude of the problem and unsustainability that will make it perish are discussed along with its current flaws. Finally, recommendations are made to address many of the weaknesses of the system with the hope to revive it.

Perspective on Systematic Medical Literature Reviews and Meta-Analyses

PURPOSE

This study sought to identify factors contributing to the inadequacies of systematic reviews and meta-analyses (SRMAs) published in the ophthalmology literature.

DESIGN

Perspective.

METHODS

Review and synthesis of selective literature, with interpretation and perspective.

RESULTS

Although recommendations for the design, conduct, assessment of quality, and risk of bias of systematic reviews have been widely available, some recent publications illustrate a serious potential failing in this domain: inclusion of refuted science, lack of citation of post-publication correspondence and failure to use ≥1 alternative search strategy.

CONCLUSIONS

Examples of inadequacies of peer review in medical literature and perpetuation of erroneous science by unfiltered inclusion in subsequent systematic reviews have been identified, and the problem can be traced to authors, peer reviewers, and editors of journals. This perspective identifies and analyzes several possible causes of the problem and recommends some specific corrective actions to improve the quality and accuracy of such reviews.

Discovering the Benefits of Group Peer Review of Submitted Manuscripts

Problem: Traditionally, journal editors expect individuals to complete peer reviews of submitted manuscripts on their own. Recently, a number of editors of health sciences journals have begun to support, and even espouse, the practice of group peer review (GPR). With GPR, multiple individuals work together to complete the review with permission from the journal editor. Motivated by the idea that GPR could provide a meaningful service learning experience for participants in an interprofessional educational scholarship course, we conducted three such reviews and subsequently reflected on our experience and the lessons we learned. We frame our reflections using guiding principles from the domains of peer review, professional development, and educational scholarship. Intervention: The course director arranged for manuscripts to review with the editors of three health sciences journals. Each GPR occurred during a separate weekly session of the course. Each GPR was completed using a similar set of steps, which included (a) gaining familiarity with review criteria, (b) reading aloud and discussing the manuscript's abstract as a class, (c) reading and critiquing assigned sections as individuals and then small groups, (d) building consensus and sharing notes, (e) having the course director synthesize notes into a single review for submission to the journal. Context: The course on educational scholarship involved 15 faculty representing faculty from the University of Utah's School of Medicine, College of Nursing, College of Pharmacy, College of Health, and School of Dentistry. The course director led three GPR sessions mid-way through the yearlong course. Impact: Participants' reflections indicate that GPR (a) conformed to principles of effective peer review; (b) resulted in a meaningful service learning experience within a formal professional development program, deepening understanding of core concepts of educational scholarship; and (c) represented an authentic example of engaging in educational scholarship (i.e., designing and evaluating an intervention while drawing upon and contributing to a body of shared understanding within a community of practice). Lessons Learned: Our principles-based approach to completing GPR within a professional development course on educational scholarship can serve as a model for others to follow. A rigorous, meaningful group review can occur in 1 hour using a combination of group and individual activities focused on matching review criteria to the submitted manuscript. As a result, we continue to include GPR in future offerings of this interprofessional course on educational scholarship, and we continue to study ways to optimize its value as a service learning experience.

Communities of Practice in Peer Review: Outlining a Group Review Process

Traditional peer review remains the gold standard for assessing the merit of scientific scholarship for publication. Challenges to this model include reliance on volunteer contributions of individuals with self-reported expertise; lack of sufficient mentoring and training of new reviewers; and the isolated, noncollaborative nature of individual reviewer processes.The authors participated in an Association of American Medical Colleges peer-review workshop in November 2015 and were intrigued by the process of group peer review. Subsequent discussions led to shared excitement about exploring this model further. The authors worked with the staff and editors of Academic Medicine to perform a group review of 4 submitted manuscripts, documenting their iterative process and analysis of outcomes, to define an optimal approach to performing group peer review.Individual recommendations for each manuscript changed as a result of the group review process. The group process led to more comprehensive reviews than each individual reviewer would have submitted independently. The time spent on group reviews decreased as the process became more refined. Recommendations aligned with journal editor findings. Shared operating principles were identified, as well as clear benefits of group peer review for reviewers, authors, and journal editors.The authors plan to continue to refine and codify an effective process for group peer review. They also aim to more formally evaluate the model, with inclusion of feedback from journal editors and authors, and to compare feedback from group peer reviews versus individual reviewer feedback. Finally, models for expansion of the group-peer-review process are proposed.

Advice for authors from the editors of Perspectives on Medical Education : Getting your research published

None

Group Peer Review

Invitation to review a manuscript for publication marks an exciting milestone in graduate and postgraduate students’ training. Yet, peer reviewing is seldom explicitly taught. First-time reviewers approaching this task often lack mentorship, guidelines, and confidence. The ongoing debate about how to judge the quality of qualitative research can further complicate the task. In this article, we introduce an innovative model for conducting group peer reviews in the context of qualitative research training. After setting out the model's principles and process, we discuss its merits and reflect on our experiences as trainees and supervisor using the approach. In addition to providing opportunities for mentorship in appraisal methods, writing strategies, and approaches for framing constructive feedback, we suggest the model has the potential to advance trainees’ development as collegial peers and their overall learning as qualitative researchers. Finally, we discuss potential ways forward to extend the model in other contexts.

Academic Primer Series: Key Papers About Peer Review

Introduction

Peer review, a cornerstone of academia, promotes rigor and relevance in scientific publishing. As educators are encouraged to adopt a more scholarly approach to medical education, peer review is becoming increasingly important. Junior educators both receive the reviews of their peers, and are also asked to participate as reviewers themselves. As such, it is imperative for junior clinician educators to be well-versed in the art of peer reviewing their colleagues’ work. In this article, our goal was to identify and summarize key papers that may be helpful for faculty members interested in learning more about the peer-review process and how to improve their reviewing skills.

Methods

The online discussions of the 2016–17 Academic Life in Emergency Medicine (ALiEM) Faculty Incubator program included a robust discussion about peer review, which highlighted a number of papers on that topic. We sought to augment this list with further suggestions by guest experts and by an open call on Twitter for other important papers. Via this process, we created a list of 24 total papers on the topic of peer review. After gathering these papers, our authorship group engaged in a consensus-building process incorporating Delphi methods to identify the papers that best described peer review, and also highlighted important tips for new reviewers.

Results

We found and reviewed 24 papers. In our results section, we present our authorship group’s top five most highly rated papers on the topic of peer review. We also summarize these papers with respect to their relevance to junior faculty members and to faculty developers.

Conclusion

We present five key papers on peer review that can be used for faculty development for novice writers and reviewers. These papers represent a mix of foundational and explanatory papers that may provide some basis from which junior faculty members might build upon as they both undergo the peer-review process and act as reviewers in turn.