Single-blind vs Double-blind Peer Review in the Setting of Author Prestige

A Comparison of Plastic Surgery Authorship Trends Under Single Versus Double-Blinded Review

INTRODUCTION

Research is key to academic advancement in plastic surgery. However, access to publication opportunities may be inequitable as seen in other fields. We compared authorship trends of plastic surgery manuscripts that underwent single-blinded review (SBR) versus double-blinded review (DBR) to identify potential disparities in publication opportunities.

METHODS

Publications from two plastic surgery journals using SBR and two using DBR from September 2019 to September 2021 were evaluated. Name and institution of the article's first and senior author and journal's editor-in-chief (EIC) were recorded. Chi-squared and Fisher's exact analyses were used to compare author characteristics between SBR and DBR articles.

RESULTS

Of 2500 manuscripts, 65.7% underwent SBR and 34.3% underwent DBR. SBR articles had higher percentages of women as first authors (31.9% versus 24.3%, P < 0.001) but lower percentages of first (50.7% versus 71.2%, P < 0.001) and senior (49.6% versus 70.3%, P < 0.001) authors from international institutions. First (26.0% versus 12.9%, P < 0.001) and senior (27.9% versus 18.0%, P = 0.007) authors of SBR articles tended to have more plastic surgery National Institutes of Health funding. Journals using SBR tended to have higher rates of authorship by EICs or authors sharing institutions with the EIC (P ≤ 0.005).

CONCLUSIONS

While associated with greater female first authorship suggesting potential efforts toward gender equity in academia, SBR of plastic surgery articles tends to favor authors from institutions with higher National Institutes of Health funding and disadvantage authors from international or lower-resourced programs. Careful consideration of current peer-review proceedings may make publication opportunities more equitable.

Gender disparity between first and senior authors on liver cancer research in the top journals of Gastroenterology and Hepatology

BACKGROUND

Gender disparity is pervasive in academic medicine. This study aimed to assess the disparity between men and women with regard to first and senior author positions in primary studies on liver cancer over the last two decades.

METHODS

We conducted a review of articles published in high-impact factor journals of the field of Gastroenterology and Hepatology in 2005, 2010, 2015 and 2020. First and senior authors of all ages were considered as the study population. The authors' genders were determined using the online artificial intelligence tool genderize.io (https://genderize.io/). The disparity between men and women authors was assessed using the linear-by-linear association test.

RESULTS

665 original articles from 10 journals were reviewed. The point prevalence of first women authors was 25.0% compared with 75.0% for men. The point prevalence of senior women authors was 16.3% compared with 83.7% for men. From 2000 to 2020, the proportion of first women authors increased 14.4% to 26.8% compared with 85.6%-73.2% for men (P = 0.009), and the proportion of senior women authors increased from 7.4% to 19.5%, compared with 92.6%-80.5% for men (P = 0.035). The factor independently associated with a reduced representation of women among first authors was the region of author. The factor independently associated with a reduced representation of women among senior authors was the impact factor of journals.

CONCLUSION

The findings indicated a remarkable increase in the proportion of women, both first and senior authors, over the past two decades in the field of liver cancers. However, the representation of women authors in this area is far less than that of men.

American Black Authorship Has Decreased Across All Clinical Specialties Despite an Increasing Number of Black Physicians Between 1990 and 2020 in the USA

PURPOSE

Many diversity, equity, and inclusion initiatives assume that attainment of a racially diverse healthcare workforce will translate to increased diversity elsewhere in the healthcare system (e.g., leadership roles or academic authorship). We sought to investigate these trends over time by examining the evolution of physician demographics in the USA, in concert with demographic changes in US authorship in US medical journals from 1990 to 2020 across 25 specialties.

METHODS

We reviewed all articles indexed in PubMed, with a primary author affiliation located in the USA and limited to journals based in the USA, relative to the proportion of medical professionals in the CMS National Provider Registry. We employed a previously peer-reviewed/validated algorithm called "averaging-of-proportions" that probabilistically predicts racial identity from surname using the US Census to assess the relationship between diversity among medical professionals and diversity in medical journal authorship.

RESULTS

Data reveals a sharp disconnect between the demographic breakdown of physicians and authors. Despite an increase in the number of Black physicians (from 8.5% in 2005 to 9.1% in 2020), there has been a decrease in Black early-career authorship from 7.2% in 1990 to 5.8% in 2020. The percentage of Black early-career authors across all specialties in 2020 is lower than the average per specialty in 1990. Similar trends were noted for Black senior authorship, decreasing from 7.6% in 1990 to 6.2% in 2020, as well as a plateau in Hispanic authorship over the same time interval despite an increasing number of Hispanic physicians.

CONCLUSION

Modest advances in physician diversity have not translated to increased diversity in academic authorship. Increasing diversity requires initiatives focused beyond recruitment of underrepresented minorities to medical schools or residencies.

The transparency of quantitative empirical legal research published in highly ranked law journals (2018-2020): an observational study

Background

Scientists are increasingly concerned with making their work easy to verify and build upon. Associated practices include sharing data, materials, and analytic scripts, and preregistering protocols. This shift towards increased transparency and rigor has been referred to as a "credibility revolution." The credibility of empirical legal research has been questioned in the past due to its distinctive peer review system and because the legal background of its researchers means that many often are not trained in study design or statistics. Still, there has been no systematic study of transparency and credibility-related characteristics of published empirical legal research.

Methods

To fill this gap and provide an estimate of current practices that can be tracked as the field evolves, we assessed 300 empirical articles from highly ranked law journals including both faculty-edited journals and student-edited journals.

Results

We found high levels of article accessibility (86%, 95% CI = [82%, 90%]), especially among student-edited journals (100%). Few articles stated that a study's data are available (19%, 95% CI = [15%, 23%]). Statements of preregistration (3%, 95% CI = [1%, 5%]) and availability of analytic scripts (6%, 95% CI = [4%, 9%]) were very uncommon. (i.e., they collected new data using the study's reported methods, but found results inconsistent or not as strong as the original).

Conclusion

We suggest that empirical legal researchers and the journals that publish their work cultivate norms and practices to encourage research credibility. Our estimates may be revisited to track the field's progress in the coming years.

Analysis of NIH K99/R00 awards and the career progression of awardees

Many postdoctoral fellows and scholars who hope to secure tenure-track faculty positions in the United States apply to the National Institutes of Health (NIH) for a Pathway to Independence Award. This award has two phases (K99 and R00) and provides funding for up to 5 years. Using NIH data for the period 2006-2022, we report that ~230 K99 awards were made every year, representing up to ~$250 million annual investment. About 40% of K99 awardees were women and ~89% of K99 awardees went on to receive an R00 award annually. Institutions with the most NIH funding produced the most recipients of K99 awards and recruited the most recipients of R00 awards. The time between a researcher starting an R00 award and receiving a major NIH award (such as an R01) ranged between 4.6 and 7.4 years, and was significantly longer for women, for those who remained at their home institution, and for those hired by an institution that was not one of the 25 institutions with the most NIH funding. Shockingly, there has yet to be a K99 awardee at a historically Black college or university. We go on to show how K99 awardees flow to faculty positions, and to identify various factors that influence the future success of individual researchers and, therefore, also influence the composition of biomedical faculty at universities in the United States.

Analysis of NIH K99/R00 Awards and the Career Progression of Awardees

Many postdoctoral fellows and scholars who hope to secure tenure-track faculty positions in the United States apply to the National Institutes of Health (NIH) for a Pathway to Independence Award. This award has two phases (K99 and R00) and provides funding for up to five years. Using NIH data for the period 2006-2022, we report that ~230 K99 awards were made every year, representing up to ~$250 million annual investment. About 40% of K99 awardees were women and ~89% of K99 awardees went on to receive an R00 award annually. Institutions with the most NIH funding produced the most recipients of K99 awards and recruited the most recipients of R00 awards. The time between a researcher starting an R00 award and receiving a major NIH award (such as an R01) ranged between 4.6 and 7.4 years, and was significantly longer for women, for those who remained at their home institution, and for those hired by an institution that was not one of the 25 institutions with the most NIH funding. Shockingly, there has yet to be a K99 awardee at a historically Black college or university. We go on to show how K99 awardees flow to faculty positions, and to identify various factors that influence the future success of individual researchers and, therefore, also influence the composition of biomedical faculty at universities in the US.

Challenges and Controversies in Peer Review: JACC Review Topic of the Week

The process of peer review has been the gold standard for evaluating medical science, but significant pressures from the recent COVID-19 pandemic, new methods of communication, larger amounts of research, and an evolving publication landscape have placed significant pressures on this system. A task force convened by the American College of Cardiology identified the 5 most significant controversies associated with the current peer-review process: the effect of preprints, reviewer blinding, reviewer selection, reviewer incentivization, and publication of peer reviewer comments. Although specific solutions to these issues will vary, regardless of how scientific communication evolves, peer review must remain an essential process for ensuring scientific integrity, timely dissemination of information, and better patient care. In medicine, the peer-review process is crucial because harm can occur if poor-quality data or incorrect conclusions are published. With the dramatic increase in scientific publications and new methods of communication, high-quality peer review is more important now than ever.

A Review and Assessment of Open Access Surgery Journals

INTRODUCTION

Open access publishing has exhibited rapid growth in recent years. However, there is uncertainty surrounding the quality of open access journals and their ability to reach target audiences. This study reviews and characterizes open access surgical journals.

MATERIALS AND METHODS

The directory of open access journals was used to search for open access surgical journals. PubMed indexing status, impact factor, article processing charge (APC), initial year of open access publishing, average weeks from manuscript submission to publication, publisher, and peer-review processes were evaluated.

RESULTS

Ninety-two open access surgical journals were identified. Most (n = 49, 53.3%) were indexed in PubMed. Journals established >10 y were more likely to be indexed in PubMed compared to journals established <5 y (28 of 41 [68.3%] versus 4 of 20 [20%], P < 0.001). 44 journals (47.8%) used a double-blind review method. 49 (53.2%) journals received an impact factor for 2021, ranging from <0.1 to 10.2 (median 1.4). The median APC was $362 United States dollar [interquartile range $0 - 1802 United States dollar]. 35 journals (38%) did not charge a processing fee. There was a significant positive correlation between the APC and impact factor (r = 0.61, P < 0.001). If accepted, the median time from manuscript submission to publication was 12 wk.

CONCLUSIONS

Open access surgical journals are largely indexed on PubMed, have transparent review processes, employ variable APCs (including no publication fees), and proceed efficiently from submission to publication. These results should increase readers' confidence in the quality of surgical literature published in open access journals.

Blinding Models for Scientific Peer-Review of Biomedical Research Proposals: A Systematic Review

Objective: The aim of this systematic review is to estimate: (i) the overall effect of blinding models on bias; (ii) the effect of each blinding model; and (iii) the effect of un-blinding on reviewer's accountability in biomedical research proposals. Methods: Systematic review of prospective or retrospective comparative studies that evaluated two or more peer review blinding models for biomedical research proposals/funding applications and reported outcomes related to peer review efficiency. Results: Three studies that met the inclusion criteria were included in this review and assessed using the QualSyst tool by two authors. Conclusion: Our systematic review is the first to assess peer review blinding models in the context of funding. While only three studies were included, this highlighted the dire need for further RCTs that generate validated evidence. We also discussed multiple aspects of peer review, such as peer review in manuscripts vs proposals and peer review in other fields.

Scientists' perception of pilot study quality was influenced by statistical significance and study design

OBJECTIVES

Preliminary studies play a key role in developing large-scale interventions but may be held to higher or lower scientific standards during the peer review process because of their preliminary study status.

STUDY DESIGN AND SETTING

Abstracts from 5 published obesity prevention preliminary studies were systematically modified to generate 16 variations of each abstract. Variations differed by 4 factors: sample size (n = 20 vs. n = 150), statistical significance (P < 0.05 vs. P > 0.05), study design (single group vs. randomized 2 groups), and preliminary study status (presence/absence of pilot language). Using an online survey, behavioral scientists were provided with a randomly selected variation of each of the 5 abstracts and blinded to the existence of other variations. Respondents rated each abstract on aspects of study quality.

RESULTS

Behavioral scientists (n = 271, 79.7% female, median age 34 years) completed 1,355 abstract ratings. Preliminary study status was not associated with perceived study quality. Statistically significant effects were rated as more scientifically significant, rigorous, innovative, clearly written, warranted further testing, and had more meaningful results. Randomized designs were rated as more rigorous, innovative, and meaningful.

CONCLUSION

Findings suggest reviewers place a greater value on statistically significant findings and randomized control design and may overlook other important study characteristics.

Blinding of Peer Review and the Impact on Geographic Diversity of Authors in the Medical Literature

Background

Blinding of reviewers is hypothesized to improve the peer review process by removing potential bias. This study aimed to evaluate the impact of blinding of peer review on the geographic diversity of authors in medical/clinical journals.

Methods

MEDLINE-indexed medical journals were evaluated, where journals that only publish in basic sciences or administration, non-English journals, journals that publish solely solicited materials, and journals that employ open review process were excluded. Journals were divided into single-blinded or double-blinded. Diversity was calculated by dividing the number of countries from which 20 evaluated articles come and multiplying by 100 (%diversity). The second method involved calculating Simpson's diversity index (SDI).

Results

Of 1054 journals, 766 employ single-blinded review and 288 were double-blinded. Journals had a median age of 28 years and were mostly international (n=355 single-blinded and 97 double-blinded). No difference was observed between the two groups in median %diversity (45 in both groups; P=0.199) and SDI (0.84 vs 0.82; P=0.128). The indexing of journals in Science Citation Index Expanded (SCIE) collection of Web of Science and Scopus, and a higher CiteScore were significantly associated with higher %diversity and SDI (P<0.05).

Conclusion

Although double blinding of peer review was not associated with higher geographic diversity of authors, several factors are also involved in the review process that could not be evaluated, such as blinding of editors. However, editors and publishers are encouraged to consider work from different countries to be able to index their journals in SCIE, Scopus, and MEDLINE where geographic diversity is a requirement.

The role of author identities in peer review

There is widespread debate on whether to anonymize author identities in peer review. The key argument for anonymization is to mitigate bias, whereas arguments against anonymization posit various uses of author identities in the review process. The Innovations in Theoretical Computer Science (ITCS) 2023 conference adopted a middle ground by initially anonymizing the author identities from reviewers, revealing them after the reviewer had submitted their initial reviews, and allowing the reviewer to change their review subsequently. We present an analysis of the reviews pertaining to the identification and use of author identities. Our key findings are: (I) A majority of reviewers self-report not knowing and being unable to guess the authors' identities for the papers they were reviewing. (II) After the initial submission of reviews, 7.1% of reviews changed their overall merit score and 3.8% changed their self-reported reviewer expertise. (III) There is a very weak and statistically insignificant correlation of the rank of authors' affiliations with the change in overall merit; there is a weak but statistically significant correlation with respect to change in reviewer expertise. We also conducted an anonymous survey to obtain opinions from reviewers and authors. The main findings from the 200 survey responses are: (i) A vast majority of participants favor anonymizing author identities in some form. (ii) The "middle-ground" initiative of ITCS 2023 was appreciated. (iii) Detecting conflicts of interest is a challenge that needs to be addressed if author identities are anonymized. Overall, these findings support anonymization of author identities in some form (e.g., as was done in ITCS 2023), as long as there is a robust and efficient way to check conflicts of interest.

The Medical Student Race to Research: Who Presents More at National Plastic Surgery Conferences?

BACKGROUND

presentations increase research output and facilitate networking for medical students applying to plastic surgery. We aim to determine predictors of increased medical student presentation at national plastic surgery conferences, identifying disparities in access to research opportunities.

METHODS

Abstracts presented at the 2 most recent meetings of the American Society of Plastic Surgeons, American Association of Plastic Surgeons, and Plastic Surgery Research Council were extracted from online archives. Presenters without MDs or other professional credentials were classified as medical students. Presenter gender, medical school ranking, plastic surgery division/department, National Institutes of Health funding, number of total and first-author publications, H-index, and research fellowship completion status were recorded. Students with 3 or more (>75th percentile) presentations were compared with those with less by χ2 tests. Univariate and multivariable regressions identified factors associated with 3 or more presentations.

RESULTS

Of 1576 abstracts, 549 (34.8%) were presented by 314 students. The gender distribution was 46.5% male and 53.5% female. Most were from the Northeast (36.9%), 35% came from top 20 medical schools, and 85% attended schools with home plastic surgery programs. While 61.8% presented once, 14.6% presented 3 or more times. Those who previously presented, completed research fellowships or had more publications or higher H-indices were likely to present more ( P ≤ 0.007). On multivariable-adjusted analysis, completing research fellowships (odds ratio [OR], 2.34-2.52; P = 0.028-0.045), affiliation with institutions having higher National Institutes of Health funding (OR, 3.47-3.73; P = 0.004-0.006), or having more total number of publications (OR, 3.81; P = 0.018) or first-author publications (OR, 3.84; P = 0.008) was associated with 3 or more presentations. Presenter gender, geographic region, medical school ranking, home program status, and H-indices were not significant predictors on multivariable analysis.

CONCLUSIONS

There are several potential inequities in access to research opportunities for medical students, disadvantaging those with less well-funded plastic surgery programs and existing research experience. Improving the equitability of these opportunities is crucial for limiting bias in trainee recruitment and diversifying representation in the field.

Tips and guidelines for being a good peer reviewer

Publication is the key means by which science spreads. The purpose of scientific journals is to publish novel and quality articles. The editors of the journals evaluate the content of the manuscripts by submitting them to a process called «peer review», considered today the gold standard to guarantee the adequate publication of scientific articles. A well-crafted and critical peer-review report is a treasure for both authors and editors. In the present manuscript we will examine the key aspects of the peer review process. We will begin by explaining what exactly this process consists of and since when it has existed, and then clarifying why it is so important. Then we will argue why we should want to be reviewers of scientific papers. We will then review what are the fundamental rules to carry out a good review of a manuscript and what aspects of it we should focus on. Later we will see what format a peer review report should have and how to write its different sections, as well as the options for its final resolution. We will pay special attention to commenting on the ethical aspects and the most frequent errors that are made in the evaluation of manuscripts. Finally, we will recognize what the fundamental limitations of peer review are, and we will end by proposing some suggestions for their improvement. Our ultimate goal is to stimulate researchers -and authors- to go one step further and undertake the challenge of being peer reviewers of scientific manuscripts.

Peer review practices in academic medicine: how the example of orthopaedic surgery may help shift the paradigm?

PURPOSE

To establish the current peer-reviewed practices in the discipline of orthopaedic surgery and correlate these to the journal's impact factor. Unfortunately, this is not receiving much attention and a critical literature gap in various disciplines; thus, determining the current practices in the discipline of orthopaedic surgery could provide valid insight that may be potentially applicable to other academic medicine disciplines as well.

METHODS

Orthopaedic surgery journals belonging to the Journal Citation Reports were queried, and the following was extracted: impact factor (IF) and blinding practices: single (SBPR), double (DBPR), triple (TBPR), quadruple (QBPR), and open (OPR) blinding review process and possibility of author-suggested reviewer (ASR) and non-preferred reviewer (NPR) options.

RESULTS

Of the 82 journals, four were excluded as they allowed submission by invitation only. In the remaining, blinding was as follows: SBPR nine (11.5%), DBPR 52 (66.7%), TBPR two (2.6%), QBPR zero (0%), and OPR three (3.8%), and in 12 (15.4%), this was unclear. ASR and NPR options were offered by 34 (43.6%) and 27 (34.6%) journals respectively, whereas ASR was mandatory in eight (10.2%). No correlation between IF and any other parameter was found.

CONCLUSION

The rules of the "game" are unclear/not disclosed in a significant number of cases, and the SBPR system, along with the ASR (mandatory sometimes) and NPR, is still extensively used with questionable integrity and fairness. Several recommendations are provided to mitigate potentially compromising practices, along with future directions to address the scarcity of research in this critical aspect of science.

Analysis of gender representation, authorship inflation, and institutional affiliation in abstract acceptance: a 5-year study

Purpose

Our goals were to characterize associations of author number, author gender, and institutional affiliation on ratings and acceptances of abstracts submitted to one surgical education conference over 5 years.

Methods

We retrospectively reviewed all abstracts submitted between 2017 and 2021 to the annual meeting of the Association for Surgical Education (ASE). Abstract data included average rater scores, acceptance status, author lists, and institutional affiliations. We cross-referenced last author affiliation with top-40 National Institutes of Health (NIH)-funded institutions and used a gender determination software to code first and last author genders.

Results

We analyzed 1,162 abstracts. Higher reviewer scores demonstrated positive, weak associations with more authors [r(1160) = 0.191, p < 0.001] and institutions [r(1160) = 0.182, p < 0.001]. Significantly higher scores were noted for abstracts with last authors affiliated with top-40 NIH-funded institutions [4.18 (SD 0.96) vs. 3.72 (SD 1.12), p < 0.001]. Women were first authors (51.8%) (n = 602) and last authors (35.4%) (n = 411) of the time. Abstracts were rated significantly higher with women rather than men as first authors [3.98 (SD 0.99) vs. 3.82 (SD 1.12), p = 0.011] or last [4.01 (SD 1.04) vs. 3.82 (SD 1.10), p = 0.005]. Across all years, abstracts were accepted more often as podium or plenary presentations when submitted by women first [n = 279, 59.7% (p = 0.002)] or last [n = 183, 38.4% (p = 0.095)] authors.

Conclusion

Abstracts whose last authors were affiliated with top-40 NIH-funded institutions received significantly higher scores, possibly indicating increased tangible or intangible resources contributing to research efforts. Abstracts with women first and last authors scored higher and were more frequently invited for plenary and podium presentations.

Influence of the COVID-19 Pandemic on Author Sex and Manuscript Acceptance Rates among Pulmonary and Critical Care Journals

Rationale: The coronavirus disease (COVID-19) pandemic has negatively affected women more than men and may influence the publication of non-COVID-19 research. Objectives: To evaluate whether the COVID-19 pandemic is associated with changes in manuscript acceptance rates among pulmonary/critical care journals and sex-based disparities in these rates. Methods: We analyzed first, senior, and corresponding author sex (female vs. male, identified by matching first names in a validated Genderize database) of manuscripts submitted to four pulmonary/critical care journals between January 1, 2018 and December 31, 2020. We constructed interrupted time series regression models to evaluate whether the proportion of female first and senior authors of non-COVID-19 original research manuscripts changed with the pandemic. Next, we performed multivariable logistic regressions to evaluate the association of author sex with acceptance of original research manuscripts. Results: Among 8,332 original research submissions, women represented 39.9% and 28.3% of first and senior authors, respectively. We found no change in the proportion of female first or senior authors of non-COVID-19 or COVID-19 submitted research manuscripts during the COVID-19 era. Non-COVID-19 manuscripts submitted during the COVID-19 era had reduced odds of acceptance, regardless of author sex (first author adjusted OR [aOR], 0.46 [95% confidence interval (CI), 0.36-0.59]; senior author aOR, 0.46 [95% CI, 0.37-0.57]). Female senior authorship was associated with decreased acceptance of non-COVID-19 research manuscripts (crude rates, 14.4% [male] vs. 13.2% [female]; aOR, 0.84 [95% CI, 0.71-0.99]). Conclusions: Although female author submissions were not disproportionately influenced by COVID-19, we found evidence suggesting sex disparities in manuscript acceptance rates. Journals may need to consider strategies to reduce this disparity, and academic institutions may need to factor our findings, including lower acceptance rates for non-COVID-19 manuscripts, into promotion decisions.

Female author representation differs between journals from the United States of America, Europe, and Asia: a 10-year comparison of five medical disciplines

To provide information for targeted support of female scientists, the objective of this study was to evaluate how female authors are represented in journals of five medical disciplines with varying rates of female physicians from the United States of America (USA), Europe, and Asia. For this retrospective bibliometric study 15 representative gynecologic, pediatric, radiologic, urologic, and surgical journals from the USA, Europe, and Asia were selected from the Web of Science database. From these, all n = 24182 publications of the years 2007/2008 and 2017/2018 were included. Gender and affiliations were assigned to first and senior authors using a software (Gender API, Passau, Germany), native speakers, and a web-based search. For statistics mixed logistic and multinomial logistic regression were applied. In pediatrics, radiology, and urology, highest female first and senior author shares were consistently found in journals from the USA. In European journals proportions across all disciplines tripled (odds ratio 2.96 [95% CI 2.60–3.37], P < .0001). Asian journals showed three-times fewer female authorships than journals from the USA or Europe and the smallest increase (1.36 [1.11–1.66], P = .0026). Compared to the proportion of female physicians within each specialty, female first authors remained underrepresented in Asian journals and female senior authors in journals of all regions. In journals from the USA most female authors originated from institutes within the USA (36.2%), in European journals from the USA (21.1%) or Europe (21.7%). Women from Asian institutes were worst represented in journals of all regions with lowest rates in Asian journals (9.4%). In conclusion female first authors remained underrepresented in Asian journals, female senior authors and women from Asian institutes in journals from all regions. Programs for gender equality in science are thus particularly necessary to support female senior authors, for Asian journals, and women from Asian institutes.

Nobel and novice: Author prominence affects peer review

Peer review is vital for research validation, but may be affected by “status bias,” the unequal treatment of papers written by prominent and less-well-known authors. We studied this bias by sending a research paper jointly authored by a Nobel Prize laureate and a relatively unknown early career research associate to many reviewers. We varied whether the prominent author’s name, the relatively unknown author’s name, or no name was revealed to reviewers. We found clear evidence for bias. More than 20% of the reviewers recommended “accept” when the Nobel laureate was shown as the author, but less than 2% did so when the research associate was shown. Our findings contribute to the debate of how best to organize the peer-review process.

Peer review is a well-established cornerstone of the scientific process, yet it is not immune to biases like status bias, which we explore in this paper. Merton described this bias as prominent researchers getting disproportionately great credit for their contribution, while relatively unknown researchers get disproportionately little credit [R. K. Merton, Science 159, 56–63 (1968)]. We measured the extent of this bias in the peer-review process through a preregistered field experiment. We invited more than 3,300 researchers to review a finance research paper jointly written by a prominent author (a Nobel laureate) and by a relatively unknown author (an early career research associate), varying whether reviewers saw the prominent author’s name, an anonymized version of the paper, or the less-well-known author’s name. We found strong evidence for the status bias: More of the invited researchers accepted to review the paper when the prominent name was shown, and while only 23% recommended “reject” when the prominent researcher was the only author shown, 48% did so when the paper was anonymized, and 65% did when the little-known author was the only author shown. Our findings complement and extend earlier results on double-anonymized vs. single-anonymized review [R. Blank, Am. Econ. Rev. 81, 1041–1067 (1991); M. A. Ucci, F. D’Antonio, V. Berghella, Am. J. Obstet. Gynecol. MFM 4, 100645 (2022)].

Research Methods: How to Perform an Effective Peer Review

Scientific peer review has existed for centuries and is a cornerstone of the scientific publication process. Because the number of scientific publications has rapidly increased over the past decades, so has the number of peer reviews and peer reviewers. In this paper, drawing on the relevant medical literature and our collective experience as peer reviewers, we provide a user guide to the peer review process, including discussion of the purpose and limitations of peer review, the qualities of a good peer reviewer, and a step-by-step process of how to conduct an effective peer review.

Quantifying hierarchy and dynamics in US faculty hiring and retention

Faculty hiring and retention determine the composition of the US academic workforce and directly shape educational outcomes¹, careers², the development and spread of ideas³ and research priorities^4,5. However, hiring and retention are dynamic, reflecting societal and academic priorities, generational turnover and efforts to diversify the professoriate along gender^6-8, racial⁹ and socioeconomic¹⁰ lines. A comprehensive study of the structure and dynamics of the US professoriate would elucidate the effects of these efforts and the processes that shape scholarship more broadly. Here we analyse the academic employment and doctoral education of tenure-track faculty at all PhD-granting US universities over the decade 2011-2020, quantifying stark inequalities in faculty production, prestige, retention and gender. Our analyses show universal inequalities in which a small minority of universities supply a large majority of faculty across fields, exacerbated by patterns of attrition and reflecting steep hierarchies of prestige. We identify markedly higher attrition rates among faculty trained outside the United States or employed by their doctoral university. Our results indicate that gains in women's representation over this decade result from demographic turnover and earlier changes made to hiring, and are unlikely to lead to long-term gender parity in most fields. These analyses quantify the dynamics of US faculty hiring and retention, and will support efforts to improve the organization, composition and scholarship of the US academic workforce.

The dementia research career pipeline: Gender disparities in publication authorships and grant funding outcomes at different career stages

Background

Multiple studies have analysed gender disparities in academic research. Here we study the gender composition of the dementia research field at different stages in the career pipeline.

Methods

We use various data sources to gain insights about the gender ratio across career stages: conference attendance data as a proxy for the field as a whole; bibliometric data to know who publishes, and who occupies positions of seniority among the listed authors; and Alzheimer's Research UK's (ARUK) internal grant funding data to understand who obtains funding. We also analyse the scoring of grant applications based on the gender of the reviewers.

Results

Our results confirm that female researchers leave dementia academic research at higher rates than men, before transitioning into senior positions. In 2020, they comprised over 60% of the field, produced 54% of first authorships, but only accounted for 38% of last authorships. Overall, women received 37% of ARUK's competitive grants, with significant differences between grant schemes awarded for early career researchers (64% female awardees) compared to grant schemes aimed at senior researchers (33% female awardees). Men and women applied for and obtained grants at significantly different rates depending on the career stage at which the grant was aimed.Finally, we also observed that male and female reviewers apply evaluation criteria differently, with men giving better scores than women on average.

Conclusions

Our study adds to the evidence that shows that women get published less, receive less funding, and transition into senior academic positions at disproportionally lower rates than men do. We briefly discuss potential reasons why gender disparities arise as researchers progress into senior positions, and offer interventions ARUK can implement in its application and evaluation process to address those disparities.

Double- vs single-blind peer review effect on acceptance rates: a systematic review and meta-analysis of randomized trials

OBJECTIVE

This study aimed to assess the changes in the acceptance rates between double- and single-blind peer review systems.

DATA SOURCES

The search was conducted using Medline, Embase, and ClinicalTrials.gov databases as electronic databases from the inception of each database to June 2021. No restriction for language or geographic location was applied.

STUDY ELIGIBILITY CRITERIA

The selection criteria included randomized controlled trials comparing the double-blind peer review process vs the single-blind peer review process.

METHODS

The primary outcome was manuscripts acceptance rates. The summary measures were reported as relative risk with 95% confidence intervals using the random-effects model meta-analyses. Between-study heterogeneity was explored using the I² statistic.

RESULTS

A total of 11 randomized controlled trials, including 3477 reviewers and 3784 manuscripts, were identified. The manuscript acceptance rates were significantly lower in the double-blind (200/1413 [14.2%]) peer review processes than in the single-blind (194/1019 [19.0%]) peer review processes (relative risk, 0.82; 95% confidence interval, 0.70-0.97; n=5 randomized controlled trials). Only 1 randomized controlled trial assessed the authors' and/or institutions' prestige on acceptance rates with results not statistically significant. Only 2 randomized controlled trials assessed the manuscript origin (US or non-US) effect on acceptance rates with results not statistically significant. Gender of the manuscript authors was assessed by only 1 randomized controlled trial, and although blinding or not female author names made no statistical difference, blinding of male author names was associated with a significant decrease in acceptance rate (139/1266 [11.0%] vs 190/1266 [15.0%]; relative risk, 0.73; 95% confidence interval, 0.59-0.90). Double-blind peer review was deemed successful by reviewers in only approximately 52% of the cases (n=5 randomized controlled trials).

CONCLUSION

The double-blind peer review process seemed to be associated with an 18% lower manuscript acceptance rate than the single-blind peer review process. However, given the large heterogeneity among the included studies, more research is needed to confirm these findings and elucidate those factors that can affect the acceptance rate in double- and single-blind peer reviews.

The impact of double-blind peer review on gender bias in scientific publishing: a systematic review

OBJECTIVE

Gender-based bias during journal peer review can lead to publication biases and perpetuate gender inequality in science. Double-blind peer review, in which the names of authors and reviewers are masked, may present an opportunity for scientific literature to increase equity and reduce gender-based biases. This systematic review of studies evaluates the impact of double-blind vs single-blind peer review on the publication rates by perceived author gender.

DATA SOURCES

The PubMed, Embase, Web of Science, and Scopus electronic databases were searched using the terms "blind," "peer review," "gender," "woman," and "author." All published literature in the English language from database inception through 2020 was queried.

STUDY ELIGIBILITY CRITERIA

Prospective experimental and observational studies comparing double-blind to single-blind peer review strategies examining impact on publication decisions by author gender were included.

STUDY APPRAISAL AND SYNTHESIS METHODS

The extracted data were primarily descriptive and included information on study design, sample size, primary outcome, major findings, and scientific discipline. The studies were characterized on the basis of design and whether the results demonstrated an impact of double-blind peer review on review scores and publication decision by perceived author gender. This study was registered with the International Prospective Register of Systematic Reviews or PROSPERO.

RESULTS

In total, 1717 articles were identified, 123 were reviewed, and 8 were included, encompassing 5 prospective experimental studies and 3 observational studies. Four studies demonstrated a difference in the acceptance rate or review score on the basis of perceived author gender, whereas the other 4 studies demonstrated no differences when the author gender was anonymized.

CONCLUSION

Studies evaluating the impact of double-blind peer review on author gender demonstrate mixed results, but there is reasonable evidence that gender bias may exist in scientific publishing and that double-blinding can mitigate its impact. Further evaluation of the processes in place to create the body of evidence that clinicians and researchers rely on is essential to reduce bias, particularly in female-majority fields such as obstetrics and gynecology.

Critical Evaluation of Clinical Research in Obstetrics and Gynecology

Within the evolving field of obstetrics and gynecology, providers should possess the ability to effectively and critically evaluate medical literature in order to best adapt and incorporate evidence-based practice. For both clinicians and researchers alike, we provide a systematic approach for reviewing a journal article published in the medical literature. We summarize the various types of study designs, with dedicated attention to observational and experimental studies, and examine sources of bias inherent to these study designs. Finally, we review important considerations when interpreting the validity and significance of the results and conclusions of a research study.

Caution, Preprint! Brief Explanations Allow Nonscientists to Differentiate Between Preprints and Peer-Reviewed Journal Articles

A growing number of psychological research findings are initially published as preprints. Preprints are not peer reviewed and thus did not undergo the established scientific quality-control process. Many researchers hence worry that these preprints reach nonscientists, such as practitioners, journalists, and policymakers, who might be unable to differentiate them from the peer-reviewed literature. Across five studies in Germany and the United States, we investigated whether this concern is warranted and whether this problem can be solved by providing nonscientists with a brief explanation of preprints and the peer-review process. Studies 1 and 2 showed that without an explanation, nonscientists perceive research findings published as preprints as equally credible as findings published as peer-reviewed articles. However, an explanation of the peer-review process reduces the credibility of preprints (Studies 3 and 4). In Study 5, we developed and tested a shortened version of this explanation, which we recommend adding to preprints. This explanation again allowed nonscientists to differentiate between preprints and the peer-reviewed literature. In sum, our research demonstrates that even a short explanation of the concept of preprints and their lack of peer review allows nonscientists who evaluate scientific findings to adjust their credibility perception accordingly. This would allow harvesting the benefits of preprints, such as faster and more accessible science communication, while reducing concerns about public overconfidence in the presented findings.

Metrics and methods in the evaluation of prestige bias in peer review: A case study in computer systems conferences

The integrity of peer review is essential for modern science. Numerous studies have therefore focused on identifying, quantifying, and mitigating biases in peer review. One of these better-known biases is prestige bias, where the recognition of a famous author or affiliation leads reviewers to subconsciously treat their submissions preferentially. A common mitigation approach for prestige bias is double-blind reviewing, where the identify of authors is hidden from reviewers. However, studies on the effectivness of this mitigation are mixed and are rarely directly comparable to each other, leading to difficulty in generalization of their results. In this paper, we explore the design space for such studies in an attempt to reach common ground. Using an observational approach with a large dataset of peer-reviewed papers in computer systems, we systematically evaluate the effects of different prestige metrics, aggregation methods, control variables, and outlier treatments. We show that depending on these choices, the data can lead to contradictory conclusions with high statistical significance. For example, authors with higher h-index often preferred to publish in competitive conferences which are also typically double-blind, whereas authors with higher paper counts often preferred the single-blind conferences. The main practical implication of our analyses is that a narrow evaluation may lead to unreliable results. A thorough evaluation of prestige bias requires a careful inventory of assumptions, metrics, and methodology, often requiring a more detailed sensitivity analysis than is normally undertaken. Importantly, two of the most commonly used metrics for prestige evaluation, past publication count and h-index, are not independent from the choice of publishing venue, which must be accounted for when comparing authors prestige across conferences.

Editors' Note: First Annual Report Regarding JAACAP's Antiracist Journey

Last year, we wrote to you of our dedication and vision for this journal "to be antiracist at every level," outlining the following 6 initiatives "to reshape the Journal to pursue this vision:" (1) Issuing a Call for Papers "on racism and its impacts on child development and children's mental health;" (2) updating our Guide for Authors "to emphasize that we will evaluate articles submitted to the Journal on whether their study designs and discussions consider and address human diversity in the context of their research questions and hypotheses; (3) assembling a special collection of "Journal articles on bias, bigotry, racism, and mental health disparities;" (4) accelerating "our efforts to make our editorial board inclusive and representative of our community of scientists and practitioners as well as the communities we all serve;" (5) engaging in "continuing education and dialogue as an Editorial Board that will include antiracism training;" and (6) critically examining "our editorial and peer review process to ensure it is antiracist.¹ In this Editors' Note, we write to update you on our progress.

An experimental test of the effects of redacting grant applicant identifiers on peer review outcomes

Background:

Blinding reviewers to applicant identity has been proposed to reduce bias in peer review.

Methods:

This experimental test used 1200 NIH grant applications, 400 from Black investigators, 400 matched applications from White investigators, and 400 randomly selected applications from White investigators. Applications were reviewed by mail in standard and redacted formats.

Results:

Redaction reduced, but did not eliminate, reviewers’ ability to correctly guess features of identity. The primary, preregistered analysis hypothesized a differential effect of redaction according to investigator race in the matched applications. A set of secondary analyses (not preregistered) used the randomly selected applications from White scientists and tested the same interaction. Both analyses revealed similar effects: Standard format applications from White investigators scored better than those from Black investigators. Redaction cut the size of the difference by about half (e.g. from a Cohen’s d of 0.20–0.10 in matched applications); redaction caused applications from White scientists to score worse but had no effect on scores for Black applications.

Conclusions:

Grant-writing considerations and halo effects are discussed as competing explanations for this pattern. The findings support further evaluation of peer review models that diminish the influence of applicant identity.

Funding:

Funding was provided by the NIH.

Peer review blinding practices of highly ranked dental journals: analysis and discussion

Objectives To evaluate the type of peer review blinding used in highly ranked dental journals and to discuss the influence of the blinding approaches on the peer review process.Methods All 91 dental journals classified by impact factor (IF) had their websites scrutinised for the type of peer review blinding used for submissions. If the information was not reported, the journals were contacted to obtain the information. Linear and logistic regression were applied to evaluate the association between type of peer review blinding and IF.Results The selected journals reported the following peer review blinding approaches: single-blind (N = 36, 39.6%), double-blind (N = 46, 50.5%), transparent (N = 2, 2.2%) and open (N = 1, 1.1%). Information from six (6.6%) journals was not available. A linear regression analysis demonstrated that journals with lower IFs were associated with double-blind review (p = 0.001). A logistic regression suggested lower odds of association between single-blind peer review and journals with IFs below a threshold of 2 (odds ratio 0.157, confidence interval 0.059 to 0.417, p <0.001).Conclusions The majority of highly ranked dental journals had single- and double-blind peer review; journals with higher IFs presented single-blind peer review and those with lower IFs reported double-blind peer review.

Effects of Experimental Interventions to Improve the Biomedical Peer-Review Process: A Systematic Review and Meta-Analysis

Background

Quality of the peer‐review process has been tested only in small studies. We describe and summarize the randomized trials that investigated interventions aimed at improving peer‐review process of biomedical manuscripts.

Methods and Results

All randomized trials comparing different peer‐review interventions at author‐, reviewer‐, and/or editor‐level were included. Differences between traditional and intervention‐modified peer‐review processes were pooled as standardized mean difference (SMD) in quality based on the definitions used in the individual studies. Main outcomes assessed were quality and duration of the peer‐review process. Five‐hundred and seventy‐five studies were retrieved, eventually yielding 24 randomized trials. Eight studies evaluated the effect of interventions at author‐level, 16 at reviewer‐level, and 3 at editor‐level. Three studies investigated interventions at multiple levels. The effects of the interventions were reported as mean change in review quality, duration of the peer‐review process, acceptance/rejection rate, manuscript quality, and number of errors detected in 13, 11, 5, 4, and 3 studies, respectively. At network meta‐analysis, reviewer‐level interventions were associated with a significant improvement in review quality (SMD, 0.20 [0.06 to 0.33]), at the cost of increased duration of the review process (SMD, 0.15 [0.01 to 0.29]), except for reviewer blinding. Author‐ and editor‐level interventions did not significantly impact peer‐review quality and duration (respectively, SMD, 0.17 [−0.16 to 0.51] and SMD, 0.19 [−0.40 to 0.79] for quality, and SMD, 0.17 [−0.16 to 0.51] and SMD, 0.19 [−0.40 to 0.79] for duration).

Conclusions

Modifications of the traditional peer‐review process at reviewer‐level are associated with improved quality, at the price of longer duration. Further studies are needed.

Registration

URL: https://www.crd.york.ac.uk/prospero; Unique identifier: CRD42020187910.

Gender disparity in critical care publications: a novel Female First Author Index

Background

Bibliometric analyses show gender bias against women in scientific publications and citations. We hypothesized that a metric of an individual senior author’s inclusivity of women as first authors in critical care publications would predict gender inequality.

Methods

Using PubMed and Web of Science, we conducted a bibliometric analysis of original research publications in critical care from 2008 to 2018 in 11 specialty and general journals. Gender for first and senior authors was assigned by a gender determination application, and manually if needed. For all senior authors we defined the novel Female First Author Index (FFA-index) = #Female first authors in publications by an individual senior author/Total # publications by that senior author. We produced a novel interactive web-based application using the R package Shiny to increase potential utilization of the FFA-index.

Results

Of 7370 publications, 30.4% had female first authors and 15.5% had female senior authors. After adjustment for impact factor, journal, year of publication, number of authors, country, and gender determination accuracy, female senior authorship was associated with a 1.9-fold increase in female first authorship [OR = 1.85 (95% CI 1.62, 2.11); p < 0.001] compared with male senior authorship. The mean (SD) FFA-index for all individual senior authors was 30.5 (42.9); with a significant difference in FFA-index between male and female senior authors (27.6 versus 42.5, respectively; p < 0.001). The interactive web-based application (FFA-index App) produces the same FFA-index output as our study results.

Conclusions

Female representation at prominent authorship positions in critical care publications is still far from achieving gender parity. By creating an authorship index score, we propose a frame of reference for the advancement of female first authorship.

Supplementary Information

The online version contains supplementary material available at 10.1186/s13613-021-00889-3.

How culture, institutions, and individuals shape the evolving gender gap in science and mathematics: an equity provocation for the scientific community

This essay contextualizes recently measured global gender gaps in science and mathematics within three different themes relevant for enhancing equity in science: journal peer review policies, academic service at US higher education institutions, and parental leave policies and usage. The article aims to problematize potential approaches for reducing such gender gaps, and thus build capacity to advance evidence-grounded transformative change. Framed as an equity provocation for the science community, this piece hopes to elicit productive thought and evidence-based action through informed perspective taking. The vision is not just to enhance gender equity in science and mathematics, but also to ensure the continued quality and relevance of our scientific endeavors for today’s diverse and global world.

Gender trends in authorships and publication impact in Academic Radiology-a 10-year perspective

Objectives

To analyze the development of publication numbers of female authors in high-, medium-, and low-impact radiological journals.

Methods

In this bibliometric analysis, gender of the first (FA) and senior author (SA) was assigned to all original research articles and reviews, published in 10 high-, medium-, and low-impact radiological journals in 2007/8 and 2017/18. The adjusted event rate (AER) and adjusted odds ratio (AOR) were calculated using mixed logistic and multinomial logistic regression models to assess and compare female publications according to impact factor, journal, author position, and combination.

Results

The proportion of female FA and female SA in N = 6979 (2007/2008) and N = 7383 (2017/2018) articles increased to 29.1% and 16.1% in 2017/2018, respectively. While most female authorships were continuously observed in medium-impact journals, the strongest increase occurred for both female FA (AOR 2.0; p < .0001) and SA (AOR 2.1; p < .0001) in low-impact journals. Female SA published significantly more often in a low- (AOR 1.5) or medium- (AOR 1.8) than in a high-ranking journal. Among the high-ranking journals, female FA published most frequently in European Radiology (32.4%; 95% CI [29.3–35.8]; p < .0001), female SA in Investigative Radiology (15.9%; 95% CI [13.7–18.4]; p < .0001). Male same-sex authorships decreased (AOR 0.9), but remained at least twice as common as all-female or mixed authorships.

Conclusion

The increase in female authorship is reflected in all impact areas. Female FA and SA increased most in low-ranking journals but are most common in medium-ranking journals. Female SA remain rare, especially in high impact journals.

Key Points

• Compared to the proportion of female radiologists worldwide, female senior authors are underrepresented in all impact areas, in particular in high-impact journals.

• Among the included high-ranking radiological journals, female first authors and senior authors were strongest represented in European Radiology and Investigative Radiology, while across all impact areas they mostly published in medium-ranking journals.

• Female author combinations were more frequent in low- and medium- than in high-ranking journals, whereas male author combinations remained more common than female senior author collaborations in all impact areas.

Comprehensive examination of the peer review process in academic medicine: Towards reaching unbiased decisions - Editorial

•The implementation of double- or triple-blind review practices will ensure that authors with worthwhile and prominent research will have fair and equitable review regardless of their prominence in the field.•Improving the quality of our reviews and raising the standings of our publishing authors. This makes way for healthy competition and a drive to produce high quality research.•It is our responsibility to limit or eliminate bias by promoting impartiality and increasing the level of transparency between the editorial teams and authors, allowing peer review to be more inclusive, instructional, and equitable.

Single-Blind and Double-Blind Peer Review: Effects on National Representation

Purpose: To assess whether the type of peer-review (single-blinded vs double-blinded) has an impact on nationality representation in journals.Methods: A cross-sectional study analyzing the top 10 nationalities contributing to the number of articles across 16 ophthalmology journals.Results: There was no difference in the percentage of articles published from the journal's country of origin between the top single-blind journals and double-blind journals (SB = 42.0%, DB = 26.6%, p = .49), but there was a significant difference between the percentage of articles from the US (SB = 48.0%, DB = 22.8%, p = .02). However, there was no difference for both country of origin (SB = 38.0%, DB = 26.6%, p = .43) and articles from the US (SB = 35.0%, DB = 22.8%, p = .21) when assessing the top eight double-blind journals matched with single-blind journals of a similar impact factor. The US (n = 16, 100%) and England (n = 16, 100%) most commonly made the top 10 lists for article contribution. This held true even for journals established outside the United States (US=11/12, England = 11/12).Conclusions: There was no significant difference in country-of-origin representation between single-blind journals and double-blind journals. However, higher income countries contributed most often to the journals studied even among journals based outside the US.

Peer review practices by medical imaging journals

OBJECTIVE

To investigate peer review practices by medical imaging journals.

METHODS

Journals in the category "radiology, nuclear medicine and medical imaging" of the 2018 Journal Citation Reports were included.

RESULTS

Of 119 included journals, 62 (52.1%) used single-blinded peer review, 49 (41.2%) used double-blinded peer review, two (1.7%) used open peer review and one (0.8%) used both single-blinded and double-blinded peer reviews, while the peer review model of five journals (4.2%) remained unclear. The use of single-blinded peer review was significantly associated with a journal's impact factor (correlation coefficient of 0.218, P = 0.022). On subgroup analysis, only subspecialty medical imaging journals had a significant association between the use of single-blinded peer review and a journal's impact factor (correlation coefficient of 0.354, P = 0.025). Forty-eight journals (40.3%) had a reviewer preference option, 48 journals (40.3%) did not have a reviewer recommendation option, and 23 journals (19.3%) obliged authors to indicate reviewers on their manuscript submission systems. Sixty-four journals (53.8%) did not provide an explicit option on their manuscript submission Web site to indicate nonpreferred reviewers, whereas 55 (46.2%) did. There were no significant associations between the option or obligation to indicate preferred or nonpreferred reviewers and a journal's impact factor.

CONCLUSION

Single-blinded peer review and the option or obligation to indicate preferred or nonpreferred reviewers are frequently employed by medical imaging journals. Single-blinded review is (weakly) associated with a higher impact factor, also for subspecialty journals. The option or obligation to indicate preferred or nonpreferred reviewers is evenly distributed among journals, regardless of impact factor.