The giant plan to track diversity in research journals

The SCOPE framework - implementing ideals of responsible research assessment

Background

Research and researchers are heavily evaluated, and over the past decade it has become widely acknowledged that the consequences of evaluating the research enterprise and particularly individual researchers are considerable. This has resulted in the publishing of several guidelines and principles to support moving towards more responsible research assessment (RRA). To ensure that research evaluation is meaningful, responsible, and effective the International Network of Research Management Societies (INORMS) Research Evaluation Group created the SCOPE framework enabling evaluators to deliver on existing principles of RRA. SCOPE bridges the gap between principles and their implementation by providing a structured five-stage framework by which evaluations can be designed and implemented, as well as evaluated.

Methods

SCOPE is a step-by-step process designed to help plan, design, and conduct research evaluations as well as check effectiveness of existing evaluations. In this article, four case studies are presented to show how SCOPE has been used in practice to provide value-based research evaluation.

Results

This article situates SCOPE within the international work towards more meaningful and robust research evaluation practices and shows through the four case studies how it can be used by different organisations to develop evaluations at different levels of granularity and in different settings.

Conclusions

The article demonstrates that the SCOPE framework is rooted firmly in the existing literature. In addition, it is argued that it does not simply translate existing principles of RRA into practice, but provides additional considerations not always addressed in existing RRA principles and practices thus playing a specific role in the delivery of RRA. Furthermore, the use cases show the value of SCOPE across a range of settings, including different institutional types, sizes, and missions.

Journals must expand access to peer review data

Empirical studies on peer review bias are primarily conducted by people from privileged groups and with affiliations with the journals studied. Data access is one major barrier to conducting peer review research. Accordingly, we propose pathways to broaden access to peer review data to people from more diverse backgrounds.

Disparities in paediatric radiology research publications from low- and lower middle-income countries: a time for change

The positive impact of diversity on health research and outcomes is well-recognised and widely published. Despite this, published evidence shows that at every step of the research pathway, issues of equity, diversity and inclusion (EDI) arise. There is evidence of a lack of diversity within research teams, in the research questions asked/research participants recruited, on grant review/funding panels, amongst funded researchers and on the editorial boards and reviewer pools of the journals to which results are submitted for peer-reviewed publication. Considering the journal Pediatric Radiology, while its editorial board of 92 members has at least one member affiliated to a country in every region of the world, the majority are in North America (n=52, 57%) and Europe (n=30, 33%) and only two (2%) are affiliated to institutions in a lower middle-income country (LMIC) (India, Nigeria), with one (1%) affiliated to an institution in an upper middle-income country (UMIC) (Peru) and none in a low-income country (LIC). Pediatric Radiology is "…the official journal of the European Society of Paediatric Radiology, the Society for Pediatric Radiology, the Asian and Oceanic Society for Pediatric Radiology and the Latin American Society of Pediatric Radiology". However, of the total number of manuscripts submitted for potential publication in the four years 2019 through 2022, only 0.03% were from a LIC and only 7.9% were from a LMIC. Further, the frequency of acceptance of manuscripts from UMIC was seven times higher than that from LMIC (no manuscripts were published from LIC). Increased collaboration is required between researchers across the globe to better understand the barriers to equity in the funding, conduct and publication of research from LIC and LMIC and to identify ways in which we can overcome them together.

Editors' Note: Third Annual Report Regarding JAACAP's Antiracist Journey

In 2020, we wrote to you of our dedication and vision for JAACAP "to be antiracist at every level."1 Over the last 3 years, we have pursued initiatives "to reshape the Journal to pursue this vision."2,3 In this article, we provide an update on these goals and initiatives (Figure 1). With the launching of our new open access journal, JAACAP Open,4 in late 2022, we now extend these initiatives to both scientific journals in the JAACAP family and aspire to be a leader among mental health journals in our intentional pursuit of antiracist policies and practices.

Positionality and Its Problems: Questioning the Value of Reflexivity Statements in Research

There has been a remarkable push for the use of positionality statements-also known as reflexivity statements-in scientific-journal articles and other research literatures. Grounded in reputable philosophical traditions, positionality statements are meant to address genuine concerns about the limits of knowledge production. However, there are at least three reasons why they should be avoided in scholarship. First, it is impossible to construct credible positionality statements because they are constrained by the very positionality they seek to address. Second, positionality statements are unnecessary because reducing bias-positional or otherwise-in scientific literatures does not hinge on the biographical details of individual scholars but on the integrity of the collective process of truth-seeking. Third, by asking scholars to disclose information about themselves, positionality statements undermine the very norms and practices that safeguard the impartiality of research. Instead of asking individual scholars to issue subjective declarations about their positionalities, scholarly communities should focus on improving the rules of intersubjective competition at the heart of scientific progress. In our view, the most productive path to increasing representation and reducing positional bias in research is to protect the freedom of scholarly inputs while insisting on methodological transparency and rigor.

The illusion of inclusion: contextual behavioral science and the Black community

Anti-racism approaches require an honest examination of cause, impact, and committed action to change, despite discomfort and without experiential avoidance. While contextual behavioral science (CBS) and third wave cognitive-behavioral modalities demonstrate efficacy among samples composed of primarily White individuals, data regarding their efficacy with people of color, and Black Americans in particular, is lacking. It is important to consider the possible effects of racial stress and trauma on Black clients, and to tailor approaches and techniques grounded in CBS accordingly. We describe how CBS has not done enough to address the needs of Black American communities, using Acceptance and Commitment Therapy (ACT) and Functional Analytic Psychotherapy (FAP) as examples. We also provide examples at the level of research representation, organizational practices, and personal experiences to illuminate covert racist policy tools that maintain inequities. Towards eradicating existing racism in the field, we conclude with suggestions for researchers and leadership in professional psychological organizations.

Author Guide for Addressing Animal Methods Bias in Publishing

There is growing recognition that animal methods bias, a preference for animal-based methods where they are not necessary or where nonanimal-based methods may already be suitable, can impact the likelihood or timeliness of a manuscript being accepted for publication. Following April 2022 workshop about animal methods bias in scientific publishing, a coalition of scientists and advocates formed a Coalition to Illuminate and Address Animal Methods Bias (COLAAB). The COLAAB has developed this guide to be used by authors who use nonanimal methods to avoid and respond to animal methods bias from manuscript reviewers. It contains information that researchers may use during 1) study design, including how to find and select appropriate nonanimal methods and preregister a research plan, 2) manuscript preparation and submission, including tips for discussing methods and choosing journals and reviewers that may be more receptive to nonanimal methods, and 3) the peer review process, providing suggested language and literature to aid authors in responding to biased reviews. The author's guide for addressing animal methods bias in publishing is a living resource also available online at animalmethodsbias.org, which aims to help ensure fair dissemination of research that uses nonanimal methods and prevent unnecessary experiments on animals.

Atopic Dermatitis Across Shades of Skin

Atopic dermatitis (AD) is a chronic, heterogeneous inflammatory skin disease that is associated with immense patient burden globally. There is increasing appreciation of disparities among patients identified as having skin of color (SOC), which often refers to patients of non-White race or non-European ancestry, but can broadly include individuals from a number of different racial, ethnic, ancestral, and skin pigmentation groups based on definition. In this narrative review, we discuss key terminology as it relates to AD across shades of skin, including modern definitions of 'race', 'ethnicity', and 'SOC'. We then synthesize the current literature describing disparities in AD prevalence, disease recognition, and burden alongside current data regarding genetic and immunologic findings across SOC populations. In the context of these findings, we highlight key concomitant social determinants of health, including environmental factors, socioeconomic status, and access to care, for which race often serves as a proxy for true biological and genetic differences. Finally, we discuss future efforts to shift to a more inclusive understanding of AD to encompass all shades of skin, to ensure equitable representation of diverse populations in high impact research, and intensify efforts to address the critical upstream factors driving observed disparities.

How to improve scientific peer review: Four schools of thought

Peer review plays an essential role as one of the cornerstones of the scholarly publishing system. There are many initiatives that aim to improve the way in which peer review is organized, resulting in a highly complex landscape of innovation in peer review. Different initiatives are based on different views on the most urgent challenges faced by the peer review system, leading to a diversity of perspectives on how the system can be improved. To provide a more systematic understanding of the landscape of innovation in peer review, we suggest that the landscape is shaped by four schools of thought: The Quality & Reproducibility school, the Democracy & Transparency school, the Equity & Inclusion school, and the Efficiency & Incentives school. Each school has a different view on the key problems of the peer review system and the innovations necessary to address these problems. The schools partly complement each other, but we argue that there are also important tensions between them. We hope that the four schools of thought offer a useful framework to facilitate conversations about the future development of the peer review system.

Outcomes of COVID-19 manuscripts submitted to the Canadian Journal of Anesthesia: a retrospective audit of author gender and person of colour status

PURPOSE

We aimed to evaluate the representation of women and persons of colour (POC) authors of COVID-19 manuscripts submitted to, accepted in, and rejected from the Journal and to evaluate trends in their representation during the pandemic.

METHODS

All COVID-19 manuscripts submitted to the Journal between 1 February 2020 and 30 April 2021 were included. Manuscript data were retrieved from Editorial Manager, and gender and POC status were obtained through: 1) e-mail communication with corresponding authors; 2) e-mail queries to other coauthors; 3) NamSor software, and 4) Internet searches. The data were described using percentages and summary statistics. A two-sample test of proportions was used for comparisons and trends were analyzed with linear regression.

RESULTS

We identified 314 manuscripts (1,555 authors), 95 (461 authors) of which were accepted for publication. Of all authors, 515 (33%) were women, and women were the lead and senior authors of 101 (32%) and 69 (23%) manuscripts, respectively. There were no differences in women's representation as authors between accepted and rejected manuscripts. Overall, 923/1,555 (59%) authors were identified as POC, with a significantly lower proportion of POC authors among accepted vs rejected manuscripts (41%, 188/461 vs 67%, 735/1,094; difference, -26%; 95% CI, -32 to -21; P < 0.001). We did not observe significant trends in the proportion of women and POC authors over the study period.

CONCLUSION

The proportion of women authors of COVID-19 manuscripts was lower than men's representation. Further research is required to determine the factors that account for the higher proportion of POC authors across rejected manuscripts.

Little transparency and equity in scientific awards for early- and mid-career researchers in ecology and evolution

Scientific awards can shape scientific careers, helping to secure jobs and grants, but can also contribute to the lack of diversity at senior levels and in the elite networks of scientists. To assess the status quo and historical trends, we evaluated 'best researcher' awards and 'best paper' early- and mid-career awards from broad-scope international journals and societies in ecology and evolution. Specifically, we collated information on eligibility rules, assessment criteria and potential gender bias. Our results reveal that, overall, few awards foster equitable access and assessment. Although many awards now explicitly allow extensions of the eligibility period for substantial career interruptions, there is a general lack of transparency in terms of assessment and consideration of other differences in access to opportunities and resources among junior researchers. Strikingly, open science practices were mentioned and valued in only one award. By highlighting instances of desirable award characteristics, we hope this work will nudge award committees to shift from simple but non-equitable award policies and practices towards strategies enhancing inclusivity and diversity. Such a shift would benefit not only those at the early- and mid-career stages but the whole research community. It is also an untapped opportunity to reward open science practices, promoting transparent and robust science.

Peer review perpetuates barriers for historically excluded groups

Peer review is central to the scientific process and scientists' career advancement, but bias at various stages of the review process disadvantages some authors. Here we use peer review data from 312,740 biological sciences manuscripts across 31 studies to (1) examine evidence for differential peer review outcomes based on author demographics, (2) evaluate the efficacy of solutions to reduce bias and (3) describe the current landscape of peer review policies for 541 ecology and evolution journals. We found notably worse review outcomes (for example, lower overall acceptance rates) for authors whose institutional affiliations were in Asia, for authors whose country's primary language is not English and in countries with relatively low Human Development Indices. We found few data evaluating efficacy of interventions outside of reducing gender bias through double-blind review or diversifying reviewer/editorial boards. Despite evidence for review outcome gaps based on author demographics, few journals currently implement policies intended to mitigate bias (for example, 15.9% of journals practised double-blind review and 2.03% had reviewer guidelines that mentioned social justice issues). The lack of demographic equity signals an urgent need to better understand and implement evidence-based bias mitigation strategies.

Herminthology: promoting gender equity in science and parasitology

Gender inequity in Science, Technology, Engineering, and Medicine (STEM) fields, including parasitology, continues to limit the participation of women in scientific leadership and development. Here we highlight the aims and activities of Herminthology, an initiative promoting the work of women in parasitology, alongside the current status quo of men and women scientists in the discipline.

New considerations of health disparities within allergy and immunology

The pandemic, political upheavals, and social justice efforts in our society have resulted in attention to persistent health disparities and the urgent need to address them. Using a scoping review, we describe published updates to address disparities and targets for interventions to improve gaps in care within allergy and immunology. These disparities-related studies provide a broad view of our current understanding of how social determinants of health threaten patient outcomes and our ability to advance health equity efforts in our field. We outline next steps to improve access to care and advance health equity for patients with allergic/immunologic diseases through actions taken at the individual, community, and policy levels, which could be applied outside of our field. Key among these are efforts to increase the diversity among our trainees, providers, and scientific teams and enhancing efforts to participate in advocacy work and public health interventions. Addressing health disparities requires advancing our understanding of the interplay between social and structural barriers to care and enacting the needed interventions in various key areas to effect change.

CRT in praxis: Library and archival collections at San José State University

Through various efforts, the staff and faculty of San José State University’s Dr. Martin Luther King, Jr. Library (King Library) are working towards creating more equitable and inclusive collections. Examining the library’s collecting practices and collections by and about African Americans, this article presents the ongoing work of a working group that was formed in 2020 in response to an Anti-Racism Action Plan developed in the library. By using some of the tenets of the CRT framework such as intersectionality, counter storytelling, and deconstructing colorblindness and white supremacy, the authors discuss the steps that are being taken to revise, review, and revisit the King Library’s collecting practices in relation to the history of SJSU’s African American Studies program, The Africana Center, and other relevant community history.

'You can't be what you can't see': equity, diversity and inclusivity of editorial teams of dental journals

Introduction Equity, diversity and inclusivity in research is critical to ensure all of society contributes.Aim This study analysed the gender and location (workplace) of editorial teams across 124 dental journals.Methods All chief editors of the 91 dental journals listed on the journal citation report for 2021 were contacted to provide details of the numbers of chief editors, other editors and the editorial board members, plus a summary of their demographic information including gender, age (group) and location. If chief editors did not respond, publicly available information was collated for the analysis. The same publicly available information of 45 PubMed indexed journals not on the citation report and three emerging online journals was also retrieved.Results Of the 159 chief editors across the 124 journals, 131 (82%) were men and 30% were based in the USA. Of the 1,265 other editors, two-thirds were from the USA, UK, Brazil and Japan. Of the 3,044 editorial board members, half were from the same four countries.Conclusions 'You can't be what you can't see': women and people from many parts of the world cannot see themselves as chief editors, other editors or editorial board members of dental journals.

Gender equity in health research publishing in Africa

Introduction

Women researchers find it more difficult to publish in academic journals than men, an inequity that affects women’s careers and was exacerbated during the pandemic, particularly for women in low-income and middle-income countries. We measured publishing by sub-Saharan African (SSA) women in prestigious authorship positions (first or last author, or single author) during the time frame 2014–2016. We also examined policies and practices at journals publishing high rates of women scientists from sub-Saharan Africa, to identify potential structural enablers affecting these women in publishing.

Methods

The study used Namsor V.2, an application programming interface, to conduct a secondary analysis of a bibliometric database. We also analysed policies and practices of ten journals with the highest number of SSA women publishing in first authorship positions.

Results

Based on regional analyses, the greatest magnitude of authorship inequity is in papers from sub-Saharan Africa, where men comprised 61% of first authors, 65% of last authors and 66% of single authors. Women from South Africa and Nigeria had greater success in publishing than those from other SSA countries, though women represented at least 20% of last authors in 25 SSA countries. The journals that published the most SSA women as prominent authors are journals based in SSA. Journals with overwhelmingly male leadership are also among those publishing the highest number of SSA women.

Conclusion

Women scholars in SSA face substantial gender inequities in publishing in prestigious authorship positions in academic journals, though there is a cadre of women research leaders across the region. Journals in SSA are important for local women scholars and the inequities SSA women researchers face are not necessarily attributable to gender discrepancy in journals’ editorial leadership.

Women Academics’ Intersectional Experiences of Policy Ineffectiveness in the European Context

Despite policy efforts targeted at making universities more inclusive and equitable, academia is still rife with harassment and bullying, and opportunities are far from equal for everyone. The present preregistered survey research (N = 91) aimed to explore whether an intersectional approach can be useful to examine the tangible effects of policy ineffectiveness, even when legislative and ideologic constraints limit the possibility to conduct a full-fledged intersectional analysis. Policy ineffectiveness was operationalized as experiences of harassment, discrimination, institutional resistance to gender equality, and retaliation against reporters of misconduct in universities. Policy ineffectiveness was negatively related to women academics’ inclination to pursue an academic career. This relationship was mediated by lower levels of psychological safety associated with policy ineffectiveness. Importantly, women academics who differ from the majority on multiple dimensions show a stronger and more negative relationship between policy ineffectiveness and psychological safety. The study further shows that self-report measures are useful to uncover intersectional privilege afforded to overrepresented groups in academia. The study discusses the benefits of intersectional approaches for designing and implementing effective policies to tackle harassment and inequality in academia, even when the available methodologies are constrained by legislation and ideology. Overall, self-report measurement can have an important function for signalling areas that warrant further intersectional inquiry to ensure that policies serve everyone.