Citation gaming induced by bibliometric evaluation: A country-level comparative analysis

The increasing authorship trend in neuroscience: A scientometric analysis across 11 countries

Previous studies have demonstrated an increasing trend of the number of authors across various fields over the years. This trend has been attributed to the necessity for larger collaborations and, at times, to ethical issues regarding authorship attribution. Our study focuses on the evolution of authorship trends in the field of Neuroscience. We conducted our analysis based on a dataset containing 580,782 neuroscience publications produced from 2000 to 2022, focusing on the publications within the Group of ten (G10) countries. Using a matrix-based methodology, we extracted and analyzed the average number of authors per country. Our findings reveal a consistent rise in authorship across all G10 countries over the past two decades. Italy emerged with the highest average number of authors, while France stood out for experiencing the most significant increase, particularly in the last decade. The countries with the lowest number of authors per publication were the USA, UK and Canada. Differences between countries could result from variations in the size of collaboration between researchers in different countries. Additionally, these differences may depend on utilitarian considerations aimed at receiving higher scores in the individual evaluation of their own work. We propose that a normalization procedure for the number of authors should be implemented to ensure a fair evaluation of researchers.

Overcoming confusion and stigma in habitat fragmentation research

Anthropogenic habitat loss is widely recognized as a primary environmental concern. By contrast, debates on the effects of habitat fragmentation persist. To facilitate overcoming these debates, here we: (i) review the state of the literature on habitat fragmentation, finding widespread confusion and stigma; (ii) identify consequences of this for biodiversity conservation and ecosystem management; and (iii) suggest ways in which research can move forward to resolve these problems. Confusion is evident from the 25 most-cited fragmentation articles published between 2017 and 2021. These articles use five distinct concepts of habitat fragmentation, only one of which clearly distinguishes habitat fragmentation from habitat area and other factors ('fragmentation per se'). Stigmatization is evident from our new findings that fragmentation papers are more charged with negative sentiments when compared to papers from other subfields in the environmental sciences, and that fragmentation papers with more negative sentiments are cited more. While most empirical studies of habitat fragmentation per se find neutral or positive effects on species and biodiversity outcomes, which implies that small habitat patches have a high cumulative value, confusion and stigma in reporting and discussing such results have led to suboptimal habitat protection policy. For example, government agencies, conservation organizations, and land trusts impose minimum habitat patch sizes on habitat protection. Given the high cumulative value of small patches, such policies mean that many opportunities for conservation are being missed. Our review highlights the importance of reducing confusion and stigma in habitat fragmentation research. To this end, we propose implementing study designs in which multiple sample landscapes are selected across independent gradients of habitat amount and fragmentation, measured as patch density. We show that such designs are possible for forest habitat across Earth's biomes. As such study designs are adopted, and as language becomes more precise, we expect that confusion and stigma in habitat fragmentation research will dissipate. We also expect important breakthroughs in understanding the situations where effects of habitat fragmentation per se are neutral, positive, or negative, and the reasons for these differences. Ultimately this will improve efficacy of area-based conservation policies, to the benefit of biodiversity and people.

Mapping the Nordic Research Landscape for the period 2016-2020: a comprehensive study of research outcomes, collaborations, and impact

Background

This article aims to study the research outcomes of five Nordic countries in terms of research publications, spend on R&D, outcomes and collaborations as these are important parameters to understand research thrust of the countries/regions, in addition to their innovation capability.

Methods

The research outcomes of the Nordic countries in terms of the total number of publications, coauthored publications, publications with corporate collaborators, citations, the Field Weighted Citation Index (FWCI) and publications in different subject areas were retrieved using Scopus and its associate SciVal. The research outcomes were extracted for five years from 2016-2020. In addition, total population, researcher population and research spend of these countries have been obtained from World Bank data available for the year 2021.

Results

The analysis showed that Sweden has the highest population and the highest number of researchers in this region. All countries have the highest number of coauthored publications with the United States, followed by the United Kingdom, except Iceland, which has the second highest number of coauthored publications with Sweden. Denmark, followed by Iceland, stands prominent with reference to having publications with corporate collaborations. Denmark and Sweden have a high percentage of articles in first quartile journals, which is above the average for Nordic countries. Iceland stands at the top with the highest citations, which is depicted by high FWCI. Across subject areas, the Nordic countries have maximum publications in life sciences. Other prominent subject areas include technology and natural sciences.

Conclusion

On analysing the research landscape of Nordic countries, maximum research output is in the field of life sciences and medicine, and most of the coauthored publications of these countries are with the United States. Denmark, with its exemplary research output, excels with maximum papers in top quartile journals and with maximum corporate collaborations and the highest FWCI.

'Special issue-ization' as a growth and revenue strategy: Reproduction by the "big five" and the risks for research integrity

The exponential growth of MDPI and Frontiers over the last decade has been powered by their extensive use of special issues. The "special issue-ization" of journal publishing has been particularly associated with new publishers and seen as potentially "questionable." Through an extended case-study analysis of three journals owned by one of the "big five" commercial publishers, this paper explores the risks that this growing use of special issues presents to research integrity. All three case-study journals show sudden and marked changes in their publication patterns. An analysis of special issue editorials and retraction notes was used to determine the specifics of special issues and reasons for retractions. Descriptive statistics were used to analyse data. Findings suggest that these commercial publishers are also promoting special issues and that article retractions are often connected to guest editor manipulation. This underlies the threat that "special issue-ization" presents to research integrity. It highlights the risks posed by the guest editor model, and the importance of extending this analysis to long-existing commercial publishers. The paper emphasizes the need for an in-depth examination of the underlying structures and political economy of science, and a discussion of the rise of gaming and manipulation within higher education systems.

Bad apples or systematic problem? Is Italy struggling with maintaining high level of research integrity?

The recent scandal involving Prof. Schillaci has raised concerns about the integrity of the Italian academic landscape, highlighting potential deeper issues within the research ecosystem. Despite the existence of comprehensive guidelines for research integrity set by the National Council of Research (CNR) and some prominent universities, the emphasis on educating research personnel about the importance of research integrity remains lacking. Additionally, prevalent issues such as nepotism and the manipulation of metrics for career advancement pose further challenges to fostering a fair and accountable research environment. While certain legislative measures have been implemented to address these issues, their effectiveness remains limited, allowing unethical practices to persist. To address these challenges, a concerted effort at the national, institutional, and individual levels is necessary. By taking these steps, Italy has the opportunity to strengthen its research ethics landscape and move toward a more transparent and ethical academic environment.

A global exploratory comparison of country self-citations 1996-2019

Self-citations are a key topic in evaluative bibliometrics because they can artificially inflate citation-related performance indicators. Recently, self-citations defined at the largest scale, i.e., country self-citations, have started to attract the attention of researchers and policymakers. According to a recent research, in fact, the anomalous trends in the country self-citation rates of some countries, such as Italy, have been induced by the distorting effect of citation metrics-centered science policies. In the present study, we investigate the trends of country self-citations in 50 countries over the world in the period 1996-2019 using Scopus data. Results show that for most countries country self-citations have decreased over time. 12 countries (Colombia, Egypt, Indonesia, Iran, Italy, Malaysia, Pakistan, Romania, Russian Federation, Saudi Arabia, Thailand, and Ukraine), however, exhibit different behavior, with anomalous trends of self-citations. We argue that these anomalies should be attributed to the aggressive science policies adopted by these countries in recent years, which are all characterized by direct or indirect incentives for citations. Our analysis confirms that when bibliometric indicators are integrated into systems of incentives, they are capable of affecting rapidly and visibly the citation behavior of entire countries.

Ranking mobility and impact inequality in early academic careers

How difficult is it for an early career academic to climb the ranks of their discipline? We tackle this question with a comprehensive bibliometric analysis of 57 disciplines, examining the publications of more than 5 million authors whose careers started between 1986 and 2008. We calibrate a simple random walk model over historical data of ranking mobility, which we use to 1) identify which strata of academic impact rankings are the most/least mobile and 2) study the temporal evolution of mobility. By focusing our analysis on cohorts of authors starting their careers in the same year, we find that ranking mobility is remarkably low for the top- and bottom-ranked authors and that this excess of stability persists throughout the entire period of our analysis. We further observe that mobility of impact rankings has increased over time, and that such rise has been accompanied by a decline of impact inequality, which is consistent with the negative correlation that we observe between such two quantities. These findings provide clarity on the opportunities of new scholars entering the academic community, with implications for academic policymaking.

The Clinical Research Bias Index (CRBI): A novel journal ranking method applied to child health respiratory studies

Background and Aims

Journal impact factor has historically been taken as a proxy for quality. However, this is open to significant manipulation and bias. There is currently not widely adopted, robust journal and paper ranking metric which is focused solely on risk of bias.

Methods

Risk of bias data was extracted from all Cochrane database systematic reviews in Child Health, Lungs, and Airways for the years 2017-2019. A novel paper quality score, the Clinical Research Bias Index (CRBI), was applied. Individual paper data were pooled for each journal. A comparison was made to journal impact factors, individual paper citations, reads, and altmetric scores.

Results

927 papers were analyzed for risk of bias. 119 (12·8%) scored a CRBI of 100%, with a mean score of 70%. A journal's overall CRBI risk of bias score was poorly correlated with impact factor (r 0.25). Citations (r 0.02), and reads (r 0.01) of individual papers showed very little association with the paper's risk of bias. Likewise, reads were not correlated with citations (r 0.03). H-index and Altmetric scores were similarly poorly correlated with CRBI.

Conclusion

The novel research quality tool CRBI demonstrates the poor correlation between journal impact factor, citations, and risk of bias. Journal and paper ranking metrics should ensure that they are fit for purpose, and enable the dissemination of high-quality research for the benefit of patients. We propose the CRBI as a potential solution which is resistant to manipulation and will reward the creation and publication of bias-free research.

Massive covidization of research citations and the citation elite

The COVID-19 pandemic saw a massive mobilization of the scientific workforce. We evaluated the citation impact of COVID-19 publications relative to all scientific work published in 2020 to 2021, finding that 20% of citations received to papers published in 2020 to 2021 were to COVID-19–related papers. Across science, 98 of the 100 most-cited papers published in 2020 to 2021 were related to COVID-19. A large number of scientists received large numbers of citations to their COVID-19 work, often exceeding the citations they had received to all their work during their entire career. We document a strong covidization of research citations across science. This may have major repercussions for research priorities and the evolution of research on COVID-19 and beyond.

Massive scientific productivity accompanied the COVID-19 pandemic. We evaluated the citation impact of COVID-19 publications relative to all scientific work published in 2020 to 2021 and assessed the impact on scientist citation profiles. Using Scopus data until August 1, 2021, COVID-19 items accounted for 4% of papers published, 20% of citations received to papers published in 2020 to 2021, and >30% of citations received in 36 of the 174 disciplines of science (up to 79.3% in general and internal medicine). Across science, 98 of the 100 most-cited papers published in 2020 to 2021 were related to COVID-19; 110 scientists received ≥10,000 citations for COVID-19 work, but none received ≥10,000 citations for non–COVID-19 work published in 2020 to 2021. For many scientists, citations to their COVID-19 work already accounted for more than half of their total career citation count. Overall, these data show a strong covidization of research citations across science, with major impact on shaping the citation elite.

Gender influences in Digital Humanities co-authorship networks

Purpose

This paper presents a co-authorship study of authors who published in Digital Humanities journals and examines the apparent influence of gender, or more specifically, the quantitatively detectable influence of gender in the networks they form.

Design/methodology/approach

This study applied co-authorship network analysis. Data has been collected from three canonical Digital Humanities journals over 52 years (1966–2017) and analysed.

Findings

The results are presented as visualised networks and suggest that female scholars in Digital Humanities play more central roles and act as the main bridges of collaborative networks even though overall female authors are fewer in number than male authors in the network.

Originality/value

This is the first co-authorship network study in Digital Humanities to examine the role that gender appears to play in these co-authorship networks using statistical analysis and visualisation.

Bibliometrics in Press. Representations and uses of bibliometric indicators in the Italian daily newspapers

Scholars in science and technology studies and bibliometricians are increasingly revealing the performative nature of bibliometric indicators. Far from being neutral technical measures, indicators such as the Impact Factor and the h-index are deeply transforming the social and epistemic structures of contemporary science. At the same time, scholars have highlighted how bibliometric indicators are endowed with social meanings that go beyond their purely technical definitions. These social representations of bibliometric indicators are constructed and negotiated between different groups of actors within several arenas. This study aims to investigate how bibliometric indicators are used in a context, which, so far, has not yet been covered by researchers, that of daily newspapers. By a content analysis of a corpus of 583 articles that appeared in four major Italian newspapers between 1990 and 2020, we chronicle the main functions that bibliometrics and bibliometric indicators played in the Italian press. Our material shows, among other things, that the public discourse developed in newspapers creates a favorable environment for bibliometrics-centered science policies, that bibliometric indicators contribute to the social construction of scientific facts in the press, especially in science news related to medicine, and that professional bibliometric expertise struggles to be represented in newspapers and hence reach the general public.

The right to refuse unwanted citations: rethinking the culture of science around the citation

Logically, and by most common standards, academics would be pleased to be cited, considering it a form of recognition of their intellect. In return, especially those with high citation counts, such as Clarivate Analytics’ Highly Cited Researchers, can benefit through peer recognition, rewards, funding, securing a better position, or expanding a collaborative network. Despite known and untold benefits, one issue has not been discussed: the right to refuse to be cited or the right to refuse a citation. Academics might not want to be cited by papers published in truly predatory journals, papers with false authors, or sting papers with falsified elements that employ underhanded ethical tactics. Currently, academics generally have the freedom to select where they publish their findings and choose studies they cite, so it is highly probable that requests to remove citations or refuse citations might never become formal publishing policy. Nonetheless, this academic discussion is worth having as valid and invalid literature increasingly gets mixed through citations, and as the grey zone between predatory/non-predatory and scholarly/unscholarly becomes increasingly difficult to distinguish.

Requiem for impact factors and high publication charges

Journal impact factors, publication charges and assessment of quality and accuracy of scientific research are critical for researchers, managers, funders, policy makers, and society. Editors and publishers compete for impact factor rankings, to demonstrate how important their journals are, and researchers strive to publish in perceived top journals, despite high publication and access charges. This raises questions of how top journals are identified, whether assessments of impacts are accurate and whether high publication charges borne by the research community are justified, bearing in mind that they also collectively provide free peer-review to the publishers. Although traditional journals accelerated peer review and publication during the COVID-19 pandemic, preprint servers made a greater impact with over 30,000 open access articles becoming available and accelerating a trend already seen in other fields of research. We review and comment on the advantages and disadvantages of a range of assessment methods and the way in which they are used by researchers, managers, employers and publishers. We argue that new approaches to assessment are required to provide a realistic and comprehensive measure of the value of research and journals and we support open access publishing at a modest, affordable price to benefit research producers and consumers.

Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World

Nowadays, the importance of bibliographic databases (DBs) has increased enormously, as they are the main providers of publication metadata and bibliometric indicators universally used both for research assessment practices and for performing daily tasks. Because the reliability of these tasks firstly depends on the data source, all users of the DBs should be able to choose the most suitable one. Web of Science (WoS) and Scopus are the two main bibliographic DBs. The comprehensive evaluation of the DBs’ coverage is practically impossible without extensive bibliometric analyses or literature reviews, but most DBs users do not have bibliometric competence and/or are not willing to invest additional time for such evaluations. Apart from that, the convenience of the DB’s interface, performance, provided impact indicators and additional tools may also influence the users’ choice. The main goal of this work is to provide all of the potential users with an all-inclusive description of the two main bibliographic DBs by gathering the findings that are presented in the most recent literature and information provided by the owners of the DBs at one place. This overview should aid all stakeholders employing publication and citation data in selecting the most suitable DB.

The Impact of Country Self-Citation Rate Among Medical Specialties in Saudi Arabia

Objectives The purpose of the study was to evaluate the impact of country self-citation rate (SCR) among medical specialties in Saudi Arabia, and to assess the impact of self-citations on the country's total cites world ranking in different specialties. Methods SCImago Journal & Country Rank (SJR) was used to collect data related to all medical specialties in Saudi Arabia for the period 1996-2019. The country SCR for the specialties was correlated with several bibliometric parameters and examined statistically. The specialties that showed a drop in Saudi Arabia's total cites world ranking following the exclusion of self-citations were identified. Results The median country SCR for 46 specialties in Saudi Arabia was 9.5% (range: 4.6-23.1%). The two specialties with the highest country SCR were Public Health (23.1%) and Family Practice (22.9%). Country SCR was significantly higher in the non-clinical specialties compared to clinical specialties (15.3% vs. 9.6%). It did not correlate significantly with any of the examined productivity indices. The exclusion of self-citations resulted in a drop in Saudi Arabia's total cites world ranking in six (13%) specialties only. There was no significant difference between the country's total cites and net total cites world rankings in the specialties. Conclusions Self-citation may be appropriate and signify an expansion of the authors' previous work. Country SCR in medical specialties in Saudi Arabia is relatively low and not affected by total documents and total cites. Non-clinical specialties tend to self-cite more. The exclusion of self-citations had minimal effect on Saudi Arabia's total cites world ranking, indicating that country SCR in the specialties is unlikely to impact its international scientific standing. Our findings do not support the argument for eliminating self-citation from citation-based metrics. We believe that more collaborative and global research practices should be encouraged.

Academic publishing in disaster risk reduction: past, present, and future

Nowadays there are approximately 80 Anglophone journals that deal primarily with disaster risk reduction (DRR) and allied fields. This large array signals a sustained, if uneven, growth in DRR scholarship but also competition between the offerings of different publishers and institutions. The purpose of this article is first to summarise the development of academic publishing on DRR from its early beginnings to the present day. The paper then evaluates the current state of publishing in this field and discusses possible future trends. Next, it identifies some possible opportunities, challenges, expectations, and commitments for journal editors both within DRR and academia more broadly, including those that refer to changes in the use of terminology, the relentless increase in the number of papers submitted, the expansion and dangers of predatory journals, different peer review models, open access versus paywalls, citations and bibliography metrics, academic social networks, and copyright and distribution issues.

Personal Motivations and Systemic Incentives: Scientists on Questionable Research Practices

As concern over the use of questionable research practices (QRPs) in academic science has increased over the last couple of decades, some reforms have been implemented and many others have been debated and recommended. While many of these proposals have merit, efforts to improve scientific practices are more likely to succeed when they are responsive to the prevailing views and concerns of scientists themselves. To date, there have been few efforts to solicit wide-ranging input from researchers on the topic of needed reforms. This article is a qualitative report of responses from federally funded scientists to the question of what should be done to address the problem of QRPs in their disciplines. Overall, participants were concerned about how institutional and career-oriented incentives encourage the use of QRPs. Compared to previous recommendations, participants had surprisingly little confidence in the ability of ethics training to improve research integrity.

How much is too much? The difference between research influence and self-citation excess

Citations can be an indicator of publication significance, utility, attention, visibility or short-term impact but analysts need to confirm whether a high citation count for an individual is a genuine reflection of influence or a consequence of extraordinary, even excessive, self-citation. It has recently been suggested there may be increasing misrepresentation of research performance by individuals who self-cite inordinately to achieve scores and win rewards. In this paper we consider self-referencing and self-citing, describe the typical shape of self-citation patterns for carefully curated publication sets authored by 3517 Highly Cited Researchers and quantify the variance in the distribution of self-citation rates within and between all 21 Essential Science Indicators’ fields. We describe both a generic level of median self-referencing rates, common to most fields, and a graphical, distribution-driven assessment of excessive self-citation that demarcates a threshold not dependent on statistical tests or percentiles (since for some fields all values are within a central ‘normal’ range). We describe this graphical procedure for identifying exceptional self-citation rates but emphasize the necessity for expert interpretation of the citation profiles of specific individuals, particularly in fields with atypical self-citation patterns.

Effective publication strategies in clinical research

Researchers in Europe are increasingly assessed by their publication metrics. To uncover the effect of quantitative assessment on the publication strategies of clinical researchers in Denmark, we interviewed 9 senior researchers at the Department of Clinical Research at the University of Southern Denmark with the lowest and highest values for a, as defined by Hirsch. Our aim is to investigate the importance of these metrics to their academic careers: h-index, number of publications, number of citations, international collaborations, local collaborations, field specific journal publishing and high journal impact factor publishing. To validate our findings we compared their publication record to their statistically analyzed stated publication strategy. Our results indicate two styles of publication strategy used by these senior researchers. Researchers with Low a engage in local collaborations, disseminate knowledge in local media and publish in field specific journals, while researchers with High a engage in international collaborations, invest significant time in publishing in the highest impact journals in their field, and acquire a greater number of citations. Both publication strategies can lead to a successful academic career, yet we have an indication through the h5-index that the practices of the High a group are more likely to nudge the h-index.