The practice of self-citations: a longitudinal study

A global exploratory comparison of country self-citations 1996-2019

Self-citations are a key topic in evaluative bibliometrics because they can artificially inflate citation-related performance indicators. Recently, self-citations defined at the largest scale, i.e., country self-citations, have started to attract the attention of researchers and policymakers. According to a recent research, in fact, the anomalous trends in the country self-citation rates of some countries, such as Italy, have been induced by the distorting effect of citation metrics-centered science policies. In the present study, we investigate the trends of country self-citations in 50 countries over the world in the period 1996-2019 using Scopus data. Results show that for most countries country self-citations have decreased over time. 12 countries (Colombia, Egypt, Indonesia, Iran, Italy, Malaysia, Pakistan, Romania, Russian Federation, Saudi Arabia, Thailand, and Ukraine), however, exhibit different behavior, with anomalous trends of self-citations. We argue that these anomalies should be attributed to the aggressive science policies adopted by these countries in recent years, which are all characterized by direct or indirect incentives for citations. Our analysis confirms that when bibliometric indicators are integrated into systems of incentives, they are capable of affecting rapidly and visibly the citation behavior of entire countries.

Do open citations give insights on the qualitative peer-review evaluation in research assessments? An analysis of the Italian National Scientific Qualification

In the past, several works have investigated ways for combining quantitative and qualitative methods in research assessment exercises. Indeed, the Italian National Scientific Qualification (NSQ), i.e. the national assessment exercise which aims at deciding whether a scholar can apply to professorial academic positions as Associate Professor and Full Professor, adopts a quantitative and qualitative evaluation process: it makes use of bibliometrics followed by a peer-review process of candidates’ CVs. The NSQ divides academic disciplines into two categories, i.e. citation-based disciplines (CDs) and non-citation-based disciplines (NDs), a division that affects the metrics used for assessing the candidates of that discipline in the first part of the process, which is based on bibliometrics. In this work, we aim at exploring whether citation-based metrics, calculated only considering open bibliographic and citation data, can support the human peer-review of NDs and yield insights on how it is conducted. To understand if and what citation-based (and, possibly, other) metrics provide relevant information, we created a series of machine learning models to replicate the decisions of the NSQ committees. As one of the main outcomes of our study, we noticed that the strength of the citational relationship between the candidate and the commission in charge of assessing his/her CV seems to play a role in the peer-review phase of the NSQ of NDs.

The role of citation networks to explain academic promotions: an empirical analysis of the Italian national scientific qualification

The aim of this paper is to study the role of citation network measures in the assessment of scientific maturity. Referring to the case of the Italian national scientific qualification (ASN), we investigate if there is a relationship between citation network indices and the results of the researchers’ evaluation procedures. In particular, we want to understand if network measures can enhance the prediction accuracy of the results of the evaluation procedures beyond basic performance indices. Moreover, we want to highlight which citation network indices prove to be more relevant in explaining the ASN results, and if quantitative indices used in the citation-based disciplines assessment can replace the citation network measures in non-citation-based disciplines. Data concerning Statistics and Computer Science disciplines are collected from different sources (ASN, Italian Ministry of University and Research, and Scopus) and processed in order to calculate the citation-based measures used in this study. Then, we apply logistic regression models to estimate the effects of network variables. We find that network measures are strongly related to the results of the ASN and significantly improve the explanatory power of the models, especially for the research fields of Statistics. Additionally, citation networks in the specific sub-disciplines are far more relevant than those in the general disciplines. Finally, results show that the citation network measures are not a substitute of the citation-based bibliometric indices.

Does the venue of scientific conferences leverage their impact? A large scale study on Computer Science conferences

Purpose

Conferences bring scientists together and provide one of the most timely means for disseminating new ideas and cutting-edge works. The importance of conferences in many scientific areas is testified by quantitative indexes. The main goal of this paper is to investigate a novel research question: is there any correlation between the impact of scientific conferences and the venue where they took place?

Design/methodology/approach

To measure the impact of conferences, the authors conducted a large scale analysis on the bibliographic data extracted from 3,838 Computer Science conference series and over 2.5 million papers spanning more than 30 years of research. To quantify the “touristicity'' of a venue, the authors exploited indexes about the attractiveness of a venue from reports of the World Economic Forum, and have extracted four country-wide and two city-wide touristic indexes, which measure the attractiveness and the touristicity of any country or city.

Findings

The authors found out that the two aspects are related, and the correlation with conference impact is stronger when considering country-wide touristic indexes, achieving a correlation value of more than 0.5 when considering the average citations, and more than 0.8 when considering the total citations. Moreover the almost linear correlation with the Tourist Service Infrastructure index attests the specific importance of tourist/accommodation facilities in a given country.

Research limitations/implications

There are two main limitations of this work: (1) the use of citations to evaluate the attractiveness of the conferences and (2) the difficulty to formally define the touristic attractiveness of a venue.

Practical implications

Starting from the results concerning the correlation between different touristicity indicators and the outcome of a conference in terms of citations, it would be possible to support conference organizers in their decisions. For instance, they could plan in advance conference venues considering the same touristicity indicators, comparing different options and selecting cities which have high scores. This will allow for rapid planning of a conference venue, encompassing the easiness of travel and the attractivity of a venue, hence increasing the potential outcomes of the conference.

Social implications

Regarding the social implications, this study will enable the possibility for municipalities and conference organizers to understand what it can be improved in a specific venue to make it more attractive. This may include better transport connections or selecting cities which show a high potential regarding the touristicity index. Regarding the willingness of a researcher to submit a paper to a specific conference, it would be unaltered, meaning that what the results show is that there is already a mental process, before submitting a paper to a conference, which considers these indicators.

Originality/value

This is the first attempt to focus on the relationship of venue characteristics to conference papers. The results open up new possibilities, such as supporting conference organizers in their organization efforts.

How self-determination of scholars outclasses shrinking public research lab budgets, supporting scientific production: a case study and R&D management implications

The main purpose of this study is to analyze how reorganization of public research organizations and shrinking public research lab budgets affect the evolution of research activity and scientific productivity. This study focuses on a case study of National Research Council of Italy (CNR), one of largest European public research organizations, to identify evolution of research activities and dynamics of scientific production from 2000 to 2019. The statistical analysis suggests that, in line with the origin of the CNR, scientific research focuses mainly on research fields of natural sciences, such as chemical, life and physical sciences, with an increasing role of scientific research in energy, engineering and mathematics. The study also shows a high intensity of collaboration of the CNR within international research networks. A key finding of this study is to show, for the first time, that although uncertain reforms and reductions of public funds, scientific productivity is growing. This novel result can be explained with self-determination of scholars as a vital determinant supporting scientific production that outclasses shrinking public research lab budgets and organizational deficiencies. The management and research policy implications of this study can be generalized to support an efficient organizational and managerial behavior, and higher scientific productivity of public research institutes in contexts of reduced public funding and market turbulence.

The Intellectual Structure of Social and Sustainable Public Procurement Research: A Co-Citation Analysis

Public procurement has recently taken into account social and sustainable factors, increasing academic literature in this field. The aim of this paper is to map the intellectual structure of social and sustainable public procurement research by means of delimiting the scientific domain’s research traditions, its disciplinary composition, and influential research topics. Given that there is a literature gap in bibliometric studies applied to this field, we conducted a co-citation analysis to identify the intellectual structure of this area of knowledge. Co-citation analyses identify networks of interconnections and, consequently, detect the most (and the least) active research areas, being a good complement to traditional literature reviews. This article contributes to science development because it is the first paper to carry out a bibliometric analysis in the field of social and sustainable public procurement, as well as the first one to conduct a co-citation analysis among public procurement research. Consequently, it is also the first article to detect which papers have become burst in this research field. The results show twelve different clusters of publications that were cited by researchers who wrote papers on social and sustainable public procurement. In other words, the sources of knowledge that scholars used as references are analysed, identifying papers that can be considered turning points, as well as those that became specially cited over a discrete period of time. Six different research trends were identified over the last decade in regard to social and sustainable public procurement research. The conclusions highlight the relevance of the findings, especially because they provide guidance to researchers when conducting literature reviews, given that the most significant journals and papers are identified.

Using altmetrics for detecting impactful research in quasi-zero-day time-windows: the case of COVID-19

On December 31st 2019, the World Health Organization China Country Office was informed of cases of pneumonia of unknown etiology detected in Wuhan City. The cause of the syndrome was a new type of coronavirus isolated on January 7th 2020 and named Severe Acute Respiratory Syndrome CoronaVirus 2 (SARS-CoV-2). SARS-CoV-2 is the cause of the coronavirus disease 2019 (COVID-19). Since January 2020 an ever increasing number of scientific works related to the new pathogen have appeared in literature. Identifying relevant research outcomes at very early stages is challenging. In this work we use COVID-19 as a use-case for investigating: (1) which tools and frameworks are mostly used for early scholarly communication; (2) to what extent altmetrics can be used to identify potential impactful research in tight (i.e. quasi-zero-day) time-windows. A literature review with rigorous eligibility criteria is performed for gathering a sample composed of scientific papers about SARS-CoV-2/COVID-19 appeared in literature in the tight time-window ranging from January 15th 2020 to February 24th 2020. This sample is used for building a knowledge graph that represents the knowledge about papers and indicators formally. This knowledge graph feeds a data analysis process which is applied for experimenting with altmetrics as impact indicators. We find moderate correlation among traditional citation count, citations on social media, and mentions on news and blogs. Additionally, correlation coefficients are not inflated by indicators associated with zero values, which are quite common at very early stages after an article has been published. This suggests there is a common intended meaning of the citational acts associated with aforementioned indicators. Then, we define a method, i.e. the Comprehensive Impact Score (CIS), that harmonises different indicators for providing a multi-dimensional impact indicator. CIS shows promising results as a tool for selecting relevant papers even in a tight time-window. Our results foster the development of automated frameworks aimed at helping the scientific community in identifying relevant work even in case of limited literature and observation time.

The Sources of Knowledge of the Economic and Social Value in Sport Industry Research: A Co-citation Analysis

The aim of this article is to map the intellectual structure of scholarship on economic and social value in the sport industry. Given that bibliometric techniques are specially appropriate for identifying the intellectual structures of a field of knowledge and complement traditional literature reviews, a co-citation bibliometric analysis has been applied. This kind of analysis identifies networks of interconnections. Therefore, we aim to detect both the most and the least active research areas in this field, as well as their sub-disciplinary composition. There is an abundance of literature on sport efficiency and economic efficiency in the sport industry, our main conclusion is the identification of a literature gap in regard to social value in sport organisations, which is expected to be a research opportunity for scholars. This is in line with the lack of standardisation in the measurement for social value in sport organisations. In fact, similar to analysis undertaken in the past few decades of other industries with contributions to stakeholders and the multi-fiduciary theory of stakeholders, both the creation of social value indicators for sport entities and the empirical analysis of social efficiency in sport institutions, are identified and outlined as future areas of research. Therefore, this bibliometric analysis will contribute to determine the future challenges that this area of research will face in the following years so as to fill the literature gap identified.