Professional and citizen bibliometrics: complementarities and ambivalences in the development and use of indicators-a state-of-the-art report

The SCOPE framework - implementing ideals of responsible research assessment

Background

Research and researchers are heavily evaluated, and over the past decade it has become widely acknowledged that the consequences of evaluating the research enterprise and particularly individual researchers are considerable. This has resulted in the publishing of several guidelines and principles to support moving towards more responsible research assessment (RRA). To ensure that research evaluation is meaningful, responsible, and effective the International Network of Research Management Societies (INORMS) Research Evaluation Group created the SCOPE framework enabling evaluators to deliver on existing principles of RRA. SCOPE bridges the gap between principles and their implementation by providing a structured five-stage framework by which evaluations can be designed and implemented, as well as evaluated.

Methods

SCOPE is a step-by-step process designed to help plan, design, and conduct research evaluations as well as check effectiveness of existing evaluations. In this article, four case studies are presented to show how SCOPE has been used in practice to provide value-based research evaluation.

Results

This article situates SCOPE within the international work towards more meaningful and robust research evaluation practices and shows through the four case studies how it can be used by different organisations to develop evaluations at different levels of granularity and in different settings.

Conclusions

The article demonstrates that the SCOPE framework is rooted firmly in the existing literature. In addition, it is argued that it does not simply translate existing principles of RRA into practice, but provides additional considerations not always addressed in existing RRA principles and practices thus playing a specific role in the delivery of RRA. Furthermore, the use cases show the value of SCOPE across a range of settings, including different institutional types, sizes, and missions.

From Research Evaluation to Research Analytics. The digitization of academic performance measurement

One could think that bibliometric measurement of academic performance has always been digital since the computer-assisted invention of the Science Citation Index. Yet, since the 2000s, the digitization of bibliometric infrastructure has accelerated at a rapid pace. Citation databases are indexing an increasing variety of publication types. Altmetric data aggregators are producing data on the reception of research outcomes. Machine-readable persistent identifiers are created to unambiguously identify researchers, research organizations, and research objects; and evaluative software tools and current research information systems are constantly enlarging their functionalities to make use of these data and extract meaning from them. In this article, we analyse how these developments in evaluative bibliometrics have contributed to an extension of indicator-based research evaluation towards data-driven research analytics. Drawing on empirical material from blogs and websites as well as from research and policy papers, we discuss how interoperability, scalability, and flexibility as material specificities of digital infrastructures generate new ways of data production and their assessment, which affect the possibilities of how academic performance can be understood and (e)valuated.

Rigor and reproducibility for data analysis and design in the study of eating disorders

OBJECTIVE

Incorporating open science practices has become a priority for submission criteria in the International Journal of Eating Disorders (IJED). In this systematic review, we used the rigor and reproducibility framework developed by Hildebrandt and Prenoveau (2020) to examine the implementation of statistically sound open science principles in IJED, determining whether the cost and effort of incorporating these practices ultimately make research more likely to be cited.

METHOD

For this systematic review, six trained coders examined 1145 articles published from January 2011 to May 2021, including the 5 years prior to the 2016 introduction of the Open Science Foundation article preregistration. We coded for the presence or absence of 10 specific open science elements and calculated citation metrics for each article.

RESULTS

There was evidence of a significant positive relationship between time and total rigor and reproducibility (Total RR) criteria included in IJED articles following the implementation of preregistration in 2016. For every increase in year from 2011 to 2016, there was a .14 decrease in Total RR criteria. From 2016 to 2021, there was a .42 increase per volume in Total RR criteria. There was no statistically significant relationship between Total RR criteria and citation impact.

DISCUSSION

Although findings indicate that statistical rigor and reproducibility in this field has increased, the lack of direct relationship between open science methods and article visibility for scientists suggests that there is a limited incentive for researchers to participate in reporting guidelines.

PUBLIC SIGNIFICANCE

Statistical controversies within science threaten the rigor and reproducibility of published research. Open science practices, including the preregistration of study hypotheses, links to statistical code, and explicit data-sharing arguably generate reliable and valid inferences. This review illustrates the rigor and reproducibility of articles published in IJED between 2011 and 2021 and identifies whether open sciences practices have become increasingly prevalent in eating disorder research.

Mapping Manuel Sandoval Vallarta (1899-1977) Scientific Contribution

This paper employs network theory, mining data and bibliometric analysis when mapping the scientific contribution of Nobel Prize candidate; Manuel Sandoval Vallarta, the first and most renowned Mexican physicist and important figure in Latin American science. Vallarta died in 1977, and the existing literature is about his life and contributions to science but not about how those are still valuable today. This paper is the first to highlight, with mapping tools, that his contributions are relevant to the international community of cosmic rays (as he was pioneer and leader), quantum mechanics and relativity. These tools delivered three findings: Identify how he built his own field of study, same as universal knowledge. Unveil that the backward and forward Vallarta citations follow a scale-free network distribution. Determine social factors that benefited or affected his scientific activities-such as World War II interrupting Vallarta's successful productivity at Massachusetts Institute of Technology. Furthermore, this study confirmed the interdisciplinary nature of the mapping studies of the scientist's contributions using scientometric tools. As a result, several interesting questions arose throughout our research, some of which were answered from the history and philosophy of science. However, others need to be analyzed by experts in the fields of Vallarta. Mapping research sends an invitation to interdisciplinary dialogue/research between experts in different areas of study to better understand the process of knowledge production both, individual and collective.

Bibliometrics in Press. Representations and uses of bibliometric indicators in the Italian daily newspapers

Scholars in science and technology studies and bibliometricians are increasingly revealing the performative nature of bibliometric indicators. Far from being neutral technical measures, indicators such as the Impact Factor and the h-index are deeply transforming the social and epistemic structures of contemporary science. At the same time, scholars have highlighted how bibliometric indicators are endowed with social meanings that go beyond their purely technical definitions. These social representations of bibliometric indicators are constructed and negotiated between different groups of actors within several arenas. This study aims to investigate how bibliometric indicators are used in a context, which, so far, has not yet been covered by researchers, that of daily newspapers. By a content analysis of a corpus of 583 articles that appeared in four major Italian newspapers between 1990 and 2020, we chronicle the main functions that bibliometrics and bibliometric indicators played in the Italian press. Our material shows, among other things, that the public discourse developed in newspapers creates a favorable environment for bibliometrics-centered science policies, that bibliometric indicators contribute to the social construction of scientific facts in the press, especially in science news related to medicine, and that professional bibliometric expertise struggles to be represented in newspapers and hence reach the general public.

What Is Quality in Research? Building a Framework of Design, Process and Impact Attributes and Evaluation Perspectives

The strategic relevance of innovation and scientific research has amplified the attention towards the definition of quality in research practice. However, despite the proliferation of evaluation metrics and procedures, there is a need to go beyond bibliometric approaches and to identify, more explicitly, what constitutes good research and which are its driving factors or determinants. This article reviews specialized research policy, science policy and scientometrics literature to extract critical dimensions associated with research quality as presented in a vast although fragmented theory background. A literature-derived framework of research quality attributes is, thus, obtained, which is subject to an expert feedback process, involving scholars and practitioners in the fields of research policy and evaluation. The results are represented by a structured taxonomy of 66 quality attributes providing a systemic definition of research quality. The attributes are aggregated into a three-dimensional framework encompassing research design (ex ante), research process (in-process) and research impact (ex post) perspectives. The main value of the study is to propose a literature-derived and comprehensive inventory of quality attributes and perspectives of evaluation. The findings can support further theoretical developments and research policy discussions on the ultimate drivers of quality and impact of scientific research. The framework can be also useful to design new exercises or procedures of research evaluation based on a multidimensional view of quality.

Impact and visibility of Norwegian, Finnish and Spanish journals in the fields of humanities

This article analyses the impact and visibility of scholarly journals in the humanities that are publishing in the national languages in Finland, Norway and Spain. Three types of publishers are considered: commercial publishers, scholarly society as publisher, and research organizations as publishers. Indicators of visibility and impact were obtained from Web of Science, SCOPUS, Google Metrics, Scimago Journal Rank and Journal Citation Report. The findings compiled show that in Spain the categories “History and Archaeology” and “Language and Literature” account for almost 70% of the journals analysed, while the other countries offer a more homogeneous distribution. In Finland, the scholarly society publisher is predominant, in Spain, research organization as publishers, mostly universities, have a greater weighting, while in Norway, the commercial publishers take centre stage. The results show that journals from Finland and Norway will have reduced possibilities in terms of impact and visibility, since the vernacular language appeals to a smaller readership. Conversely, the Spanish journals are more attractive for indexing in commercial databases. Distribution in open access ranges from 64 to 70% in Norwegian and Finish journals, and to 91% in Spanish journals. The existence of DOI range from 31 to 41% in Nordic journals to 60% in Spanish journals and has a more widespread bearing on the citations received in all three countries (journals with DOI and open access are cited more frequently).

Four challenges when conducting bibliometric reviews and how to deal with them

The evidence base in environmental sciences is increasing steadily. Environmental researchers have been challenged to handle massive volumes of data to support more comprehensive studies, assess the current status of science, and move research towards future progress. Bibliometrics can provide important insights into the research directions by providing summarized information for several end users. Here, we present an in-depth discussion on the use of bibliometric indicators to evaluate research outputs through four case studies comprising disciplines in environmental sciences. We discuss four big challenges researchers may face when conducting bibliometric reviews and how to deal with them. We also address some primary questions researchers may answer with bibliometric mapping, drawing lessons from the case studies. Lastly, we clarify some misuses of review concepts and suggest methodological principles of systematic reviews and maps to improve the overall quality of bibliometric studies.

Researchers' attitudes towards the h-index on Twitter 2007-2020: criticism and acceptance

The h-index is an indicator of the scientific impact of an academic publishing career. Its hybrid publishing/citation nature and inherent bias against younger researchers, women, people in low resourced countries, and those not prioritizing publishing arguably give it little value for most formal and informal research evaluations. Nevertheless, it is well-known by academics, used in some promotion decisions, and is prominent in bibliometric databases, such as Google Scholar. In the context of this apparent conflict, it is important to understand researchers’ attitudes towards the h-index. This article used public tweets in English to analyse how scholars discuss the h-index in public: is it mentioned, are tweets about it positive or negative, and has interest decreased since its shortcomings were exposed? The January 2021 Twitter Academic Research initiative was harnessed to download all English tweets mentioning the h-index from the 2006 start of Twitter until the end of 2020. The results showed a constantly increasing number of tweets. Whilst the most popular tweets unapologetically used the h-index as an indicator of research performance, 28.5% of tweets were critical of its simplistic nature and others joked about it (8%). The results suggest that interest in the h-index is still increasing online despite scientists willing to evaluate the h-index in public tending to be critical. Nevertheless, in limited situations it may be effective at succinctly conveying the message that a researcher has had a successful publishing career.

Obsolescence of the literature: A study of included studies in Cochrane reviews

Ageing or obsolescence describes the process of declining use of a particular publication over time and can affect the results of a citation analyses as the length of citation window can change rankings. Obsolescence may not only vary across fields but also across subfields or sub-disciplines. The aim of this study is to determine the sub-disciplinary differences of obsolescence on a larger scale allowing for differences over time as well. The study presents the results of an analysis of 82,759 references across 53 healthcare and health policy topics. The references in this study are extracted from systematic reviews published from 2012 to 2016. The analyses of obsolescence include median citation age and mean citation age. This study finds that the median citation age and the mean citation age differ considerably across groups. For the latter indicator, an analysis of the confidence intervals confirms these differences. Using the subfield categorisation from Cochrane review groups, we found larger differences across subfields than in the citing half-lives published by Journal Citation Reports. Obsolescence is important to consider when setting the length of the citation windows. This study emphasises the vast differences across health sciences subfields. The length of the citation period is thus highly important for the results of a bibliometric evaluation or study covering fields with very varying obsolescence rates.

Requiem for impact factors and high publication charges

Journal impact factors, publication charges and assessment of quality and accuracy of scientific research are critical for researchers, managers, funders, policy makers, and society. Editors and publishers compete for impact factor rankings, to demonstrate how important their journals are, and researchers strive to publish in perceived top journals, despite high publication and access charges. This raises questions of how top journals are identified, whether assessments of impacts are accurate and whether high publication charges borne by the research community are justified, bearing in mind that they also collectively provide free peer-review to the publishers. Although traditional journals accelerated peer review and publication during the COVID-19 pandemic, preprint servers made a greater impact with over 30,000 open access articles becoming available and accelerating a trend already seen in other fields of research. We review and comment on the advantages and disadvantages of a range of assessment methods and the way in which they are used by researchers, managers, employers and publishers. We argue that new approaches to assessment are required to provide a realistic and comprehensive measure of the value of research and journals and we support open access publishing at a modest, affordable price to benefit research producers and consumers.

Web of Science (WoS) and Scopus: The Titans of Bibliographic Information in Today’s Academic World

Nowadays, the importance of bibliographic databases (DBs) has increased enormously, as they are the main providers of publication metadata and bibliometric indicators universally used both for research assessment practices and for performing daily tasks. Because the reliability of these tasks firstly depends on the data source, all users of the DBs should be able to choose the most suitable one. Web of Science (WoS) and Scopus are the two main bibliographic DBs. The comprehensive evaluation of the DBs’ coverage is practically impossible without extensive bibliometric analyses or literature reviews, but most DBs users do not have bibliometric competence and/or are not willing to invest additional time for such evaluations. Apart from that, the convenience of the DB’s interface, performance, provided impact indicators and additional tools may also influence the users’ choice. The main goal of this work is to provide all of the potential users with an all-inclusive description of the two main bibliographic DBs by gathering the findings that are presented in the most recent literature and information provided by the owners of the DBs at one place. This overview should aid all stakeholders employing publication and citation data in selecting the most suitable DB.

Are University Rankings Statistically Significant? A Comparison among Chinese Universities and with the USA

Purpose

Building on Leydesdorff, Bornmann, and Mingers (2019), we elaborate the differences between Tsinghua and Zhejiang University as an empirical example. We address the question of whether differences are statistically significant in the rankings of Chinese universities. We propose methods for measuring statistical significance among different universities within or among countries.

Design/methodology/approach

Based on z-testing and overlapping confidence intervals, and using data about 205 Chinese universities included in the Leiden Rankings 2020, we argue that three main groups of Chinese research universities can be distinguished (low, middle, and high).

Findings

When the sample of 205 Chinese universities is merged with the 197 US universities included in Leiden Rankings 2020, the results similarly indicate three main groups: low, middle, and high. Using this data (Leiden Rankings and Web of Science), the z-scores of the Chinese universities are significantly below those of the US universities albeit with some overlap.

Research limitations

We show empirically that differences in ranking may be due to changes in the data, the models, or the modeling effects on the data. The scientometric groupings are not always stable when we use different methods.

Practical implications

Differences among universities can be tested for their statistical significance. The statistics relativize the values of decimals in the rankings. One can operate with a scheme of low/middle/high in policy debates and leave the more fine-grained rankings of individual universities to operational management and local settings.

Originality/value

In the discussion about the rankings of universities, the question of whether differences are statistically significant, has, in our opinion, insufficiently been addressed in research evaluations.

What is meaningful research and how should we measure it?

We discuss the trend towards using quantitative metrics for evaluating research. We claim that, rather than promoting meaningful research, purely metric-based research evaluation schemes potentially lead to a dystopian academic reality, leaving no space for creativity and intellectual initiative. After sketching what the future could look like if quantitative metrics are allowed to proliferate, we provide a more detailed discussion on why research is so difficult to evaluate and outline approaches for avoiding such a situation. In particular, we characterize meaningful research as an essentially contested concept and argue that quantitative metrics should always be accompanied by operationalized instructions for their proper use and continuously evaluated via feedback loops. Additionally, we analyze a dataset containing information about computer science publications and their citation history and indicate how quantitative metrics could potentially be calibrated via alternative evaluation methods such as test of time awards. Finally, we argue that, instead of over-relying on indicators, research environments should primarily be based on trust and personal responsibility.

Use of the journal impact factor for assessing individual articles: Statistically flawed or not?

Most scientometricians reject the use of the journal impact factor for assessing individual articles and their authors. The well-known San Francisco Declaration on Research Assessment also strongly objects against this way of using the impact factor. Arguments against the use of the impact factor at the level of individual articles are often based on statistical considerations. The skewness of journal citation distributions typically plays a central role in these arguments. We present a theoretical analysis of statistical arguments against the use of the impact factor at the level of individual articles. Our analysis shows that these arguments do not support the conclusion that the impact factor should not be used for assessing individual articles. Using computer simulations, we demonstrate that under certain conditions the number of citations an article has received is a more accurate indicator of the value of the article than the impact factor. However, under other conditions, the impact factor is a more accurate indicator. It is important to critically discuss the dominant role of the impact factor in research evaluations, but the discussion should not be based on misplaced statistical arguments. Instead, the primary focus should be on the socio-technical implications of the use of the impact factor.

Use of the journal impact factor for assessing individual articles need not be statistically wrong

Most scientometricians reject the use of the journal impact factor for assessing individual articles and their authors. The well-known San Francisco Declaration on Research Assessment also strongly objects against this way of using the impact factor. Arguments against the use of the impact factor at the level of individual articles are often based on statistical considerations. The skewness of journal citation distributions typically plays a central role in these arguments. We present a theoretical analysis of statistical arguments against the use of the impact factor at the level of individual articles. Our analysis shows that these arguments do not support the conclusion that the impact factor should not be used for assessing individual articles. In fact, our computer simulations demonstrate the possibility that the impact factor is a more accurate indicator of the value of an article than the number of citations the article has received. It is important to critically discuss the dominant role of the impact factor in research evaluations, but the discussion should not be based on misplaced statistical arguments. Instead, the primary focus should be on the socio-technical implications of the use of the impact factor.

Bibliometrics-based decision trees (BBDTs) based on bibliometrics-based heuristics (BBHs): Visualized guidelines for the use of bibliometrics in research evaluation

Fast-and-frugal heuristics are simple strategies that base decisions on only a few predictor variables. In so doing, heuristics may not only reduce complexity but also boost the accuracy of decisions, their speed, and transparency. In this paper, bibliometrics-based decision trees (BBDTs) are introduced for research evaluation purposes. BBDTs visualize bibliometrics-based heuristics (BBHs), which are judgment strategies solely using publication and citation data. The BBDT exemplar presented in this paper can be used as guidance to find an answer on the question in which situations simple indicators such as mean citation rates are reasonable and in which situations more elaborated indicators (i.e., [sub-]field-normalized indicators) should be applied.

Scientific output scales with resources. A comparison of US and European universities

By using a comprehensive dataset of US and European universities, we demonstrate super-linear scaling between university revenues and their volume of publications and (field-normalized) citations. We show that this relationship holds both in the US and in Europe. In terms of resources, our data show that three characteristics differentiate the US system: (1) a significantly higher level of resources for the entire system, (2) a clearer distinction between education-oriented institutions and doctoral universities and (3) a higher concentration of resources among doctoral universities. Accordingly, a group of US universities receive a much larger amount of resources and have a far higher number of publications and citations when compared to their European counterparts. These results demonstrate empirically that international rankings are by and large richness measures and, therefore, can be interpreted only by introducing a measure of resources. Implications for public policies and institutional evaluation are finally discussed.

Comparing the topological rank of journals in Web of Science and Mendeley

Recently, there has been a surge of interest in new data emerged due to the rapid development of the information technologies in scholarly communication. Since the 2010s, altmetrics has become a common trend in scientometric research. However, researchers have not treated in much detail the question of the probability distributions underlying these new data. The principal objective of this study was to investigate one of the classic problems of scientometrics-the problem of citation and readership distributions. The study is based on the data obtained from two information systems: Web of Science and Mendeley. Here we based on the concept of the cumulative empirical distribution function to explore the differences and similarities between citations and readership counts of biological journals indexed in Web of Science and Mendeley. The basic idea was to determine, for any journal, a "size" (it is said to be the topological rank) of citation and readership empirical cumulative distributions, and then to compare distributions of the topological ranks of Web of Science and Mendeley. In order to verify our model, we employ it to the bibliometric and altmetric research of 305 biological journals indexed in Journal Citation Reports 2015. The findings show that both distributions of the topological rank of biological journals are statistically close to the Wakeby distribution. The findings presented in this study add to our understanding of information processes of the scholarly communication in the new digital environment.

Mining university rankings: Publication output and citation impact as their basis

World university rankings have become well-established tools that students, university managers, and policy makers read and use. Each ranking claims to have a unique methodology capable of measuring the ‘quality’ of universities. The purpose of this article is to analyze to which extent these different rankings measure the same phenomenon and what it is that they are measuring. For this, we selected a total of seven world university rankings and performed a principal component analysis. After ensuring that despite their methodological differences, they all come together to a single component; we hypothesized that bibliometric indicators could explain what is being measured. Our analyses show that ranking scores from whichever of the seven league tables under study can be explained by the number of publications and citations received by the institution. We conclude by discussing policy implications and opportunities on how a nuanced and responsible use of rankings can help decision-making at the institutional level

Basic bibliometrics for dummies and others: an overview of some journal-level indicators in physical and rehabilitation medicine

This report aims to complement and update a series of papers published in the last decade on bibliometrics regarding journals related to physical and rehabilitation medicine (PRM). It targets clinicians and researchers (academic and non-academic) in our discipline who would like to use bibliometric indicators as a complementary "tool" to integrate into their expert practice of journal evaluation. Different journal-based metrics are analysed in order to provide a wide (albeit general) view of the performance of top PRM journals. First, we provide some brief preliminary remarks useful for an informed understanding of our results: 1) an update on bibliometric indicators and multidisciplinary databases of peer-reviewed literature; 2) the meaning of some bibliometric indicators; 3) the practical message related to this report: "keep it simple." Then, we profile the performance of 22 PRM core journals, according to six widely used bibliometric indicators. Indicators are grouped into three categories defined by their quartile classification (Three Star: top quartile; Two Star: second upper quartile; One Star: under the median). In conclusion, bibliometrics is just one of the key methods used for measuring the (supposed) 'impact' of scholarly publications and it represents only a raw proxy for the real impact or value of the research. This report wishes to add a small contribution for a simplified understanding of journal-level indicators in PRM, to support informed decisions on which high-level journals merit special attention by clinicians and researchers working in our discipline.

Hot and cold spots in the US research: A spatial analysis of bibliometric data on the institutional level

Spatial bibliometrics addresses the spatial aspects of scientific research activities. In this case study, we use the Getis–Ord G∗ i ( d) statistic for bibliometric data on US institutions to identify hot spots of institutions on a map publishing many high-impact papers. The study is based on a dataset with performance data (proportion and number of papers belonging to the 10% most frequently cited papers) and geo-coordinates for all institutions in the United States from the SCImago group (and Scopus). The Getis-Ord Gi* statistic returns, for each institution on a map, a z score. Higher z scores point to intense clustering of institutions, which have published a large proportion or number of highly cited papers (hot spots). The US maps, which we generate as examples in this study, point to four regions. These regions can be labelled as hot spots: around San Francisco, Los Angeles, Boston and Washington, DC. The empirical focus on institutional hot spots in a country using bibliometric data is of specific importance for science policy, because geospatial proximity is shown as an important factor for innovation processes.

Measuring Individual Performance with Comprehensive Bibliometric Reports as an Alternative to h-Index Values

The h-index is frequently used to measure the performance of single scientists in Korea (and beyond). No single indicator alone, however, is able to provide a stable and complete assessment of performance. The Stata command bibrep.ado is introduced which automatically produces bibliometric reports for single researchers (senior researchers working in the natural or life sciences). The user of the command receives a comprehensive bibliometric report which can be used in research evaluation instead of the h-index.

Which research institution performs better than average in a subject category or better than selected other institutions?

Purpose

Institutional bibliometric analyses compare as a rule the performance of different institutions. The purpose of this paper is to use a statistical approach which not only allows a comparison of the citation impact of papers from selected institutions, but also a comparison of the citation impact of the papers of these institutions with all other papers published in a particular time frame.

Design/methodology/approach

The study is based on a randomly selected cluster sample (n=4,327,013 articles and reviews from 2000 to 2004), which is drawn from a bibliometric in-house database including Web of Science data. Regression models are used to analyze citation impact scores. Subsequent to the models, average predictions at specific interesting values are calculated to analyze which factors could have an effect on the impact scores-the journal impact factor (JIF), of the journals which published the papers and the number of affiliations given in a paper.

Findings

Three anonymous German institutions are compared with one another and with the set of all other papers in the time frame. As an indicator of institutional performance, fractionally counted PPtop 50% on the level of individual papers are used. This indicator is a normalized impact score whereas each paper is fractionally assigned to the 50 percent most frequently cited papers within its subject category and publication year. The results show that the JIF and the number of affiliations have a statistically significant effect on the institutional performance.

Originality/value

Fractional regression models are introduced to analyze the fractionally counted PPtop 50% on the level of individual papers.

Count highly-cited papers instead of papers with h citations: use normalized citation counts and compare "like with like"!

Teixeira da Silva and Dobránszki (Scientometrics. 10.1007/s11192-018-2680-3, 2018) describe practical problems in using the h-index for the purpose of research evaluation. For example, they discuss the h-index differences among the bibliometric databases. In this Letter to the Editor, we argue for abstaining from using the h-index. One can use normalized indicators instead.

Plots for visualizing paper impact and journal impact of single researchers in a single graph

In research evaluation of single researchers, the assessment of paper and journal impact is of interest. High journal impact reflects the ability of researchers to convince strict reviewers, and high paper impact reflects the usefulness of papers for future research. In many bibliometric studies, metrics for journal and paper impact are separately presented. In this paper, we introduce two graph types, which combine both metrics in a single graph. The graphs can be used in research evaluation to visualize the performance of single researchers comprehensively.