Normalisation of citation impact in economics

Bibliometric analysis of real-time PCR-based pathogen detection in plant protection research: a comprehensive study

Introduction

The discovery of RT-PCR-based pathogen detection and gene expression analysis has had a transformative impact on the field of plant protection. This study aims to analyze the global research conducted between 2001 and 2021, focusing on the utilization of RT-PCR techniques for diagnostic assays and gene expression level studies. By retrieving data from the 'Dimensions' database and employing bibliometric visualization software, this analysis provides insights into the major publishing journals, institutions involved, leading journals, influential authors, most cited articles, and common keywords.

Methods

The 'Dimensions' database was utilized to retrieve relevant literature on RT-PCR-based pathogen detection. Fourteen distinct search queries were employed, and the resulting dataset was analyzed for trends in scholarly publications over time. The bibliometric visualization software facilitated the identification of major publishing journals, institutions, leading journals, influential authors, most cited articles, and common keywords. The study's search query was based on the conjunction 'AND', ensuring a comprehensive analysis of the literature.

Results

The analysis revealed a significant increase in the number of scholarly publications on RT-PCR-based pathogen detection over the years, indicating a growing interest and investment in research within the field. This finding emphasizes the importance of ongoing investigation and development, highlighting the potential for further advancements in knowledge and understanding. In terms of publishing journals, Plos One emerged as the leading journal, closely followed by BMC Genomics and Phytopathology. Among the highly cited journals were the European Journal of Plant Pathology, BMC Genomics, and Fungal Genetics and Biology. The publications with the highest number of citations and publications were associated with the United Nations and China. Furthermore, a network visualization map of co-authorship analysis provided intriguing insights into the collaborative nature of the research. Out of 2,636 authors analyzed, 50 surpassed the level threshold, suggesting active collaboration among researchers in the field.

Discussion

Overall, this bibliometric analysis demonstrates that the research on RT-PCR-based pathogen detection is thriving. However, there is a need for further strengthening using modern diagnostic tools and promoting collaboration among well-equipped laboratories. The findings underscore the significance of RT-PCR-based pathogen detection in plant protection and highlight the potential for continued advancements in this field. Continued research and collaboration are vital for enhancing knowledge, developing innovative diagnostic tools, and effectively protecting plants from pathogens.

SciSciNet: A large-scale open data lake for the science of science research

The science of science has attracted growing research interests, partly due to the increasing availability of large-scale datasets capturing the innerworkings of science. These datasets, and the numerous linkages among them, enable researchers to ask a range of fascinating questions about how science works and where innovation occurs. Yet as datasets grow, it becomes increasingly difficult to track available sources and linkages across datasets. Here we present SciSciNet, a large-scale open data lake for the science of science research, covering over 134M scientific publications and millions of external linkages to funding and public uses. We offer detailed documentation of pre-processing steps and analytical choices in constructing the data lake. We further supplement the data lake by computing frequently used measures in the literature, illustrating how researchers may contribute collectively to enriching the data lake. Overall, this data lake serves as an initial but useful resource for the field, by lowering the barrier to entry, reducing duplication of efforts in data processing and measurements, improving the robustness and replicability of empirical claims, and broadening the diversity and representation of ideas in the field.

Anchoring effects in the assessment of papers: An empirical survey of citing authors

In our study, we have empirically studied the assessment of cited papers within the framework of the anchoring-and-adjustment heuristic. We are interested in the question whether the assessment of a paper can be influenced by numerical information that act as an anchor (e.g. citation impact). We have undertaken a survey of corresponding authors with an available email address in the Web of Science database. The authors were asked to assess the quality of papers that they cited in previous papers. Some authors were assigned to three treatment groups that receive further information alongside the cited paper: citation impact information, information on the publishing journal (journal impact factor) or a numerical access code to enter the survey. The control group did not receive any further numerical information. We are interested in whether possible adjustments in the assessments can not only be produced by quality-related information (citation impact or journal impact), but also by numbers that are not related to quality, i.e. the access code. Our results show that the quality assessments of papers seem to depend on the citation impact information of single papers. The other information (anchors) such as an arbitrary number (an access code) and journal impact information did not play a (important) role in the assessments of papers. The results point to a possible anchoring bias caused by insufficient adjustment: it seems that the respondents assessed cited papers in another way when they observed paper impact values in the survey. We conclude that initiatives aiming at reducing the use of journal impact information in research evaluation either were already successful or overestimated the influence of this information.

Alphabetized co-authorship in economics reconsidered

In this article, we revisit the analysis of Laband and Tollison (Appl Econ 38(14):1649–1653, 2006) who documented that articles with two authors in alphabetical order are cited much more often than non-alphabetized papers with two authors in the American Economic Review and the American Journal of Agricultural Economics. Using more than 120,000 multi-authored articles from the Web of Science economics subject category, we demonstrate first that the alphabetization rate in economics has declined over the last decade. Second, we find no statistically significant relationship between alphabetized co-authorship and citations in economics using six different regression settings (the coefficients are very small). This result holds mostly true when accounting both for journal heterogeneity and intentionally or incidentally alphabetical ordering of authors. We find some evidence that alphabetization in case of two authos increases citations rates for very high-impact journals. Third, we show that the likelihood of non-alphabetized co-authorship increases the more authors an article has.

Do managers learn more about successful project management methods from articles in high impact factor journals?

BACKGROUND: In recent years, the need to develop performance-based measurement systems to improve project management outcomes has dramatically increased. Managers still take various risks during the course of managing projects which lead to ineffective decision making. A range of theories discuss such behaviors. These theories demonstrate that the discussion of risk embedded in non-optimal decision-making processes is based on theory rather than practical knowledge. However, various components of project management can be derived from academic best practices for decision making. OBJECTIVE: The study aims to explore whether articles in high impact journals tend to embody practical, rather than theoretical, knowledge thus closing the gap between academia and industry. The study is based on SEM and various machine learning classification methods. METHOD: The study was conducted using an NLP analysis of 1461 academic journals in the field of project management. RESULTS: Results show a significant positive relationship between the success of projects and the impact of new practical procedures. In contrast, a negative correlation was found between theories that use non-practical processes of effective project management. CONCLUSION: Managers can learn about new methods for project management from articles in high impact factor journals.

Automatic Identification of Addresses: A Systematic Literature Review

Address matching continues to play a central role at various levels, through geocoding and data integration from different sources, with a view to promote activities such as urban planning, location-based services, and the construction of databases like those used in census operations. However, the task of address matching continues to face several challenges, such as non-standard or incomplete address records or addresses written in more complex languages. In order to better understand how current limitations can be overcome, this paper conducted a systematic literature review focused on automated approaches to address matching and their evolution across time. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines were followed, resulting in a final set of 41 papers published between 2002 and 2021, the great majority of which are after 2017, with Chinese authors leading the way. The main findings revealed a consistent move from more traditional approaches to deep learning methods based on semantics, encoder-decoder architectures, and attention mechanisms, as well as the very recent adoption of hybrid approaches making an increased use of spatial constraints and entities. The adoption of evolutionary-based approaches and privacy preserving methods stand as some of the research gaps to address in future studies.

Anchoring effects in the assessment of papers: The proposal for an empirical survey of citing authors

In our planned study, we shall empirically study the assessment of cited papers within the framework of the anchoring-and-adjustment heuristic. We are interested in the question whether citation decisions are (mainly) driven by the quality of cited references. The design of our study is oriented towards the study by Teplitskiy, Duede [10]. We shall undertake a survey of corresponding authors with an available email address in the Web of Science database. The authors are asked to assess the quality of papers that they cited in previous papers. Some authors will be assigned to three treatment groups that receive further information alongside the cited paper: citation information, information on the publishing journal (journal impact factor), or a numerical access code to enter the survey. The control group will not receive any further numerical information. In the statistical analyses, we estimate how (strongly) the quality assessments of the cited papers are adjusted by the respondents to the anchor value (citation, journal, or access code). Thus, we are interested in whether possible adjustments in the assessments can not only be produced by quality-related information (citation or journal), but also by numbers that are not related to quality, i.e. the access code. The results of the study may have important implications for quality assessments of papers by researchers and the role of numbers, citations, and journal metrics in assessment processes.

Obsolescence of the literature: A study of included studies in Cochrane reviews

Ageing or obsolescence describes the process of declining use of a particular publication over time and can affect the results of a citation analyses as the length of citation window can change rankings. Obsolescence may not only vary across fields but also across subfields or sub-disciplines. The aim of this study is to determine the sub-disciplinary differences of obsolescence on a larger scale allowing for differences over time as well. The study presents the results of an analysis of 82,759 references across 53 healthcare and health policy topics. The references in this study are extracted from systematic reviews published from 2012 to 2016. The analyses of obsolescence include median citation age and mean citation age. This study finds that the median citation age and the mean citation age differ considerably across groups. For the latter indicator, an analysis of the confidence intervals confirms these differences. Using the subfield categorisation from Cochrane review groups, we found larger differences across subfields than in the citing half-lives published by Journal Citation Reports. Obsolescence is important to consider when setting the length of the citation windows. This study emphasises the vast differences across health sciences subfields. The length of the citation period is thus highly important for the results of a bibliometric evaluation or study covering fields with very varying obsolescence rates.

Article-Level Metrics

In the era of digitization and Open Access, article-level metrics are increasingly employed to distinguish influential research works and adjust research management strategies. Tagging individual articles with digital object identifiers allows exposing them to numerous channels of scholarly communication and quantifying related activities. The aim of this article was to overview currently available article-level metrics and highlight their advantages and limitations. Article views and downloads, citations, and social media metrics are increasingly employed by publishers to move away from the dominance and inappropriate use of journal metrics. Quantitative article metrics are complementary to one another and often require qualitative expert evaluations. Expert evaluations may help to avoid manipulations with indiscriminate social media activities that artificially boost altmetrics. Values of article metrics should be interpreted in view of confounders such as patterns of citation and social media activities across countries and academic disciplines.

How can citation impact in bibliometrics be normalized? A new approach combining citing-side normalization and citation percentiles

Since the 1980s, many different methods have been proposed to field-normalize citations. In this study, an approach is introduced that combines two previously introduced methods: citing-side normalization and citation percentiles. The advantage of combining two methods is that their advantages can be integrated in one solution. Based on citing-side normalization, each citation is field weighted and, therefore, contextualized in its field. The most important advantage of citing-side normalization is that it is not necessary to work with a specific field categorization scheme for the normalization procedure. The disadvantages of citing-side normalization—the calculation is complex and the numbers are elusive—can be compensated for by calculating percentiles based on weighted citations that result from citing-side normalization. On the one hand, percentiles are easy to understand: They are the percentage of papers published in the same year with a lower citation impact. On the other hand, weighted citation distributions are skewed distributions with outliers. Percentiles are well suited to assigning the position of a focal paper in such distributions of comparable papers. The new approach of calculating percentiles based on weighted citations is demonstrated in this study on the basis of a citation impact comparison between several countries.

Uncertainty and the ranking of economics journals

Journal rankings often show significant changes compared to previous rankings. This gives rise to the question of how well estimated the rank of a journal is. In this contribution, we consider uncertainty in a ranking of economics journals. We use the invariant method of Pinski and Narin to rank the journals. We propose an uncertainty measure, which is based on a bootstrap approach. The measure is the average absolute change in rank, which we see as a reasonable uncertainty measure regarding rankings. We further calculate, based on the bootstrap method, 95% confidence interval for the observed values of the invariant method. We show that ranks of the highest, as well as the lowest, ranked journals are well estimated, while there is a high degree of uncertainty regarding the rank of many mid-ranked journals. The distribution of the underlying measure is useful for identifying groups of journals that are more or less of the same quality (from the point of view of the invariant measure). The journal with the highest observed value of the invariant measure, Journal of Political Economy, has the best performance and constitutes a singleton, whereas Quarterly Journal of Economics and Econometrica form the next group (there is a slight overlap between the two with respect to confidence intervals). The journals ranked between about 190–230 form another group in which there are no major quality differences between the journals, as the confidence intervals are overlapping.

Mapping Social Impact: A Bibliometric Analysis

Social dimension is a fundamental element in the evaluation of initiatives and policies that are demanded and promoted by public and private organizations as well as society as a whole. Thus, there is a thriving interest in social impact research, especially from the point of view of its measurement and valuation. In this work, we explored the rising attention on the concept of social impact to identify salient agents in the field and categorize the conceptual structure of research. To achieve this, we used evaluative and relational techniques combining traditional bibliometric analysis using VOSviewer and a text mining analysis based on natural processing language (NLP) to search for documents with the term “social impact” in the title. The documents were extracted from the database Web of Science (WoS) for the period of 1938–2020. As a result, we mapped the concept of social impact from up to 1677 documents, providing an overview of the topics in which the concept was used (e.g., health, finance, environment and development, etc.) and the trends of research. This work seeks to serve as a roadmap that reflects not only the evolution of social impact but also future lines of research that require attention.

Where have all the working papers gone? Evidence from four major economics working paper series

Working papers or preprints have become an important part in the scientific landscape. Such papers present research before (potentially) being published in refereed journals. But is every working paper finally published in a journal? We answer this question for four major working paper series in economics. Based on linked data in RePEc and a random sample we provide an estimate of 66.5% of more than 28,000 investigated working papers that are published in a journal. About 8% are released as a book chapter. For the remaining 25.5% we find no evidence for what happened to the article.

Using archetypoid analysis to classify institutions and faculties of economics

We use archetypoid analysis as a new tool to categorize institutions and faculties of economics. The approach identifies typical characteristics of extreme (archetypal) values in a multivariate data set. Each entity under investigation is assigned relative shares of the identified archetypoid, which show the affiliation of the entity to the archetypoid. In contrast to its predecessor, the archetypal analysis, archetypoids always represent actual observed units in the data. The approach therefore allows to classify institutions in a rarely used way. While the method has been recognized in the literature, it is the first time that it is used in higher education research and as in our case for institutions and faculties of economics. Our dataset contains seven bibliometric indicators for 298 top-level institutions obtained from the RePEc database. We identify three archetypoids, which are characterized as the top-, the low- and the medium-performer. We discuss the assignment of shares of the identified archetypoids to the institutions in detail. As a sensitivity analysis we show how the classification changes when for four and five archetypoids are considered.

Bibliometrics-based decision trees (BBDTs) based on bibliometrics-based heuristics (BBHs): Visualized guidelines for the use of bibliometrics in research evaluation

Fast-and-frugal heuristics are simple strategies that base decisions on only a few predictor variables. In so doing, heuristics may not only reduce complexity but also boost the accuracy of decisions, their speed, and transparency. In this paper, bibliometrics-based decision trees (BBDTs) are introduced for research evaluation purposes. BBDTs visualize bibliometrics-based heuristics (BBHs), which are judgment strategies solely using publication and citation data. The BBDT exemplar presented in this paper can be used as guidance to find an answer on the question in which situations simple indicators such as mean citation rates are reasonable and in which situations more elaborated indicators (i.e., [sub-]field-normalized indicators) should be applied.