Citation inequality and the Journal Impact Factor: median, mean, (does it) matter?

Determinants of impact factor and Eigenfactor score in otolaryngology journals

Objective

This study aims to identify determinants of high impact, measured by Impact Factor (IF) and Eigenfactor score, among otolaryngology journals.

Methods

Bibliometric data of "otorhinolaryngology" journals were collected from the Journal Citation Reports (JCR) database. For the years 2009-2020, we collected normalized Eigenfactor score, 5-year IF, immediacy index, fraction of IF from journal-self citation, proportion and magnitude of published citable articles, and total citation counts. High-IF and -Eigenfactor journals were considered those within the top-quartile of that metric each respective year.

Results

High-IF and -Eigenfactor otolaryngology journals displayed higher 5-year IFs, immediacy indexes, and IF without self-citation (p < .05 for all years) including total citations counts and citable articles when ranked by Eigenfactor (p < .05 for all years). Otolaryngology IF correlated with 5-year IF and immediacy index within the same year (p < .05 for all years) and from previous years (p < .05 for all years; p < .05 for 2017-2018; p > .05 for 2009-2016). Eigenfactor correlated with 5-year IF, total citation counts, and citable articles within the same year (p < .05 for all years) and previous years (p < .05 for 2013-2018). Multilinear regression revealed that 5-year IF (p < .05 for 2009-2018) and immediacy index from the prior 2 years (p < .05 for 2017-2018; p > .05 for 2009-2016) predicted 2019 IF. Similarly, 5-year IF, total citation counts, and citable articles (p < .05 for 2013-2018) predicted 2019 Eigenfactor score.

Conclusion

Sustained publication of impactful articles is the dominant driver of high IF and Eigenfactor score. Eigenfactor score reflects a unique evaluation of otolaryngology journals; ranking otolaryngology journals by their Eigenfactor scores significantly alters journal ranking compared to ranking by IF.

Level of evidence

NA.

Correlating article citedness and journal impact: an empirical investigation by field on a large-scale dataset

In spite of previous research demonstrating the risks involved, and counsel against the practice as early as 1997, some research evaluations continue to use journal impact alone as a surrogate of the number of citations of hosted articles to assess the latter’s impact. Such usage is also taken up by research administrators and policy-makers, with very serious implications. The aim of this work is to investigate the correlation between the citedness of a publication and the impact of the host journal. We extend the analyses of previous literature to all STEM fields. Then we also aim to assess whether this correlation varies across fields and is stronger for highly cited authors than for lowly cited ones. Our dataset consists of a total of almost one million authorships of 2010–2019 publications authored by about 28,000 professors in 230 research fields. Results show a low correlation between the two indicators, more so for lowly cited authors as compared to highly cited ones, although differences occur across fields.

Distribution of Article Citation Frequency, Citation Skew, and Impact Factor in Otolaryngology Journals

The objectives of this study were to determine the degree of citation skew in otolaryngology journals and how article citation distribution affects their impact factors (IFs). Forty-one "otorhinolaryngology" journals in the Clarivate Journal Citation Reports database were identified and their article citation data from 2017 to 2020 extracted. Article citation frequency was remarkably similar across most journals, consistent with the narrow distribution of IFs (mean [SD], 1.9 [0.9]). Although the percentage of a journal's articles cited more than its IF during the IF citation window (mean [SD] of 32.4% [7.9%] of journals' articles)-reflecting citation skew-was not correlated with IF, the percentage of a journal's articles that were cited at least once (mean [SD] of 62.5% [15.3%] of articles) was highly correlated (ρ = 0.92, P < .001) with its IF. Although citation skew exists, otolaryngology journals' overall portfolios of published works-not a small number of highly cited articles causing citation skew-likely predominate their IFs.

Faculty appointment and promotion in Taiwan's medical schools, a systematic analysis

BACKGROUND

A rigorous faculty appointment and promotion (FAP) system is vital for the success of any academic institution. However, studies examining the FAP system in Asian universities are lacking. We surveyed the FAP policies of Taiwan's medical schools and identified an overreliance on the CJA score (manuscript Category, Journal quality, and Author order). The potential shortcomings of this metric and recommendations for refinement were discussed.

METHODS

We obtained the FAP documents from all 12 medical schools in Taiwan, and analyzed their use of traditional versus non-traditional criteria for FAP according to a published methodology. The influence of the journal impact factor (JIF) on the FAP process was quantified by comparing its relative weight between papers with two extreme JIFs. To better understand the research impact and international standing of each school, we utilized the public bibliographic database to rank universities by the number of papers, and the proportions of papers within the top 10% or 50% citation.

RESULTS

Compared with other countries, Taiwan's medical schools focus more on the quantifiable quality of the research, mostly using a "CJA" score that integrates the category, JIF or ranking, and authorship of a paper, with the JIF being the most influential factor. The CJA score for an article with a JIF of 20 can be up to three times the threshold for promotion to Assistant Professor. The emphasis on JIF is based on a presumed correlation between JIF and citation counts. However, our analysis shows that Taiwan's medical schools have lower-than-average citation counts despite a competitive rank in the number of publications.

CONCLUSIONS

The JIF plays an unrivaled role in determining the outcome of FAP in Taiwan's medical schools, mostly via the CJA system. The questionable effectiveness of the current system in elevating the international standing of Taiwan's higher-education institutions calls for a re-examination of the FAP system. We recommend a reduction in the relative importance of CJA score in the FAP system, adopting more rigorous metrics such as the h-index for evaluating research quality, and supporting more research aimed at improving the FAP system.

Web of Science's Citation Median Metrics Overcome the Major Constraints of the Journal Impact Factor

There are many metrics to evaluate the performance and status of journals. Among these, the journal impact factor (JIF) has become the dominant metric. The influence of JIF is illustrated by its widespread use to evaluate academic status, compensation, and funding decisions. However, as noted by Clarivate Analytics, the parent company of the Web of Science (WoS), the JIF should not be used without careful attention to the many phenomena that influence citation rates. To facilitate transparency, Clarivate Analytics provides all data used to determine the JIF. In addition, WoS provides other metrics for journal evaluation, including the article citation median and the review citation median. These metrics are represented as medians to minimize the confounding influence of a small number of highly cited articles that may occur when data are represented as means. Another feature of these WoS metrics is that data are separated according to different publication types of article (original research and review). To systematically compare these selected metrics, we used the data provided on the WoS web site to analyze 25 top ranked cardiovascular journals in the same mode as represented in the WoS citation distribution window. The results indicate that the article citation median and review citation median overcome several concerns that have been raised about the JIF and seem to provide enhanced objectivity as an indicator of journal impact in publishing original research and reviews. Therefore, we advocate that these additional WoS metrics might be preferentially considered as indicators of journal performance.

The weighted impact factor: the paper evaluation index based on the citation ratio

Purpose

This study distinguishes the academic influence of different papers published in journals of the same subject or field based on the modification of the journal impact factor.

Design/methodology/approach

Taking SSCI journals in library and information science (LIS) as the research object, the authors first explore the skewness degree of the citation distribution of journal articles. Then, we define the paper citation ratio as the weight of impact factor to modify the journal impact factor for the evaluation of papers, namely the weighted impact factor. The authors further explore the feasibility of the weighted impact factor in evaluating papers.

Findings

The research results show that different types of skewness exist in the citation distribution of journal papers. Particularly, 94% of journal paper citations are highly skewed, while the rest are moderately skewed. The weighted impact factor has a closer correlation with the citation frequency of papers than the journal impact factor. It resolves the issue that the journal impact factor tends to exaggerate the influence of low-cited papers in journals with high impact factors or weaken the influence of high-cited papers in journals with low impact factors.

Originality/value

The weighted impact factor is constructed based on the skewness of the citation distribution of journal articles. It provides a new method to distinguish the academic influence of different papers published in journals of the same subject or field, then avoids the situation that papers published in the same journal having the same academic impact.

Mapping environmental impact assessment research landscapes in the Arab world using visualization and bibliometric techniques

Interests and concerns on environmental issues have attracted much attention over the past few decades. This is in harmony with the growing understanding of environmental impacts associated with human activities and their role in degrading ecosystems. In line with these concerns, considerable advances in science and technology to assess, mitigate, or lessen these adverse impacts have emerged (i.e., environmental impact assessment (EIA) methodologies). The involvement of EIA in sustainable development has become a prevalent topic in research in either developed and developing countries. The present work investigated the research status, development trends, and hotspots of EIA in a region with massive environmental challenges; the Arab world. Bibliometric analysis and visualization mapping were utilized with an objective of revealing and evaluating the developments in knowledge on EIA from the Arab world. A sum of 595 documents was the productivity of the Arab world on EIA (2.1% of total global productivity). Most of the studies were performed by scholars in Egypt (143 documents; 24.0%), followed by Saudi Arabia (96 documents; 16.1%), and Tunisia (68 documents; 11.4%). France, the USA, and the UK were, respectively, the most collaborated countries with the Arab world on EIA. Most of the publications on EIA were in prestigious journals in relation to environmental sciences. King Abdulaziz University, Saudi Arabia and University of Kuwait were the most productive institutions (24 documents/institution). Topics in relation to assessing different environmental impacts on the quality and quantity of water will continue to be vital themes of research. While, the utilization of remote sensing, geographic information systems, risk assessment, life cycle assessment, bioaccumulation, and biomarkers techniques in assessing environmental impacts will continue to be dominant as efficient tools in conducting EIA related research. The outcomes displayed, in general, a rapidly and steadily rising interests on EIA. However, the development of regional experience, increasing of funds and advancing of competencies will further promote research activities on EIA.