Hirsch-Type Index Values for Organic Chemistry Journals: A Comparison of New Metrics with the Journal Impact Factor

Evaluating Journal Impact Factor: a systematic survey of the pros and cons, and overview of alternative measures

Background

Journal Impact Factor (JIF) has several intrinsic flaws, which highlight its inability to adequately measure citation distributions or indicate journal quality. Despite these flaws, JIF is still widely used within the academic community, resulting in the propagation of potentially misleading information. A critical review of the usefulness of JIF is needed including an overview of the literature to identify viable alternative metrics. The objectives of this study are: (1) to assess the usefulness of JIF by compiling and comparing its advantages and disadvantages; (2) to record the differential uses of JIF within research environments; and (3) to summarize and compare viable alternative measures to JIF.

Methods

Three separate literature search strategies using MEDLINE and Web of Science were completed to address the three study objectives. Each search was completed in accordance with PRISMA guidelines. Results were compiled in tabular format and analyzed based on reporting frequency.

Results

For objective (1), 84 studies were included in qualitative analysis. It was found that the recorded advantages of JIF were outweighed by disadvantages (18 disadvantages vs. 9 advantages). For objective (2), 653 records were included in a qualitative analysis. JIF was found to be most commonly used in journal ranking (n = 653, 100%) and calculation of scientific research productivity (n = 367, 56.2%). For objective (3), 65 works were included in qualitative analysis. These articles revealed 45 alternatives, which includes 18 alternatives that improve on highly reported disadvantages of JIF.

Conclusion

JIF has many disadvantages and is applied beyond its original intent, leading to inaccurate information. Several metrics have been identified to improve on certain disadvantages of JIF. Integrated Impact Indicator (I3) shows great promise as an alternative to JIF. However, further scientometric analysis is needed to assess its properties.

A theoretical model of the relationship between the h-index and other simple citation indicators

Of the existing theoretical formulas for the h-index, those recently suggested by Burrell (J Informetr 7:774-783, 2013b) and by Bertoli-Barsotti and Lando (J Informetr 9(4):762-776, 2015) have proved very effective in estimating the actual value of the h-index Hirsch (Proc Natl Acad Sci USA 102:16569-16572, 2005), at least at the level of the individual scientist. These approaches lead (or may lead) to two slightly different formulas, being based, respectively, on a "standard" and a "shifted" version of the geometric distribution. In this paper, we review the genesis of these two formulas-which we shall call the "basic" and "improved" Lambert-W formula for the h-index-and compare their effectiveness with that of a number of instances taken from the well-known Glänzel-Schubert class of models for the h-index (based, instead, on a Paretian model) by means of an empirical study. All the formulas considered in the comparison are "ready-to-use", i.e., functions of simple citation indicators such as: the total number of publications; the total number of citations; the total number of cited paper; the number of citations of the most cited paper. The empirical study is based on citation data obtained from two different sets of journals belonging to two different scientific fields: more specifically, 231 journals from the area of "Statistics and Mathematical Methods" and 100 journals from the area of "Economics, Econometrics and Finance", totaling almost 100,000 and 20,000 publications, respectively. The citation data refer to different publication/citation time windows, different types of "citable" documents, and alternative approaches to the analysis of the citation process ("prospective" and "retrospective"). We conclude that, especially in its improved version, the Lambert-W formula for the h-index provides a quite robust and effective ready-to-use rule that should be preferred to other known formulas if one's goal is (simply) to derive a reliable estimate of the h-index.

A hybrid indicator for journal ranking

Purpose

– Although journal rankings are important for authors, readers, publishers, promotion, and tenure committees, it has been argued that the use of different measures (e.g. the journal impact factor (JIF), and Hirsch’s h-index) often lead to different journal rankings, which render it difficult to make an appropriate decision. A hybrid ranking method based on the Borda count approach, the Standardized Average Index (SA index), was introduced to solve this problem. The paper aims to discuss these issues.

Design/methodology/approach

– Citations received by the articles published in 85 Health Care Sciences and Services (HCSS) journals in the period of 2009-2013 were analyzed with the use of the JIF, the h-index, and the SA index.

Findings

– The SA index exhibits a high correlation with the JIF and the h-index (γ > 0.9, p < 0.01) and yields results with higher accuracy than the h-index. The new, comprehensive citation impact analysis of the 85 HCSS journals shows that the SA index can help researchers to find journals with both high JIFs and high h-indices more easily, thereby harvesting references for paper submissions and research directions.

Originality/value

– The contribution of this study is the application of the Borda count approach to combine the HCSS journal rankings produced by the two widely accepted indices of the JIF and the h-index. The new HCSS journal rankings can be used by publishers, journal editors, researchers, policymakers, librarians, and practitioners as a reference for journal selection and the establishment of decisions and professional judgment.

Food for thought: comparison of citations received from articles appearing in specialized eating disorder journals versus general psychiatry journals

OBJECTIVE

To conduct a bibliometric analysis of eating disorder journals to guide journal readers and researchers when submitting their manuscripts.

METHOD

Several indices were used to compare journal impact and citations of articles appearing between 1996 and 2010 in six eating disorders journals and six leading general psychiatry journals.

RESULTS

The International Journal of Eating Disorders (IJED) had the highest journal impact factor (JIF, 2.278) of the six eating disorders' journals. The general psychiatry journals had higher JIFs and received more citations per eating disorder article than the specialized journals. However, IJED published the highest number of eating disorder articles between 1996 and 2010, and 35 of these articles received at least 100 citations.

DISCUSSION

Using the JIF alone to decide where to submit a manuscript is a poor strategy, as this does not take into consideration the impact an article can have within the eating disorder's field over time.

Psychiatry and the Hirsch h-index: The relationship between journal impact factors and accrued citations

There is considerable debate on the use and abuse of journal impact factors and on selecting the most appropriate indicator to assess research outcome for an individual or group of scientists. Internet searches using Web of Science and Scopus were conducted to retrieve citation data for an individual in order to calculate nine variants of Hirsch's h-index. Citations to articles published in a wide range of psychiatric journals in the periods 1995-99 and 2000-05 were analyzed using Web of Science. Comparisons were made between journal impact factor, h-index of citations from publication to 2008, and the proportion of articles cited at least 30 or 50 times. For up to 14 years post-publication, there was a strong positive relationship between journal impact factor and h-index for citations received. Journal impact factor was also compared to the percentage of articles cited at least 30 or 50 times-a comparison that showed wide variations between journals with similar impact factors. This study found that 40%-50% of the articles published in the top ten psychiatry journals ranked by impact factor acquire 30 to 50 citations within ten to fifteen years. Despite certain flaws and weaknesses, the h-index provides a better way to assess long-term performance of articles or authors than using a journal's impact factor, and it provides an alternative way to assess a journal's long-term ranking.

[New bibliometric indicators. Is this the end of the impact factor era?]

More and more bibliometric indicators emerge beside the impact factor, and a few of them are promising performance indicators. The aim of the study is to show briefly, but clearly the newest bibliometric measure numbers - like the recent enhancements of Journal Citation Report Web, Hirsch-index, Eigenfactor and SCImago Journal & Country Rank algorithms - focusing on the ranking of oncological journals, and to demonstrate the similarities and differences of the results with some examples. There are unified and well-structured web pages behind the newly appeared indicators, and some of the new numbers are presented in the well-known databases (Web of Science, Scopus) as scientific measures. The ranking of the compared oncological journals based on the different indicators show more similarities than differences, but more in-depth studies are required to find out how the results converge on any scientific area. There are some other methods, such as usage factor of the journals, beyond the indicators based on the citations. But the main point is that any ranking system should be a valid and correct representation of the scientific quality.