Effectiveness of journal ranking schemes as a tool for locating information

Machine learning misclassification networks reveal a citation advantage of interdisciplinary publications only in high-impact journals

Given a large enough volume of data and precise, meaningful categories, training a statistical model to solve a classification problem is straightforward and has become a standard application of machine learning (ML). If the categories are not precise, but rather fuzzy, as in the case of scientific disciplines, the systematic failures of ML classification can be informative about properties of the underlying categories. Here we classify a large volume of academic publications using only the abstract as information. From the publications that are classified differently by journal categories and ML categories (i.e., misclassified publications, when using the journal assignment as ground truth) we construct a network among disciplines. Analysis of these misclassifications provides insight in two topics at the core of the science of science: (1) Mapping out the interplay of disciplines. We show that this misclassification network is informative about the interplay of academic disciplines and it is similar to, but distinct from, a citation-based map of science, where nodes are scientific disciplines and an edge indicates a strong co-citation count between publications in these disciplines. (2) Analyzing the success of interdisciplinarity. By evaluating the citation patterns of publications, we show that misclassification can be linked to interdisciplinarity and, furthermore, that misclassified articles have different citation frequencies than correctly classified articles: In the highest 10 percent of journals in each discipline, these misclassified articles are on average cited more frequently, while in the rest of the journals they are cited less frequently.

Bayesian discrete lognormal regression model for genomic prediction

KEY MESSAGE

Genomic prediction models for quantitative traits assume continuous and normally distributed phenotypes. In this research, we proposed a novel Bayesian discrete lognormal regression model. Genomic selection is a powerful tool in modern breeding programs that uses genomic information to predict the performance of individuals and select those with desirable traits. It has revolutionized animal and plant breeding, as it allows breeders to identify the best candidates without labor-intensive and time-consuming phenotypic evaluations. While several statistical models have been developed, most of them have been for quantitative continuous traits and only a few for count responses. In this paper, we propose a discrete lognormal regression model in the Bayesian context, that with a Gibbs sampler to explore the corresponding posterior distribution and make the predictions. Two datasets of resistance disease is used in the wheat crop and are then evaluated against the traditional Gaussian model and a lognormal model. The results indicate the proposed model is a competitive and natural model for predicting count genomic traits.

A network-based normalized impact measure reveals successful periods of scientific discovery across discipline

The impact of a scientific publication is often measured by the number of citations it receives from the scientific community. However, citation count is susceptible to well-documented variations in citation practices across time and discipline, limiting our ability to compare different scientific achievements. Previous efforts to account for citation variations often rely on a priori discipline labels of papers, assuming that all papers in a discipline are identical in their subject matter. Here, we propose a network-based methodology to quantify the impact of an article by comparing it with locally comparable research, thereby eliminating the discipline label requirement. We show that the developed measure is not susceptible to discipline bias and follows a universal distribution for all articles published in different years, offering an unbiased indicator for impact across time and discipline. We then use the indicator to identify science-wide high impact research in the past half century and quantify its temporal production dynamics across disciplines, helping us identifying breakthroughs from diverse, smaller disciplines, such as geosciences, radiology, and optics, as opposed to citation-rich biomedical sciences. Our work provides insights into the evolution of science and paves a way for fair comparisons of the impact of diverse contributions across many fields.

Early-career factors largely determine the future impact of prominent researchers: evidence across eight scientific fields

Can we help predict the future impact of researchers using early-career factors? We analyze early-career factors of the world's 100 most prominent researchers across 8 scientific fields and identify four key drivers in researchers' initial career: working at a top 25 ranked university, publishing a paper in a top 5 ranked journal, publishing most papers in top quartile (high-impact) journals and co-authoring with other prominent researchers in their field. We find that over 95% of prominent researchers across multiple fields had at least one of these four features in the first 5 years of their career. We find that the most prominent scientists who had an early career advantage in terms of citations and h-index are more likely to have had all four features, and that this advantage persists throughout their career after 10, 15 and 20 years. Our findings show that these few early-career factors help predict researchers' impact later in their careers. Our research thus points to the need to enhance fairness and career mobility among scientists who have not had a jump start early on.

Research hotspots and future trends of hot corrosion research: a bibliometric analysis

Hot corrosion has attracted researchers due to its complexity of mechanisms leading to a critical challenge for energy efficiency advancement. Literature on hot corrosion spans a wide range of discussions in materials, including metals or non-metals and operating environmental conditions. Hence it was difficult to overshadow the current status and future trends of hot corrosion research. Here we pioneered a bibliometric analysis to identify the research hotspot and possible anticipated future direction of the hot corrosion study. The results showed that at least six research hotspots can be derived after carefully classifying hot corrosion research literature based on their discussion and key findings. Some hotspots were inactive in recent years and brought complications in research direction prediction. Nevertheless, several future trends of hot corrosion research are suggested. This study provides beneficial ideas in enlightening hot corrosion research development.

Research Status and Trends of Underwater Photosynthesis

Underwater photosynthesis is the most important metabolic activity for submerged plants since it could utilize carbon fixation to replenish lost carbohydrates and improve internal aeration by producing O2. The present study used bibliometric methods to quantify the annual number of publications related to underwater photosynthesis. CiteSpace, as a visual analytic software for the literature, was employed to analyze the distribution of the subject categories, author collaborations, institution collaborations, international (regional) collaborations, and cocitation and keyword burst. The results show the basic characteristics of the literature, the main intellectual base, and the main research powers of underwater photosynthesis. Meanwhile, this paper revealed the research hotspots and trends of this field. This study provides an objective and comprehensive analysis of underwater photosynthesis from a bibliometric perspective. It is expected to provide reference information for scholars in related fields to refine the research direction, solve specific scientific problems, and assist scholars in seeking/establishing relevant collaborations in their areas of interest.

A big picture: bibliometric study of academic publications from post-Soviet countries

The world’s largest community of scientists disintegrated following the dissolution of the Soviet Union. With extremely scarce resources and limited academic freedom as starting points, researchers in this region have been creating new knowledge; they have been building on rich scientific traditions in selected disciplines and, at times, paving new paths in non-traditional disciplines. At present, the cumulative contribution of post-Soviet countries to global research output is only three percent, indicating that these countries are not key players on the global research scene. This study uses bibliometric methods to offer novel empirical insight into the quantity and impact of academic publications; it also looks at the quality of journals in which the output is published. The findings reveal that fifteen post-Soviet countries differ considerably in terms of how much they have prioritised research, as well as the quantity, quality, and impact of their publications. The research productivity across the region has not been high and, taken together, these countries have produced publications of considerably lower quality and lower impact when viewed in the context of global research output. At the same time, researchers from post-Soviet countries tap into international collaborative networks actively, resulting in an exceptionally large proportion of publications from this region being internationally co-authored. In the historical context of Soviet research being known as one of the least collaborative globally, this finding indicates that researchers in the region are attractive to international collaborators and may be seeking such partnerships due to relatively modest research capacity at home.

Machine learning misclassification of academic publications reveals non-trivial interdependencies of scientific disciplines

Exploring the production of knowledge with quantitative methods is the foundation of scientometrics. In an application of machine learning to scientometrics, we here consider the classification problem of the mapping of academic publications to the subcategories of a multidisciplinary journal—and hence to scientific disciplines—based on the information contained in the abstract. In contrast to standard classification tasks, we are not interested in maximizing the accuracy, but rather we ask, whether the failures of an automatic classification are systematic and contain information about the system under investigation. These failures can be represented as a ’misclassification network’ inter-relating scientific disciplines. Here we show that this misclassification network (1) gives a markedly different pattern of interdependencies among scientific disciplines than common ’maps of science’, (2) reveals a statistical association between misclassification and citation frequencies, and (3) allows disciplines to be classified as ’method lenders’ and ’content explorers’, based on their in-degree out-degree asymmetry. On a more general level, in a wide range of machine learning applications misclassification networks have the potential of extracting systemic information from the failed classifications, thus allowing to visualize and quantitatively assess those aspects of a complex system, which are not machine learnable.

The Trends in Research on the Effects of Biochar on Soil

The present study used bibliometric methods to analyze the literature regarding the biochar effects on soil that are included in the Web of Science Core Collection database and quantified the annual number of publications in the field and distribution of publications. Using CiteSpace as a visual analytic software for the literature, the distribution of the subject categories, author collaborations, institution collaborations, international (regional) collaborations, and cocitation and keyword clustering were analyzed. The results showed the basic characteristics of the literature related to the effects of biochar on soil. Furthermore, the main research powers in this field were identified. Then, we recognized the main intellectual base in the domain of biochar effects on soil. Meanwhile, this paper revealed the research hotspots and trends of this field. Furthermore, focuses of future research in this field are discussed. The present study quantitatively and objectively describes the research status and trends of biochar effects on soil from the bibliometric perspective to promote in-depth research in this field and provide reference information for scholars in the relevant fields to refine their research directions, address specific scientific issues, and help scholars to seek/establish relevant collaborations in their fields of interests.

Towards a More Realistic Citation Model: The Key Role of Research Team Sizes

We propose a new citation model which builds on the existing models that explicitly or implicitly include “direct” and “indirect” (learning about a cited paper’s existence from references in another paper) citation mechanisms. Our model departs from the usual, unrealistic assumption of uniform probability of direct citation, in which initial differences in citation arise purely randomly. Instead, we demonstrate that a two-mechanism model in which the probability of direct citation is proportional to the number of authors on a paper (team size) is able to reproduce the empirical citation distributions of articles published in the field of astronomy remarkably well, and at different points in time. Interpretation of our model is that the intrinsic citation capacity, and hence the initial visibility of a paper, will be enhanced when more people are intimately familiar with some work, favoring papers from larger teams. While the intrinsic citation capacity cannot depend only on the team size, our model demonstrates that it must be to some degree correlated with it, and distributed in a similar way, i.e., having a power-law tail. Consequently, our team-size model qualitatively explains the existence of a correlation between the number of citations and the number of authors on a paper.

Recency predicts bursts in the evolution of author citations

The citations process for scientific papers has been studied extensively. But while the citations accrued by authors are the sum of the citations of their papers, translating the dynamics of citation accumulation from the paper to the author level is not trivial. Here we conduct a systematic study of the evolution of author citations, and in particular their bursty dynamics. We find empirical evidence of a correlation between the number of citations most recently accrued by an author and the number of citations they receive in the future. Using a simple model where the probability for an author to receive new citations depends only on the number of citations collected in the previous 12–24 months, we are able to reproduce both the citation and burst size distributions of authors across multiple decades.

Informed peer review for publication assessments: Are improved impact measures worth the hassle?

In this work we ask whether and to what extent applying a predictor of a publication’s impact that is better than early citations has an effect on the assessment of the research performance of individual scientists. Specifically, we measure the total impact of Italian professors in the sciences and economics over time, valuing their publications first by early citations and then by a weighted combination of early citations and the impact factor of the hosting journal. As expected, the scores and ranks of the two indicators show a very strong correlation, but significant shifts occur in many fields, mainly in economics and statistics, and mathematics and computer science. The higher the share of uncited professors in a field and the shorter the citation time window, the more recommendable is recourse to the above combination.

Describing Citations as a Function of Time

Purpose

Providing an overview of types of citation curves.

Design/methodology/approach

The terms citation curves or citation graphs are made explicit.

Findings

A framework for the study of diachronous (and synchronous) citation curves is proposed.

Research limitations

No new practical applications are given.

Practical implications

This short note about citation curves will help readers to make the optimal choice for their applications.

Originality/value

A new scheme for the meaning of the term “citation curve” is designed.

Use of the journal impact factor for assessing individual articles: Statistically flawed or not?

Most scientometricians reject the use of the journal impact factor for assessing individual articles and their authors. The well-known San Francisco Declaration on Research Assessment also strongly objects against this way of using the impact factor. Arguments against the use of the impact factor at the level of individual articles are often based on statistical considerations. The skewness of journal citation distributions typically plays a central role in these arguments. We present a theoretical analysis of statistical arguments against the use of the impact factor at the level of individual articles. Our analysis shows that these arguments do not support the conclusion that the impact factor should not be used for assessing individual articles. Using computer simulations, we demonstrate that under certain conditions the number of citations an article has received is a more accurate indicator of the value of the article than the impact factor. However, under other conditions, the impact factor is a more accurate indicator. It is important to critically discuss the dominant role of the impact factor in research evaluations, but the discussion should not be based on misplaced statistical arguments. Instead, the primary focus should be on the socio-technical implications of the use of the impact factor.

Use of the journal impact factor for assessing individual articles need not be statistically wrong

Most scientometricians reject the use of the journal impact factor for assessing individual articles and their authors. The well-known San Francisco Declaration on Research Assessment also strongly objects against this way of using the impact factor. Arguments against the use of the impact factor at the level of individual articles are often based on statistical considerations. The skewness of journal citation distributions typically plays a central role in these arguments. We present a theoretical analysis of statistical arguments against the use of the impact factor at the level of individual articles. Our analysis shows that these arguments do not support the conclusion that the impact factor should not be used for assessing individual articles. In fact, our computer simulations demonstrate the possibility that the impact factor is a more accurate indicator of the value of an article than the number of citations the article has received. It is important to critically discuss the dominant role of the impact factor in research evaluations, but the discussion should not be based on misplaced statistical arguments. Instead, the primary focus should be on the socio-technical implications of the use of the impact factor.

Incorporating Word Significance into Aspect-Level Sentiment Analysis

Aspect-level sentiment analysis has drawn growing attention in recent years, with higher performance achieved through the attention mechanism. Despite this, previous research does not consider some human psychological evidence relating to language interpretation. This results in attention being paid to less significant words especially when the aspect word is far from the relevant context word or when an important context word is found at the end of a long sentence. We design a novel model using word significance to direct attention towards the most significant words, with novelty decay and incremental interpretation factors working together as an alternative for position based models. The interpretation factor represents the maximization of the degree each new encountered word contributes to the sentiment polarity and a counter balancing stretched exponential novelty decay factor represents decaying human reaction as a sentence gets longer. Our findings support the hypothesis that the attention mechanism needs to be applied to the most significant words for sentiment interpretation and that novelty decay is applicable in aspect-level sentiment analysis with a decay factor β = 0.7 .

Large-scale analysis of micro-level citation patterns reveals nuanced selection criteria

The analysis of citations to scientific publications has become a tool that is used in the evaluation of a researcher's work; especially in the face of an ever-increasing production volume^1-6. Despite the acknowledged shortcomings of citation analysis and the ongoing debate on the meaning of citations^7,8, citations are still primarily viewed as endorsements and as indicators of the influence of the cited reference, regardless of the context of the citation. However, only recently has attention^9,10 been given to the connection between contextual information and the success of citing and cited papers, primarily because of the lack of extensive databases that cover both types of metadata. Here we address this issue by studying the usage of citations throughout the full text of 156,558 articles published by the Public Library of Science (PLoS), and by tracing their bibliometric history from among 60 million records obtained from the Web of Science. We find universal patterns of variation in the usage of citations across paper sections¹¹. Notably, we find differences in microlevel citation patterns that were dependent on the ultimate impact of the citing paper itself; publications from high-impact groups tend to cite younger references, as well as more very young and better-cited references. Our study provides a quantitative approach to addressing the long-standing issue that not all citations count the same.

Hot streaks in artistic, cultural, and scientific careers

The hot streak-loosely defined as 'winning begets more winnings'-highlights a specific period during which an individual's performance is substantially better than his or her typical performance. Although hot streaks have been widely debated in sports^1,2, gambling^3-5 and financial markets^6,7 over the past several decades, little is known about whether they apply to individual careers. Here, building on rich literature on the lifecycle of creativity^8-22, we collected large-scale career histories of individual artists, film directors and scientists, tracing the artworks, films and scientific publications they produced. We find that, across all three domains, hit works within a career show a high degree of temporal regularity, with each career being characterized by bursts of high-impact works occurring in sequence. We demonstrate that these observations can be explained by a simple hot-streak model, allowing us to probe quantitatively the hot streak phenomenon governing individual careers. We find this phenomemon to be remarkably universal across diverse domains: hot streaks are ubiquitous yet usually unique across different careers. The hot streak emerges randomly within an individual's sequence of works, is temporally localized, and is not associated with any detectable change in productivity. We show that, because works produced during hot streaks garner substantially more impact, the uncovered hot streaks fundamentally drive the collective impact of an individual, and ignoring this leads us to systematically overestimate or underestimate the future impact of a career. These results not only deepen our quantitative understanding of patterns that govern individual ingenuity and success, but also may have implications for identifying and nurturing individuals whose work will have lasting impact.

Measuring discursive influence across scholarship

Scientific and scholarly influence is multifaceted, shifts over time, and varies across disciplines. We present a dynamic topic model to credit documents with influence that shapes future discourse based on their content and contextual features. We trace discursive innovation in scholarship and identify the influence of particular articles along with their authors, affiliations, and journals. In collections of science, social science, and humanities research spanning over a century, our measure helps predict citations and reveals signals that recognize authors who make diverse contributions and whose contributions take longer to be appreciated, allowing us to compensate for bias in citation behavior.

Assessing scholarly influence is critical for understanding the collective system of scholarship and the history of academic inquiry. Influence is multifaceted, and citations reveal only part of it. Citation counts exhibit preferential attachment and follow a rigid “news cycle” that can miss sustained and indirect forms of influence. Building on dynamic topic models that track distributional shifts in discourse over time, we introduce a variant that incorporates features, such as authorship, affiliation, and publication venue, to assess how these contexts interact with content to shape future scholarship. We perform in-depth analyses on collections of physics research (500,000 abstracts; 102 years) and scholarship generally (JSTOR repository: 2 million full-text articles; 130 years). Our measure of document influence helps predict citations and shows how outcomes, such as winning a Nobel Prize or affiliation with a highly ranked institution, boost influence. Analysis of citations alongside discursive influence reveals that citations tend to credit authors who persist in their fields over time and discount credit for works that are influential over many topics or are “ahead of their time.” In this way, our measures provide a way to acknowledge diverse contributions that take longer and travel farther to achieve scholarly appreciation, enabling us to correct citation biases and enhance sensitivity to the full spectrum of scholarly impact.

Science of science

Identifying fundamental drivers of science and developing predictive models to capture its evolution are instrumental for the design of policies that can improve the scientific enterprise-for example, through enhanced career paths for scientists, better performance evaluation for organizations hosting research, discovery of novel effective funding vehicles, and even identification of promising regions along the scientific frontier. The science of science uses large-scale data on the production of science to search for universal and domain-specific patterns. Here, we review recent developments in this transdisciplinary field.

Article-level assessment of influence and translation in biomedical research

Given the vast scale of the modern scientific enterprise, it can be difficult for scientists to make judgments about the work of others through careful analysis of the entirety of the relevant literature. This has led to a reliance on metrics that are mathematically flawed and insufficiently diverse to account for the variety of ways in which investigators contribute to scientific progress. An urgent, critical first step in solving this problem is replacing the Journal Impact Factor with an article-level alternative. The Relative Citation Ratio (RCR), a metric that was designed to serve in that capacity, measures the influence of each publication on its respective area of research. RCR can serve as one component of a multifaceted metric that provides an effective data-driven supplement to expert opinion. Developing validated methods that quantify scientific progress can help to optimize the management of research investments and accelerate the acquisition of knowledge that improves human health.

Power-law citation distributions are not scale-free

We analyze time evolution of statistical distributions of citations to scientific papers published in the same year. While these distributions seem to follow the power-law dependence we find that they are nonstationary and the exponent of the power-law fit decreases with time and does not come to saturation. We attribute the nonstationarity of citation distributions to different longevity of the low-cited and highly cited papers. By measuring citation trajectories of papers we found that citation careers of the low-cited papers come to saturation after 10-15 years while those of the highly cited papers continue to increase indefinitely: The papers that exceed some citation threshold become runaways. Thus, we show that although citation distribution can look as a power-law dependence, it is not scale free and there is a hidden dynamic scale associated with the onset of runaways. We compare our measurements to our recently developed model of citation dynamics based on copying-redirection-triadic closure and find explanations to our empirical observations.

Growing complex network of citations of scientific papers: Modeling and measurements

We consider the network of citations of scientific papers and use a combination of the theoretical and experimental tools to uncover microscopic details of this network growth. Namely, we develop a stochastic model of citation dynamics based on the copying-redirection-triadic closure mechanism. In a complementary and coherent way, the model accounts both for statistics of references of scientific papers and for their citation dynamics. Originating in empirical measurements, the model is cast in such a way that it can be verified quantitatively in every aspect. Such validation is performed by measuring citation dynamics of physics papers. The measurements revealed nonlinear citation dynamics, the nonlinearity being intricately related to network topology. The nonlinearity has far-reaching consequences including nonstationary citation distributions, diverging citation trajectories of similar papers, runaways or "immortal papers" with infinite citation lifetime, etc. Thus nonlinearity in complex network growth is our most important finding. In a more specific context, our results can be a basis for quantitative probabilistic prediction of citation dynamics of individual papers and of the journal impact factor.

Relative Citation Ratio (RCR): A New Metric That Uses Citation Rates to Measure Influence at the Article Level

Despite their recognized limitations, bibliometric assessments of scientific productivity have been widely adopted. We describe here an improved method to quantify the influence of a research article by making novel use of its co-citation network to field-normalize the number of citations it has received. Article citation rates are divided by an expected citation rate that is derived from performance of articles in the same field and benchmarked to a peer comparison group. The resulting Relative Citation Ratio is article level and field independent and provides an alternative to the invalid practice of using journal impact factors to identify influential papers. To illustrate one application of our method, we analyzed 88,835 articles published between 2003 and 2010 and found that the National Institutes of Health awardees who authored those papers occupy relatively stable positions of influence across all disciplines. We demonstrate that the values generated by this method strongly correlate with the opinions of subject matter experts in biomedical research and suggest that the same approach should be generally applicable to articles published in all areas of science. A beta version of iCite, our web tool for calculating Relative Citation Ratios of articles listed in PubMed, is available at https://icite.od.nih.gov.

Multiple Citation Indicators and Their Composite across Scientific Disciplines

Many fields face an increasing prevalence of multi-authorship, and this poses challenges in assessing citation metrics. Here, we explore multiple citation indicators that address total impact (number of citations, Hirsch H index [H]), co-authorship adjustment (Schreiber Hm index [Hm]), and author order (total citations to papers as single; single or first; or single, first, or last author). We demonstrate the correlation patterns between these indicators across 84,116 scientists (those among the top 30,000 for impact in a single year [2013] in at least one of these indicators) and separately across 12 scientific fields. Correlation patterns vary across these 12 fields. In physics, total citations are highly negatively correlated with indicators of co-authorship adjustment and of author order, while in other sciences the negative correlation is seen only for total citation impact and citations to papers as single author. We propose a composite score that sums standardized values of these six log-transformed indicators. Of the 1,000 top-ranked scientists with the composite score, only 322 are in the top 1,000 based on total citations. Many Nobel laureates and other extremely influential scientists rank among the top-1,000 with the composite indicator, but would rank much lower based on total citations. Conversely, many of the top 1,000 authors on total citations have had no single/first/last-authored cited paper. More Nobel laureates of 2011–2015 are among the top authors when authors are ranked by the composite score than by total citations, H index, or Hm index; 40/47 of these laureates are among the top 30,000 by at least one of the six indicators. We also explore the sensitivity of indicators to self-citation and alphabetic ordering of authors in papers across different scientific fields. Multiple indicators and their composite may give a more comprehensive picture of impact, although no citation indicator, single or composite, can be expected to select all the best scientists.

Citation indicators addressing total impact, co-authorship, and author positions offer complementary insights about impact. This article shows that a composite score including six citation indicators identifies extremely influential scientists better than single indicators.

Multiple citation indicators are used in science and scientific evaluation. With an increasing proportion of papers co-authored by many researchers, it is important to account for the relative contributions of different co-authors. We explored multiple citation indicators that address total impact, co-authorship adjustment, and author order (in particular, single, first, or last position authorships, since these positions suggest pivotal contributions to the work). We evaluated the top 30,000 scientists in 2013 based on each of six citation indicators (84,116 total scientists assessed) and also developed a composite score that combines the six indicators. Different scientists populated the top ranks when different indicators were used. Many Nobel laureates and other influential scientists rank among the top 1,000 with the composite indicator, but rank much lower based on total citations. Conversely, many of the top 1,000 authors on total citations had no single/first/last-authored cited paper. More Nobel laureates are among the top authors when authors are ranked by the composite score than by single indicators. Multiple indicators and their composite give a more comprehensive picture of impact, although no method can pick all the best scientists.

Identifying key papers within a journal via network centrality measures

This article examines the extent to which existing network centrality measures can be used (1) as filters to identify a set of papers to start reading within a journal and (2) as article-level metrics to identify the relative importance of a paper within a journal. We represent a dataset of published papers in the Public Library of Science (PLOS) via a co-citation network and compute three established centrality metrics for each paper in the network: closeness, betweenness, and eigenvector. Our results show that the network of papers in a journal is scale-free and that eigenvector centrality (1) is an effective filter and article-level metric and (2) correlates well with citation counts within a given journal. However, closeness centrality is a poor filter because articles fit within a small range of citations. We also show that betweenness centrality is a poor filter for journals with a narrow focus and a good filter for multidisciplinary journals where communities of papers can be identified.

The Distribution of the Asymptotic Number of Citations to Sets of Publications by a Researcher or from an Academic Department Are Consistent with a Discrete Lognormal Model

How to quantify the impact of a researcher's or an institution's body of work is a matter of increasing importance to scientists, funding agencies, and hiring committees. The use of bibliometric indicators, such as the h-index or the Journal Impact Factor, have become widespread despite their known limitations. We argue that most existing bibliometric indicators are inconsistent, biased, and, worst of all, susceptible to manipulation. Here, we pursue a principled approach to the development of an indicator to quantify the scientific impact of both individual researchers and research institutions grounded on the functional form of the distribution of the asymptotic number of citations. We validate our approach using the publication records of 1,283 researchers from seven scientific and engineering disciplines and the chemistry departments at the 106 U.S. research institutions classified as "very high research activity". Our approach has three distinct advantages. First, it accurately captures the overall scientific impact of researchers at all career stages, as measured by asymptotic citation counts. Second, unlike other measures, our indicator is resistant to manipulation and rewards publication quality over quantity. Third, our approach captures the time-evolution of the scientific impact of research institutions.

Is quality and completeness of reporting of systematic reviews and meta-analyses published in high impact radiology journals associated with citation rates?

Purpose

The purpose of this study is to determine whether study quality and completeness of reporting of systematic reviews (SR) and meta-analyses (MA) published in high impact factor (IF) radiology journals is associated with citation rates.

Methods

All SR and MA published in English between Jan 2007–Dec 2011, in radiology journals with an IF >2.75, were identified on Ovid MEDLINE. The Assessing the Methodologic Quality of Systematic Reviews (AMSTAR) checklist for study quality, and the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) checklist for study completeness, was applied to each SR & MA. Each SR & MA was then searched in Google Scholar to yield a citation rate. Spearman correlation coefficients were used to assess the relationship between AMSTAR and PRISMA results with citation rate. Multivariate analyses were performed to account for the effect of journal IF and journal 5-year IF on correlation with citation rate. Values were reported as medians with interquartile range (IQR) provided.

Results

129 studies from 11 journals were included (50 SR and 79 MA). Median AMSTAR result was 8.0/11 (IQR: 5–9) and median PRISMA result was 23.0/27 (IQR: 21–25). The median citation rate for SR & MA was 0.73 citations/month post-publication (IQR: 0.40–1.17). There was a positive correlation between both AMSTAR and PRISMA results and SR & MA citation rate; ρ=0.323 (P=0.0002) and ρ=0.327 (P=0.0002) respectively. Positive correlation persisted for AMSTAR and PRISMA results after journal IF was partialed out; ρ=0.243 (P=0.006) and ρ=0.256 (P=0.004), and after journal 5-year IF was partialed out; ρ=0.235 (P=0.008) and ρ=0.243 (P=0.006) respectively.

Conclusion

There is a positive correlation between the quality and the completeness of a reported SR or MA with citation rate which persists when adjusted for journal IF and journal 5-year IF.

Power laws in citation distributions: evidence from Scopus

Modeling distributions of citations to scientific papers is crucial for understanding how science develops. However, there is a considerable empirical controversy on which statistical model fits the citation distributions best. This paper is concerned with rigorous empirical detection of power-law behaviour in the distribution of citations received by the most highly cited scientific papers. We have used a large, novel data set on citations to scientific papers published between 1998 and 2002 drawn from Scopus. The power-law model is compared with a number of alternative models using a likelihood ratio test. We have found that the power-law hypothesis is rejected for around half of the Scopus fields of science. For these fields of science, the Yule, power-law with exponential cut-off and log-normal distributions seem to fit the data better than the pure power-law model. On the other hand, when the power-law hypothesis is not rejected, it is usually empirically indistinguishable from most of the alternative models. The pure power-law model seems to be the best model only for the most highly cited papers in “Physics and Astronomy”. Overall, our results seem to support theories implying that the most highly cited scientific papers follow the Yule, power-law with exponential cut-off or log-normal distribution. Our findings suggest also that power laws in citation distributions, when present, account only for a very small fraction of the published papers (less than 1 % for most of science fields) and that the power-law scaling parameter (exponent) is substantially higher (from around 3.2 to around 4.7) than found in the older literature.

High-ranked social science journal articles can be identified from early citation information

Do citations accumulate too slowly in the social sciences to be used to assess the quality of recent articles? I investigate whether this is the case using citation data for all articles in economics and political science published in 2006 and indexed in the Web of Science. I find that citations in the first two years after publication explain more than half of the variation in cumulative citations received over a longer period. Journal impact factors improve the correlation between the predicted and actual future ranks of journal articles when using citation data from 2006 alone but the effect declines sharply thereafter. Finally, more than half of the papers in the top 20% in 2012 were already in the top 20% in the year of publication (2006).

Determinants of the citation rate of medical research publications from a developing country

Background

The number of citations received by an article is considered as an objective marker judging the importance and the quality of the research work. The present study aims to study the determinants of citations for research articles published by Sri Lankan authors.

Methods

Papers were selectively retrieved from the SciVerse Scopus® (Elsevier Properties S.A, USA) database for 10 years from 1st January 1997 to 31st December 2006, of which 50% were selected for inclusion by simple random sampling. The primary outcome measure was citation rate (defined as the number of citations during the 2 subsequent years after publication). Citation data was collected using the SciVerse Scopus® Citation Analyzer and self citations were excluded. A linear regression analysis was performed with ‘number of citations’ as the continuous dependent variable and other independent variables.

Result

The number of publications has steadily increased during the period of study. Over three quarter of papers were published in international journals. More than half of publications were research studies (55.3%), and most of the research studies were descriptive cross-sectional studies (27.1%). The mean number of citations within 2 years of publication was 1.7 and 52.1% of papers were not cited within the first two years of publication. The mean number of citations for collaborative studies (2.74) was significantly higher than that of non-collaborative studies (0.66). The mean number of citations did not significantly change depending on whether the publication had a positive result (2.08) or not (2.92) and was also not influenced by the presence (2.30) or absence (1.99) of the main study conclusion in the title of the article. In the linear regression model, the journal rank, number of authors, conducting the study abroad, being a research study or systematic review/meta-analysis and having regional and/or international collaboration all significantly increased the number of citations.

Conclusion

The journal rank, number of authors, conducting the study abroad, being a research study or systematic review/meta-analysis and having regional and/or international collaboration all significantly increased the number of citations. However, the presence of a positive result in the study did not influence the citation rate.

A proposal for a novel impact factor as an alternative to the JCR impact factor

One disadvantage of the JCR impact factor, the most commonly used assessment tool for ranking and evaluating scientific journals, is its inability in distinguishing among different shapes of citation distribution curves, leading to unfair evaluation of journals in some cases. This paper aims to put forward an alternative impact factor (IF′) that can properly reflect citation distributions. The two impact factors are linearly and positively correlated, and have roughly the same order of magnitude. Because of the ability of IF′ in distinguishing among different shapes of citation distribution curves, IF′ may properly reflect the academic performance of a scientific journal in a way that is different from the JCR impact factor with some unique features that reward journals with highly cited papers. Therefore, it is suggested that IF′ could be used to complement the JCR impact factor.