A comparison of 17 author-level bibliometric indicators for researchers in Astronomy, Environmental Science, Philosophy and Public Health in Web of Science and Google Scholar

An Automated Customizable Live Web Crawler for Curation of Comparative Pharmacokinetic Data: An Intelligent Compilation of Research-Based Comprehensive Article Repository

Data curation has significant research implications irrespective of application areas. As most curated studies rely on databases for data extraction, the availability of data resources is extremely important. Taking a perspective from pharmacology, extracted data contribute to improved drug treatment outcomes and well-being but with some challenges. Considering available pharmacology literature, it is necessary to review articles and other scientific documents carefully. A typical method of accessing articles on journal websites is through long-established searches. In addition to being labor-intensive, this conventional approach often leads to incomplete-content downloads. This paper presents a new methodology with user-friendly models to accept search keywords according to the investigators' research fields for metadata and full-text articles. To accomplish this, scientifically published records on the pharmacokinetics of drugs were extracted from several sources using our navigating tool called the Web Crawler for Pharmacokinetics (WCPK). The results of metadata extraction provided 74,867 publications for four drug classes. Full-text extractions performed with WCPK revealed that the system is highly competent, extracting over 97% of records. This model helps establish keyword-based article repositories, contributing to comprehensive databases for article curation projects. This paper also explains the procedures adopted to build the proposed customizable-live WCPK, from system design and development to deployment phases.

Types of Errors Hiding in Google Scholar Data

Google Scholar (GS) is a free tool that may be used by researchers to analyze citations; find appropriate literature; or evaluate the quality of an author or a contender for tenure, promotion, a faculty position, funding, or research grants. GS has become a major bibliographic and citation database. For assessing the literature, databases, such as PubMed, PsycINFO, Scopus, and Web of Science, can be used in place of GS because they are more reliable. The aim of this study was to examine the accuracy of citation data collected from GS and provide a comprehensive description of the errors and miscounts identified. For this purpose, 281 documents that cited 2 specific works were retrieved via Publish or Perish software (PoP) and were examined. This work studied the false-positive issue inherent in the analysis of neuroimaging data. The results revealed an unprecedented error rate, with 279 of 281 (99.3%) examined references containing at least one error. Nonacademic documents tended to contain more errors than academic publications (U=5117.0; P<.001). This viewpoint article, based on a case study examining GS data accuracy, shows that GS data not only fail to be accurate but also potentially expose researchers, who would use these data without verification, to substantial biases in their analyses and results. Further work must be conducted to assess the consequences of using GS data extracted by PoP.

Academic vs. biological age in research on academic careers: a large-scale study with implications for scientifically developing systems

Biological age is an important sociodemographic factor in studies on academic careers (research productivity, scholarly impact, and collaboration patterns). It is assumed that the academic age, or the time elapsed from the first publication, is a good proxy for biological age. In this study, we analyze the limitations of the proxy in academic career studies, using as an example the entire population of Polish academic scientists and scholars visible in the last decade in global science and holding at least a PhD (N = 20,569). The proxy works well for science, technology, engineering, mathematics, and medicine (STEMM) disciplines; however, for non-STEMM disciplines (particularly for humanities and social sciences), it has a dramatically worse performance. This negative conclusion is particularly important for systems that have only recently visible in global academic journals. The micro-level data suggest a delayed participation of social scientists and humanists in global science networks, with practical implications for predicting biological age from academic age. We calculate correlation coefficients, present contingency analysis of academic career stages with academic positions and age groups, and create a linear multivariate regression model. Our research suggests that in scientifically developing countries, academic age as a proxy for biological age should be used more cautiously than in advanced countries: ideally, it should be used only for STEMM disciplines.

Four challenges when conducting bibliometric reviews and how to deal with them

The evidence base in environmental sciences is increasing steadily. Environmental researchers have been challenged to handle massive volumes of data to support more comprehensive studies, assess the current status of science, and move research towards future progress. Bibliometrics can provide important insights into the research directions by providing summarized information for several end users. Here, we present an in-depth discussion on the use of bibliometric indicators to evaluate research outputs through four case studies comprising disciplines in environmental sciences. We discuss four big challenges researchers may face when conducting bibliometric reviews and how to deal with them. We also address some primary questions researchers may answer with bibliometric mapping, drawing lessons from the case studies. Lastly, we clarify some misuses of review concepts and suggest methodological principles of systematic reviews and maps to improve the overall quality of bibliometric studies.

Research productivity of radiation therapy physics faculty in the United States

Purpose

Research productivity metrics are important for decisions regarding hiring, retention, and promotion in academic medicine, and these metrics can vary widely among different disciplines. This article examines productivity metrics for radiation therapy physicists (RTP) in the United States.

Methods and materials

Database searches were performed for RTP faculty at US institutions that have RTP residencies accredited by the Commission on Accreditation of Medical Physics Education Programs (CAMPEP). Demographics, academic rank, number of publications, academic career length, Hirsch index (h‐index), m‐quotient, and history of National Institutes of Health (NIH) funding as a principal investigator (PI) were collected for each RTP. Logistic regression was performed to determine the probability of academic rank as a function of h‐index and m‐quotient. Statistical tests used included the Wilcoxon ranked sum test and the Pearson χ² test.

Results

A total of 1038 faculty and staff were identified at 78 institutions with CAMPEP‐accredited residencies. The average RTP academic career duration is 13.5 years, with 46.7 total publications, h‐index of 10.7, and m‐quotient of 0.66. Additionally, 10.5% of RTP have a history of NIH funding as a PI. Large disparities were found in academic productivity of doctoral‐prepared physicists compared to those with a terminal master's degree. For differences in junior and senior faculty, statistical tests yielded significance in career duration, number of publications, h‐index, and m‐quotient. Gender disparities were identified in the overall distribution of RTP consistent with the membership of the American Association of Physicists in Medicine. Further gender disparities were found in the number of doctoral‐prepared RTP and physicists in senior faculty roles.

Conclusions

This manuscript provides objective benchmark data regarding research productivity of academic RTP. These data may be of interest to faculty preparing for promotion, and also to institutional leadership.

Requiem for impact factors and high publication charges

Journal impact factors, publication charges and assessment of quality and accuracy of scientific research are critical for researchers, managers, funders, policy makers, and society. Editors and publishers compete for impact factor rankings, to demonstrate how important their journals are, and researchers strive to publish in perceived top journals, despite high publication and access charges. This raises questions of how top journals are identified, whether assessments of impacts are accurate and whether high publication charges borne by the research community are justified, bearing in mind that they also collectively provide free peer-review to the publishers. Although traditional journals accelerated peer review and publication during the COVID-19 pandemic, preprint servers made a greater impact with over 30,000 open access articles becoming available and accelerating a trend already seen in other fields of research. We review and comment on the advantages and disadvantages of a range of assessment methods and the way in which they are used by researchers, managers, employers and publishers. We argue that new approaches to assessment are required to provide a realistic and comprehensive measure of the value of research and journals and we support open access publishing at a modest, affordable price to benefit research producers and consumers.

Scientific Production and Productivity for Characterizing an Author’s Publication History: Simple and Nested Gini’s and Hirsch’s Indexes Combined

In this study, I developed operational versions of Gini’s and Hirsch’s indexes that can be applied to characterize each researcher’s publication history (PH) as heterodox, orthodox, and interdisciplinary. In addition, the new indicators warn against anomalies that potentially arise from tactical or opportunistic citation and publication behaviors by authors and editors, and can be calculated from readily available information. I split the original Hirsch index into nested indexes to isolate networking activity, as well as to distinguish scientific production (number of articles) from scientific productivity (rate of production), and used nested Gini indexes to identify intentional and successful intertopical and interdisciplinary research. I applied the most popular standardizations (i.e., per author and per year), and used simple methodologies (i.e., least-squares linear and cubic fitting, whole-career vs. subperiods, two-dimensional graphs). I provide three representative numerical examples based on an orthodox multidisciplinary PH, a heterodox PH from the social sciences, and an orthodox unidisciplinary PH from the physical sciences. Two additional numerical examples based on PHs from the life and health sciences show that the suggested PH characterization can be applied to different disciplines where different publication and citation practices prevail. Software is provided to help readers explore the use of these indicators.

Researcher and Author Impact Metrics: Variety, Value, and Context

Numerous quantitative indicators are currently available for evaluating research productivity. No single metric is suitable for comprehensive evaluation of the author-level impact. The choice of particular metrics depends on the purpose and context of the evaluation. The aim of this article is to overview some of the widely employed author impact metrics and highlight perspectives of their optimal use. The h-index is one of the most popular metrics for research evaluation, which is easy to calculate and understandable for non-experts. It is automatically displayed on researcher and author profiles on citation databases such as Scopus and Web of Science. Its main advantage relates to the combined approach to the quantification of publication and citation counts. This index is increasingly cited globally. Being an appropriate indicator of publication and citation activity of highly productive and successfully promoted authors, the h-index has been criticized primarily for disadvantaging early career researchers and authors with a few indexed publications. Numerous variants of the index have been proposed to overcome its limitations. Alternative metrics have also emerged to highlight 'societal impact.' However, each of these traditional and alternative metrics has its own drawbacks, necessitating careful analyses of the context of social attention and value of publication and citation sets. Perspectives of the optimal use of researcher and author metrics is dependent on evaluation purposes and compounded by information sourced from various global, national, and specialist bibliographic databases.

Aberration of the Citation

Multiple inherent biases related to different citation practices (for e.g., self-citations, negative citations, wrong citations, multi-authorship-biased citations, honorary citations, circumstantial citations, discriminatory citations, selective and arbitrary citations, etc.) make citation-based bibliometrics strongly flawed and defective measures. A paper can be highly cited for a while (for e.g., under circumstantial or transitional knowledge), but years later it may appear that its findings, paradigms, or theories were untrue or invalid anymore. By contrast, a paper may remain shelved or overlooked for years or decades, but new studies or discoveries may actualize its subject at any moment. As citation-based metrics are transformed into "commercial activities," the "citation credit" should be considered on a commercial basis too, in the sense that "citation credit" should be shared out as a "citation dividend" by shareholders (coauthors) averagely or proportionally to their contributions but not fully appropriated by each of them. At equal numbers of citations, the greater number of authors, the lower "citation credit" should be and vice versa. Overlooking the presence of distorted and subjective citation practices makes many people and administrators "obsessed" with the number of citations to such an extent to run after "highly cited" authors and to create specialized citation databases for commercial purposes. Citation-based bibliometrics, however, are unreliable and unscientific measures; citation counts do not mean that a more cited work is of a higher quality or accuracy than a less cited work because citations do not measure the quality or accuracy. Citations do not mean that a highly cited author or journal is more commendable than a less cited author or journal. Citations are not more than countable numbers: no more, no less.

Multiple Citation Indicators and Their Composite across Scientific Disciplines

Many fields face an increasing prevalence of multi-authorship, and this poses challenges in assessing citation metrics. Here, we explore multiple citation indicators that address total impact (number of citations, Hirsch H index [H]), co-authorship adjustment (Schreiber Hm index [Hm]), and author order (total citations to papers as single; single or first; or single, first, or last author). We demonstrate the correlation patterns between these indicators across 84,116 scientists (those among the top 30,000 for impact in a single year [2013] in at least one of these indicators) and separately across 12 scientific fields. Correlation patterns vary across these 12 fields. In physics, total citations are highly negatively correlated with indicators of co-authorship adjustment and of author order, while in other sciences the negative correlation is seen only for total citation impact and citations to papers as single author. We propose a composite score that sums standardized values of these six log-transformed indicators. Of the 1,000 top-ranked scientists with the composite score, only 322 are in the top 1,000 based on total citations. Many Nobel laureates and other extremely influential scientists rank among the top-1,000 with the composite indicator, but would rank much lower based on total citations. Conversely, many of the top 1,000 authors on total citations have had no single/first/last-authored cited paper. More Nobel laureates of 2011–2015 are among the top authors when authors are ranked by the composite score than by total citations, H index, or Hm index; 40/47 of these laureates are among the top 30,000 by at least one of the six indicators. We also explore the sensitivity of indicators to self-citation and alphabetic ordering of authors in papers across different scientific fields. Multiple indicators and their composite may give a more comprehensive picture of impact, although no citation indicator, single or composite, can be expected to select all the best scientists.

Citation indicators addressing total impact, co-authorship, and author positions offer complementary insights about impact. This article shows that a composite score including six citation indicators identifies extremely influential scientists better than single indicators.

Multiple citation indicators are used in science and scientific evaluation. With an increasing proportion of papers co-authored by many researchers, it is important to account for the relative contributions of different co-authors. We explored multiple citation indicators that address total impact, co-authorship adjustment, and author order (in particular, single, first, or last position authorships, since these positions suggest pivotal contributions to the work). We evaluated the top 30,000 scientists in 2013 based on each of six citation indicators (84,116 total scientists assessed) and also developed a composite score that combines the six indicators. Different scientists populated the top ranks when different indicators were used. Many Nobel laureates and other influential scientists rank among the top 1,000 with the composite indicator, but rank much lower based on total citations. Conversely, many of the top 1,000 authors on total citations had no single/first/last-authored cited paper. More Nobel laureates are among the top authors when authors are ranked by the composite score than by single indicators. Multiple indicators and their composite give a more comprehensive picture of impact, although no method can pick all the best scientists.