How much is too much? The difference between research influence and self-citation excess

Return on investment in science: twenty years of European Commission funded research in Alzheimer's dementia, breast cancer and prostate cancer

Alzheimer's disease (AD), breast cancer (BC) and prostate cancer (PC) continue to be high in the research and innovation agenda of the European Commission (EC). This is due to their exceptionally large burden to the national health systems, the profound economic effects of opportunity costs attributable to decreased working ability, premature mortality and the ever-increasing demand for both hospital and home-based medical care. Over the last two decades, the EC has been steadily increasing both the number of proposals being funded and the amounts of financial resources being allocated to these fields of research. This trend has continued throughout four consecutive science funding cycles, namely framework programme (FP)5, FP6, FP7 and Horizon 2020 (H2020). We performed a retrospective assessment of the outputs and outcomes of EC funding in AD, BC and PC research over the 1999-2019 period by means of selected indicators. These indicators were assessed for their ability to screen the past, present and future for an array of causal relationships and long-term trends in clinical, epidemiological and public health sphere, while considering also the broader socioeconomic impact of funded research on the society at large. This analysis shows that public-private partnerships with large industry and university-based consortia have led to some of the most impactful proposals being funded over the analysed time period. New pharmaceuticals, small molecules and monoclonal antibodies alike, along with screening and prevention, have been the most prominent sources of innovation in BC and PC, extending patients' survival and enhancing their quality of life. Unlike oncology, dementia drug development has been way less successful, with only minor improvements related to the quality of supportive medical care for symptoms and more sensitive diagnostics, without any ground-breaking disease-modifying treatment(s). Significant progresses in imaging diagnostics and nanotechnology have been largely driven by the participation of medical device industry multinational companies. Clinical trials funded by the EC were conducted, leading to the development of brand-new drug molecules featuring novel mechanisms of action. Some prominent cases of breakthrough discoveries serve as evidence for the European capability to generate cutting-edge technological innovation in biomedicine. Less productive areas of research may be reconsidered as priorities when shaping the new agenda for forthcoming science funding programmes.

Utilizing a composite citation index for evaluating clinical ophthalmology research: insights into gender, nationality, and self-citation among top ophthalmology researchers

OBJECTIVES

To compare the performance of a composite citation score (c-score) and its six constituent citation indices, including H-index, in predicting winners of the Weisenfeld Award in ophthalmologic research. Secondary objectives were to explore career and demographic characteristics of the most highly cited researchers in ophthalmology.

METHODS

A publicly available database was accessed to compile a set of top researchers in the field of clinical ophthalmology and optometry based on Scopus data from 1996 to 2021. Each citation index was used to construct a multivariable model adjusted for author demographic characteristics. Using area under the receiver operating curve (AUC) analysis, each index's model was evaluated for its ability to predict winners of the Weisenfeld Award in Ophthalmology, a research distinction presented by the Association for Research in Vision and Ophthalmology (ARVO). Secondary analyses investigated authors' self-citation rates, career length, gender, and country affiliation over time.

RESULTS

Approximately one thousand unique authors publishing primarily in clinical ophthalmology/optometry were analyzed. The c-score outperformed all other citation indices at predicting Weisenfeld Awardees, with an AUC of 0.99 (95% CI: 0.97-1.0). The H-index had an AUC of 0.89 (95% CI: 0.83-0.96). Authors with higher c-scores tended to have longer career lengths and similar self-citation rates compared to other authors. Sixteen percent of authors in the database were identified as female, and 64% were affiliated with the United States of America.

CONCLUSION

The c-score is an effective metric for assessing research impact in ophthalmology, as seen through its ability to predict Weisenfeld Awardees.

Novel approach for quantitative and qualitative authors research profiling using feature fusion and tree-based learning approach

Article citation creates a link between the cited and citing articles and is used as a basis for several parameters like author and journal impact factor, H-index, i10 index, etc., for scientific achievements. Citations also include self-citation which refers to article citation by the author himself. Self-citation is important to evaluate an author's research profile and has gained popularity recently. Although different criteria are found in the literature regarding appropriate self-citation, self-citation does have a huge impact on a researcher's scientific profile. This study carries out two cases in this regard. In case 1, the qualitative aspect of the author's profile is analyzed using hand-crafted feature engineering techniques. The sentiments conveyed through citations are integral in assessing research quality, as they can signify appreciation, critique, or serve as a foundation for further research. Analyzing sentiments within in-text citations remains a formidable challenge, even with the utilization of automated sentiment annotations. For this purpose, this study employs machine learning models using term frequency (TF) and term frequency-inverse document frequency (TF-IDF). Random forest using TF with Synthetic Minority Oversampling Technique (SMOTE) achieved a 0.9727 score of accuracy. Case 2 deals with quantitative analysis and investigates direct and indirect self-citation. In this study, the top 2% of researchers in 2020 is considered as a baseline. For this purpose, the data of the top 25 Pakistani researchers are manually retrieved from this dataset, in addition to the citation information from the Web of Science (WoS). The self-citation is estimated using the proposed model and results are compared with those obtained from WoS. Experimental results show a substantial difference between the two, as the ratio of self-citation from the proposed approach is higher than WoS. It is observed that the citations from the WoS for authors are overstated. For a comprehensive evaluation of the researcher's profile, both direct and indirect self-citation must be included.

Self-referencing rates in biological disciplines

The use of citation counts (among other bibliometrics) as a facet of academic research evaluation can influence citation behavior in scientific publications. One possible unintended consequence of this bibliometric is excessive self-referencing, where an author favors referencing their own publications over related publications from different research groups. Peer reviewers are often prompted by journals to determine whether references listed in the manuscript under review are unbiased, but there is no consensus on what is considered "excessive" self-referencing. Here, self-referencing rates are examined across multiple journals in the fields of biology, genetics, computational biology, medicine, pathology, and cell biology. Median self-referencing rates are between 8-13% across a range of journals within these disciplines. However, self-referencing rates vary as a function of total number of references, number of authors, author status/rank, author position, and total number of publications for each author. Importantly, these relationships exhibit interdisciplinary and journal-dependent differences that are not captured by examining broader fields in aggregate (e.g., Biology, Chemistry, Physics, etc.). These results provide useful statistical guidelines for authors, editors, reviewers, and journals when considering referencing practices for individual publications, and highlight the effects of additional factors influencing self-referencing rates.

Productivity metrics and hiring rubrics are warped by cultural schemas of merit

Academic science is committed to objective hiring and promotion, yet diversification has stalled. Conventional approaches to improve diversity overlook the valorized cultural schemas scientists use to assess, and often mismeasure, merit. These schemas warp the design and use of productivity metrics and rubric scoring. We suggest interventions to unwarp them.

Evaluating Research Impact Based on Semantic Scholar Highly Influential Citations, Total Citations, and Altmetric Attention Scores: The Quest for Refined Measures Remains Illusive

Background: The evaluation of scholarly articles’ impact has been heavily based on the citation metrics despite the limitations of this approach. Therefore, the quest for meticulous and refined measures to evaluate publications’ impact is warranted. Semantic Scholar (SS) is an artificial intelligence-based database that allegedly identifies influential citations defined as “Highly Influential Citations” (HICs). Citations are considered highly influential according to SS when the cited publication has a significant impact on the citing publication (i.e., the citer uses or extends the cited work). Altmetrics are measures of online attention to research mined from activity in online tools and environments. Aims: The current study aimed to explore whether SS HICs provide an added value when it comes to measuring research impact compared to total citation counts and Altmetric Attention Score (AAS). Methods: Dimensions was used to generate the dataset for this study, which included COVID-19-related scholarly articles published by researchers affiliated to Jordanian institutions. Altmetric Explorer was selected as an altmetrics harvesting tool, while Semantic Scholar was used to extract details related to HICs. A total of 618 publications comprised the final dataset. Results: Only 4.57% (413/9029) of the total SS citations compiled in this study were classified as SS HICs. Based on SS categories of citations intent, 2626 were background citations (29.08%, providing historical context, justification of importance, and/or additional information related to the cited paper), 358 were result citations (3.97%, that extend on findings from research that was previously conducted), and 263 were method citations (2.91%, that use the previously established procedures or experiments to determine whether the results are consistent with findings in related studies). No correlation was found between HICs and AAS (r = 0.094). Manual inspection of the results revealed substantial contradictions, flaws, and inconsistencies in the SS HICs tool. Conclusions: The use of SS HICs in gauging research impact is significantly limited due to the enigmatic method of its calculation and total dependence on artificial intelligence. Along with the already documented drawbacks of total citation counts and AASs, continuous evaluation of the existing tools and the conception of novel approaches are highly recommended to improve the reliability of publication impact assessment.

Global research status of anterior cruciate ligament reconstruction: a bibliometric analysis

Purpose

The aim of this study is to comprehensively analyze the publications of anterior cruciate ligament reconstruction (ACLR) research and display the current research status in this field.

Methods

Articles regarding ACLR research published before October 7, 2021, were downloaded from the Web of Science Core Collection. Excel 2016 and Bibliometric website were used to analyze the annual article trends and international cooperation network. CiteSpace V and VOSviewer were used to perform co-occurrence and citation analyses for journals, institutions, authors, cocitation authors and keywords. Burst keyword detection was also performed with CiteSpace V.

Results

A total of 12 223 ACLR articles were identified. The American Journal of Sports Medicine (1636 publications, 92,310 citations), the Pennsylvania Commonwealth System of Higher Education (624 publications, 25,304 citations) and Freddie H. Fu (321 publications, 15,245 citations) were journals, institutions and authors with the most publications and citations, respectively. Patellar tendon was the keyword with the most occurrences (1618 times) and return to sport was the keyword with the most burst strength (burst strength: 46.99).

Conclusion

ACLR-related publications showed a rapid increasing trend since 1990. A large number of articles have been published by authors from different institutions and countries, some of which have gained great academic influence. Based on keyword analysis, patellar tendon is identified as the research hotspot and return to sport is identified as the current research frontier.

Recent trends in the use of social media in parasitology and the application of alternative metrics

In recent times, the use of social media for the dissemination of “news and views” in parasitology has increased in popularity. News, Twitter and Blogs have emerged as commonplace vehicles in the knowledge dissemination and transfer process. Alternative metrics (“altmetrics”), based on social media mentions have been proposed as a measure of societal impact, although firm evidence for this relationship is yet to be found. Nevertheless, increasing amounts of data on “altmetrics” are being analysed to identify the nature of the unknown impact that social media is generating. Here, we examine the recent, and increasing use of social media in the field of parasitology and the relationship of “altmetrics” with more traditional bibliometric indicators, such as article citations and journal metrics. The analyses document the rise and dominance of Twitter as the main form of social media occurring in the discipline of parasitology and note the contribution to this trend of Twitter bots that automatically tweet about publications. We also report on the use of the social referencing platform Mendeley and its correlation to article citations; Mendeley reader numbers are now considered to provide firm evidence on the early impact of research. Finally, we consider the Twitter profile of 31 journals publishing parasitology research articles (by volume of papers published); we show that 13 journals are associated with prolific Twitter activity about parasitology. We hope this study will stimulate not only the continued and responsible use of social media to disseminate knowledge about parasitology for the greater good, but also encourage others to further investigate the impact and benefits that altmetrics may bring to this discipline.

•

We highlight and document the rise of social media and its use in parasitology.

•

Twitter activity within the parasitology community has increased significantly over the last 10 years.

•

Mendeley reader activity is strongly correlated with an article's citations.

•

Thirteen journals are associated with prolific Twitter activity about parasitology.

•

A Journalʼs social media strategy is important to authors.

Are publications on zoological taxonomy under attack?

Taxonomy is essential to biological sciences and the priority field in face of the biodiversity crisis. The industry of scientific publications has made extensive promotion and display of bibliometric indexes, resulting in side effects such as the Journal Impact Factor™ (JIF) mania. Inadequacies of the widely used indexes to assess taxonomic publications are among the impediments for the progress of this field. Based on an unusually high proportion of self-citations, the mega-journal Zootaxa, focused on zoological taxonomy, was suppressed from the Journal Citation Reports (JCR, Clarivate™). A prompt reaction from the scientific community against this decision took place exposing myths and misuses of bibliometrics. Our goal is to shed light on the impact of misuse of bibliometrics to the production in taxonomy. We explored JCR's metrics for 2010-2018 of 123 zoological journals publishing taxonomic studies. Zootaxa, with around 15 000 citations, received 311% more citations than the second most cited journal, and shows higher levels of self-citations than similar journals. We consider Zootaxa's scope and the fact that it is a mega-journal are insufficient to explain its high level of self-citation. Instead, this result is related to the 'Zootaxa phenomenon', a sociological bias that includes visibility and potentially harmful misconceptions that portray the journal as the only one that publishes taxonomic studies. Menaces to taxonomy come from many sources and the low bibliometric indexes, including JIF, are only one factor among a range of threats. Instead of being focused on statistically illiterate journal metrics endorsing the villainy of policies imposed by profit-motivated companies, taxonomists should be engaged with renewed strength in actions directly connected to the promotion and practice of this science without regard for citation analysis.