Research evaluation and citation analysis: key issues and implications

Scientific text citation analysis using CNN features and ensemble learning model

Citation illustrates the link between citing and cited documents. Different aspects of achievements like the journal's impact factor, author's ranking, and peers' judgment are analyzed using citations. However, citations are given the same weight for determining these important metrics. However academics contend that not all citations can ever have equal weight. Predominantly, such rankings are based on quantitative measures and the qualitative aspect is completely ignored. For a fair evaluation, qualitative evaluation of citations is needed in addition to quantitative ones. Many existing works that use qualitative evaluation consider binary class and categorize citations as important or unimportant. This study considers multi-class tasks for citation sentiments on imbalanced data and presents a novel framework for sentiment analysis in in-text citations of research articles. In the proposed technique, features are retrieved using a convolutional neural network (CNN), and classification is performed using a voting classifier that combines Logistic Regression (LR) and Stochastic Gradient Descent (SGD). The class imbalance problem is handled by the synthetic minority oversampling technique (SMOTE). Extensive experiments are performed in comparison with the proposed approach using SMOTE-generated data and machine learning models by term frequency (TF), and term frequency-inverse document frequency (TF-IDF) to evaluate the efficacy of the proposed approach for citation analysis. It is found that the proposed voting classifier using CNN features achieves an accuracy, precision, recall, and F1 score of 0.99 for all. This work not only advances the field of sentiment analysis in academic citations but also underscores the importance of incorporating qualitative aspects in evaluating the impact and sentiments conveyed through citations.

Novel approach for quantitative and qualitative authors research profiling using feature fusion and tree-based learning approach

Article citation creates a link between the cited and citing articles and is used as a basis for several parameters like author and journal impact factor, H-index, i10 index, etc., for scientific achievements. Citations also include self-citation which refers to article citation by the author himself. Self-citation is important to evaluate an author's research profile and has gained popularity recently. Although different criteria are found in the literature regarding appropriate self-citation, self-citation does have a huge impact on a researcher's scientific profile. This study carries out two cases in this regard. In case 1, the qualitative aspect of the author's profile is analyzed using hand-crafted feature engineering techniques. The sentiments conveyed through citations are integral in assessing research quality, as they can signify appreciation, critique, or serve as a foundation for further research. Analyzing sentiments within in-text citations remains a formidable challenge, even with the utilization of automated sentiment annotations. For this purpose, this study employs machine learning models using term frequency (TF) and term frequency-inverse document frequency (TF-IDF). Random forest using TF with Synthetic Minority Oversampling Technique (SMOTE) achieved a 0.9727 score of accuracy. Case 2 deals with quantitative analysis and investigates direct and indirect self-citation. In this study, the top 2% of researchers in 2020 is considered as a baseline. For this purpose, the data of the top 25 Pakistani researchers are manually retrieved from this dataset, in addition to the citation information from the Web of Science (WoS). The self-citation is estimated using the proposed model and results are compared with those obtained from WoS. Experimental results show a substantial difference between the two, as the ratio of self-citation from the proposed approach is higher than WoS. It is observed that the citations from the WoS for authors are overstated. For a comprehensive evaluation of the researcher's profile, both direct and indirect self-citation must be included.

Global research in sarcopenia: High-cited papers, research institutions, funding agencies and collaborations, 1993-2022

BACKGROUND AND AIMS

This bibliometric analysis aims to analyze the high-cited papers (HCPs), those which have received >100 citations) on Sarcopenia to provide insight into publication performances and research characteristics of the literature.

METHODS

Global HCPs on Sarcopenia research were identified from the Scopus database from January 1993 to August 2022. VOSviewer, and Biblioshiny software were used to visualize the collaborative interaction among most productive countries, organizations, authors, and keywords. Select bibliometric measures were applied to evaluate the publication productivity and their influence in this area.

RESULTS

Out of 6219 publications on Sarcopenia only 398 were HCPs. These HCPs received an average of 271.7 citations per publication (CPP). The most productive organizations were the Università Cattolica del Sacro Cuore, Italy, and Jean Mayer USDA Human Nutrition Center on Aging, USA. The most impactful organizations in terms of CPP and relative citation index were CHU de Toulouse, France, and Università Cattolica del Sacro Cuore, Italy. The most productive authors were Landi F and Morley JE, and Anker SD from Italy and Germany respectively. The most impactful authors were: Cederholm T (Sweden), Cruz JAJ (Spain) and Rolland Y (France). There were few/no HCPs from South America, Africa, South Asia, and USSR.

CONCLUSION

Sarcopenia research has been predominantly done in USA, Europe, and China, and rarely from low and middle-income countries. Further focus of research should be on its etiopathogenesis (especially at the molecular level), prevalence in different communities, methods to diagnose it in early stages, and its cost-effective management.

Proceedings of the annual meetings of the association for information science and technology: analysis of two decades of published research

Purpose

The purpose of this study is to investigate the research and publication trends in the articles published in the conference proceedings of the Association for Information Science & Technology (ASIS&T) since the year 2000.

Design/methodology/approach

We analyzed two decades of ASIS&T proceedings to uncover bibliometric patterns. This study uses two bibliometric procedures applied to the publications in the ASIS&T conference proceedings – a bibliometrics analysis using three data sources (Scopus, ASIS&T proceedings website and Scimago journal ranking) and a scientific mapping analysis using VOSViewer.

Findings

We found 3,129 publications from 2000 to 2020, with more than three-quarters jointly authored. Most authors are from the United States, Canada and China. Social media and information behavior are the top-researched areas. The top-cited journals are the Journal of the Association for Information Science & Technology, Information Processing and Management and Library and Information Science Research.

Research limitations/implications

The study will help information professionals understand patterns in recent research, which should help guide them in their future research directions.

Practical implications

The findings affirm ASIS&T’s move to an international association and point to the growing importance of collaborative work and social media.

Originality/value

ASIS&T has been holding annual meetings since the 1950s. While there have been various bibliometric studies analyzing publication trends in different journals in the field of information science, none of these studies have analyzed the ASIS&T conference proceedings.

Bibliometric analysis of the VINE Journal of Information and Knowledge Management Systems: 2000–2020

Purpose

VINE Journal of Information and Knowledge Management Systems (VJIKMS) is one of the oldest and leading journals in the knowledge management (KM) field. The purpose of this study is to conduct bibliometrics analysis of publications published in VJIKMS for the past two decades. For doing this, this paper covered the past two decades of publications and carried out a science mapping analysis of publications.

Design/methodology/approach

The methodology included bibliometrics and the science mapping analysis process. This paper imported the bibliographic information of VJIKMS from the abstract and citation database Scopus. Through bibliometrics method, this paper examined the citation results, author productivity, authorship pattern, research collaboration and other parameters of the selected publications. Afterward, this paper used VOSviewer software to carry out the science mapping of bibliometric networks.

Findings

The findings showed that VJIKMS published 718 publications during 2000–2020, which got cited 4,311 times (6 times per article) till date. Joint authorship and international collaboration have been increasing where 1,417 authors from 66 countries have published. The USA, the UK and Iran lead the KM publications in this journal. Nonaka’s publications and Journal of Knowledge Management (JKM) are highly cited references and journals in the VJIKMS.

Research limitations/implications

The findings of this study would help the KM students, researchers and practitioners to see the diffusion of KM globally, what are the promising areas to work and helps to know the various patterns of publications if they aim to publish in VJIKMS.

Originality/value

This is the first time a bibliometric analysis has been conducted to analysis of research publications published in VJIKMS. This presents a comprehensive analysis of publications between 2000 and 2020.

Scholarly research in Library and Information Science: an analysis based on ISI Web of Science

Purpose

The purpose of this paper is to present a quantitative analysis of the Library and Information Science (LIS) scholarly literature published worldwide.

Design/methodology/approach

The ISI Web of Science database was used as a source for extracting the data of published documents during the period 2003–2017. The analysis of the published literature was based on the following indicators: research productivity of each county, annual publications, annual citations, highly cited articles, highly cited LIS journals, most productive institutions in the field of LIS and most prolific authors. The types of documents included in this study were research articles, conference proceeding papers, book reviews and editorials.

Findings

The findings revealed that the USA had the highest overall output of LIS scholarly publications. The year 2016 was identified as the most productive year for the number of publications, whereas 2017 was identified as the most productive year for the number of citations. “Impact of data sources on citation counts and rankings of LIS faculty: Web of Science versus Scopus and Google Scholar” was the most highly cited article. The Journal of Medical Library Association was the most highly cited journal in LIS. Indiana University from the USA was the most productive LIS research institution. Mike Thelwall, from the UK, is identified as the most prolific author in LIS.

Originality/value

The study will be of interest for those researchers, who intend to conduct bibliometric research studies in LIS.

Competencies for bibliometrics

Universities are increasingly offering support services for bibliometrics, often based in the library. This paper describes work done to produce a competency model for those supporting bibliometrics. The results of a questionnaire in which current practitioners rated bibliometric tasks as entry level, core or specialist are reported. Entry level competencies identified were explaining bibliometric concepts, doing basic calculations and some professional skills. Activities identified by participants as core are outlined. Reflecting on items that were considered in scope but specialist there was less stress on evaluating scholars, work at a strategic level, working with data outside proprietary bibliometric tools and consultancy-type services as opposed to training for disintermediated use. A competency model is presented as an appendix.

Subject vs. functional: Should subject librarians be replaced by functional specialists in academic libraries?

This paper focuses on the move by some academic libraries to replace subject-based organisational structures with new functional structures, introducing functional specialists instead of subject librarians. It reports 11 in-depth semi-structured interviews with senior managers in UK libraries discussing their attitudes to this change. Key drivers for change included the desire to align the library more closely with institutional strategy and provide enhanced research support. However, there were also major concerns, particularly the loss of close relationships with academic departments. Little consensus emerged apart from agreement that this is a major current debate impacting significantly on the future positioning of libraries in their institutions.

The bibliometric analysis of scholarly production: How great is the impact?

Bibliometric methods or "analysis" are now firmly established as scientific specialties and are an integral part of research evaluation methodology especially within the scientific and applied fields. The methods are used increasingly when studying various aspects of science and also in the way institutions and universities are ranked worldwide. A sufficient number of studies have been completed, and with the resulting literature, it is now possible to analyse the bibliometric method by using its own methodology. The bibliometric literature in this study, which was extracted from Web of Science, is divided into two parts using a method comparable to the method of Jonkers et al. (Characteristics of bibliometrics articles in library and information sciences (LIS) and other journals, pp. 449-551, 2012: The publications either lie within the Information and Library Science (ILS) category or within the non-ILS category which includes more applied, "subject" based studies. The impact in the different groupings is judged by means of citation analysis using normalized data and an almost linear increase can be observed from 1994 onwards in the non-ILS category. The implication for the dissemination and use of the bibliometric methods in the different contexts is discussed. A keyword analysis identifies the most popular subjects covered by bibliometric analysis, and multidisciplinary articles are shown to have the highest impact. A noticeable shift is observed in those countries which contribute to the pool of bibliometric analysis, as well as a self-perpetuating effect in giving and taking references.

Tenure metrics: bibliometric education and services for academic faculty

Bibliometric indicators can assist biomedical researchers in determining the nature of their influence and impact of their research. In recent years, demands by academic faculty for bibliometric analyses have risen considerably at the University at Buffalo (UB). Consequently, library staff at the Health Sciences Library (HSL) began to offer bibliometric services, a series of workshops and consultations to meet faculty requests. This column highlights the implementation, assessment, and challenges of UB HSL's "Tenure Metrics" program.