Ranking university departments using the mean h-index

Using Bibliometric Analysis to Map the Impact of a Simulation Program's Published Scholarly Works

SUMMARY STATEMENT

Bibliometrics quantitatively evaluates the targeted literature sources and can help define research and scholarly publications' impact and demonstrate connections for authors, departments, or universities. This article presents a methodology for simulation programs to evaluate their influence in terms of both impact and scope of their published simulation-based healthcare scholarly output. Using the authors' home university and healthcare system as an example, the article outlines a methodology to map research and scholarly works networks within the systems, identify and map connections outside the system, and quantifiably score the overall impact of the simulation program's scholarly output using a common scoring metric, the h-index. This generates an objective measure of impact, rather than a subjective opinion of an organization's research and scholarly impact. The combination of an institutional h-index with mapping of simulation-based healthcare scholarly output provides a full, objective description of the institution's output and provides a benchmark for other simulation programs for comparison.

Higher education research performance parameters classified by systems theory: antecedents for the development of assessment models

Purpose

This paper aims to identify and classify the parameters that construct the input, processes, output, productivity and outcome variables that affect performance. These parameters are used in the evaluation model to measure research performance in universities so that they can be used as the basis for making leadership policies both at the national and institutional levels.

Design/methodology/approach

The design of this research is a quantitative research method using a survey questionnaire that was sent to the heads of research institutions at universities in Indonesia. To obtain these parameters, a test for determining the value of the loading factor was used.

Findings

The authors found that input variable parameters consisted of 10 parameters; process variable consisted of 22 parameters; output variable parameters consisted of 8 parameters; productivity variable consisted of 4 parameters; and outcome variable parameters consisted of 10 parameters.

Originality/value

One approach to obtain parameters is through systems theory, where every element that makes up the organization contributes to the achievement of goals. This study attempted to develop parameters in the performance appraisal model of systems theory-based research institutions that are adapted to trends in the direction of research in universities. These parameters are based on aspects of input, process, output, productivity and outcome.

Do authors play fair or manipulate Google Scholar h-index?

Purpose

The study's main purpose is to scrutinize Google Scholar profiles and find the answer to the question, “Do authors play fair or manipulate Google Scholar Bibliometric Indicators like h-index and i10-index?”

Design/methodology/approach

The authors scrutinized the Google Scholar profiles of the top 50 library and science researchers claiming authorship of 21,022 publications. The bibliographic information of all the 21,022 publications like authorship and subject details were verified to identify accuracy, discrepancies and manipulation in their authorship claims. The actual and fabricated entries of all the authors along with their citations were recorded in the Microsoft Office Excel 2007 for further analyses and interpretation using simple arithmetic calculations.

Findings

The results show that the h-index of authors obtained from the Google Scholar should not be approved at its face value as the variations exist in the publication count and citations, which ultimately affect their h-index and i10 index. The results reveal that the majority of the authors have variations in publication count (58%), citations (58%), h-index (42%) and i10-index (54%). The magnitude of variation in the number of publications, citations, h-index and i10-index is very high, especially for the top-ranked authors.

Research limitations/implications

The scope of the study is strictly restricted to the faculty members of library and information science and cannot be generalized across disciplines. Further, the scope of the study is limited to Google Scholar and caution needs to be taken to extend results to other databases like Web of Science and Scopus.

Practical implications

The study has practical implications for authors, publishers, and academic institutions. Authors must stop the unethical research practices; publishers must adopt techniques to overcome the problem and academic institutions need to take precautions before hiring, recruiting, promoting and allocating resources to the candidates on the face value of the Google Scholar h-index. Besides, Google needs to work on the weak areas of Google Scholar to improve its efficacy.

Originality/value

The study brings to light the new ways of manipulating bibliometric indicators like h-index, and i10-index provided by Google Scholar using false authorship claims.

Important citations identification by exploiting generative model into discriminative model

Although the citations between scientific documents are deemed as a vehicle for dissemination, inheritance and development of scientific knowledge, not all citations are well-positioned to be equal. A plethora of taxonomies and machine-learning models have been implemented to tackle the task of citation function and importance classification from qualitative aspect. Inspired by the success of kernel functions from resulting general models to promote the performance of the support vector machine (SVM) model, this work exploits the potential of combining generative and discriminative models for the task of citation importance classification. In more detail, generative features are generated from a topic model, citation influence model (CIM) and then fed to two discriminative traditional machine-learning models, SVM and RF (random forest), and a deep learning model, convolutional neural network (CNN), with other 13 traditional features to identify important citations. The extensive experiments are performed on two data sets with different characteristics. These three models perform better on the data set from one discipline. It is very possible that the patterns for important citations may vary by the fields, which disable machine-learning models to learn effectively the discriminative patterns from publications from multiple domains. The RF classifier outperforms the SVM classifier, which accords with many prior studies. However, the CNN model does not achieve the desired performance due to small-scaled data set. Furthermore, our CIM model–based features improve further the performance for identifying important citations.

Trends, patterns and determinants of research productivity at the Technical University of Kenya

This paper explores the trends, patterns and determinants of research productivity at the Technical University of Kenya (TU-K). It analyses the research output of full-time academic staff collected from Google Scholar using Harzing’s “Publish or Perish” software. Further information was obtained from the top ten researchers based on productivity from the analysed research output. Additional information was obtained from purposively sampled officers of the university, including the Deputy Vice-Chancellor in charge of Academics, Research and Students; three executive deans of faculties; and the Director of Research and Knowledge Exchange. The study revealed gradual growth in research productivity at TU-K. The study also revealed that insufficient funding, lack of motivation, limited time and lack of equipment are some of the factors affecting research productivity at TU-K. Full implementation of performance contracting targets and establishment of the Directorate of Research and Knowledge Exchange are some of the strategies being implemented to increase research output, visibility and impact. The authors recommend that the university should nurture interest in, as well as enhance capability for inquiry and investigation to increase research productivity. The university should also create mechanisms for building intrinsic motivation and address the extrinsic factors in order to catapult research productivity.

Ranking Multi-Metric Scientific Achievements Using a Concept of Pareto Optimality

The ranking of multi-metric scientific achievements is a challenging task. For example, the scientific ranking of researchers utilizes two major types of indicators; namely, number of publications and citations. In fact, they focus on how to select proper indicators, considering only one indicator or combination of them. The majority of ranking methods combine several indicators, but these methods are faced with a challenging concern—the assignment of suitable/optimal weights to the targeted indicators. Pareto optimality is defined as a measure of efficiency in the multi-objective optimization which seeks the optimal solutions by considering multiple criteria/objectives simultaneously. The performance of the basic Pareto dominance depth ranking strategy decreases by increasing the number of criteria (generally speaking, when it is more than three criteria). In this paper, a new, modified Pareto dominance depth ranking strategy is proposed which uses some dominance metrics obtained from the basic Pareto dominance depth ranking and some sorted statistical metrics to rank the scientific achievements. It attempts to find the clusters of compared data by using all of indicators simultaneously. Furthermore, we apply the proposed method to address the multi-source ranking resolution problem which is very common these days; for example, there are several world-wide institutions which rank the world’s universities every year, but their rankings are not consistent. As our case studies, the proposed method was used to rank several scientific datasets (i.e., researchers, universities, and countries) for proof of concept.

Statistical evaluation of research performance of young university scholars: A case study

Abstract The research performance of a small group of 49 young scholars, such as doctoral students, postdoctoral and junior researchers, working in different technical and scientific fields, was evaluated based on 11 types of research outputs. The scholars worked at a technical university in the fields of Civil Engineering, Ecology, Economics, Informatics, Materials Engineering, Mechanical Engineering, and Safety Engineering. Principal Component Analysis was used to statistically analyze the research outputs and its results were compared with factor and cluster analysis. The metrics of research productivity describing the types of research outputs included the number of papers, books and chapters published in books, the number of patents, utility models and function samples, and the number of research projects conducted. The metrics of citation impact included the number of citations and h-index. From these metrics – the variables – the principal component analysis extracted 4 main principal components. The 1st principal component characterized the cited publications in high-impact journals indexed by the Web of Science. The 2nd principal component represented the outputs of applied research and the 3rd and 4th principal components represented other kinds of publications. The results of the principal component analysis were compared with the hierarchical clustering using Ward’s method. The scatter plots of the principal component analysis and the Mahalanobis distances were calculated from the 4 main principal component scores, which allowed us to statistically evaluate the research performance of individual scholars. Using variance analysis, no influence of the field of research on the overall research performance was found. Unlike the statistical analysis of individual research metrics, the approach based on the principal component analysis can provide a complex view of the research systems.

Departmental h-Index: Evidence for Publishing Less?

Purpose

The h-index is an established method for determining an individual faculty member's impact on the scientific literature. The purpose of this study was to measure and describe over time the combined h-index of a large university medical imaging department.

Materials and Methods

All faculty members from the Department of Medical Imaging, University of Toronto, were identified from administrative records for 6 separate years between 2000-2014. Individual members' and the departmental h-index were calculated using citation data from the Scopus database. Descriptive univariate statistics were reported. Factors contributing to the change in departmental h-index over time were assessed using linear regression analysis.

Results

The number of faculty members increased from 117 in 2000 to 186 in 2014. The departmental h-index increased from 48 in 2000 to 142 in 2014. During this time period, the median h-index for faculty members increased from 4 (interquartile range 2-8) to 10 (interquartile range 5-19). Regression analysis revealed that for every additional staff member, the departmental h-index increased by 1.4 (standard error = 0.1, P < .01), whereas, by increasing the median h-index of members by 1 the departmental h-index increased by 15.7 (standard error = 0.6, P < .01).

Conclusion

Our study suggests that to increase a department's h-index, it is important to foster impactful research from within the faculty ranks of the department. The h-index of academic radiology departments is a meaningful tool that allows for evaluation from within and against other academic centres.

The construct validity of theh-index

Purpose

The purpose of this paper is to show how bibliometrics would benefit from a stronger programme of construct validity.

Design/methodology/approach

The value of the construct validity concept is demonstrated by applying this approach to the evaluation of theh-index, a widely used metric.

Findings

The paper demonstrates that theh-index comprehensively fails any test of construct validity. In simple terms, the metric does not measure what it purports to measure. This conclusion suggests that the current popularity of theh-index as a topic for bibliometric research represents wasted effort, which might have been avoided if researchers had adopted the approach suggested in this paper.

Research limitations/implications

This study is based on the analysis of a single bibliometric concept.

Practical implications

The conclusion that theh-index fails any test in terms of construct validity implies that the widespread use of this metric within the higher education sector as a management tool represents poor practice, and almost certainly results in the misallocation of resources.

Social implications

This paper suggests that the current enthusiasm for theh-index within the higher education sector is misplaced. The implication is that universities, grant funding bodies and faculty administrators should abandon the use of theh-index as a management tool. Such a change would have a significant effect on current hiring, promotion and tenure practices within the sector, as well as current attitudes towards the measurement of academic performance.

Originality/value

The originality of the paper lies in the systematic application of the concept of construct validity to bibliometric enquiry.

Bibliometric factors associated with h-index of Peruvian researchers with publications indexed on Web of Science and Scopus databases

The objective of this article is: a) to identify Peruvian researchers with high, medium and low impact factor according to Web of Science and Scopus databases; b) to identify the bibliometric factor with the highest influence on h-index of Peruvian esearchers; c) to compare h-index between Web of Science and Scopus, at an individual and institutional level. Data were collected from Web of Science and Scopus (189 Peruvian researchers, 28 institutions on Web of Science and 33 on Scopus), between September 1823, 2013. Then, institutional registries were created and linear regression analysis with stepwise procedure was run to identify bibliometric factors with higher influence on the h-index of Peruvian researchers. Web of Science and Scopus showed interesting simmilarities in the h-index of Peruvian academic institutions. At individual level, documents indexed in citation database had the highest influence on the h-index. Regression model identified bibliometric factors with higher influence on the h-index of Peruvian researchers, however further large scale studies are needed to improve external validity.