National Lists of Scholarly Publication Channels: An Overview and Recommendations for Their Construction and Maintenance

Inconsistent quality signals: evidence from the regional journals

Nowadays many countries and institutions use bibliometric assessment of journal quality in their research evaluation policies. However, bibliometric measures, such as impact factor or quartile, may provide a biased quality assessment for relatively new, regional, or non-mainstream journals, as these outlets usually do not possess a longstanding history, and may not be included into indexing databases. To reduce the information asymmetry between academic community (researchers, editors, policymakers) and journal management, we propose an alternative approach to evaluate journals quality signals using previous publication track record of authors. We explore the difference in the quality signals sent by regional journals. Traditional, journal-level, bibliometric measures are contrasted with generalised measures of authors' publishing records. We used a set of 50,477 articles and reviews in 83 regional journals in Physics and Astronomy (2014-2019) to extract and process data on 73 866 authors and their additional 329,245 publications in other Scopus-indexed journals. We found that traditional journal-level measures (such as journal quartile, CiteScore percentile, Scimago Journal Rank) tend to under-evaluate journal quality, thus contributing to an image of low-quality research venues. Author-level measures (including the share of papers in the Nature Index journals) send positive signals of journal quality and allow us to subdivide regional journals by their publishing strategies. These results suggest that research evaluation policies might consider attributing greater weight to regional journals, not only for the training purposes of doctoral students but also for gaining international visibility and impact.

What makes databases tick: Logics that underpin national databases for research output in Europe

The increasing use of research metrics in the monitoring and evaluation of research has led to the expansion of research information infrastructure, as indicated by the rising number of national databases for research output and research information systems. At the same time, debates on requirements of national databases for research output are typically guided by simplistic assumptions about what an infrastructure is and how it shapes our understanding of different phenomena. We present an alternative framing of databases wherein infrastructure and context are seen as mutually constitutive. Drawing upon qualitative data on 12 national databases in Europe, we investigate the role that databases for research output play in research metrics and consequently in our understanding of research activities. Specifically, we bring to the surface database features that are typically left implicit and yet are crucial for understanding these databases in context. A key outcome of this study is an identification of two database logics—New Public Management and Enlightenment—that underpin database designs and the organization of database work. These logics do not merely guide practical database work (e.g. by determining the features of highest priority), but through the outcomes of that work they both enact and distribute a particular conception of what research is. In this article, we show how the two logics manifest and how they are related to each other. Situating these findings in the landscape of developments in research information infrastructure, we discuss implications for research evaluation and the understanding of research in general.

Research performance and scholarly communication profile of competitive research funding: the case of Academy of Finland

The Academy of Finland (AKA), Finland’s major public research funding agency, uses a Web of Science (WoS) based bibliometric indicator to assess the performance of research it has funded. We use an alternative methodology to compare (1) the research performance and (2) the scholarly communication profile of AKA-funded research to the Finnish universities’ entire output across the major fields of arts and sciences. Our data consists of 142,742 publications (years 2015–2018) registered in the national information service, which integrates Current Research Information System (CRIS) data of 13 Finnish universities. Research performance is analyzed using the Finnish community-curated expert-based rating of publication channels (so-called JUFO). Our results show that compared to the Finnish universities’ entire output a larger share of AKA-funded research is published in leading JUFO rated journals and book publishers. JUFO and WoS-based indicators produced consonant results regarding the performance of AKA-funded research. Analysis of publication profiles shows that AKA-funded research is more focused than the universities’ output on using peer-reviewed publications, articles published in journals, English language, foreign publishers and open access publishing. We conclude that the CRIS-based publication data can support multidimensional assessments of research performance and scholarly communication profiles, potentially also in other countries and institutions. CRIS development and maintenance require multi-stakeholder commitment, resources and incentives to ensure data quality and coverage. To fully recognize diverse open science practices and to enable international comparisons, CRISs need further development and integration as data sources.

Implications of Publication Requirements for the Research Output of Ukrainian Academics in Scopus in 1999–2019

Purpose

This article explores the implications of publication requirements for the research output of Ukrainian academics in Scopus in 1999–2019. As such it contributes to the existing body of knowledge on quantitative and qualitative effects of research evaluation policies.

Design/methodology/approach

Three metrics were chosen to analyse the implications of publication requirements for the quality of research output: publications in predatory journals, publications in local journals and publications per SNIP quartile from the disciplinary perspective.

Findings

Study results highlight, that, firstly, publications of Ukrainian authors in predatory journals rose to 1% in 2019. Secondly, the share of publications in local journals reached the peak of 47.3% in 2015. In 2019 it fell to 31.8%. Thirdly, though the total number of publications has risen dramatically since 2011, but the share of Q3+Q4 has exceeded the share of Q1+Q2. To summarise, the study findings highligh, that research evaluation policies are required to contain not only quantitative but also qualitative criteria.

Research limitation

The study does not explore in detail the effects of a particular type of publication requirements.

Practical implications

The findings of the study have practical implications for policymakers and university managers aimed to develop research evaluation policies.

Originality/value

This paper gains insights into the effects of publication requirements on the research output of Ukrainian academics in Scopus.

Impact and visibility of Norwegian, Finnish and Spanish journals in the fields of humanities

This article analyses the impact and visibility of scholarly journals in the humanities that are publishing in the national languages in Finland, Norway and Spain. Three types of publishers are considered: commercial publishers, scholarly society as publisher, and research organizations as publishers. Indicators of visibility and impact were obtained from Web of Science, SCOPUS, Google Metrics, Scimago Journal Rank and Journal Citation Report. The findings compiled show that in Spain the categories “History and Archaeology” and “Language and Literature” account for almost 70% of the journals analysed, while the other countries offer a more homogeneous distribution. In Finland, the scholarly society publisher is predominant, in Spain, research organization as publishers, mostly universities, have a greater weighting, while in Norway, the commercial publishers take centre stage. The results show that journals from Finland and Norway will have reduced possibilities in terms of impact and visibility, since the vernacular language appeals to a smaller readership. Conversely, the Spanish journals are more attractive for indexing in commercial databases. Distribution in open access ranges from 64 to 70% in Norwegian and Finish journals, and to 91% in Spanish journals. The existence of DOI range from 31 to 41% in Nordic journals to 60% in Spanish journals and has a more widespread bearing on the citations received in all three countries (journals with DOI and open access are cited more frequently).