Quality Issues of CRIS Data: An Exploratory Investigation with Universities from Twelve Countries

Research performance and scholarly communication profile of competitive research funding: the case of Academy of Finland

The Academy of Finland (AKA), Finland’s major public research funding agency, uses a Web of Science (WoS) based bibliometric indicator to assess the performance of research it has funded. We use an alternative methodology to compare (1) the research performance and (2) the scholarly communication profile of AKA-funded research to the Finnish universities’ entire output across the major fields of arts and sciences. Our data consists of 142,742 publications (years 2015–2018) registered in the national information service, which integrates Current Research Information System (CRIS) data of 13 Finnish universities. Research performance is analyzed using the Finnish community-curated expert-based rating of publication channels (so-called JUFO). Our results show that compared to the Finnish universities’ entire output a larger share of AKA-funded research is published in leading JUFO rated journals and book publishers. JUFO and WoS-based indicators produced consonant results regarding the performance of AKA-funded research. Analysis of publication profiles shows that AKA-funded research is more focused than the universities’ output on using peer-reviewed publications, articles published in journals, English language, foreign publishers and open access publishing. We conclude that the CRIS-based publication data can support multidimensional assessments of research performance and scholarly communication profiles, potentially also in other countries and institutions. CRIS development and maintenance require multi-stakeholder commitment, resources and incentives to ensure data quality and coverage. To fully recognize diverse open science practices and to enable international comparisons, CRISs need further development and integration as data sources.

Open access at the national level: A comprehensive analysis of publications by Finnish researchers

Open access (OA) has mostly been studied by relying on publication data from selective international databases, notably Web of Science (WoS) and Scopus. The aim of our study is to show that it is possible to achieve a national estimate of the number and share of OA based on institutional publication data providing a comprehensive coverage of the peer-reviewed outputs across fields, publication types, and languages. Our data consists of 48,177 journal, conference, and book publications from 14 Finnish universities in 2016–2017, including information about OA status, as self-reported by researchers and validated by data-collection personnel through their Current Research Information System (CRIS). We investigate the WoS, Scopus, and DOI coverage, as well as the share of OA outputs between different fields, publication types, languages, OA mechanisms (gold, hybrid, and green), and OA information sources (DOAJ, Bielefeld list, and Sherpa/Romeo). We also estimate the role of the largest international commercial publishers compared to the not-for-profit Finnish national publishers of journals and books. We conclude that institutional data, integrated at national and international level, provides one of the building blocks of a large-scale data infrastructure needed for comprehensive assessment and monitoring of OA across countries, for example at the European level.

How to Inspect and Measure Data Quality about Scientific Publications: Use Case of Wikipedia and CRIS Databases

The quality assurance of publication data in collaborative knowledge bases and in current research information systems (CRIS) becomes more and more relevant by the use of freely available spatial information in different application scenarios. When integrating this data into CRIS, it is necessary to be able to recognize and assess their quality. Only then is it possible to compile a result from the available data that fulfills its purpose for the user, namely to deliver reliable data and information. This paper discussed the quality problems of source metadata in Wikipedia and CRIS. Based on real data from over 40 million Wikipedia articles in various languages, we performed preliminary quality analysis of the metadata of scientific publications using a data quality tool. So far, no data quality measurements have been programmed with Python to assess the quality of metadata from scientific publications in Wikipedia and CRIS. With this in mind, we programmed the methods and algorithms as code, but presented it in the form of pseudocode in this paper to measure the quality related to objective data quality dimensions such as completeness, correctness, consistency, and timeliness. This was prepared as a macro service so that the users can use the measurement results with the program code to make a statement about their scientific publications metadata so that the management can rely on high-quality data when making decisions.

Data Quality as a Critical Success Factor for User Acceptance of Research Information Systems

In our present paper, the influence of data quality on the success of the user acceptance of research information systems (RIS) is investigated and determined. Until today, only a little research has been done on this topic and no studies have been carried out. So far, just the importance of data quality in RIS, the investigation of its dimensions and techniques for measuring, improving, and increasing data quality in RIS (such as data profiling, data cleansing, data wrangling, and text data mining) has been focused. With this work, we try to derive an answer to the question of the impact of data quality on the success of RIS user acceptance. An acceptance of RIS users is achieved when the research institutions decide to replace the RIS and replace it with a new one. The result is a statement about the extent to which data quality influences the success of users’ acceptance of RIS.

Influence of Information Quality via Implemented German RCD Standard in Research Information Systems

With the steady increase in the number of data sources to be stored and processed by higher education and research institutions, it has become necessary to develop Research Information Systems, which will store this research information in the long term and make it accessible for further use, such as reporting and evaluation processes, institutional decision making and the presentation of research performance. In order to retain control while integrating research information from heterogeneous internal and external data sources and disparate interfaces into RIS and to maximize the benefits of the research information, ensuring data quality in RIS is critical. To facilitate a common understanding of the research information collected and to harmonize data collection processes, various standardization initiatives have emerged in recent decades. These standards support the use of research information in RIS and enable compatibility and interoperability between different information systems. This paper examines the process of securing data quality in RIS and the impact of research information standards on data quality in RIS. We focus on the recently developed German Research Core Dataset standard as a case of application.

Text and Data Quality Mining in CRIS

To provide scientific institutions with comprehensive and well-maintained documentation of their research information in a current research information system (CRIS), they have the best prerequisites for the implementation of text and data mining (TDM) methods. Using TDM helps to better identify and eliminate errors, improve the process, develop the business, and make informed decisions. In addition, TDM increases understanding of the data and its context. This not only improves the quality of the data itself, but also the institution’s handling of the data and consequently the analyses. This present paper deploys TDM in CRIS to analyze, quantify, and correct the unstructured data and its quality issues. Bad data leads to increased costs or wrong decisions. Ensuring high data quality is an essential requirement when creating a CRIS project. User acceptance in a CRIS depends, among other things, on data quality. Not only is the objective data quality the decisive criterion, but also the subjective quality that the individual user assigns to the data.

Implementation and user acceptance of research information systems

Purpose

The purpose of this paper is to present empirical evidence on the implementation, acceptance and quality-related aspects of research information systems (RIS) in academic institutions.

Design/methodology/approach

The study is based on a 2018 survey with 160 German universities and research institutions.

Findings

The paper presents recent figures about the implementation of RIS in German academic institutions, including results on the satisfaction, perceived usefulness and ease of use. It contains also information about the perceived data quality and the preferred quality management. RIS acceptance can be achieved only if the highest possible quality of the data is to be ensured. For this reason, the impact of data quality on the technology acceptance model (TAM) is examined, and the relation between the level of data quality and user acceptance of the associated institutional RIS is addressed.

Research limitations/implications

The data provide empirical elements for a better understanding of the role of the data quality for the acceptance of RIS, in the framework of a TAM. The study puts the focus on commercial and open-source solutions while in-house developments have been excluded. Also, mainly because of the small sample size, the data analysis was limited to descriptive statistics.

Practical implications

The results are helpful for the management of RIS projects, to increase acceptance and satisfaction with the system, and for the further development of RIS functionalities.

Originality/value

The number of empirical studies on the implementation and acceptance of RIS is low, and very few address in this context the question of data quality. The study tries to fill the gap.

ETL Best Practices for Data Quality Checks in RIS Databases

The topic of data integration from external data sources or independent IT-systems has received increasing attention recently in IT departments as well as at management level, in particular concerning data integration in federated database systems. An example of the latter are commercial research information systems (RIS), which regularly import, cleanse, transform and prepare the analysis research information of the institutions of a variety of databases. In addition, all these so-called steps must be provided in a secured quality. As several internal and external data sources are loaded for integration into the RIS, ensuring information quality is becoming increasingly challenging for the research institutions. Before the research information is transferred to a RIS, it must be checked and cleaned up. An important factor for successful or competent data integration is therefore always the data quality. The removal of data errors (such as duplicates and harmonization of the data structure, inconsistent data and outdated data, etc.) are essential tasks of data integration using extract, transform, and load (ETL) processes. Data is extracted from the source systems, transformed and loaded into the RIS. At this point conflicts between different data sources are controlled and solved, as well as data quality issues during data integration are eliminated. Against this background, our paper presents the process of data transformation in the context of RIS which gains an overview of the quality of research information in an institution’s internal and external data sources during its integration into RIS. In addition, the question of how to control and improve the quality issues during the integration process in RIS will be addressed.