Identifying publications in questionable journals in the context of performance-based research funding

Profile of authors publishing in ‘predatory’ journals and causal factors behind their decision: A systematic review

Intensified pressure to publish is a hallmark of a rapidly evolving higher education field where the faculty of any hue cannot avoid the ‘publish or perish’ treadmill. Growing need to publish more and to do so fast have resulted in the proliferation of pseudo scholarly publications many regards as ‘predatory’. This article provides a systematic review of research studies on so-called ‘predatory’ publishing, a new but fast-growing area of research, with a particular focus on the awareness of prospective authors about so-called ‘predatory’ publishing, the profile of authors publishing in ‘predatory’ journals and the causal factors encouraging authors to publish in such outlets. It synthetizes the results of research studies on the topic to identify gaps and trends in the existing knowledgebase to guide further research. Results indicate so-called ‘predatory’ articles are authored by scholars from all fields and levels of academic experience rather than by inexperienced scholars only and ‘predatory’ contributions are not limited to developing countries, suggesting geographical location and author experience fail to explain the author profile of ‘predatory’ articles. Findings of this review suggest causal factors include research evaluation policies and publication pressure that emerge from the research environment in which scholars operate authors’ limited capacity to publish in ‘legitimate’ journals and conventions of so-called ‘predatory’ publishers. This indicates meaningful action might address all these factors in combination, rather than focus on them in isolation.

Citation patterns between impact-factor and questionable journals

One of the most fundamental issues in academia today is understanding the differences between legitimate and questionable publishing. While decision-makers and managers consider journals indexed in popular citation indexes such as Web of Science or Scopus as legitimate, they use two lists of questionable journals (Beall’s and Cabell’s), one of which has not been updated for a few years, to identify the so-called predatory journals. The main aim of our study is to reveal the contribution of the journals accepted as legitimate by the authorities to the visibility of questionable journals. For this purpose, 65 questionable journals from social sciences and 2338 Web-of-Science-indexed journals that cited these questionable journals were examined in-depth in terms of index coverages, subject categories, impact factors and self-citation patterns. We have analysed 3234 unique cited papers from questionable journals and 5964 unique citing papers (6750 citations of cited papers) from Web of Science journals. We found that 13% of the questionable papers were cited by WoS journals and 37% of the citations were from impact-factor journals. The findings show that neither the impact factor of citing journals nor the size of cited journals is a good predictor of the number of citations to the questionable journals.

A credit-like rating system to determine the legitimacy of scientific journals and publishers

The predatory nature of a journal is in constant debate because it depends on multiple factors, which keep evolving. The classification of a journal as being predatory, or not, is no longer exclusively associated with its open access status, by inclusion or exclusion on perceived reputable academic indexes and/or on whitelists or blacklists. Inclusion in the latter may itself be determined by a host of criteria, may be riddled with type I errors (e.g., erroneous inclusion of a truly predatory journal in a whitelist) and/or type II errors (e.g., erroneous exclusion of a truly valid scholarly journal in a whitelist). While extreme cases of predatory publishing behavior may be clear cut, with true predatory journals displaying ample predatory properties, journals in non-binary grey zones of predatory criteria are difficult to classify. They may have some legitimate properties, but also some illegitimate ones. In such cases, it might be too extreme to refer to such entities as "predatory". Simply referring to them as "potentially predatory" or "borderline predatory" also does little justice to discern a predatory entity from an unscholarly, low-quality, unprofessional, or exploitative one. Faced with the limitations caused by this gradient of predatory dimensionality, this paper introduces a novel credit-like rating system, based in part on well-known financial credit ratings companies used to assess investment risk and creditworthiness, to assess journal or publisher quality. Cognizant of the weaknesses and criticisms of these rating systems, we suggest their use as a new way to view the scholarly nature of a journal or publisher. When used as a tool to supplement, replace, or reinforce current sets of criteria used for whitelists and blacklists, this system may provide a fresh perspective to gain a better understanding of predatory publishing behavior. Our tool does not propose to offer a definitive solution to this problem.

"Evidence-based checklists" for identifying predatory journals have not been assessed for reliability or validity: An analysis and proposal for moving forward

BACKGROUND

Predatory journals (PJs) pose a threat to the quality/integrity of scientific publishing. Checklists have been proposed to identify PJs, but few are "evidence-based". This study's objective was to search for/assess evidence-based checklists (EBCs) for reliability and validity, based on a new consensus definition of PJs.

METHODS

A published methods guideline for scale development was used to identify how many steps were completed in the generation of identified EBCs. Items from each EBC were compared against the consensus definition, and a list of items to be considered in the creation of a composite EBC to identify PJs was generated.

RESULTS

Four EBCs were identified. None of these had completed the first of the nine steps for scale development and validation. Forty-seven items from the four EBCs were assessed against the consensus definition, of which 28 items fell within the definition. A proposed composite EBC was created from items matching components of the consensus definition.

CONCLUSION

EBCs to detect PJs lack assessment of reliability and validity. To a varying degree, the EBCs contain items that match the scope of the new consensus definition of PJs. With the recent consensus definition, EBCs creators now have a clearer target, and can make adjustments.

One year of unsolicited e-mails: The modus operandi of predatory journals and publishers

OBJECTIVES

To quantify, characterize and analyze e-mail from predatory journals (PJ) received by an academic in dentistry.

METHODS

E-mails received in 2019 and suspected of being potentially predatory were pre-selected. The Ottawa Hospital Research Institute (OHRI) checklist was applied to identify the suspected biomedical PJ, including the following criteria: article processing charge (APC), fake impact factor, the journal being listed in the Directory of Open Access Journals (DOAJ) and the Committee on Publication Ethics (COPE). We also extracted information on the lack of an impact factor on Journal Citations Reports, non-journal affiliated contact e-mail address, flattering language, article and/or personal citation, unsubscribe link, being listed in the National Library of Medicine (NLM) current catalog and indexed on Medline.

RESULTS

A total of 2812 unsolicited suspected e-mails were received, and 1837 requested some sort of manuscript; among these, 1751 met some of the OHRI criteria. Less than half (780/1837, 42 %) referred to some area of dentistry. The median APC was US$399. A false impact factor was mentioned in 11 % (201/1837) of the e-mails, and 27 % (504/1837) corresponded to journals currently listed in the NLM catalog. Journals listed in DOAJ and COPE sent 89 e-mails.

CONCLUSIONS

The email campaign from PJ was high and recurrent. Researchers should be well informed about PJ' modus operandi to protect their own reputation as authors and that of science.

CLINICAL SIGNIFICANCE

Peer review and established academic practices and etiquette contribute to ensuring scientific progress, which is essential to protect the health of patients in particular and of people in general. Predatory journals constitute a threat to peer review and scientific etiquette and, as such, may hinder scientific progress and public health.

National Lists of Scholarly Publication Channels: An Overview and Recommendations for Their Construction and Maintenance

Purpose

This paper presents an overview of different kinds of lists of scholarly publication channels and of experiences related to the construction and maintenance of national lists supporting performance-based research funding systems. It also contributes with a set of recommendations for the construction and maintenance of national lists of journals and book publishers.

Design/methodology/approach

The study is based on analysis of previously published studies, policy papers, and reported experiences related to the construction and use of lists of scholarly publication channels.

Findings

Several countries have systems for research funding and/or evaluation, that involve the use of national lists of scholarly publication channels (mainly journals and publishers). Typically, such lists are selective (do not include all scholarly or non-scholarly channels) and differentiated (distinguish between channels of different levels and quality). At the same time, most lists are embedded in a system that encompasses multiple or all disciplines. This raises the question how such lists can be organized and maintained to ensure that all relevant disciplines and all types of research are adequately represented.

Research limitation

The conclusions and recommendations of the study are based on the authors’ interpretation of a complex and sometimes controversial process with many different stakeholders involved.

Practical implications

The recommendations and the related background information provided in this paper enable mutual learning that may feed into improvements in the construction and maintenance of national and other lists of scholarly publication channels in any geographical context. This may foster a development of responsible evaluation practices.

Originality/value

This paper presents the first general overview and typology of different kinds of publication channel lists, provides insights on expert-based versus metrics-based evaluation, and formulates a set of recommendations for the responsible construction and maintenance of publication channel lists.

Methodological considerations for identifying questionable publishing in a national context: The case of Swedish Higher Education Institutions

The overall scope of this study is an attempt at a methodological framework for matching publication lists at the national level against a combined set of blacklists for questionable publishing. Using the total verified set of publications from Swedish Higher Education Institutions (HEI) as a case, we examined the number, distribution, and proportion of publishing in questionable journals at the national level. Journal publication data was extracted from the national SwePub database and matched against three curated blacklists of questionable publishing. For the period 2012–2017, we identified 1,743 published papers in blacklisted journals, equal to an average of 0.5–0.9% of the total publications from Swedish HEIs. There was high variability between different HEI categories, with more established universities at the lower end of the spectrum, while university colleges and new universities had a much higher proportion (∼2%). There was a general decreasing trend during the study period (ρ = 0.83) for all categories of HEIs. The study presents a methodology to identify questionable publishing in academia that could be applied to other countries with similar infrastructure. Thus, it could serve as a starting point for the development of a general framework for cross-national quantitative estimation of questionable publishing.

Identifying publications in questionable journals in the context of performance-based research funding

In this article we discuss the five yearly screenings for publications in questionable journals which have been carried out in the context of the performance-based research funding model in Flanders, Belgium. The Flemish funding model expanded from 2010 onwards, with a comprehensive bibliographic database for research output in the social sciences and humanities. Along with an overview of the procedures followed during the screenings for articles in questionable journals submitted for inclusion in this database, we present a bibliographic analysis of the publications identified. First, we show how the yearly number of publications in questionable journals has evolved over the period 2003–2016. Second, we present a disciplinary classification of the identified journals. In the third part of the results section, three authorship characteristics are discussed: multi-authorship, the seniority–or experience level–of authors in general and of the first author in particular, and the relation of the disciplinary scope of the journal (cognitive classification) with the departmental affiliation of the authors (organizational classification). Our results regarding yearly rates of publications in questionable journals indicate that awareness of the risks of questionable journals does not lead to a turn away from open access in general. The number of publications in open access journals rises every year, while the number of publications in questionable journals decreases from 2012 onwards. We find further that both early career and more senior researchers publish in questionable journals. We show that the average proportion of senior authors contributing to publications in questionable journals is somewhat higher than that for publications in open access journals. In addition, this paper yields insight into the extent to which publications in questionable journals pose a threat to the public and political legitimacy of a performance-based research funding system of a western European region. We include concrete suggestions for those tasked with maintaining bibliographic databases and screening for publications in questionable journals.