Reproducibility issues with correlating Beall-listed publications and research awards at a small Canadian business school

A credit-like rating system to determine the legitimacy of scientific journals and publishers

The predatory nature of a journal is in constant debate because it depends on multiple factors, which keep evolving. The classification of a journal as being predatory, or not, is no longer exclusively associated with its open access status, by inclusion or exclusion on perceived reputable academic indexes and/or on whitelists or blacklists. Inclusion in the latter may itself be determined by a host of criteria, may be riddled with type I errors (e.g., erroneous inclusion of a truly predatory journal in a whitelist) and/or type II errors (e.g., erroneous exclusion of a truly valid scholarly journal in a whitelist). While extreme cases of predatory publishing behavior may be clear cut, with true predatory journals displaying ample predatory properties, journals in non-binary grey zones of predatory criteria are difficult to classify. They may have some legitimate properties, but also some illegitimate ones. In such cases, it might be too extreme to refer to such entities as "predatory". Simply referring to them as "potentially predatory" or "borderline predatory" also does little justice to discern a predatory entity from an unscholarly, low-quality, unprofessional, or exploitative one. Faced with the limitations caused by this gradient of predatory dimensionality, this paper introduces a novel credit-like rating system, based in part on well-known financial credit ratings companies used to assess investment risk and creditworthiness, to assess journal or publisher quality. Cognizant of the weaknesses and criticisms of these rating systems, we suggest their use as a new way to view the scholarly nature of a journal or publisher. When used as a tool to supplement, replace, or reinforce current sets of criteria used for whitelists and blacklists, this system may provide a fresh perspective to gain a better understanding of predatory publishing behavior. Our tool does not propose to offer a definitive solution to this problem.

Bibliometric analysis of a controversial paper on predatory publishing

Purpose

In 2017, one study (Derek Pyne; Journal of Scholarly Publishing; DOI: 10.3138/jsp.48.3.137; University of Toronto Press) in the “predatory” publishing literature attracted global media attention. Now, over three years, according to adjusted Google Scholar data, with 53 citations (34 in Clarivate Analytics' Web of Science), that paper became that author's most cited paper, accounting for one-third of his Google Scholar citations.

Design/methodology/approach

In this paper, the authors conducted a bibliometric analysis of the authors who cited that paper.

Findings

We found that out of the 39 English peer-reviewed journal papers, 11 papers (28%) critically assessed Pyne's findings, some of which even refuted those findings. The 2019 citations of the Pyne (2017) paper caused a 43% increase in the Journal of Scholarly Publishing 2019 Journal Impact Factor, which was 0.956, and a 7.7% increase in the 2019 CiteScore.

Originality/value

The authors are of the opinion that scholars and numerous media that cited the Pyne (2017) paper were unaware of its flawed findings.