Questionnaires mentioned in academic research 1996–2019: Rapid increase but declining citation impact

Terms in journal articles associating with high quality: can qualitative research be world-leading?

Purpose

Scholars often aim to conduct high quality research and their success is judged primarily by peer reviewers. Research quality is difficult for either group to identify, however and misunderstandings can reduce the efficiency of the scientific enterprise. In response, we use a novel term association strategy to seek quantitative evidence of aspects of research that are associated with high or low quality.

Design/methodology/approach

We extracted the words and 2–5-word phrases most strongly associated with different quality scores in each of 34 Units of Assessment (UoAs) in the Research Excellence Framework (REF) 2021. We extracted the terms from 122,331 journal articles 2014–2020 with individual REF2021 quality scores.

Findings

The terms associating with high- or low-quality scores vary between fields but relate to writing styles, methods and topics. We show that the first-person writing style strongly associates with higher quality research in many areas because it is the norm for a set of large prestigious journals. We found methods and topics that associate with both high- and low-quality scores. Worryingly, terms associated with educational and qualitative research attract lower quality scores in multiple areas. REF experts may rarely give high scores to qualitative or educational research because the authors tend to be less competent, because it is harder to do world leading research with these themes, or because they do not value them.

Originality/value

This is the first investigation of journal article terms associating with research quality.

Comparison between two asynchronous teaching methods in an undergraduate dental course: a pilot study

BACKGROUND

Properly designed and implemented eLearning can lead to improvement of dental teaching quality. Various strategies have been proposed to increase the effectiveness of eLearning in dental education, however, there is a lack of research to assess the effectiveness of these strategies.

OBJECTIVE

To investigate dental students' learning performance and perception of a virtual flipped learning format compared to a virtual traditional learning method.

METHODS

A crossover pilot study was conducted at the College of Dentistry, Princes Sattam Bin Abdulaziz University, Saudi Arabia. Computer-generated randomization, blinded from researchers who analyzed the results, was performed to allocate 32 participants (aged 23.27 ± 0.86 years) to one of two groups. Participants in the control group were taught through the virtual traditional learning method (VTL) using live video lectures. In contrast, participants in the intervention group were taught through the virtual flipped learning method (VFL) using recorded online lectures and post-lecture virtual discussions. Learning gain and preference were measured by pre- and post-test average score differences and a modified validated survey, respectively.

RESULTS

There was no significant difference in learning performance between VFL and VTL groups (P > 0.05). However, students preferred VFL over VTL and the differences were significant among all survey items, except for the opportunity to ask questions.

CONCLUSION

Health professions educators are encouraged to carefully design online curricula with efficient learning strategies that help students improve learning performance and foster self-directed learning skills while valuing active learning in an online environment.

TRIAL REGISTRATION

NCT04692142 , 31/12/2020.

Can the quality of published academic journal articles be assessed with machine learning?

Formal assessments of the quality of the research produced by departments and universities are now conducted by many countries to monitor achievements and allocate performance-related funding. These evaluations are hugely time consuming if conducted by post-publication peer review and are simplistic if based on citations or journal impact factors. This article investigates whether machine learning could help reduce the burden of peer review by using citations and metadata to learn how to score articles from a sample assessed by peer review. An experiment is used to underpin the discussion, attempting to predict journal citation thirds, as a proxy for article quality scores, for all Scopus narrow fields from 2014 to 2020. The results show that these proxy quality thirds can be predicted with above baseline accuracy in all 326 narrow fields, with Gradient Boosting Classifier, Random Forest Classifier, or Multinomial Naïve Bayes being the most accurate in nearly all cases. Nevertheless, the results partly leverage journal writing styles and topics, which are unwanted for some practical applications and cause substantial shifts in average scores between countries and between institutions within a country. There may be scope for predicting articles scores when the predictions have the highest probability.

Peer Review

https://publons.com/publon/10.1162/qss_a_00185

Scopus 1900–2020: Growth in articles, abstracts, countries, fields, and journals

Scientometric research often relies on large-scale bibliometric databases of academic journal articles. Long term and longitudinal research can be affected if the composition of a database varies over time, and text processing research can be affected if the percentage of articles with abstracts changes. This article therefore assesses changes in the magnitude of the coverage of a major citation index, Scopus, over 121 years from 1900. The results show sustained exponential growth from 1900, except for dips during both world wars, and with increased growth after 2004. Over the same period, the percentage of articles with 500+ character abstracts increased from 1% to 95%. The number of different journals in Scopus also increased exponentially, but slowing down from 2010, with the number of articles per journal being approximately constant until 1980, then tripling due to megajournals and online-only publishing. The breadth of Scopus, in terms of the number of narrow fields with substantial numbers of articles, simultaneously increased from one field having 1000 articles in 1945 to 308 in 2020. Scopus’s international character also radically changed from 68% of first authors from Germany and the USA in 1900 to just 17% in 2020, with China dominating (25%).

Peer Review

https://publons.com/publon/10.1162/qss_a_00177