How do media mention research papers? Structural analysis of blogs and news networks using citation coupling

Evaluative altmetrics: is there evidence for its application to research evaluation?

Introduction

Altmetrics have been demonstrated as a promising tool for analyzing scientific communication on social media. Nevertheless, its application for research evaluation remains underdeveloped, despite the advancement of research in the study of diverse scientific interactions.

Methods

This paper develops a method for applying altmetrics in the evaluation of researchers, focusing on a case study of the Environment/Ecology ESI field publications by researchers at the University of Granada. We considered Twitter as a mirror of social attention, news outlets as media, and Wikipedia as educational, exploring mentions from these three sources and the associated actors in their respective media, contextualizing them using various metrics.

Results

Our analysis evaluated different dimensions such as the type of audience, local attention, engagement generated around the mention, and the profile of the actor. Our methodology effectively provided dashboards that gave a comprehensive view of the different instances of social attention at the author level.

Discussion

The use of altmetrics for research evaluation presents significant potential, as shown by our case study. While this is a novel method, our results suggest that altmetrics could provide valuable insights into the social attention that researchers garner. This can be an important tool for research evaluation, expanding our understanding beyond traditional metrics.

Identifying science in the news: An assessment of the precision and recall of Altmetric.com news mention data

The company Altmetric is often used to collect mentions of research in online news stories, yet there have been concerns about the quality of this data. This study investigates these concerns. Using a manual content analysis of 400 news stories as a comparison method, we analyzed the precision and recall with which Altmetric identified mentions of research in 8 news outlets. We also used logistic regression to identify the characteristics of research mentions that influence their likelihood of being successfully identified. We find that, for a predefined set of outlets, Altmetric’s news mention data were relatively accurate (F-score = 0.80), with very high precision (0.95) and acceptable recall (0.70), although recall is below 0.50 for some news outlets. Altmetric is more likely to successfully identify mentions of research that include a hyperlink to the research item, an author name, and/or the title of a publication venue. This data source appears to be less reliable for mentions of research that provide little or no bibliometric information, as well as for identifying mentions of scholarly monographs, conference presentations, dissertations, and non-English research articles. Our findings suggest that, with caveats, scholars can use Altmetric news mention data as a relatively reliable source to identify research mentions across a range of outlets with high precision and acceptable recall, offering scholars the potential to conserve resources during data collection. Our study does not, however, offer an assessment of completeness or accuracy of Altmetric news data overall.

Predictors of societal and professional impact of Endodontology research articles. A multivariate scientometric analysis

AIM

To identify factors that are predictive of short-term professional and societal impact of research within the specialty of Endodontology, and to identify the top-10 articles that achieved the greatest societal impact and describe their characteristics.

METHODOLOGY

Research articles in the field of Endodontology published in 2019 were eligible for inclusion, with the sample identified using Medline. Following screening of titles and abstracts, bibliometric data of the identified articles were exported into a spreadsheet, where further data related to continental origin, type of article, type of journal (endodontic or non-endodontic) and grant-funding were collated, with additional data concerning presence of journal impact factor, citations, news mentions and Altmetric-tracked-mentions and scores compiled using Clarivate, Scopus, ProQuest and Altmetric Explorer, respectively. Data analysis comprised descriptive statistics, frequency distributions, cross-tabulations and un/adjusted negative binomial regression models (P<0.05).

RESULTS

The search retrieved 30,443 articles; 951 were included for analysis. Most articles originated from Asia (43.2%) with over 51.5% of articles published in endodontic journals and 80% published in a journal with an impact factor. Over three-quarters of articles were primary research, 141 were grant-funded, 165 had a news mention, 338 achieved an Altmetric Attention Score and 808 were cited. The five-highest ranked articles were mentioned in general news bulletins, five of the top-10 articles were primary research and six of the top-10 were published in endodontic journals. The highest Altmetric Attention Score and citation count were 100 and 87, respectively. Adjusted models demonstrated that the type of article, publication in a journal with an impact factor, absence of grant-funding and coverage within general news bulletins predicted the Altmetric Attention Scores (P<0.001). Article type, publication in an impact-factor journal and presence of an Altmetric Attention Score were predictive of citations (P<0.001).

CONCLUSIONS

Article type and publication in a journal with an impact factor were significant predictors of both societal and professional impact of research articles within Endodontology in the short-term. Non grant-funded research and coverage in general news bulletins achieved greater societal impact, whereas an article achieving an Altmetric Attention Score was also strongly related to professional impact.