Consuming Article-Level Metrics: Observations and Lessons

How much research shared on Facebook happens outside of public pages and groups? A comparison of public and private online activity around PLOS ONE papers

Despite its undisputed position as the biggest social media platform, Facebook has never entered the main stage of altmetrics research. In this study, we argue that the lack of attention by altmetrics researchers is due, in part, to the challenges in collecting Facebook data regarding activity that takes place outside of public pages and groups. We present a new method of collecting aggregate counts of shares, reactions, and comments across the platform—including users’ personal timelines—and use it to gather data for all articles published between 2015 to 2017 in the journal PLOS ONE. We compare the gathered data with altmetrics collected and aggregated by Altmetric. The results show that 58.7% of papers shared on Facebook happen outside of public spaces and that, when collecting all shares, the volume of activity approximates patterns of engagement previously only observed for Twitter. Both results suggest that the role and impact of Facebook as a medium for science and scholarly communication has been underestimated. Furthermore, they emphasize the importance of openness and transparency around the collection and aggregation of altmetrics.

Measuring the social impact of nursing research: An insight into altmetrics

AIMS

The objectives of this study were to (a) identify nursing journal articles that provoked the most online activity and discussion and (b) assess the association between these articles' altmetric scores and publication characteristics, citation counts; and publishing journals metrics.

BACKGROUND

Altmetrics, or alternative metrics, have recently emerged as a complementary way of measuring the societal impact of research by assessing the public engagement with research output. To date, no studies have yet investigated the online attention about scientific papers published in the nursing field.

DESIGN

Integration of quantitative and qualitative synthesized evidence.

DATA SOURCES AND REVIEW METHODS

InCites Journal Citation Report was used to identify a list of nursing journals indexed in the Web of Science Core Collection. Altmetric Explorer was selected as an altmetrics harvesting tool. The search in Altmetric Explorer yielded 66,608 research outputs from 118 nursing journals. The articles with the top 100 altmetric attention score (AAS) were identified and a new search, limited to only those 100 articles, was run to produce aggregate metrics specific to those articles. The articles were also exported for thematic analysis.

RESULTS

The median AAS for the 100 articles was 248, ranging from 138 - 649. The articles were mostly discussed on Twitter, followed by news outlets and Mendeley. Articles indexed in the nursing journals category attracted low online attention compared with articles published in other health journal categories. Twitter remained the dominant source of attention over the years 2012-2018, followed distantly by news outlets. Most online attention came from the USA and the UK. Of the top 100 articles included in the study, the Journal of Advanced Nursing published the highest number of articles (N = 26; Median AAS = 179). The AAS was not significantly different between articles published in Q1 journals and those published in Q2 and Q3 journals. There was a significant relationship between articles' AASs and their citation counts on Scopus and Web of Science. Publication date was significantly related to citation counts on Scopus and Web of Science but not with AASs.

CONCLUSION

Altmetrics will likely continue to evolve alongside the rapidly expanding use of social media and online platforms. As nursing continues to strive to have our research and scholarship inform policy, translated into practice and recognized for its scientific merit, we have to remain vigilant about the best ways to disseminate the important work we are doing. Research, such as this study, will allow nursing scholars to benchmark our progress as we adapt to the changing environment for measuring impact and quality in the digital age.

General discussion of data quality challenges in social media metrics: Extensive comparison of four major altmetric data aggregators

The data collection and reporting approaches of four major altmetric data aggregators are studied. The main aim of this study is to understand how differences in social media tracking and data collection methodologies can have effects on the analytical use of altmetric data. For this purpose, discrepancies in the metrics across aggregators have been studied in order to understand how the methodological choices adopted by these aggregators can explain the discrepancies found. Our results show that different forms of accessing the data from diverse social media platforms, together with different approaches of collecting, processing, summarizing, and updating social media metrics cause substantial differences in the data and metrics offered by these aggregators. These results highlight the importance that methodological choices in the tracking, collecting, and reporting of altmetric data can have in the analytical value of the data. Some recommendations for altmetric users and data aggregators are proposed and discussed.

To what extent does the Leiden manifesto also apply to altmetrics? A discussion of the manifesto against the background of research into altmetrics

Purpose

– Hicks et al. (2015) have formulated the so-called Leiden manifesto, in which they have assembled the ten principles for a meaningful evaluation of research on the basis of bibliometric data. The paper aims to discuss this issue.

Design/methodology/approach

– In this work the attempt is made to indicate the relevance of the Leiden manifesto for altmetrics.

Findings

– As shown by the discussion of the ten principles against the background of the knowledge about and the research into altmetrics, the principles also have a great importance for altmetrics and should be taken into account in their application.

Originality/value

– Altmetrics is already frequently used in the area of research evaluation. Thus, it is important that the user of altmetrics data knows the relevance of the Leiden manifesto also in this area.

How to normalize Twitter counts? A first attempt based on journals in the Twitter Index

One possible way of measuring the broad impact of research (societal impact) quantitatively is the use of alternative metrics (altmetrics). An important source of altmetrics is Twitter, which is a popular microblogging service. In bibliometrics, it is standard to normalize citations for cross-field comparisons. This study deals with the normalization of Twitter counts (TC). The problem with Twitter data is that many papers receive zero tweets or only one tweet. In order to restrict the impact analysis on only those journals producing a considerable Twitter impact, we defined the Twitter Index (TI) containing journals with at least 80 % of the papers with at least 1 tweet each. For all papers in each TI journal, we calculated normalized Twitter percentiles (TP) which range from 0 (no impact) to 100 (highest impact). Thus, the highest impact accounts for the paper with the most tweets compared to the other papers in the journal. TP are proposed to be used for cross-field comparisons. We studied the field-independency of TP in comparison with TC. The results point out that the TP can validly be used particularly in biomedical and health sciences, life and earth sciences, mathematics and computer science, as well as physical sciences and engineering. In a first application of TP, we calculated percentiles for countries. The results show that Denmark, Finland, and Norway are the countries with the most tweeted papers (measured by TP).