Examining the correlation between Altmetric score and citation count in the anaesthesiology literature

A comparison of the top 500 papers in Clinical Nutrition ranked by citation and Altmetric Attention Scores

BACKGROUND AND AIMS

Citation scores (CS) have been traditionally used to measure the impact of scientific publications. Sourced from the Internet, Altmetric Attention Scores (AAS) are complementary metrics that assess how often publications are discussed and used globally. We compared by rank the top 500 papers by CS and AAS published in Clinical Nutrition with corresponding AAS and CS.

METHODS

A search for all publications in Clinical Nutrition was performed on Dimensions (https://app.dimensions.ai/discover/publication) on 3rd April 2024. Outputs were ranked according to CS and then by AAS with the top 500 in each category selected. Scores, year and type of publication were recorded. Correlation was expressed as the Spearman's rank coefficient (ϱ).

RESULTS

We identified 18,790 outputs. Within the top 500 publications ranked by CS, there was a significant weak positive correlation (ϱ = 0.235, P < 0.0001) between CS [median (IQR) 149 (116-223)] and AAS [7 (3-22)]. Ranked by AAS, there was a non-significant very weak positive correlation (ϱ = 0.072, P = 0.106) between AAS [55.5 (36-115)] and CS [42 (16.5-94.5)]. Trends remained similar when grouped by publication type. Guidelines, ranked by CS, had the highest CS and ranked by AAS, the highest CS and AAS. Publications per year, by year, ranked by CS, had a negatively skewed distribution peaking in 2012 and declined thereafter, but when ranked by AAS, had a moderately positive linear trend from 2001 to 2024 (ϱ = 0.513, P < 0.0001).

CONCLUSION

Correlation between CS and AAS was weak. Guidelines had the highest CS and AAS. CS are iterative taking years to mature while AAS are immediate.

The Future of Social Media, Anesthesiology, and the Perioperative Physician

Social media has rapidly developed in the past decade to become a powerful and influential force for patients, physicians, health systems, and the academic community. While the use of social media in health care has produced many positive changes, such as rapid dissemination of information, crowd-sourced sharing of knowledge, learning, and social interaction, social media in health care has also negative effects. Recent examples of negative impacts of social media include rapid and unchecked information dissemination leading to patient misinformation and inadvertent reputational harm for health care professionals due to engaging in controversial topics on public platforms. Members of the anesthesiology community, like other medical specialties, have rapidly adopted social media at many different levels. However, most anesthesiologists, health systems, and academic communities have little education, preparation, and guidelines on optimizing the use of social media technology while minimizing the risks of social media. Anesthesiology has been and will continue to be impacted by the forces of technology and the cultural influences of social media for the foreseeable future. The purpose of this article was to examine the recent history of social media adoption in anesthesiology and perioperative medicine, understand the current impact of social media across our specialty, and consider how the future development of technology and evolving social and cultural dynamic influences of social media will have on anesthesiology over the next quarter century.

Modern Impact of Surgery Journals: Associations Between Impact Factor, H5-Index, and Altmetric Score

INTRODUCTION

While impact factor (IF) remains the "gold standard" metric for journal quality, newer metrics are gaining popularity. These include the H5-index and journal Altmetric Attention Score (AAS). We explored the relationship between the IF, H5-index, and AAS for core general surgery (GS) and subspecialty journals.

METHODS

For all GS and subspecialty journals with a Clarivate IF, H5-index (January 1, 2017, to December 31, 2021) and journal AAS were obtained. Journal Twitter presence and activity was sourced from Twitter and the Twitter application programming interface. Spearman's correlations were assessed for numeric variables.

RESULTS

A total of 105 journals were included, around half (49/105; 46.7%) of which were core GS journals. Median IF was 2.48 and median H5-index 19. Journal IF demonstrated a strong correlation with H5-index overall (r = 0.81), though this ranged from r = 0.95 (P < 0.01) for vascular surgery to r = 0.77 (P < 0.01) for plastic surgery journals. AAS was moderately correlated with the IF and H5-index (r = 0.59 and 0.62, respectively; both P < 0.01). R² values ranging indicated that 66% of the variation in the H5-index and 35% of the variation in AAS was explained by the IF. Just over half the journals had a Twitter account (54/105; 51.4%). Journals with a Twitter account also had a significantly higher IF, H5-index, and AAS than those without a Twitter account (all P < 0.01). AAS was moderately correlated with Twitter activity (r = 0.59) and Twitter followers (r = 0.69).

CONCLUSIONS

Across GS and subspecialty journals, journal IF correlates strongly with the H5-index and moderately with AAS. However, only 35% of variation in AAS and 66% of variation in the H5-index is explained by the IF, indicating that these metrics measure unique aspects of journal quality. The future growth of surgical journals should be geared towards improving across multiple metrics, including both the conventional and the contemporary, while leveraging social media to improve readership and eventual academic impact.

Analysing the relationship between altmetric attention score (AAS) and citation: a correlational study

Purpose

The purpose of this study is to assess the value of altmetrics or other indicators, showcasing the impact of academic output, which is seen too often correlated with the citation count.

Design/methodology/approach

This study considered three reputed journals of Library and Information Science (LIS) published by Elsevier. A total of 1,164 articles were found in these journals from 2016 to 2020 and the relationships between altmetric attention scores (AAS) and citations were examined. The analysis was extended to compare the grouped data set based on percentile ranks of AAS like top 50%, top 25%, top 10% and top 1%.

Findings

Using Spearman correlation analysis, the findings reveal a positive correlation between AAS and citations with different significant levels for all articles, and articles with AAS, as well as for normalized AAS in the top 50%, top 25%, top 10% and top 1% data set. For the three journals International Journal of Information Management (IJIM), Journal of Informetrics (JIF) and Library and Information Science Research (LISR), a significant positive correlation is observed across all data sets. But an unexpected result was observed: in the case of the top 50% of articles for the IJIM and JIF showed no significant correlation but the LISR journal showed a positive correlation for the whole data set. This journal though has fewer articles in comparison to the other two.

Research limitations/implications

A source item that is highly cited may not be having high social media attention as reflected in the findings. This demarcates AAS with citations implying various factors on which these measurements are dependent. The study distinguishes these metrics lucidly. There is not a single guideline or uniformity in assessing the correlation found. But the problem is that the interpretation of the correlation strength affects the conclusion of the study. Moreover, this study will be a role model as a draft for librarians to select relevant journals for their libraries and will facilitate authors in the choice of the publication outlets for their papers, particularly concerning the journals that have both visibility and research impact.

Originality/value

The study reported devising a comprehensive tool to validate AAS as a measure of scholarly impact to include appropriate social media sources and verify its relationship with other metrics. To the best of the authors’ knowledge, this paper is the first attempt to discover the correlation between AAS and citations for the highly impactful LIS journal published by Elsevier. The empirical evidence lies in the citation and altmetric data extracted from the dimension database.

AI-SPedia: a novel ontology to evaluate the impact of research in the field of artificial intelligence

Background

Sharing knowledge such as resources, research results, and scholarly documents, is of key importance to improving collaboration between researchers worldwide. Research results from the field of artificial intelligence (AI) are vital to share because of the extensive applicability of AI to several other fields of research. This has led to a significant increase in the number of AI publications over the past decade. The metadata of AI publications, including bibliometrics and altmetrics indicators, can be accessed by searching familiar bibliographical databases such as Web of Science (WoS), which enables the impact of research to be evaluated and identify rising researchers and trending topics in the field of AI.

Problem description

In general, bibliographical databases have two limitations in terms of the type and form of metadata we aim to improve. First, most bibliographical databases, such as WoS, are more concerned with bibliometric indicators and do not offer a wide range of altmetric indicators to complement traditional bibliometric indicators. Second, the traditional format in which data is downloaded from bibliographical databases limits users to keyword-based searches without considering the semantics of the data.

Proposed solution

To overcome these limitations, we developed a repository, named AI-SPedia. The repository contains semantic knowledge of scientific publications concerned with AI and considers both the bibliometric and altmetric indicators. Moreover, it uses semantic web technology to produce and store data to enable semantic-based searches. Furthermore, we devised related competency questions to be answered by posing smart queries against the AI-SPedia datasets.

Results

The results revealed that AI-SPedia can evaluate the impact of AI research by exploiting knowledge that is not explicitly mentioned but extracted using the power of semantics. Moreover, a simple analysis was performed based on the answered questions to help make research policy decisions in the AI domain. The end product, AI-SPedia, is considered the first attempt to evaluate the impacts of AI scientific publications using both bibliometric and altmetric indicators and the power of semantic web technology.

Altmetric score, reads, and citations in paediatric cardiology: do they correlate and what do they mean?

INTRODUCTION

The primary objective of this study was to determine whether Altmetric score, number of reads, and citations for paediatric cardiology manuscripts correlate with one another. A secondary objective was to determine the extent to which factors mediated citation number for paediatric cardiology manuscripts.

METHODS

Data for this study came from manuscripts published in Cardiology in the Young (2010-2021). Data were extracted by using data shared on the journal website. Spearman's correlation analyses were conducted between manuscript reads, citations, and Altmetric score. Regression analyses were conducted with number of citations as the dependent variable and year of publication, publication type, number of reads, and Altmetric score as independent variables.

RESULTS

A total of 2642 manuscripts were included in the final analyses. Reads and citations had poor correlation (r-value 0.32); reads and Altmetric score had negligible correlation (r-value 0.26); and Altmetric score and citations had negligible correlation (r-value 0.07). Year of publication was independently associated with number of citations (β -0.95, p-value <0.01). Manuscript type was independently associated with number of citations (β 1.04, p-value <0.01). Number of reads was independently associated with citations (β 0.01, p-value <0.01). Altmetric score was independently associated with number of citations (β 0.05, p-value <0.01).

CONCLUSION

This study describes the correlation of reads, citations, and Altmetric score in manuscripts published in Cardiology in the Young, demonstrating poor correlation, at best, between these metrics. Each bibliometric index seems to represent a different phenomenon of manuscript consumption. No single bibliometric index in isolation offers ample representation of manuscript consumption.

Altmetric and bibliometric analysis of obstetrics and gynecology research: influence of public engagement on citation potential

BACKGROUND

Whether research engagement on social media and other public platforms results in increased citations in obstetrics and gynecology remains uncertain. The Altmetric Attention Score is a metric of research influence based on mentions on social media and public platforms, such as newsfeeds and Wikipedia. The correlation between Altmetric Attention Scores, absolute citation rates, and the Relative Citation Ratio (a novel metric of research engagement also based on citation rates) in obstetrics and gynecology research is uncertain.

OBJECTIVE

To evaluate the correlation between Altmetric Attention Score, absolute citation rate, and Relative Citation Ratio for articles published in obstetrics and gynecology journals from 2004 to 2019. Our second objective was to identify, characterize, and compare the 100 articles with highest Altmetric Attention Scores, the 100 most-cited articles, and the 100 articles with highest Relative Citation Ratios.

STUDY DESIGN

We performed a cross-sectional altmetric and bibliometric study of all obstetrics and gynecology articles indexed in the National Institutes of Health Open Citation Collection from 2004 to 2019. Articles were included if they were published in obstetrics and gynecology journals according to InCites Journal Citation Reports indexing. Citation data, including citation numbers and Relative Citation Ratios, were downloaded on March 20, 2021 and merged with altmetric data from the Altmetric Explorer on the basis of each article's unique PubMed identification number. We assessed correlation between Altmetric Attention Scores and number of citations and Altmetric Attention Scores and Relative Citation Ratios by calculating the Pearson correlation coefficient. The 100 articles with highest Altmetric Attention Scores, the 100 most-cited articles, and the 100 articles with highest Relative Citation Ratios were characterized and compared using means (standard deviations) and mean differences (95% confidence intervals).

RESULTS

There were 156,592 articles published in 82 obstetrics and gynecology journals and indexed in the National Institutes of Health Open Citation Collection between 2004 and 2019. The correlation coefficient was 0.18 (95% confidence interval, 0.17-0.19) for Altmetric Attention Scores vs number of citations and 0.10 (95% confidence interval, 0.09-0.11) for Altmetric Attention Scores vs Relative Citation Ratios. There was no overlap among the 100 articles on the highest Altmetric Attention Score list and the 100 most-cited list, and there was minimal overlap among the 100 articles on the highest Altmetric Attention Score list and the 100 highest Relative Citation Ratio list (98 unique articles on each list). Articles with highest Altmetric Attention Scores generated substantially more engagement on social media and other public platforms than most-cited articles (mean Altmetric Attention Score, 763.1 [standard deviation, 520.8] vs 49.9 [standard deviation, 81.6]; mean difference, -713.2 [95% confidence interval, -819.9 to -606.6]) and highest Relative Citation Ratio articles (mean, 116.2 [standard deviation, 415.9]; mean difference, -661.5 [95% confidence interval, -746.2 to -576.9]). In contrast, the articles with highest Altmetric Attention Scores generated far fewer citations than most-cited articles (mean, 39.7 [standard deviation, 47.6] vs 541.8 [standard deviation, 312.8]; mean difference, 502.0 [95% confidence interval, 439.0-565.0]) and highest Relative Citation Ratio articles (mean, 458.9 [standard deviation, 363.5]; mean difference, 427.7 [95% confidence interval, 353.8-501.6]). Nearly half of articles with highest Altmetric Attention Scores were basic/translational studies, often about menopause and environmental factors impacting fertility, whereas most-cited articles and articles with highest Relative Citation Ratios were more likely to be reviews and consensus statements, respectively, often about placentation and polycystic ovary syndrome, respectively. Articles with highest Altmetric Attention Scores were more likely to be published as open-access.

CONCLUSION

There seems to be weak short-term correlation between Altmetric Attention Scores and citation rates. Further study is warranted to ascertain whether there may be long-term correlation between alternative metrics and citation rates in obstetrics and gynecology.

Social media attention and citations of published outputs from re-use of clinical trial data: a matched comparison with articles published in the same journals

BACKGROUND

Data-sharing policies in randomized clinical trials (RCTs) should have an evaluation component. The main objective of this case-control study was to assess the impact of published re-uses of RCT data in terms of media attention (Altmetric) and citation rates.

METHODS

Re-uses of RCT data published up to December 2019 (cases) were searched for by two reviewers on 3 repositories (CSDR, YODA project, and Vivli) and matched to control papers published in the same journal. The Altmetric Attention Score (primary outcome), components of this score (e.g. mention of policy sources, media attention) and the total number of citations were compared between these two groups.

RESULTS

89 re-uses were identified: 48 (53.9%) secondary analyses, 34 (38.2%) meta-analyses, 4 (4.5%) methodological analyses and 3 (3.4%) re-analyses. The median (interquartile range) Altmetric Attention Scores were 5.9 (1.3-22.2) for re-use and 2.8 (0.3-12.3) for controls (p = 0.14). No statistical difference was found on any of the components of in the Altmetric Attention Score. The median (interquartile range) numbers of citations were 3 (1-8) for reuses and 4 (1 - 11.5) for controls (p = 0.30). Only 6/89 re-uses (6.7%) were cited in a policy source.

CONCLUSIONS

Using all available re-uses of RCT data to date from major data repositories, we were not able to demonstrate that re-uses attracted more attention than a matched sample of studies published in the same journals. Small average differences are still possible, as the sample size was limited. However matching choices have some limitations so results should be interpreted very cautiously. Also, citations by policy sources for re-uses were rare.

TRIAL REGISTRATION

Registration: osf.io/fp62e.

Examining the correlation between Altmetric Attention Score and citation count in the gynecologic oncology literature: Does it have an impact?

•

The AAS is a useful tool to predict future citation counts for oncology publications in gynecologic oncology.

•

A hybrid metric using impact factor and AAS may help to predict academic influence on a broader scale.

•

In addition to impact factor, AAS may be used by journals to enhance interest in subscriptions.

•

AAS may be another metric used to assess the quality of publications.

We sought to determine the correlation between Altmetric Attention Score and traditional bibliometrics in the gynecologic oncology literature. We identified the 10 most-cited gynecologic oncology articles from 5 major gynecology journals and 10 major “oncology” journals that publish on gynecologic oncology during 2014, 2016, and 2018. Article citation count and Altmetric Attention Score (AAS), as well as journal impact factor (IF) and date of Twitter account development were recorded. Pearson's correlation coefficient was used to describe the relationship between AAS, tweets, IF, and citation count. While the median citation counts significantly decreased for the top-cited gynecologic oncology articles from 2014 to 2018 (p < 0.001), the corresponding median AAS continuously increased during this period (p = 0.008). For articles published in 2014 and 2018, there was a strong positive relationship between the median citation count and the median AAS (2014: r = 0.92; 2018: r = 0.97), as well as between the IF (r = 0.78 and r = 0.89, respectively); these correlations were moderate to weak in 2016 (r = 0.5 and r = 0.41, respectively). There was a continuously increasing strong positive correlation from 2014 to 2018 between journal IF and median AAS (2014: r = 0.75; 2016: r = 0.82; 2018: r = 0.92). Gynecologic oncology articles published in higher impact journals are associated with increased social media visibility and attention. Our data support the idea that early online attention scores, like the AAS, might be useful for predicting future citation counts for oncology publications in general and gynecologic oncology specifically.

Changes in Social Media Impact of the Radiological Literature During the Covid-19 Pandemic

PURPOSE

The objective of this study was to determine how the social media impact of the radiological literature has changed during the Covid-19 pandemic.

METHODS

Altmetric Attention scores were collected for all articles in five leading radiology journals over a 5-year period ending in June 2020, and temporal smoothing yielded the filtered Altmetric Attention (fAA) score. Natural language processing methods were used to label articles with major topic areas. A forecasting model was used to identify periods of outlier behavior in the fAA score aggregated across all journals, for each journal individually, and stratified by article topic area. The distributions of fAA scores prior to the onset of the pandemic were statistically compared to those during the pandemic. For journals exhibiting increased fAA scores, the frequency distributions of articles not related to Covid-19 was compared to that prior to the pandemic.

RESULTS

During the pandemic, we found sustained outliers and statistically significant increases in the aggregate fAA score across all five journals, as well as for Radiology, American Journal of Roentgenology, and Academic Radiology individually. Articles related to Covid-19, thoracic imaging, and radiology education also experienced significantly increased fAA scores during the pandemic period. We did not find significantly decreased rates of publication of non-Covid articles in the journals experiencing elevated fAA scores.

CONCLUSION

Social media engagement with the radiological literature significantly increased during the Covid-19 pandemic. This preferentially affected certain journals and articles addressing specific topics, reflecting the intense public interest in the diagnosis and treatment of Covid-19.