The Correlation between Altmetric Score and Traditional Bibliometrics in Orthopaedic Literature

A comparison of the top 500 papers in Clinical Nutrition ranked by citation and Altmetric Attention Scores

BACKGROUND AND AIMS

Citation scores (CS) have been traditionally used to measure the impact of scientific publications. Sourced from the Internet, Altmetric Attention Scores (AAS) are complementary metrics that assess how often publications are discussed and used globally. We compared by rank the top 500 papers by CS and AAS published in Clinical Nutrition with corresponding AAS and CS.

METHODS

A search for all publications in Clinical Nutrition was performed on Dimensions (https://app.dimensions.ai/discover/publication) on 3rd April 2024. Outputs were ranked according to CS and then by AAS with the top 500 in each category selected. Scores, year and type of publication were recorded. Correlation was expressed as the Spearman's rank coefficient (ϱ).

RESULTS

We identified 18,790 outputs. Within the top 500 publications ranked by CS, there was a significant weak positive correlation (ϱ = 0.235, P < 0.0001) between CS [median (IQR) 149 (116-223)] and AAS [7 (3-22)]. Ranked by AAS, there was a non-significant very weak positive correlation (ϱ = 0.072, P = 0.106) between AAS [55.5 (36-115)] and CS [42 (16.5-94.5)]. Trends remained similar when grouped by publication type. Guidelines, ranked by CS, had the highest CS and ranked by AAS, the highest CS and AAS. Publications per year, by year, ranked by CS, had a negatively skewed distribution peaking in 2012 and declined thereafter, but when ranked by AAS, had a moderately positive linear trend from 2001 to 2024 (ϱ = 0.513, P < 0.0001).

CONCLUSION

Correlation between CS and AAS was weak. Guidelines had the highest CS and AAS. CS are iterative taking years to mature while AAS are immediate.

Five decades of research on immediate implant therapy: A modern bibliometric network analysis via Altmetric and level of evidence mapping

AIM

To conduct a bibliometric network analysis to explore the research landscape of immediate implant placement (IIP) and provide insights into its trends and contributors.

MATERIALS AND METHODS

The Scopus database was utilized as the bibliographic source, and a search strategy was implemented to identify relevant research articles. Various bibliometric parameters were extracted, including publication year, journal, authors, citations, and funding. The analysis involved examining authorship patterns, international collaborations, level of evidence, Altmetric data, and funding analysis.

RESULTS

We identified a steady annual growth rate of 6.49% in IIP research. The top three countries contributing to research output were the USA, Italy, and China. Prolific authors were identified based on publication and citation metrics. International collaborations among different countries were observed. The level of evidence analysis revealed that over 30% of the articles fell into higher levels of evidence (levels 1 and 2). Altmetric data analysis indicated no significant correlations between citation counts and Altmetric Attention Score (AAS), and conversely a significant association with Mendeley readers count. Funding and open access did not significantly impact the bibliometric indices of the papers.

CONCLUSIONS

The focus of research on IIP has been evolving as indicated by an exponential growth rate in this study. Only approximately 16% of the articles fit into level 1 evidence, therefore, emphasizing on higher quality level research study shortage in this field. Modern indices can be used as new bibliometric indicators as they also cover social media and online attention scores.

The Top Ranked 101 Articles in Pediatric Surgical Journals from an Altmetric Perspective

INTRODUCTION

 In the era of scientific digitalization, online media platforms gain increasing popularity to accomplish research output awareness. The Altmetric Attention Score AAS weights these online mentions based on a privy algorithm. We aimed to characterize the top 100 articles with the highest (AAS) published in pediatric surgery journals.

MATERIALS AND METHODS

 Publications from six core pediatric surgery journals were retrieved from www.altmetric.com in January 2023 and ranked by their AAS. The top 101 publications were analyzed for their bibliometric measures, study design, and quality as well as online media mentions.

RESULTS

 The top 101 AAS articles were published between 1974 and 2022, preferentially from the United States (64%) and mainly in Journal of Pediatric Surgery (73%), followed by Journal of Pediatric Surgery Case Reports, Pediatric Surgery International, Seminars in Pediatric Surgery, and European Journal of Pediatric Surgery. Their AAS ranged between 21 and 389 (median: 33), with Twitter/X being mostly responsible for online mentions (n = 2,189; 75%). The number of citations in peer-reviewed journals ranged between 0 and 358 (median: 16) and did not correlate to AAS. Retrospective study design (33%) with low evidence level IV (43%) dominated.

CONCLUSION

 The Journal of Pediatric Surgery is the main source of high-profile AAS publications in pediatric surgery. The altmetric popularity of articles is predominantly achieved by their propagation via X, irrespective of the study quality and recognition in the scientific community. Thus, active "twitterism" may play the key role to reach high AAS scores.

The bibliometric and altmetric analysis of chronic traumatic encephalopathy research: how great is the impact?

Background

The study of chronic traumatic encephalopathy (CTE) has received great attention from academia and the general public. This study aims to analyze the research productivity on CTE and investigate the most discussed articles in academia and the general public by conducting bibliometric and altmetric analyses.

Methods

Data of articles were obtained from the Web of Science Core Databases and Altmetric Explore. VOSviewer and CiteSpace software were used to analyze and visualize the articles. The correlation between Altmetric attention scores (AAS) and citation counts were assessed by Spearman correlation coefficient.

Results

788 publications of CTE were eventually gathered and analyzed, and 100 articles with highest citation counts (Top-cited) and 100 articles with highest AASs (Top-AAS) were then identified. The keywords density map showed both the general public and the scientists were particularly interested in the risk factors and pathology of CTE, and scientists were interested in the causes and characteristics of neurodegenerative diseases while the public became increasingly concerned about the detection and prevention of CTE. By examining the shared characteristics of the 44 articles (High-High articles) that overlapped between Top-cited and Top-AAS articles, we identified certain traits that may potentially contribute to their high citation rates and high AASs. Besides, significant positive correlations with varied strength between AAS and citation were observed in the 788 articles, Top-cited, Top-AAS and High-High datasets.

Conclusion

This study is the first to link bibliometric and altmetric analyses for CTE publications, which may provide deeper understanding of the attention of the scientists and the general public pay to the study of CTE, and offer some guidance and inspiration for future CTE in the selection of research topics and directions.

Publishing on Topical Subjects in Total Joint Arthroplasty Is Associated With Increased Social Media Attention

BACKGROUND

Social media platforms are often used for research dissemination and collaboration. Given the increased prevalence of online-only publications, understanding what drives research dissemination is important. Here, we analyzed factors associated with increased social media attention among peer-reviewed publications in total knee arthroplasty, total hip arthroplasty, and unicompartmental knee arthroplasty.

METHODS

We analyzed publications about total knee arthroplasty, total hip arthroplasty, or unicompartmental knee arthroplasty from 2010 to 2022 using a national database. We analyzed a weighted count of social media mentions, using negative binomial regressions adjusting for days since publication. Publications on "hot topics" in arthroplasty were examined including navigation/robotics, COVID-19, race/ethnicity, body mass index, and reimbursement. There were 9,542 publications included, 4,216 (44%) were open access (OA), 338 (3.5%) included navigation, 32 (0.34%) discussed race/ethnicity, 20 (0.2%) discussed COVID-19, 3,840 (40%) were randomized studies, 30 (0.3%) discussed reimbursement, and 2,867 (30%) were in top-10 orthopaedic journals.

RESULTS

Factors associated with higher weighted score included studies about COVID-19 (50 versus 6.0, P < .001), race/ethnicity (15.8 versus 6.0, P < .001), OA status (6.3 versus 5.8, P = .001), and randomized studies (6.5 versus 5.7, P < .001). Studies from top-10 journals had a lower score (5.8 versus 6.2, P = .025), as did studies about body mass index (3.4 versus 6.1, P = .001). Studies about navigation and reimbursement did not have significantly different scores.

CONCLUSIONS

Studies on COVID-19, race/ethnicity, randomized studies, and OA publication were associated with increased social media while those in top-10 orthopaedic journals had lower scores.

LEVEL OF EVIDENCE

Level IV, Prognostic Study.

The Most Impactful Articles on the Shoulder Labrum From the United States or Europe, and Decreased Level of Evidence Is Associated With Increased Altmetric Attention Score

Purpose

To use the top 100 articles pertaining to the shoulder labrum to understand the impact that social medial platforms have on the dissemination of shoulder research and to highlight bibliometric factors associated with Altmetric Attention Scores (AAS) to offer insight into the impact that social media platforms have on the dissemination, attention, and citation of shoulder research publications.

Methods

This was a cross-sectional study. In January 2023, the Altmetric database was searched using the PubMed Medical Subject Headings terms "shoulder labrum." Articles with the greatest AAS were screened to exclude other topics unrelated to the labrum of the shoulder. The top 100 articles that met inclusion criteria were used in the final analysis. Bibliometric factors pertaining to each study were collected for further analysis of article characteristics in accordance with previous studies.

Results

The Altmetric Database query yielded 619 studies. The top 100 articles with highest AAS were identified, mean Attestation Score was 24.85 ± 55.51, with a range of 7 to 460. The included articles represented 35 journals, with 57 articles attributed to 3 journals: American Journal of Sports Medicine (AJSM; 29%), Arthroscopy: The Journal of Arthroscopic & Related Surgery (Arthroscopy; 19%), and the Journal of Shoulder and Elbow Surgery (JSES; 9%). There was a significant increase in AAS for every decrease in the numerical Level of Evidence value for a study (P = .011) but no association between score and citation rate (P > 005).

Conclusions

Top articles on the shoulder labrum, as defined by high AAS score, are most commonly original clinical research published in 1 of 3 sports medicine journals and performed in the United States or Europe. A decreased numerical Level of Evidence is associated with an increase in AAS score, but there is no association between AAS score and citation rate.

Clinical Relevance

The increasing amount of science and health information shared freely through open-access journals, online servers, and numerous social media channels makes it difficult to measure the impact of research. Using measures such as the Altmetric Attention Score, in isolation or addition to measures of researcher or journal impact, has the potential to provide comprehensive information about the impact of research in the modern world.

Comparing Orthopaedic Randomized Control Trials Published in High-Impact Medical and Orthopaedic Journals

INTRODUCTION

Orthopaedic studies published in high-impact medical journals are often believed to have a high prevalence of negative or neutral results and possess methodological characteristics that may bias toward nonsurgical treatments. The purpose of this study was to compare study characteristics, methodologic quality, exposure, and outcome direction among orthopaedic randomized control trials (RCTs) published in high-impact medical and orthopaedic journals and to identify study attributes associated with greater impact.

METHODS

RCTs published between January 2010 and December 2020 in the five medical journals and 10 orthopaedic journals with the highest 5-year impact factors were analyzed. Inclusion criteria were RCTs reporting on orthopaedic surgical intervention compared with nonsurgical or less-invasive surgical procedures. Study characteristics, methodologic quality (Jadad scale), outcomes, and altmetric data were collected. Primary outcomes were categorized as positive (favoring surgical/more-extensive surgery), negative (favoring nonsurgical/less-extensive surgery), or neutral.

RESULTS

One hundred twenty-eight RCTs were analyzed; 26 from medical and 102 from orthopaedic journals. Studies published in medical journals included more authors ( P < 0.001), larger sample sizes ( P < 0.001), more institutions ( P < 0.001), and more often received funding ( P < 0.001). The average Jadad scale did not significantly differ between journals ( P = 0.14). The direction of the primary study outcome did not differ between journals ( P = 0.22). Average AAS and annual citation rates were higher in RCTs published in medical journals ( P < 0.001). Publication in a medical journal was the only covariate associated with higher annual citation rates ( P < 0.001) and AAS ( P < 0.001) on multivariable analyses.

DISCUSSION

High-impact medical journals do not publish orthopaedic RCTs with negative or neutral findings at a rate that significantly differs from orthopaedic journals. However, the higher impact and digital coverage of the studies published in medical journals may disproportionally influence the practices of nonorthopaedic providers. Raising awareness of critical findings published in orthopaedic journals may be particularly important for improving healthcare policies and orthopaedic referral patterns for musculoskeletal problems.

Altmetric analysis of the most mentioned articles online in the orthopaedic literature

Social media has become a valuable tool for the distribution of scientific content. This investigation examined the most mentioned articles in orthopaedics and identified factors concurrent with greater dissemination. Altmetric Attention Score (AAS), which measures the online attention of an article, was examined for 88,683 publications in the top 50 orthopaedic journals by 2021 impact factor. Comparative and correlational analyses were completed. Overall, average AAS and citation count of the 100 most mentioned articles were 606.7 (SD ± 272.0) and 67.55 (SD ± 98.01), respectively. These articles had primarily male first authors (75%), were produced predominantly in the United States (60%), concerned the subspecialty of sports medicine (28%), and were cross-sectional analyses (22%). There were significant differences in AAS between publications (p < 0.05) by the degree of collaboration and the type of article. Such insights can guide authors when considering avenues to increase the impact of their research.

Article-level metrics: A new approach to quantify reach and impact of published research

A spectrum of measuring tools are available to evaluate the impact of published literature and the journals they are published in. Journal Level Metrics (JLM) such as Journal Impact Factor (JIF) or CiteScore assess the reputation of peer-reviewed journals based on citation analysis. Whereas, Article Level Metrics (ALM) quantify the importance, reach and impact of a particular article, and are a new approach to quantifying the reach and impact of published research. Traditionally JLM has served as a proxy for an individual publication's significance, however, the introduction of contemporary and evolution of Alternative metrics measuring digital or societal influence of a particular article has gained popularity in recent times. These metrics help in rapid dissemination of research, development of newer research strategies and individual academic progress. We highlight the characteristics and importance of currently available ALM, and the newer ones influenced by social media, digital media and Open Access publishing models.

Impact of Large Database Studies on Orthopedic Surgery Literature: Are We Advancing the Field?

Background: While database studies have become more prevalent in the literature, there is concern over their value. In addition, the questions they are suitable to answer are limited. Questions/Purposes: We sought to determine the incidence of database studies in the orthopedic literature and in each subspecialty. In addition, we wanted to assess the impact of database studies on the literature by determining whether citations and Altmetric Attention Scores (AAS) varied by study type (studies using internal or external databases and those not using databases). Methods: We searched PubMed for articles published in impactful orthopedic surgery journals in the year 2018. All articles were discoverable on the Altmetric explorer portal database. Impact was determined by journal impact factor. Study design, subspecialty, number of citations, and AAS were obtained. Univariable analyses were conducted between study type, demographic variables, and the outcome of either citation count or AAS. Multivariable analyses were performed to identify independent predictors of the primary outcomes. Subgroup analyses were performed to differentiate the impact of external and internal database studies compared with non-database studies. Results: A total of 2684 total articles were eligible for inclusion. Of these, 366 studies (13.6%) were database studies. Hip and knee articles had the greatest incidence of database studies. Database studies had significantly more citations (5.9 vs 4.0) and significantly higher AAS (12.8 vs 11.3) compared with non-database studies. External database studies had significantly more citations (6.7 vs 4.8) and significantly higher AAS (14.0 vs 10.7) than internal database studies. Internal database studies had higher traditional citation counts but similar AAS to non-database studies. Conclusions: In 2018, database studies in well-reputed orthopedic journals had a greater number of citations but similar AAS compared with non-database studies. Further studies are warranted.

Relationship of Altmetric Attention Score, Twitter Performance, and Dimensions Badge Value With Traditional Metrics in Top-Cited Anterior Cruciate Ligament Research Studies

Background

Alternative citation metrics-such as the Altmetric Attention Score (AAS), number of tweets (TN), and dimensions badge value (DBV)-are emerging as new options for assessing the value of scientific works.

Purpose

To analyze the AAS of highly cited articles on the anterior cruciate ligament (ACL) and to assess the relationship between alternative and traditional metrics such as journal rankings and article citation performance.

Study Design

Cross-sectional study.

Methods

A search was conducted using the Web of Science Core Collection of databases with "anterior cruciate ligament" as the search term. Full-text articles published between 2011 and 2021 were reviewed, and the top 100 cited articles were determined. The articles were analyzed by publication year, study design, research topic, journal impact factor, journal h-index, number of total citations (TC), recent citations (RC) (ie, citations in the latest 2 years), and average citation per year (ACY), as well as AAS, TN, and DBV.

Results

For articles in the top 100 list, the median TC was 160 (interquartile range [IQR], 117-561) and the median AAS was 24.50 (IQR, 1-730). A higher AAS score was achieved by articles on return to sports and anterolateral ligament (P < .05). The AAS and TN were significantly and positively correlated with the RC (r = 0.459 and P = .001; r = 0.438 and P = .001, respectively) and ACY (r = 0.363 and P = .001; r = 0.393 and P = .001, respectively).

Conclusion

Alternative metrics were linked to traditional metrics but were not a direct representation of bibliometrics. The AAS was not correlated with TC numbers in the ACL research. Higher AAS and Twitter popularity of an article were related to receiving high-volume RC performance. The DBV represented a stronger correlation with traditional metrics than the AAS.

The Correlation between Altmetric Attention Score and Traditional Bibliometrics in Top Nursing Journal Articles

Background. Altmetric Attention Score (AAS) is a quantitative measurement of the online impact of research and has a potential correlation with traditional bibliometrics. However, the correlation for nursing journal articles is still unknown. The objective of the study was to analyze the correlation between AAS and traditional bibliometrics in the top nursing journal articles. Materials and Methods. Articles published in top nursing journals (the journals with the top 20 5-year impact factors) from 2010 to 2019 were included. The correlations between AAS and citations, AAS and Relative Citation Ratio (RCR) score, AAS and Category Normalized Citation Impact (CNCI) value, and AAS and impact factors were analyzed. Statistical analyses were performed using Stata 25.0 software. Results. A total of 15,212 journal articles were included in the study. Very weak correlations were found between AASs and citations [0.124 (95% CI, 0.108–0.14)], AASs and RCRs [0.26 (95% CI, 0.244–0.275)], and AASs and CNCIs [0.207 (95% CI, 0.192–0.223)]. The weak correlations were also found between AASs and impact factors in several journals. The weak correlations between AASs and citations, AASs and CNCIs, and AASs and RCRs were also found for most journals based on subgroup analysis. Conclusions. There is very weak correlations between AASs and traditional bibliometrics in top nursing journal articles. More studies should be conducted to assess how AAS influence bibliometrics, and how they can help manage nursing journal articles and research.

Open Access Articles Garner Increased Social Media Attention and Citation Rates Compared With Subscription Access Research Articles: An Altmetrics-Based Analysis

BACKGROUND

To better understand the research impact on social media, alternative web-based metrics (Altmetrics) were developed. Open access (OA) publishing, which allows for widespread distribution of scientific content, has become increasingly common in the medical literature. However, the relationship between OA publishing and social media impact remains unclear.

PURPOSE

To compare social media attention and citation rates between OA and subscription access (SA) research articles within the orthopaedic and sports medicine literature.

STUDY DESIGN

Cross-sectional study.

METHODS

Articles published as either OA or SA in 5 high-impact hybrid orthopaedic journals between January 2019 and December 2019 were analyzed. The primary outcome was the Altmetric Attention Score (AAS), a validated measure of social media attention. Secondary outcomes included citation rates, article characteristics, and the number of shares on social media. Independent t tests and chi-square analyses were used to compare outcomes between OA and SA articles. A multivariable linear regression analysis was performed to determine the association between article type and AAS while controlling for bibliometric characteristics.

RESULTS

A total of 2143 articles (246 OA articles, 11.5%; 1897 SA articles, 88.5%) were included. The mean AAS among all OA articles was 62.4 ± 184.6 (range, 0-2032), whereas the mean AAS among all SA articles was 18.4 ± 109.8 (range, 0-3425), representing a statistically significant difference (P < .001). The mean citation rate among OA articles was significantly higher (17.0 ± 22.5; range, 0-139) than that of SA articles (8.6 ± 13.4; range, 0-169) (P < .001). Multivariable linear regression analysis demonstrated that OA status (β = 15.15; P = .044), number of institutions (β = 2.13; P = .023), studies classified as epidemiological investigations (β = 107.40; P < .001), and disclosure of a conflict of interest (β = -11.18; P = .032) were significantly associated with a higher AAS.

CONCLUSION

OA articles resulted in significantly greater AAS and citations in comparison with SA articles. Articles published through the OA option in hybrid journals as well as those with a higher number of institutions, those that disclosed a conflict of interest, and those classified as epidemiological investigations were positively associated with greater AAS in addition to a greater number of citations. The potential for more extensive research dissemination inherent in the OA option may therefore translate into greater reach and social media attention.

Deep Impact: A Study on the Impact of Data Papers and Datasets in the Humanities and Social Sciences

The humanities and social sciences (HSS) have recently witnessed an exponential growth in data-driven research. In response, attention has been afforded to datasets and accompanying data papers as outputs of the research and dissemination ecosystem. In 2015, two data journals dedicated to HSS disciplines appeared in this landscape: Journal of Open Humanities Data (JOHD) and Research Data Journal for the Humanities and Social Sciences (RDJ). In this paper, we analyse the state of the art in the landscape of data journals in HSS using JOHD and RDJ as exemplars by measuring performance and the deep impact of data-driven projects, including metrics (citation count; Altmetrics, views, downloads, tweets) of data papers in relation to associated research papers and the reuse of associated datasets. Our findings indicate: that data papers are published following the deposit of datasets in a repository and usually following research articles; that data papers have a positive impact on both the metrics of research papers associated with them and on data reuse; and that Twitter hashtags targeted at specific research campaigns can lead to increases in data papers’ views and downloads. HSS data papers improve the visibility of datasets they describe, support accompanying research articles, and add to transparency and the open research agenda.

AI-SPedia: a novel ontology to evaluate the impact of research in the field of artificial intelligence

Background

Sharing knowledge such as resources, research results, and scholarly documents, is of key importance to improving collaboration between researchers worldwide. Research results from the field of artificial intelligence (AI) are vital to share because of the extensive applicability of AI to several other fields of research. This has led to a significant increase in the number of AI publications over the past decade. The metadata of AI publications, including bibliometrics and altmetrics indicators, can be accessed by searching familiar bibliographical databases such as Web of Science (WoS), which enables the impact of research to be evaluated and identify rising researchers and trending topics in the field of AI.

Problem description

In general, bibliographical databases have two limitations in terms of the type and form of metadata we aim to improve. First, most bibliographical databases, such as WoS, are more concerned with bibliometric indicators and do not offer a wide range of altmetric indicators to complement traditional bibliometric indicators. Second, the traditional format in which data is downloaded from bibliographical databases limits users to keyword-based searches without considering the semantics of the data.

Proposed solution

To overcome these limitations, we developed a repository, named AI-SPedia. The repository contains semantic knowledge of scientific publications concerned with AI and considers both the bibliometric and altmetric indicators. Moreover, it uses semantic web technology to produce and store data to enable semantic-based searches. Furthermore, we devised related competency questions to be answered by posing smart queries against the AI-SPedia datasets.

Results

The results revealed that AI-SPedia can evaluate the impact of AI research by exploiting knowledge that is not explicitly mentioned but extracted using the power of semantics. Moreover, a simple analysis was performed based on the answered questions to help make research policy decisions in the AI domain. The end product, AI-SPedia, is considered the first attempt to evaluate the impacts of AI scientific publications using both bibliometric and altmetric indicators and the power of semantic web technology.

Altmetric score, reads, and citations in paediatric cardiology: do they correlate and what do they mean?

INTRODUCTION

The primary objective of this study was to determine whether Altmetric score, number of reads, and citations for paediatric cardiology manuscripts correlate with one another. A secondary objective was to determine the extent to which factors mediated citation number for paediatric cardiology manuscripts.

METHODS

Data for this study came from manuscripts published in Cardiology in the Young (2010-2021). Data were extracted by using data shared on the journal website. Spearman's correlation analyses were conducted between manuscript reads, citations, and Altmetric score. Regression analyses were conducted with number of citations as the dependent variable and year of publication, publication type, number of reads, and Altmetric score as independent variables.

RESULTS

A total of 2642 manuscripts were included in the final analyses. Reads and citations had poor correlation (r-value 0.32); reads and Altmetric score had negligible correlation (r-value 0.26); and Altmetric score and citations had negligible correlation (r-value 0.07). Year of publication was independently associated with number of citations (β -0.95, p-value <0.01). Manuscript type was independently associated with number of citations (β 1.04, p-value <0.01). Number of reads was independently associated with citations (β 0.01, p-value <0.01). Altmetric score was independently associated with number of citations (β 0.05, p-value <0.01).

CONCLUSION

This study describes the correlation of reads, citations, and Altmetric score in manuscripts published in Cardiology in the Young, demonstrating poor correlation, at best, between these metrics. Each bibliometric index seems to represent a different phenomenon of manuscript consumption. No single bibliometric index in isolation offers ample representation of manuscript consumption.

Effect of Altmetric score on manuscript citations: A randomized-controlled trial

BACKGROUND

Alternative metrics to traditional, citation-based metrics are increasingly being used. These are complementary to traditional metrics, like downloads and citations, and give information on how often a given journal article is discussed and used in professional (reference managers) and social networks, such as mainstream media and Twitter. Altmetrics is used in most journals and is available in all indexed headache medicine journals. Whether Altmetrics have an input on traditional, citation-based metrics or whether it is a stand-alone metric system is not clear. Actively promoting a paper through media channels will probably increase the Altmetric score but the question arises whether this will also increase citations and downloads of this individual paper.

METHODS

Focusing on this point we performed a randomized study in order to test the hypothesis that a promotion intervention would improve citations and other science metric scores. We selected 48 papers published in Cephalalgia from July 2019 to January 2020 and randomized them to either receive an active promotion through social media channels or not. The primary outcome used was the difference between mean article citations with versus without intervention 12 months after the intervention period.

RESULTS

The results show that the alternative metrics significantly increased for those papers randomly selected to receive an intervention compared to those who did not. This effect was observed in the first 12 months, right after the boosting strategy was performed. The higher promoted paper diffusion in social media lead to a significantly higher number of citations and downloads.

CONCLUSION

Further promotion strategies should be studied in order to tailor the best cost-benefit intervention.

Correlation Between Altmetric Attention Scores and Citations for Articles Published in High-Impact Factor Ophthalmology Journals From 2018 to 2019

Importance

The Altmetric attention score (AAS) provides new information to gauge the impact of a research article not found through typical metrics, such as impact factor or citation counts.

Objective

To explore the association between AAS and common impact markers among high-impact ophthalmology journals from 2018 to 2019.

Design, Setting, and Participants

All articles published in the American Journal of Ophthalmology (AJO), JAMA Ophthalmology (JAMAO), and Ophthalmology (OPH) from January 1, 2018, to December 31, 2019, were collected for this cross-sectional study. Excluded articles were those missing Altmetric data at the time of data collection. The AAS and associated social media impact for each article were collected with the AAS calculator bookmarklet. Spearman rank correlation analyses and analysis of variance tests were conducted to assess differences in various metrics between AJO, JAMAO, and OPH. The study included articles published of all document types (article, conference paper, editorial, erratum, letter, note, retracted, review, and short survey) and access status (open access and not open access).

Main Outcomes and Measures

The correlation between citation counts and Altmetric variables including AAS.

Results

A total of 2467 articles were published in the study period. There were 351 articles excluded owing to missing Altmetric data. Of the 2116 articles included in the analysis, 1039 (49.1%) were published in 2018, and 1077 (50.9%) were published in 2019; the mean number of citations was 8.8 (95% CI, 7.9-9.6) for AJO, 6.2 (95% CI, 5.3-7.1) for JAMAO, and 15.1 (95% CI, 13.3-17.0) for OPH. The mean AAS was 4.5 (95% CI, 3.3-5.6) for AJO (723 publications), 27.4 (95% CI, 22.1-32.8) for JAMAO (758 publications), and 15.1 (95% CI, 10.9-19.3) for OPH (635 publications). Citation rate was moderately correlated with AAS across the 3 journals (AJO, ρ = 0.39; P < .001; JAMAO, ρ = 0.41; P < .001; OPH, ρ = 0.40; P < .001), as well as minimally or moderately correlated with engagement or mention by Facebook posts (AJO, ρ = 0.38; P < .001; JAMAO, ρ = 0.24; P < .001; OPH, ρ = 0.20; P < .001), news outlet reporting (AJO, ρ = 0.12; P < .001; JAMAO, ρ = 0.38; P < .001; OPH, ρ = 0.19; P < .001), and Twitter posts (AJO, ρ = 0.40; P < .001; JAMAO, ρ = 0.38; P < .001; OPH, ρ = 0.42; P < .001).

Conclusions and Relevance

Results of this cross-sectional study suggest that citation rate has a moderate positive correlation with online and social media sharing of research in ophthalmology literature. Peer-reviewed journals may increase their reach and impact by sharing their literature through social media and online platforms.

Predicting Citation Impact from Altmetric Attention in Clinical and Translational Research: Do Big Splashes Lead to Ripple Effects?

Publications are commonly used to evaluate the productivity and impact of research programs. Traditional metrics examine publication impact through slowly accumulating academic citations. “Altmetrics” are a new way to describe early publication influence in nonacademic media/community spheres (news, tweets, and blogs). Articles with significant altmetric attention make a big splash of immediate impact, whereas papers with high rates of academic citation reflect ripple effects of influence over time. Past research has found weak associations between altmetrics and academic citations. However, no previous research has focused on clinical/translational research, which aims to translate scientific discoveries to public use. Further, no previous research has assessed the relationship between altmetrics and modern citation impact factors like the National Institutes of Health (NIH)’s Relative Citation Ratio (RCR). It is also unclear whether publication in journals with higher journal impact factors (JIFs) may drive both public attention and academic impact. We investigated whether early altmetric indicators of splash predict citation ripples, beyond the effect of the JIF. For a portfolio of 2188 publications supported by the NIH’s Georgia Clinical and Translational Science Alliance from 2007–2020, we collected 2020 Altmetric Attention Scores (AAS), 2020 JIFs, and 2021 RCRs. All three were significantly correlated with one another. Regression analyses revealed that AAS significantly predicts later RCR, controlling for JIF and publication year. Findings indicate that in clinical/translational science, articles that make a big splash of altmetric attention have ripple effects through increased citation influence, which is not entirely due to publication in higher impact journals. Altmetric attention may be a useful early indicator of eventual influence and potential for translational advancement.