Some thoughts on bibliometrics, usage metrics and altmetrics concerning the International Journal of Rehabilitation Research

Subject matter predicts where top pediatric spine articles are shared: citations vs. social media

STUDY DESIGN

Systematic review. The purpose of this study was to compare the top 25 articles on pediatric spine surgery by number of citations and Altmetric score. All published articles pertaining to pediatric spine surgery from 2010 to 2021 were assessed for: Altmetric scores, Altmetric score breakdown (e.g. Twitter, News), citation counts, and article topics. The top 25 Altmetric articles and top 25 cited articles were identified. Out of the 50 total articles, only 3 (6.0%) overlapped between the two groups. The top Altmetric articles had averages (mean ± SD) of 167 ± 130 Altmetric score and 66 ± 135 citations, while the top citation articles had averages of 22 ± 45 Altmetric score and 196 ± 114 citations. When evaluating article topics, articles on 'back pain' (36% vs. 4%; P  = 0.003) and 'backpacks' (16% vs. 0%; P  = 0.030) were published significantly more in the top Altmetric group, while articles on 'scoliosis' (93% vs. 36%; P  < 0.001) and 'growth friendly surgery' (24% vs. 4%; P  = 0.041) were published significantly more in the top citation group. The total number of citations and online mentions for both groups are presented in Table 2. The biggest differences were the top Altmetric score articles receiving greater percentages of Twitter mentions relative to overall mentions (87% vs. 57%). The most socially popular articles focused on back pain and backpacks, and the most cited articles focused on scoliosis and growth-friendly surgery. Twitter had the most mentions of all social media for both the top cited articles and the top Altmetric articles.

Evaluating Funding for Authors Achieving High Citations vs. Online Popularity

INTRODUCTION

The Altmetric score is a validated tool that measures online attention of scientific studies. A relationship between government/industry funding for authors and their study's citations have been previously demonstrated. It is not known whether funding is related to greater online attention. We hypothesize authors publishing studies achieving greater online attention (higher Altmetric scores) receive greater monetary support from industry while authors publishing studies achieving critical acclaim (more citations) receive greater monetary support from the National Institute of Health (NIH).

METHODS

Top spine surgery studies between 2010 and 2021 were selected based on Altmetric scores and citation number. The Open Payments Database was accessed to evaluate industry financial relationships while the NIH Research Portfolio Online Reporting Tool was accessed to evaluate NIH funding. Payments were compared between groups and analyzed with the Student t-test, analysis of variance, and chi square analysis. Alpha <0.05.

RESULTS

There were 60 and 51 authors with payment data in the top 50 Altmetric and top 50 citation studies, respectively, with eight authors having studies in both groups. Total industry payments between groups were not markedly different. The eight authors with studies in both groups received markedly more industry payments for consulting, travel/lodging, and faculty/speaking fees. Authors with articles in both groups (50%) were significantly more likely to receive NIH support, compared with authors of the top Altmetric articles (5%; P < 0.001) and top citation articles (12%; P < 0.001). Authors receiving NIH support received significantly less industry payments compared with authors not receiving NIH support ($148,544 versus $2,159,526; P < 0.001).

DISCUSSION

These findings reject our hypothesis: no notable differences for industry payments and NIH funding between authors for top Altmetric and citation studies. Authors receiving funding from industry versus the NIH are generally two distinct groups, but there is a small group supported by both. These studies achieve both critical acclaim (citations) and online popularity (Altmetric scores).

DATA AVAILABILITY

Data can be available on reasonable request.

The Altmetric Attention Score Is Associated With Citation Rates and May Reflect Academic Impact in the Total Joint Arthroplasty Literature

Background: Given the increasing interest and potential use of social media for the promotion of orthopedic literature, there is a need to better understand Altmetrics. Purposes: We sought to determine the relationship between the Altmetric Attention Score (AAS) and the number of citations for articles on total joint arthroplasty (TJA) published in orthopedics journals. We also sought to determine the predictors of greater social media attention for these articles. Methods: Articles on TJA published in Bone and Joint Journal (BJJ), Journal of Bone and Joint Surgery (JBJS), Clinical Orthopedics and Related Research (CORR), Journal of Arthroplasty, Journal of Knee Surgery, Hip International, and Acta Orthopaedica in 2016 were extracted (n = 498). One-way analysis of variance with Bonferroni corrections was used to compare AAS and citations across journals. Multivariate regressions were used to determine predictors of social media attention and number of citations. Results: The mean AAS and number of citations were 7.5 (range: 0-289) and 16.7 (range: 0-156), respectively. Significant between-group effects were observed according to journal for AAS and number of citations. Publishing an article in JBJS was the strongest predictor of higher number of citations. Publishing an article in BJJ was the only independent predictor of higher AAS, while publishing an article in JBJS or CORR trended toward statistical significance. A higher AAS was a significant predictor of a higher number of citations. Number of citations and number of study references were positive predictors of greater social media attention on Twitter and Facebook. Conclusions: In articles on TJA published in 7 journals in 2016, a higher AAS was a associated with a higher number of citations. Various bibliometric characteristics were found to be significantly associated with greater social media attention; the most common influences were number of citations and number of references. Researchers in orthopedics can use this information when considering how to assess the impact of their work.

Twitter Use Among Orthopedic Surgery Journals Correlates With Increased Citation Rates

Use of alternative online means of distribution for research is a rapidly evolving area in orthopedics. In this study, we investigate the impact of Twitter on research citation rates and the association between Altmetric Attention Score (AAS) and citation rates across major orthopedic journals. Original research articles published in a selection of 10 journals between January 2018 and December 2018 were analyzed for number and source of tweets, AAS, and number of citations. We determined the relationship between Twitter use of different sources and number of citations across all journals and within each individual journal, any statistical differences in citations and AAS for tweeted vs nontweeted articles, and the correlation between journal impact factor and average number of monthly tweets. We included 2916 articles in our study, at an average of 26 months after publication. The mean AAS was 9.7 (SD, 45.1; range, 0-1125), and mean citation rate was 6.4 (SD, 8.5; range, 0-166). Impact factor was strongly correlated with average monthly tweets (r²=0.77). Tweeted articles received 8.51 citations on average, compared with 2.56 citations for nontweeted articles (P<.05). Tweets sent by the author/department had the highest impact on future citations (P<.01). Tweets by news outlets had a smaller, positive effect on citations (P<.01). Tweets sent by Arthroscopy (P<.01) and Journal of Pediatric Orthopaedics (P<.01) had high impacts compared with other journals. [Orthopedics. 2023;46(1):e38-e44.].

Technique Articles Are More Effective at Increasing Social Media Attention in Comparison With Original Research Articles: An Altmetrics-Based Analysis

Purpose

To compare social media attention and citation rates between technique articles and matched original research articles (ORAs) regarding surgical procedures.

Methods

All technique articles published from August 2019 through July 2020 in the free, electronic versions of Arthroscopy Techniques and JBJS Essential Surgical Techniques were matched by topic to articles in the “Original Research” sections of Arthroscopy, Sports Medicine, and Rehabilitation and JBJS Open Access in a 4:1 ratio within this time frame. The primary outcome was the Altmetric Attention Score (AAS). Secondary outcomes included citations, bibliometrics, and social media metrics. Independent t tests were used to compare primary and secondary outcomes between technique articles and ORAs. A multivariate linear regression analysis was performed to determine the association between article type and social media attention while controlling for confounding bibliometric characteristics.

Results

A total of 285 matched research articles (n = 57, 20.0%) and technique articles (n = 228, 80.0%) were included. The mean AAS among all technique articles was 3.63 ± 10.08 (range, 0-96) whereas the mean AAS among all ORAs was 1.30 ± 3.98 (range, 0-25), representing a statistically significant difference (P = .016). The mean citation rate among all technique articles was not significantly different from that among ORAs (P = .73). Multivariate linear regression analysis showed a statistically significant positive association between AAS and article type, with an additional mean increase in the AAS of 2.91 (95% confidence interval, 0.04-5.77; P = .047) for every technique article compared with an ORA. Furthermore, a significant positive relation was noted between the article origin and the AAS, with an increase in the AAS of 3.00 (95% confidence interval, 0.82-5.17; P = .007) for every article published in North America compared with an article originating from another continent.

Conclusions

Technique articles resulted in significantly greater AASs and social media attention in comparison with open-access ORAs on similar topics. Publications that described technical procedures in a technique journal and studies from North America were positively associated with greater AASs and greater numbers of citations received by articles.

Clinical Relevance

An improved understanding of how much attention is given to technique articles versus matched ORAs by social media may influence the methods authors and journals use for distributing content. The present study suggests that one option to increase the amount of social media attention received for a particular study may be to utilize an accompanying surgical technique video or illustrations as these are easily shareable on social media and offer rapid dissemination of knowledge, similar to that of an infographic. However, physicians who view multimedia within technique articles should be encouraged to review the accompanying articles and the supporting original research as a primary source before making changes in their clinical practice.

Infographics Are More Effective at Increasing Social Media Attention in Comparison With Original Research Articles: An Altmetrics-Based Analysis

PURPOSE

To compare social media attention and citation rates between infographics (visual abstracts) and original research articles.

METHODS

All infographics in 2019 from electronic versions of Arthroscopy were matched by topic to articles in the "Original Research" section of the journal in a 4:1 ratio within the same year. The primary outcome was the Altmetric Attention Score (AAS), a cumulative measure of social media attention from various platforms such as Twitter and Facebook. Secondary outcomes included citation rates, article characteristics, and number of shares on social media platforms. Independent t tests and χ² analyses were used to compare primary and secondary outcomes between infographics and control articles. Multivariate linear regression analysis was performed to determine the association between article type and social media attention while controlling for bibliometric characteristics.

RESULTS

A total of 60 matched research articles (n = 48, 80.0%) and infographics (n = 12, 20.0%) published in 2019 in Arthroscopy were included. The mean AAS among all infographics was 29.75 ± 32.84 (range, 3-118), whereas the mean AAS among all control research articles was 5.75 ± 8.90 (range, 0-41), representing a statistically significant difference (P < .001). Infographics had significantly more Twitter mentions (100% vs 70.8%, P < .001) and Facebook mentions (75% vs. 6.2%, P < .001) compared with original articles. Multivariate linear regression analysis demonstrated a statistically significant and positive association between AAS and article type, with an additional mean increase in the AAS of 33.7 (95% confidence interval 11.6-50.6; P = .003) for every infographic article compared with an original research article. The mean citation rate among all infographics was 2.4 ± 2.4 (range, 0-7), whereas the mean citation rate among all control research articles was 2.2 ± 4.0 (range, 0-27), which was not a significant difference (P = .69).

CONCLUSIONS

Infographics resulted in significantly greater AAS and social media attention in comparison with original research articles of similar topics. We recommend the routine creation of infographics by journals to increase the social media attention that their research and chosen topics of interest receive. However, viewers of infographics should read them out of interest but turn their attention toward the original article or a source of more detailed information before making changes in clinical decision-making or practice, as they can be oversimplified.

CLINICAL RELEVANCE

Infographics are an increasingly used by journals as a form of depicting research findings from select studies. By producing infographics, journals may increase the amount of social media attention received for a particular study or topic of interest.

A Bibliometric Analysis Using Alternative Metrics for Articles in the Annals of Rehabilitation Medicine

OBJECTIVE

To investigate the articles in the Annals of Rehabilitation Medicine (ARM) using a bibliometric analysis to verify whether there is a correlation between the topics of interest for expert groups and the public media.

METHODS

A total of 1,088 ARM articles from the third issue of 2011 to the third issue of 2019 were analyzed. We conducted a bibliometric analysis of the articles using conventional metrics (CM) and alternative metrics (AM). The CM was investigated by collating the type of publication, number of citations, and the specific field of rehabilitation medicine for each article. The AM was analyzed using the Altmetric Attention Score (AAS) provided by Altmetric, the leading AM company. The correlation between the number of citations and the AAS was tested using the Spearman rank correlation coefficient.

RESULTS

The combined ratio of original articles and case reports was over 90% in this study; however, the total distribution was significantly different compared to previous bibliometric studies (p<0.05). There were 233 articles that satisfied both conditions of at least one citation and at least one AAS point. The number of citations and the AAS were found to have a statistically significant positive linear correlation on a scatter plot (r=0.216, p=0.001).

CONCLUSION

There is a significant correlation between AM and CM, which means itis important to increase the dissemination of academic knowledge through the public media and increase the status of the journal by increasing the citation-related index.

What Is the Predictive Ability and Academic Impact of the Altmetrics Score and Social Media Attention?

BACKGROUND

Citation rate and journal impact factor have traditionally been used to assess research impact; however, these may fail to represent impact beyond the sphere of academics. Given that social media is now used to disseminate research, alternative web-based metrics (altmetrics) were recently developed to better understand research impact on social media. However, the relationship between altmetrics and traditional bibliometrics in orthopaedic literature is poorly understood.

PURPOSE

To (1) assess the extent that altmetrics correlate with traditional bibliometrics and (2) identify publication characteristics that predict greater altmetrics scores.

STUDY DESIGN

Cross-sectional study.

METHODS

Articles published in The American Journal of Sports Medicine (AJSM), The Journal of Bone and Joint Surgery, Clinical Orthopaedics and Related Research, Acta Orthopaedica, and Knee Surgery, Sports Traumatology, Arthroscopy between January 2016 and December 2016 were analyzed. Among the extracted publication characteristics were journal, number of authors, geographic region of origin, highest degree of first author, study subject and design, sample size, conflicts of interest, and level of evidence; number of references, institutions, citations, tweets, Facebook mentions, and news mentions; and Altmetric Attention Score (AAS). Multivariate regressions were used to determine (1) publication characteristics predictive of AAS and social media attention (mentions on Twitter, Facebook, and the news) and (2) the relationship between AAS and citation rate.

RESULTS

A total of 496 published articles were included, with a mean AAS of 8.6 (SD, 31.7; range, 0-501) and a mean citation rate of 15.0 (SD, 16.1; range, 0-178). Articles in AJSM (β = 19.9; P < .001), publications from North America (β = 8.5; P = .033), and studies concerning measure validation/reliability (β = 25.5; P = .004) were independently associated with higher AAS. Greater AAS score significantly predicted a greater citation rate (β = 0.16; P < .0001). The citation rate was an independent predictor of greater social media attention on Twitter, Facebook, and the news (odds ratio range, 1.02-1.03; P < .05 all).

CONCLUSION

AAS had a significant positive association with citation rates of articles in 5 high-impact orthopaedic journals. Articles in AJSM, studies concerning measure validation and reliability, and publications from North America were positively associated with greater AAS. A greater number of citations was consistently associated with publication attention received on social media platforms.