High social media attention scores are not reflective of study quality: an altmetrics-based content analysis

Journal Metrics of Orthopaedic Journals between 2016 and 2021: Trends and Comparisons with Other Specialties

Bibliometric analysis plays a crucial role in elucidating publication trends and aids scholars in gauging the reach of prospective journals for their research dissemination. Concerns with impact factor (IF) have led us to examine the trends in IF, corrected IF (cIF), and Citescore in orthopaedic journals from 2016 to 2021 and compare them with internal medicine and general surgery journals. Journal IF and cIF were obtained from Journal Citation Reports and Citescore data from the Elsevier Scopus database for the years 2016 to 2021. Orthopaedic journals were categorized, and 10 medicine and surgery journals were selected for comparison. Mean values were analyzed to identify trends. The study included 52 orthopaedic journals, evenly split between the United States and the rest of the world, predominantly publishing in English. Mean IF in orthopaedic journals increased from 1.93 (2016) to 2.78 (2021), with similar rises in cIF and Citescore. These trends were consistent in specialty and general orthopaedic journals. No significant differences were found in mean IF between these categories. Medicine and surgery journals also experienced significant IF increases. Orthopaedic journals have experienced growing esteem and extent from 2016 to 2021. Specialty and general orthopaedic journals showed parallel growth. Researchers can utilize this analysis for informed publishing decisions, potentially expanding their readership.

Scholarly Impact of Quality Improvement Reports in Neonatology

OBJECTIVE

 Neonatology quality improvement (QI) projects can improve the safety and value of health care, but the scholarly impact of published QI projects is unclear. We measured scholarly citation and media attention garnered by published neonatology QI projects and analyzed project or publication characteristics associated with increased impact metrics.

STUDY DESIGN

 We identified publications between 2016 and 2019 using mapping review methodology. We correlated project characteristics with measures of scholarly citation in Scopus and Google Scholar, and media attention as measured by Altmetrics. We collected Citation and Altmetric data in 2023.

RESULTS

 The search identified 148 eligible articles, with a median citation count of 7 based on Scopus (or 12, based on Google Scholar) and a median Altmetric score of 2. Notably, 66% of articles published in a journal with an Impact Factor (IF) had more citations per year than would be expected from the IF value. Higher scientific citations were associated with articles reporting process and cost outcomes; implementing interventions that addressed family education or organizational change; and using regression analysis. Higher media attention was associated with multicenter projects, longer intervention periods, and projects scoring higher on the Quality Improvement Minimum Quality Criteria Set (QI-MQCS) rubric.

CONCLUSION

 Published neonatology QI projects are well cited in subsequent scientific publications, with the choice of project outcome, interventions, and analytic strategy influencing citation metrics. Adherence to QI-MQCS guidelines was favorably associated with media attention, but not with scholarly citations.

KEY POINTS

· Neonatology QI publications are frequently cited.. · Projects with cost data receive more citations.. · Citation and media mention predictors differ..

Examining the Effectiveness of Social Media for the Dissemination of Research Evidence for Health and Social Care Practitioners: Systematic Review and Meta-Analysis

BACKGROUND

Social media use has potential to facilitate the rapid dissemination of research evidence to busy health and social care practitioners.

OBJECTIVE

This study aims to quantitatively synthesize evidence of the between- and within-group effectiveness of social media for dissemination of research evidence to health and social care practitioners. It also compared effectiveness between different social media platforms, formats, and strategies.

METHODS

We searched electronic databases for articles in English that were published between January 1, 2010, and January 10, 2023, and that evaluated social media interventions for disseminating research evidence to qualified, postregistration health and social care practitioners in measures of reach, engagement, direct dissemination, or impact. Screening, data extraction, and risk of bias assessments were carried out by at least 2 independent reviewers. Meta-analyses of standardized pooled effects were carried out for between- and within-group effectiveness of social media and comparisons between platforms, formats, and strategies. Certainty of evidence for outcomes was assessed using the GRADE (Grading of Recommendations, Assessment, Development, and Evaluations) framework.

RESULTS

In total, 50 mixed-quality articles that were heterogeneous in design and outcome were included (n=9, 18% were randomized controlled trials [RCTs]). Reach (measured in number of practitioners, impressions, or post views) was reported in 26 studies. Engagement (measured in likes or post interactions) was evaluated in 21 studies. Direct dissemination (measured in link clicks, article views, downloads, or altmetric attention score) was analyzed in 23 studies (8 RCTs). Impact (measured in citations or measures of thinking and practice) was reported in 13 studies. Included studies almost universally indicated effects in favor of social media interventions, although effect sizes varied. Cumulative evidence indicated moderate certainty of large and moderate between-group effects of social media interventions on direct dissemination (standardized mean difference [SMD] 0.88; P=.02) and impact (SMD 0.76; P<.001). After social media interventions, cumulative evidence showed moderate certainty of large within-group effects on reach (SMD 1.99; P<.001), engagement (SMD 3.74; P<.001), and direct dissemination (SMD 0.82; P=.004) and low certainty of a small within-group effect on impacting thinking or practice (SMD 0.45; P=.02). There was also evidence for the effectiveness of using multiple social media platforms (including Twitter, subsequently rebranded X; and Facebook), images (particularly infographics), and intensive social media strategies with frequent, daily posts and involving influential others. No included studies tested the dissemination of research evidence to social care practitioners.

CONCLUSIONS

Social media was effective for disseminating research evidence to health care practitioners. More intense social media campaigns using specific platforms, formats, and strategies may be more effective than less intense interventions. Implications include recommendations for effective dissemination of research evidence to health care practitioners and further RCTs in this field, particularly investigating the dissemination of social care research.

TRIAL REGISTRATION

PROSPERO International Prospective Register of Systematic Reviews CRD42022378793; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=378793.

INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID)

RR2-10.2196/45684.

The online attention analysis on orthognathic surgery research

OBJECTIVES

Altmetrics is one of the fields of bibliometrics that seeks to assess the impact and interest of a given subject through Internet users. The aim of this study is to make an altmetric analysis of the orthognathic surgery literature.

METHODS

A literature search was conducted using Dimensions app up to December 2023. A list of the 100 most mentioned articles on the topic was compiled. A Google Trends search was performed with same strategy to visualize important data regarding internet search. Charts and tables were created using Microsoft Excel and VOSviewer software to allow bibliometric visualization.

RESULTS

There was a very poor correlation between the number of mentions and the number of citations (r = 0.0202). Most articles discussed on technical innovations associated to orthognathic surgery, majority related to virtual planning (n = 26). Other topics considered interesting to internet readers were complications (n = 18), surgical technique (n = 14), and psychological aspects/quality of life (n = 13).

CONCLUSION

Online interest in orthognathic surgery closely aligns with the level of academic interest but is also influenced by factors such as location and economic status. The internet is a powerful tool for disseminating scientific research to a broad audience, making it more accessible and engaging than traditional academic channels.

The correlation between altmetric score and traditional measures of article impact for studies pertaining to spine trauma

PURPOSE

It is becoming increasingly common for researchers to share scientific literature via social media. Traditional bibliometrics have long been utilized to measure a study's academic impact, but they fail to capture the impact generated through social media sharing. Altmetric Attention Score (AAS) is a weighted count of all the online attention garnered by a study, and it is currently unclear whether a relationship with traditional bibliometrics exists.

METHODS

We identified the five highest-rated spine-specific and five highest-rated general orthopedic journals by Scopus CiteScore 2020. We then identified all the spine trauma studies across a 5-year span (2016-2020) within these journals and compared AAS with traditional bibliometrics using Independent t-tests and Pearson's correlational analyses.

RESULTS

No statistically significant relationships were identified between AAS and traditional bibliometrics for articles pertaining to spine trauma: Level of Evidence (R = - 0.02, p = 0.34), H-Index Primary Author (R =  < - 0.01, p = 0.50), H-Index Senior Author (R = - 0.04, p = 0.24), and Number of Citations (R = 0.01, p = 0.40). The top five articles by AAS include those pertaining to motorcycle injuries (AAS = 687), orthosis in thoracolumbar fractures (AAS = 199), golfing injuries (AAS = 166), smartphone-based teleradiology (AAS = 41), and auto racing injuries (AAS = 39).

CONCLUSION

The lack of overlap between these types of metrics suggests that AAS or similar alternative metrics should be used to measure an article's social impact. The social impact of an article should likewise be a factor in determining an article's overall impact along with its academic impact as measured by bibliometrics.

The correlation between altmetric score and traditional measures of article impact for studies published within the Surgeon Journal

Alternative metrics, or altmetrics, have emerged as a promising tool for measuring the social impact of research, which is increasingly important in today's digital and social media-driven world. Altmetric Attention Score (AAS) is a weighted count of all the online attention garnered by a study, and it is currently unclear whether a relationship with traditional bibliometrics exists. The purpose of this article was to retrospectively review articles published in the Surgeon Journal from 2003 to 2020 to compare AAS with bibliometric parameters using an Independent t-test and Pearson's correlation analysis. There were statistically significant weakly positive relationships between AAS and sample size, number of reads, and number of citations. There was no statistically significant relationship between AAS and number of authors, H-index, or level of evidence. This study highlights the potential value of altmetrics by measuring the social impact of research as altmetrics can provide valuable information not captured by traditional metrics. It is currently unclear what the optimal balance of social and academic impact is in evaluating research impact and how altmetrics can be integrated into existing research frameworks.

Effective communications strategies to increase the impact of environmental health research

BACKGROUND

Per- and polyfluoroalkyl substances (PFAS) are the subject of a growing body of research with the potential to positively impact public and ecological health. However, to effect positive change, findings must be communicated beyond the scientific community.

OBJECTIVE

We sought to (a) evaluate the relationships between communications strategy, media attention, and scholarly citations of PFAS research and (b) offer guidance for researchers and communications professionals who would like to publicize future work and increase its impact.

METHODS

We analyzed 273 peer-reviewed epidemiological studies on PFAS human health impacts with publication years 2018-2020, as collected by a pre-existing database. We investigated whether a press release was issued, open-access status, abstract and press release readability, timing of publication and press release distribution, journal impact factor, study type and sample size, statistical significance of finding(s), number of scholarly citations, and the Altmetric Attention Score (a measure of media attention).

DISCUSSION

Of papers reporting a statistically significant association with health harm, those with a press release received 20 times more media attention (as assessed by Altmetric scores) than those that did not. However, only 6.2% of all papers and 7.8% of significant papers issued one. Among papers with a press release, media attention was positively correlated with better abstract and press release readability and speed in issuing the press release. Scholarly citations were positively correlated with media attention, presence of a press release, and open-access status.

CONCLUSION

Most papers with significant findings on PFAS are published without a press release and receive little or no media attention. This reduces the likelihood that important research is reaching the public and decisionmakers who can translate science into action. Issuing a press release and receiving media attention also appear to increase scholarly citations. We provide recommendations for authors to increase the reach and impact of future papers.

The Altmetric Attention Score Is Associated With Citation Rates and May Reflect Academic Impact in the Total Joint Arthroplasty Literature

Background: Given the increasing interest and potential use of social media for the promotion of orthopedic literature, there is a need to better understand Altmetrics. Purposes: We sought to determine the relationship between the Altmetric Attention Score (AAS) and the number of citations for articles on total joint arthroplasty (TJA) published in orthopedics journals. We also sought to determine the predictors of greater social media attention for these articles. Methods: Articles on TJA published in Bone and Joint Journal (BJJ), Journal of Bone and Joint Surgery (JBJS), Clinical Orthopedics and Related Research (CORR), Journal of Arthroplasty, Journal of Knee Surgery, Hip International, and Acta Orthopaedica in 2016 were extracted (n = 498). One-way analysis of variance with Bonferroni corrections was used to compare AAS and citations across journals. Multivariate regressions were used to determine predictors of social media attention and number of citations. Results: The mean AAS and number of citations were 7.5 (range: 0-289) and 16.7 (range: 0-156), respectively. Significant between-group effects were observed according to journal for AAS and number of citations. Publishing an article in JBJS was the strongest predictor of higher number of citations. Publishing an article in BJJ was the only independent predictor of higher AAS, while publishing an article in JBJS or CORR trended toward statistical significance. A higher AAS was a significant predictor of a higher number of citations. Number of citations and number of study references were positive predictors of greater social media attention on Twitter and Facebook. Conclusions: In articles on TJA published in 7 journals in 2016, a higher AAS was a associated with a higher number of citations. Various bibliometric characteristics were found to be significantly associated with greater social media attention; the most common influences were number of citations and number of references. Researchers in orthopedics can use this information when considering how to assess the impact of their work.

Prophylactic tamsulosin can reduce the risk of urinary retention after surgery in male patients: A systematic review and meta-analysis

Objective

The meta-analysis aimed to estimate the efficacy of prophylactic tamsulosin on postoperative urinary retention (POUR) in male patients.

Methods

Papers were searched in the PubMed, Embase, Web of Science, and Cochrane Library databases with predetermined keywords up to March 1, 2022. The studies reporting the preventive efficacy of prophylactic tamsulosin on POUR among men were identified. Pooled risk ratios (RRs) were calculated based on the random-effects model. Meta-regression was performed to explore potential sources of heterogeneity.

Results

There were 11 studies with 1,046 patients in the tamsulosin group and 1,113 patients in the control group. The risk of POUR was significantly lower in the tamsulosin group (123/1,046 [11.8%] vs. 238/1,119 [19.0%]; RR = 0.61; 95% confidence interval [CI] 0.43 to 0.87; P = 0.006; heterogeneity: I2 = 57%; P = 0.009). Administration of tamsulosin was related to higher risk of adverse events (57/688 [8.3%] vs. 33/624 [5.3%]; RR = 1.68; 95% CI: 1.13 to 2.48; P = 0.010; heterogeneity: I2 = 33%; P = 0.20). The level of evidence and mean age of the included patients were identified as the potential sources of heterogeneity.

Conclusion

The present meta-analysis indicated that prophylactic tamsulosin helps in preventing POUR and younger patients might benefit more from this preventive regimen. Administrating tamsulosin was also associated with a possibly higher risk of adverse events.

Open Access Articles Garner Increased Social Media Attention and Citation Rates Compared With Subscription Access Research Articles: An Altmetrics-Based Analysis

BACKGROUND

To better understand the research impact on social media, alternative web-based metrics (Altmetrics) were developed. Open access (OA) publishing, which allows for widespread distribution of scientific content, has become increasingly common in the medical literature. However, the relationship between OA publishing and social media impact remains unclear.

PURPOSE

To compare social media attention and citation rates between OA and subscription access (SA) research articles within the orthopaedic and sports medicine literature.

STUDY DESIGN

Cross-sectional study.

METHODS

Articles published as either OA or SA in 5 high-impact hybrid orthopaedic journals between January 2019 and December 2019 were analyzed. The primary outcome was the Altmetric Attention Score (AAS), a validated measure of social media attention. Secondary outcomes included citation rates, article characteristics, and the number of shares on social media. Independent t tests and chi-square analyses were used to compare outcomes between OA and SA articles. A multivariable linear regression analysis was performed to determine the association between article type and AAS while controlling for bibliometric characteristics.

RESULTS

A total of 2143 articles (246 OA articles, 11.5%; 1897 SA articles, 88.5%) were included. The mean AAS among all OA articles was 62.4 ± 184.6 (range, 0-2032), whereas the mean AAS among all SA articles was 18.4 ± 109.8 (range, 0-3425), representing a statistically significant difference (P < .001). The mean citation rate among OA articles was significantly higher (17.0 ± 22.5; range, 0-139) than that of SA articles (8.6 ± 13.4; range, 0-169) (P < .001). Multivariable linear regression analysis demonstrated that OA status (β = 15.15; P = .044), number of institutions (β = 2.13; P = .023), studies classified as epidemiological investigations (β = 107.40; P < .001), and disclosure of a conflict of interest (β = -11.18; P = .032) were significantly associated with a higher AAS.

CONCLUSION

OA articles resulted in significantly greater AAS and citations in comparison with SA articles. Articles published through the OA option in hybrid journals as well as those with a higher number of institutions, those that disclosed a conflict of interest, and those classified as epidemiological investigations were positively associated with greater AAS in addition to a greater number of citations. The potential for more extensive research dissemination inherent in the OA option may therefore translate into greater reach and social media attention.

Meaningless Applications and Misguided Methodologies in Artificial Intelligence-Related Orthopaedic Research Propagates Hype Over Hope

There exists great hope and hype in the literature surrounding applications of artificial intelligence (AI) to orthopaedic surgery. Between 2018 and 2021, a total of 178 AI-related articles were published in orthopaedics. However, for every 2 original research papers that apply AI to orthopaedics, a commentary or review is published (30.3%). AI-related research in orthopaedics frequently fails to provide use cases that offer the uninitiated an opportunity to appraise the importance of AI by studying meaningful questions, evaluating unknown hypotheses, or analyzing quality data. The hype perpetuates a feed-forward cycle that relegates AI to a meaningless buzzword by rewarding those with nascent understanding and rudimentary technical knowhow into committing several basic errors: (1) inappropriately conflating vernacular ("AI/machine learning"), (2) repackaging registry data, (3) prematurely releasing internally validated algorithms, (4) overstating the "black box phenomenon" by failing to provide weighted analysis, (5) claiming to evaluate AI rather than the data itself, and (6) withholding full model architecture code. Relevant AI-specific guidelines are forthcoming, but forced application of the original Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis guidelines designed for regression analyses is irrelevant and misleading. To safeguard meaningful use, AI-related research efforts in orthopaedics should be (1) directed toward administrative support over clinical evaluation and management, (2) require the use of the advanced model, and (3) answer a question that was previously unknown, unanswered, or unquantifiable.