Practical publication metrics for academics

Online Visibility and Scientific Relevance of Strabismus Research: Bibliometric Analysis

BACKGROUND

Quality and accuracy of online scientific data are crucial, given that the internet and social media serve nowadays as primary sources of medical knowledge.

OBJECTIVE

This study aims to analyze the relationship between scientific relevance and online visibility of strabismus research to answer the following questions: (1) Are the most popular strabismus papers scientifically relevant? (2) Are the most high-impact strabismus studies shared enough online?

METHODS

The Altmetric Attention Score (AAS) was used as a proxy for online visibility, whereas citations and the journal's impact factor (IF) served as a metric for scientific relevance. Using "strabismus" as a keyword, 100 papers with the highest AAS and 100 papers with the highest number of citations were identified. Statistical analyses, including the Spearman rank test, linear regression, and factor analysis, were performed to assess the relationship between AAS, citations, a journal's IF, and mentions across 18 individual Web 2.0 platforms.

RESULTS

A weak, positive, statistically significant correlation was observed between normalized AAS and normalized citations (P<.001; r=0.27) for papers with high visibility. Only Twitter mentions and Mendeley readers correlated significantly with normalized citations (P=.02 and P<.001, respectively) and IF (P=.04 and P=.009, respectively), with Twitter being the strongest significant predictor of citation numbers (r=0.53). For high-impact papers, no correlation was found between normalized citations and normalized AAS (P=.12) or the IF of the journal (P=.55).

CONCLUSIONS

While clinical relevance influences online attention, most high-impact research related to strabismus is not sufficiently shared on the web. Therefore, researchers should make a greater effort to share high-impact papers related to strabismus on online media platforms to improve accessibility and quality of evidence-based knowledge for patients.

Journal metrics of the top-ranked Orthopaedic, Medical, and Surgical journals - A cross-sectional, comparative study

PURPOSE

In academic publishing, research metrics play a crucial role in assessing the scientific impact and performance of the published literature, as well as of the journals in which they are published. Several journal-level metrics (JLM) such as the h-index of the analysed journals, total citations, total documents, citable documents, references and external citations per document are considered crucial indicators of the importance and reputation of the journals. We hypothesize that journals in the field of Medicine receive more citations than those in Surgical journals like Orthopaedic surgery, and hence have better JLM. This study aims to to assess and compare the JLM of Medical and Surgical journals between two time zones 2017-2019 vs. 2020-2022, i.e., pre and post-COVID-19 pandemic period.

METHODS

A cross-sectional bibliometric analysis of the top-ranked Orthopaedic, Medical, and Surgical journals was undertaken based on traditional JLM, using the SCImago database from 2017 to 2022. Our analysis focused on identifying trends in the h-index of the analysed journals, total citations, total documents, citable documents, references and external citations per document.

RESULTS

Overall Medical journals were found to have higher JLM than the Surgical and Orthopaedic journals. The h-index of Surgical journals, Medical journals and Orthopaedic journals were comparable between the two periods (pre and -post-COVID-19 pandemic); Total Cites (3 years), total documents (2017), total documents (3 years), total references, and citable documents (3 years) of Surgical journals, Medical journals and Orthopaedic journals were significantly higher in the period 2020-2022.

CONCLUSION

There has been a steady increase in the number of publications from post COVID-19 period. Medical journals have higher JLM than Surgical and Orthopaedic journals. Journal of Bone and Joint Surgery (Am), Annals of Surgery and Diabetes Care were the most published journals in Orthopaedics, General Surgery and Medicine-related topics respectively.

Article-level metrics: A new approach to quantify reach and impact of published research

A spectrum of measuring tools are available to evaluate the impact of published literature and the journals they are published in. Journal Level Metrics (JLM) such as Journal Impact Factor (JIF) or CiteScore assess the reputation of peer-reviewed journals based on citation analysis. Whereas, Article Level Metrics (ALM) quantify the importance, reach and impact of a particular article, and are a new approach to quantifying the reach and impact of published research. Traditionally JLM has served as a proxy for an individual publication's significance, however, the introduction of contemporary and evolution of Alternative metrics measuring digital or societal influence of a particular article has gained popularity in recent times. These metrics help in rapid dissemination of research, development of newer research strategies and individual academic progress. We highlight the characteristics and importance of currently available ALM, and the newer ones influenced by social media, digital media and Open Access publishing models.

Big splashes & ripple effects: a narrative review of the short- & long-term impact of publications supported by an NIH CTSA pediatrics program

BACKGROUND AND OBJECTIVE

This review examines a promising new framework for analyzing outputs of pediatric research in the context of translational advancement. We demonstrate a method for evaluating the impact of an NIH Clinical and Translational Science Award's (CTSA) Pediatrics Program through publications that have emerged from supported research. The Georgia CTSA Pediatrics Program provides training, funding, and infrastructure to ensure that researchers have the resources to advance pediatric health. Internal evaluations found that research supported by this program is exceptionally impactful within the academic community and commands high interest within the lay community. Therefore, we examined the impact of this research in both traditional academic and broader community spheres using bibliometrics-the study of supported publications. Bibliometrics describe a pivotal stage in the translational process of bringing scientific discoveries to clinical/community use and include both academic citations and 'altmetric' or non-academic attention. These complementary approaches combine to shed light on the short- and long-term impact of the research on segments of the translational pipeline, including academic literature, community discourse, technological advancement, and public health policy.

METHODS

The authors identified a portfolio of 250 articles supported by the Georgia CTSA Pediatrics Program from 2007-2020. We utilized various bibliometrics to analyze both short-term attention, or 'splash' made by articles, and long-term influence, or 'ripples' made across both academic and public spheres.

KEY CONTENT AND FINDINGS

The short-term splash of the portfolio was indicated through publication in high-impact factor journals, peer faculty recommendations, and Mendeley readership, as well as by early altmetric attention in news stories, blogs, and Twitter posts. The portfolio's long-term ripples were demonstrated by high absolute and relative rates of academic citation and by downstream altmetric influence in public-facing documents, including Wikipedia articles, patent applications, and policy documents.

CONCLUSIONS

This article reviews a useful bibliometric methodology for illustrating the waves of impact made by pediatric research. Whereas splash provides a picture of early interest in a publication, a preliminary indicator of eventual utility and impact, ripples provide a measure of the cumulative influence of an article over time. Both reflect opportunities for a line of research to advance along the translational spectrum.

Predicting Citation Impact from Altmetric Attention in Clinical and Translational Research: Do Big Splashes Lead to Ripple Effects?

Publications are commonly used to evaluate the productivity and impact of research programs. Traditional metrics examine publication impact through slowly accumulating academic citations. “Altmetrics” are a new way to describe early publication influence in nonacademic media/community spheres (news, tweets, and blogs). Articles with significant altmetric attention make a big splash of immediate impact, whereas papers with high rates of academic citation reflect ripple effects of influence over time. Past research has found weak associations between altmetrics and academic citations. However, no previous research has focused on clinical/translational research, which aims to translate scientific discoveries to public use. Further, no previous research has assessed the relationship between altmetrics and modern citation impact factors like the National Institutes of Health (NIH)’s Relative Citation Ratio (RCR). It is also unclear whether publication in journals with higher journal impact factors (JIFs) may drive both public attention and academic impact. We investigated whether early altmetric indicators of splash predict citation ripples, beyond the effect of the JIF. For a portfolio of 2188 publications supported by the NIH’s Georgia Clinical and Translational Science Alliance from 2007–2020, we collected 2020 Altmetric Attention Scores (AAS), 2020 JIFs, and 2021 RCRs. All three were significantly correlated with one another. Regression analyses revealed that AAS significantly predicts later RCR, controlling for JIF and publication year. Findings indicate that in clinical/translational science, articles that make a big splash of altmetric attention have ripple effects through increased citation influence, which is not entirely due to publication in higher impact journals. Altmetric attention may be a useful early indicator of eventual influence and potential for translational advancement.