Using altmetrics for detecting impactful research in quasi-zero-day time-windows: the case of COVID-19

Altmetric Analysis of Dermatology Manuscript Dissemination During the COVID-19 Era: Cross-Sectional Study

BACKGROUND

Alternative bibliometrics or altmetrics, is a measure of an academic article's impact on social media outlets, which is quantified by the Altmetric Attention score (AAS). Given a lack of data for altmetric trends during the COVID-19 pandemic, we conducted a comprehensive, multivariable analysis of top dermatology manuscripts published during this time period.

OBJECTIVE

We aim to assess (1) the relationship between traditional bibiliometrics and Altmetrics and (2) factors associated with high AAS.

METHODS

All abstracted articles published in the top-5 (ranked by SCImago Journal Rankings) peer-reviewed dermatology journals published in 2021 were included in our study. We collected AAS as the dependent variable and categorical predictor variables included journal title, whether a conflict of interest existed, open access status, whether the article was related to COVID-19 or skin-of-color research, and the type of research (eg, clinical, basic science, review, etc). Numerical predictor variables consisted of the impact factor of journal, total citations, and number of authors. Multivariable linear or logistic regression models were used.

RESULTS

The relationship between AAS and citation number was significant by multivariable analysis during the COVID-19 pandemic (P<.001). Numerous factors, including studies related to COVID-19, whether the article was open access, title of the journal, and journal impact factor were also independently related to higher AAS (P<.002).

CONCLUSIONS

Our results validate the use of altmetrics as a complement to traditional bibliometrics, especially in times of widespread scientific interest. Despite existing in a complex realm of bibliometrics, there are also discernable patterns associated with higher AAS. This is especially relevant in the era of growing technologic importance and utility to assess the impact of scientific works within the general public.

Research on the relationships between discourse leading indicators and citations: perspectives from altmetrics indicators of international multidisciplinary academic journals

Purpose

This paper aims to analyze the relationships between discourse leading indicators and citations from perspectives of integrating altmetrics indicators and tries to provide references for comprehending the quantitative indicators of scientific communication in the era of open science, constructing the evaluation indicator system of the discourse leading for academic journals and then improving the discourse leading of academic journals.

Design/methodology/approach

Based on the theory of communication and the new pattern of scientific communication, this paper explores the formation process of academic journals' discourse leading. This paper obtains 874,119 citations and 6,378,843 altmetrics indicators data from 65 international multidisciplinary academic journals. The relationships between indicators of discourse leading (altmetrics) and citations are studied by using descriptive statistical analysis, correlation analysis, principal component analysis, negative binomial regression analysis and marginal effects analysis. Meanwhile, the connotation and essential characteristics of the indicators, the strength and influence of the relationships are further analyzed and explored. It is proposed that academic journals' discourse leading is composed of news discourse leading, social media discourse leading, peer review discourse leading, encyclopedic discourse leading, video discourse leading and policy discourse leading.

Findings

It is discovered that the 15 altmetrics indicators data have a low degree of centralization to the center and a high degree of polarization dispersion overall; their distribution patterns do not follow the normal distributions, and their distributions have the characteristics of long-tailed right-peaked curves. Overall, 15 indicators show positive correlations and wide gaps exist in the number of mentions and coverage. The academic journals' discourse leading significantly affects total cites. When altmetrics indicators of international mainstream academic and social media platforms are used to explore the connotation and characteristics of academic journals' discourse leading, the influence or contribution of social media discourse, news discourse, video discourse, policy discourse, peer review discourse and encyclopedia discourse on the citations decreases in turn.

Originality/value

This study is innovative from the academic journal level to analyze the deep relationships between altmetrics indicators and citations from the perspective of correlation. First, this paper explores the formation process of academic journals' discourse leading. Second, this paper integrates altmetrics indicators to study the correlation between discourse leading indicators and citations. This study will help to enrich and improve basic theoretical issues and indicators’ composition, provide theoretical support for the construction of the discourse leading evaluation system for academic journals and provide ideas for the evaluation practice activities.

Characteristics analysis and evaluation of discourse leading for academic journals: perspectives from multiple integration of altmetrics indicators and evaluation methods

Purpose

Under the background of open science, this paper integrates altmetrics data and combines multiple evaluation methods to analyze and evaluate the indicators' characteristics of discourse leading for academic journals, which is of great significance to enrich and improve the evaluation theory and indicator system of academic journals.

Design/methodology/approach

This paper obtained 795,631 citations and 10.3 million altmetrics indicators data for 126,424 published papers from 151 medicine, general and internal academic journals. In this paper, descriptive statistical analysis and distribution rules of evaluation indicators are first carried out at the macro level. The distribution characteristics of evaluation indicators under different international collaboration conditions are analyzed at the micro level. Second, according to the characteristics and connotation of the evaluation indicators, the evaluation indicator system is constructed. Third, correlation analysis, factor analysis, entropy weight method and TOPSIS method are adopted to evaluate and analyze the discourse leading in medicine, general and internal academic journals by integrating altmetrics. At the same time, this paper verifies the reliability of the evaluation results.

Findings

Six features of discourse leading integrated with altmetrics indicators are obtained. In the era of open science, online academic exchanges are becoming more and more popular. The evaluation activities based on altmetrics have fine-grained and procedural advantages. It is feasible and necessary to integrate altmetrics indicators and combine the advantages of multiple methods to evaluate the academic journals' discourse leading of which are in a diversified academic ecosystem.

Originality/value

This paper uses descriptive statistical analysis to analyze the distribution characteristics and distribution rules of discourse leading indicators of academic journals and to explore the availability of altmetrics indicators and the effectiveness of constructing an evaluation system. Then, combining the advantages of multiple evaluation methods, The author integrates altmetrics indicators to comprehensively evaluate the discourse leading of academic journals and verify the reliability of the evaluation results. This paper aims to provide references for enriching and improving the evaluation theory and indicator system of academic journals.

The role of blogs and news sites in science communication during the COVID-19 pandemic

We present a brief review of literature related to blogs and news sites; our focus is on publications related to COVID-19. We primarily focus on the role of blogs and news sites in disseminating research on COVID-19 to the wider public, that is knowledge transfer channels. The review is for researchers and practitioners in scholarly communication and social media studies of science who would like to find out more about the role of blogs and news sites during the COVID-19 pandemic. From our review, we see that blogs and news sites are widely used as scholarly communication channels and are closely related to each other. That is, the same research might be reported in blogs and news sites at the same time. They both play a particular role in higher education and research systems, due to the increasing blogging and science communication activity of researchers and higher education institutions (HEIs). We conclude that these two media types have been playing an important role for a long time in disseminating research, which even increased during the COVID-19 pandemic. This can be verified, for example, through knowledge graphs on COVID-19 publications that contain a significant amount of scientific publications mentioned in blogs and news sites.

Research on the discourse power evaluation of academic journals from the perspective of multiple fusion: Taking Medicine, General and Internal journals as an example

In the open science environment, this article evaluates the discourse power of academic journals from the perspective of multiple integration. It is conducive to scientific research management and provides a reference for enriching and perfecting the evaluation theory and indicators system of academic journals. Based on the theory of evaluation science, discourse power theory and communication theory, first, this article discusses the basic issues of the discourse power evaluation for academic journals. It defines the academic discourse power and the discourse power of academic journals. It is proposed that the discourse power of academic journals is composed of discourse influence and discourse leading. Discourse influence is composed of discourse influence ability and discourse power, and discourse leading is composed of social media discourse, news and policy discourse, encyclopaedia discourse, peer-review discourse and video discourse leading. This article explores the formation process and operation mechanism of the discourse power for academic journals. Then, this article constructs the discourse power evaluation model of academic journals. Second, this article integrates multi-source heterogeneous data, then adopts correlation analysis, integrated factor analysis, entropy weight method, Technique for Order Preference by Similarity to an Ideal Solution (TOPSIS) method and two-dimensional four-quadrant mapping method to conduct empirical research on the discourse power evaluation of Medicine, General and Internal journals from the perspectives of multi-dimensions, multi-factors, multi-indicators and multi-methods fusion. The results show that the research on the discourse power evaluation for academic journals based on the theory, method and application logic is practical, comprehensive and reliable.

Semi-Automatic Systematic Literature Reviews and Information Extraction of COVID-19 Scientific Evidence: Description and Preliminary Results of the COKE Project

The COVID-19 pandemic highlighted the importance of validated and updated scientific information to help policy makers, healthcare professionals, and the public. The speed in disseminating reliable information and the subsequent guidelines and policy implementation are also essential to save as many lives as possible. Trustworthy guidelines should be based on a systematic evidence review which uses reproducible analytical methods to collect secondary data and analyse them. However, the guidelines’ drafting process is time consuming and requires a great deal of resources. This paper aims to highlight the importance of accelerating and streamlining the extraction and synthesis of scientific evidence, specifically within the systematic review process. To do so, this paper describes the COKE (COVID-19 Knowledge Extraction framework for next generation discovery science) Project, which involves the use of machine reading and deep learning to design and implement a semi-automated system that supports and enhances the systematic literature review and guideline drafting processes. Specifically, we propose a framework for aiding in the literature selection and navigation process that employs natural language processing and clustering techniques for selecting and organizing the literature for human consultation, according to PICO (Population/Problem, Intervention, Comparison, and Outcome) elements. We show some preliminary results of the automatic classification of sentences on a dataset of abstracts related to COVID-19.

Alternative publication metrics in the time of COVID-19

Alternative metrics are unique bibliometrics comprising social, news, and other sources of media outside of traditional academic citations. Some have suggested that these metrics can complement traditional metrics of research impact, including public engagement with research. The COVID-19 pandemic provides a unique opportunity to study alternative metrics and the dissemination of scientific research given the heightened academic and public interest. This study analyzed Altmetric Attention Scores for the top 25 publications on COVID-19 and the top 25 non-COVID-19 publications in 2020. There were significantly higher levels of social attention scores across multiple metrics for COVID-19 articles than for non-COVID-19 articles for that year. There was a slightly higher goodness of fit between Altmetric Attention Scores and academic citations for COVID-19 publications than for non-COVID-19 publications, although trendline differences were not significant. These results suggest that researchers should be aware that their studies can become highly visible on publicly available social and news media platforms, especially during events of high interest (such as a global pandemic).

O impacto do acesso aberto na produção e difusão de conhecimento sobre a Covid-19

A corrida científica ocasionada pelo surto do novo coronavírus (SARS-CoV-2) tem posto em evidência a relevância do compartilhamento de dados e resultados de pesquisa de forma mais rápida e aberta do que a do atual sistema de comunicação científica, bem como o papel das mídias sociais nesse contexto. O objetivo deste artigo é analisar o impacto do acesso aberto (open access) na produção e difusão do conhecimento sobre a COVID-19 nos meios tradicionais de comunicação científica e na web social. Via indicadores bibliométricos e altmétricos, analisamos um conjunto de 6.631 artigos e revisões das bases PubMed e Scopus, publicados entre janeiro e abril de 2020. Os resultados evidenciaram que o volume das publicações científicas relacionadas à COVID-19 aumentou a uma taxa média mensal de 166% no período analisado, com destaque às publicações na modalidade acesso aberto. Os países com maior número de publicações são aqueles que tiveram mais casos registrados da COVID-19. No geral, observou-se maior atenção da mídia social e redes de cooperação mais intensas nas publicações científicas de acesso aberto se comparadas às de acesso restrito. Ao combinar indicadores bibliométricos e altmétricos, o artigo permitiu não somente caracterizar a evolução e difusão da produção científica sobre a COVID-19, mas compreender a relação entre a pandemia provocada pelo novo coronavírus e a interação da sociedade em torno dos produtos de pesquisa. Nossos resultados reforçam a importância de se implementar práticas de acesso aberto para criar e/ou fortalecer redes de colaboração de pesquisa e estimular publicações sobre temas emergentes e de interesse mundial