Effect of Altmetric score on manuscript citations: A randomized-controlled trial

Characteristics and Impact of Randomized Trials on Drugs or Devices in Cardiovascular Medicine

INTRODUCTION

Clinical trials, essential for medical advancement, vary significantly in methodology and regulatory pathways depending on the type of therapeutic intervention (i.e., drugs or devices). This study aimed to determine whether the drug or device intervention types influence the impact of randomized trials in cardiovascular medicine.

METHODS

We analyzed late-breaking randomized controlled trials presented at major cardiology conferences from 2015 to 2021. The primary endpoint was the total number of citations obtained. Secondary endpoints included the number of citations at 1 and 2 years, number of total and 1-year mentions, and several metrics of study conduct and publication. Statistical analysis included tests for comparisons of continuous or categorical variables, based on their distribution, as appropriate. To adjust the results for potential confounders, univariable and multivariable regression models were utilized. Additionally, sensitivity analyses were conducted to explore both the effect of neutral or positive study outcomes on the comparative impact of drug versus device trials and the impact of the coronavirus disease 2019 (COVID-19) pandemic on the primary endpoint.

RESULTS

Of 382 eligible randomized trials, 227 (59.4%) were trials of drugs and 155 (40.6%) were trials of devices. Drug trials had a higher median number of total citations compared to device studies (93 [interquartile range {IQR} 48-137] vs. 82 [IQR 39-192]; p = 0.025). This difference was consistent at 1 and 2 years and was also observed in the number of total mentions and mentions at 1 year. All the metrics of study conduct and publication were similar, except for drug studies being more often stopped prematurely (8.8 vs. 1.9%; p = 0.006). After adjusting for multiple potential confounders, the difference in citations and mentions was no longer statistically significant. However, drug trials remained more likely to be stopped prematurely (adjusted odds ratio = 1.15; 95% confidence interval 1.03-1.28; p = 0.009). Positive study outcomes significantly influenced the number of citations and the likelihood of a trial being stopped prematurely.

CONCLUSIONS

Drug trials are often stopped early and receive more citations and mentions than device trials. However, these differences are mainly due to factors other than the treatment itself. Studies published simultaneously tend to get more attention, and drug trials with positive results are cited more often than those with neutral results.

A comparison of the top 500 papers in Clinical Nutrition ranked by citation and Altmetric Attention Scores

BACKGROUND AND AIMS

Citation scores (CS) have been traditionally used to measure the impact of scientific publications. Sourced from the Internet, Altmetric Attention Scores (AAS) are complementary metrics that assess how often publications are discussed and used globally. We compared by rank the top 500 papers by CS and AAS published in Clinical Nutrition with corresponding AAS and CS.

METHODS

A search for all publications in Clinical Nutrition was performed on Dimensions (https://app.dimensions.ai/discover/publication) on 3rd April 2024. Outputs were ranked according to CS and then by AAS with the top 500 in each category selected. Scores, year and type of publication were recorded. Correlation was expressed as the Spearman's rank coefficient (ϱ).

RESULTS

We identified 18,790 outputs. Within the top 500 publications ranked by CS, there was a significant weak positive correlation (ϱ = 0.235, P < 0.0001) between CS [median (IQR) 149 (116-223)] and AAS [7 (3-22)]. Ranked by AAS, there was a non-significant very weak positive correlation (ϱ = 0.072, P = 0.106) between AAS [55.5 (36-115)] and CS [42 (16.5-94.5)]. Trends remained similar when grouped by publication type. Guidelines, ranked by CS, had the highest CS and ranked by AAS, the highest CS and AAS. Publications per year, by year, ranked by CS, had a negatively skewed distribution peaking in 2012 and declined thereafter, but when ranked by AAS, had a moderately positive linear trend from 2001 to 2024 (ϱ = 0.513, P < 0.0001).

CONCLUSION

Correlation between CS and AAS was weak. Guidelines had the highest CS and AAS. CS are iterative taking years to mature while AAS are immediate.

The changes in zoological publication rates and focal subdisciplines between 1960 and 2022

Since ancient times, zoology, as the branch of biology dealing with animals, has been a cornerstone of natural science and has developed substantially over the last century. We conducted a bibliometric analysis using structural topic modeling (STM) to determine changes in the representation of principal zoological subdisciplines in the literature between 1960 and 2022. We collated a corpus of 217 414 articles from 88 top-ranked zoology journals and identified three main fields: (i) ecology, (ii) evolution, and (iii) applied research. Within these, we identified 10 major subdisciplines. The number of studies published per year grew from 118 in 1960 to 6635 in 2022. Macroscale-related subdisciplines increased while classical and traditional subdisciplines decreased. Mammals (34.4%) and insects (18.1%) were the dominant taxa covered, followed by birds (15.2%) and fish (8.0%). Research on mammals, insects, and fish involved a broad range of subdisciplines, whereas studies of birds focused on ecological subdisciplines. Most publications were from the United States, followed by the United Kingdom, Germany, Canada, Australia, China, and Japan, with two developing countries, China and South Africa among the top 15 countries. There were different subdiscipline biases between countries, and the gross domestic product of each country correlated positively with its publication output (R2 = 0.681). We discuss our findings in the context of advances in technological innovations and computing power, as well as the emergence of ecology as a formal sister discipline, driven by changing environmental pressures and societal values. We caution that valuable publications from traditional zoological fields must not be completely supplanted by more contemporary topics and increasingly sophisticated analyses.

Controlled experiment finds no detectable citation bump from Twitter promotion

Multiple studies across a variety of scientific disciplines have shown that the number of times that a paper is shared on Twitter (now called X) is correlated with the number of citations that paper receives. However, these studies were not designed to answer whether tweeting about scientific papers causes an increase in citations, or whether they were simply highlighting that some papers have higher relevance, importance or quality and are therefore both tweeted about more and cited more. The authors of this study are leading science communicators on Twitter from several life science disciplines, with substantially higher follower counts than the average scientist, making us uniquely placed to address this question. We conducted a three-year-long controlled experiment, randomly selecting five articles published in the same month and journal, and randomly tweeting one while retaining the others as controls. This process was repeated for 10 articles from each of 11 journals, recording Altmetric scores, number of tweets, and citation counts before and after tweeting. Randomization tests revealed that tweeted articles were downloaded 2.6-3.9 times more often than controls immediately after tweeting, and retained significantly higher Altmetric scores (+81%) and number of tweets (+105%) three years after tweeting. However, while some tweeted papers were cited more than their respective control papers published in the same journal and month, the overall increase in citation counts after three years (+7% for Web of Science and +12% for Google Scholar) was not statistically significant (p > 0.15). Therefore while discussing science on social media has many professional and societal benefits (and has been a lot of fun), increasing the citation rate of a scientist's papers is likely not among them.

Factors Associated With Altmetric Attention Scores for Randomized Phase III Cancer Clinical Trials

PURPOSE

Altmetric Attention Scores (Altmetrics) are real-time measures of scientific impact and attention through various public outlets, including news, blogs, and social media. Herein, we aimed to describe and characterize the relationship between Altmetrics, conventional impact metrics, and features of published cancer clinical trials.

METHODS

We identified two-arm phase III cancer randomized clinical trials with a superiority end point and publication date between 2015 and 2020 from HemOnc and tabulated the following data: Altmetric, study positivity, US Food and Drug Administration (FDA) registration trial status, cancer site/category, treatment context (curative or palliative), trial design, primary end point type, experimental/control arm modality, and journal tier. We further collected conventional bibliometrics including the number of citations and relative citation ratio (RCR) for all published studies. Multiple linear regression modeling identified clinical trial factors predictive of Altmetrics, with alpha = .05 defining statistical significance.

RESULTS

Altmetrics were found for 681 (98%) of 698 publications, with a median score of 38.5 (IQR, 13-132.8). FDA registration studies (β [95% CI], 84.7 [48.8 to 120.6]; P < .001), studies reporting on curative (as opposed to palliative) interventions (-29 [-53.7 to -4.4]; P = .02), genitourinary trials (73.2 [28.1 to 118.2]; P = .001), studies published in tier 1 journals (P < .001), and those with an increased number of citations per year (0.81 [0.66 to 0.95]; P < .001) were significantly associated with increased engagement as measured by Altmetrics. Furthermore, there was a strong correlation between all collected bibliometrics and Altmetrics (R2 = 0.63, 0.68, and 0.67; P < .001 for citation count, citations per year, and RCR, respectively).

CONCLUSION

FDA registration trials describing curative interventions, studies published in traditionally defined high-impact journals, and genitourinary trial publications tend to have the greatest Altmetrics. We observed a strong relationship between Altmetrics and conventional bibliometrics. The significance and consequences of these relationships warrant further investigation.

Headache education by leaflet distribution during COVID-19 vaccination and school-based on-demand e-learning: Itoigawa Geopark Headache Awareness Campaign

OBJECTIVE

We prospectively performed the Itoigawa Headache Awareness Campaign from August 2021 to June 2022, with two main interventions, and evaluated its effectiveness.

BACKGROUND

Headache is a common public health problem, but its burden could be reduced by raising awareness about headache and the appropriate use of acute and prophylactic medication. However, few studies on raising headache awareness in the general public have been reported.

METHODS

The target group was the general public aged 15-64. We performed two main interventions synergistically supported by other small interventions. Intervention 1 included leaflet distribution and a paper-based questionnaire about headache during COVID-19 vaccination, and intervention 2 included on-demand e-learning and online survey through schools. In these interventions, we emphasize the six important topics for the general public that were described in the Clinical Practice Guideline for Headache Disorders 2021. Each response among the two interventions' cohorts was collected on pre and post occasions. The awareness of the six topics before and after the campaign was evaluated.

RESULTS

We obtained 4016 valid responses from 6382 individuals who underwent vaccination in intervention 1 and 2577 from 594 students and 1983 parents in intervention 2; thus, 6593 of 20,458 (32.2%) of the overall working-age population in Itoigawa city experienced these interventions. The percentage of individuals' aware of the six topics significantly increased after the two main interventions ranging from 6.6% (39/594)-40.0% (1606/4016) to 64.1% (381/594)-92.6% (1836/1983) (p < 0.001, all).

CONCLUSIONS

We conducted this campaign through two main interventions with an improved percentage of individuals who know about headache. The two methods of community-based interventions could raise headache awareness effectively. Furthermore, we can achieve outstanding results by doing something to raise disease awareness during mass vaccination, when almost all residents gather in a certain place, and school-based e-learning without face-to-face instruction due to the COVID-19 pandemic.