Factors Associated With the Highest and Lowest Cited Research Articles in Urology Journals

Anesthesiology Articles Published in 2020: A Review and Characterization of COVID-19 Versus Non-COVID-19 Publications in Top Anesthesiology Journals

The coronavirus disease 2019 (COVID-19) pandemic has had a significant impact on the practice of medicine worldwide, particularly in anesthesiology. As the clinical realm has rapidly adjusted to the realities of the pandemic, anesthesiology literature has also changed significantly to reflect this. The purpose of this study was to characterize the effects the COVID-19 pandemic has had on anesthesiology literature. Specifically, it was hypothesized that the COVID-19-related literature in the anesthesiology community would gain more interest than non-COVID-19-related articles. A total of 15 anesthesiology-related journals with the highest impact factor in 2019, according to the Journal Citation Reports (JCR), were selected for data collection. An advanced PubMed search identified 5,722 COVID-19-related articles published by these journals in 2020. Next, articles with titles including “corona,” “COVID,” “COVID-19,” “pandemic,” “SARS,” or “SARS-CoV-2” were selected for inclusion in the study, which resulted in 676 (12%) articles. A Kruskal-Wallis test was used to assess the Altmetric score, which is a weighted calculation of the attention an article receives online, for COVID-19 versus non-COVID-19 articles. Articles were then further characterized across multiple different variables, including country of origin, month published, type of article, and subspecialty of anesthesiology it pertained to. Of the 15 journals investigated, 676 (12%) articles of the 5,722 total articles published were found to be COVID-19-related material. The majority of the articles were found to be published in April (18%), May (19.5%), and June (14%). The majority of these articles were related either to general anesthesia (operating room anesthesiology that is not tied to a particular subspecialty fellowship track) (48%) or critical care (39%). By article type, most were determined to be editorial (71%) in nature, followed by original research articles (21%), of which most were cross-sectional (55%) studies. When compared with non-COVID-19-related articles, COVID-19-related articles had a significantly greater Altmetric score (29.518 versus 8.6333, p < 0.001). Of the COVID-19-related articles, original articles had the greatest Altmetric score, when compared to editorials and guidelines (54.794 versus 20.777 versus 40.643, p < 0.002). The response of the academic anesthesiology community to the COVID-19 pandemic was strong and timely, with a particularly strong focus on critical care anesthesia. The impact of the pandemic was strongly felt by the anesthesiology community, and their timely response served to guide our country and world through an incredibly challenging time. The pandemic highlighted the value of anesthesiologists worldwide, not only in the operating room setting but particularly as critical care physicians.

Characteristics of the least-cited and most-cited articles in ophthalmology journals: A pilot study

BACKGROUND

Limited research has examined differences between uncited papers and their most-cited counterparts. By comparing characteristics of each cohort, it is possible to better determine factors associated with increased citation count in the ophthalmology literature.

METHODS

We initially identified all research articles published in six popular general ophthalmology journals (Ophthalmology, JAMA Ophthalmology, Investigative Ophthalmology and Visual Sciences, American Journal of Ophthalmology, British Journal of Ophthalmology, and Graefe's Archive for Clinical and Experimental Ophthalmology) between 2001 and 2011. Forty-nine articles were identified as having accrued zero citations as of March 2021 and were compared with an equivalent number of articles with the highest number of citations published in the same journals and time period. Significance (p < 0.05) for comparisons was determined using the Mann-Whitney U test and Fisher's exact test.

RESULTS

Compared to the least-cited articles, the most-cited articles were significantly more likely to be clinical, multi-institutional, and multi-national in scope, report a statistically significant result, have a conflict of interest, state a funding source, and have higher sample sizes. These publications had significantly more words in the abstract and manuscript and more references. Overall, the first authors of the most-cited articles were significantly more likely to be female and report greater prior research productivity, as assessed by the relative citation ratio (RCR).

CONCLUSION

Considering a small number of articles were uncited at least a decade after publication, it appears most research is useful for future investigations. However, there remain distinct differences between uncited articles and their most-cited equivalents in ophthalmology.

Evaluating the Highest- and Lowest-cited Research Articles in the Cardiothoracic Surgery Literature

BACKGROUND

Understanding the differences between articles that amass a high number of citations and those that receive very few allows investigators to write journal articles that maximize the impact of their research. There are minimal data regarding these two cohorts in the cardiothoracic surgery literature.

METHODS

We identified all primary research articles from 1998 to 2008 from The Journal of Thoracic and Cardiovascular Surgery, The Journal of Cardiac Surgery, The Annals of Thoracic Surgery, and The European Journal of Cardio-Thoracic Surgery (n = 4276). Eighty-seven of these articles accrued 0 or only 1 citation within 10 y of publication. We compared this "low citation" cohort to the "high citation" cohort made up of the 87 highest-cited articles from the same journals over the same time period.

RESULTS

When compared with the low-citation articles, high-citation articles were significantly more likely to be clinical in nature (P < 0.0001), have observational study design (P < 0.0001), involve multidisciplinary authorship (P < 0.0001), and have more funding reported (P = 0.0039). With regard to technical aspects of the article, the high-citation articles were likely to have longer titles (P = 0.0086), punctuation in the title (P = 0.0027), longer abstracts (P = 0.0007), more words in the manuscript (P < 0.0001), more authors (P < 0.0001), more declared conflict of interests (P = 0.0167), more references (P < 0.0001), more tables (P < 0.0001), more figures (P = 0.0024), and more pages (P < 0.0001). There was no significant difference in the year of publication among both cohorts.

CONCLUSIONS

This review suggests that there are several important distinguishing characteristics that should be considered by investigators when designing and implementing cardiothoracic research studies to maximize the impact of their published research.

Defining Common Features in High Impact and Highly Cited Journal Articles on Pancreatic Tumors: An Analysis of 1044 Studies over the Past Decade

INTRODUCTION

Surgical researchers seek to publish their findings in esteemed surgical journals to advance science and their careers. A detailed investigation of study and manuscript attributes in a specific research area, like pancreatic neoplasia, may yield informative insights for researchers looking to maximize research impact.

OBJECTIVES

We analyzed publications related to pancreatic surgery primarily focused on pancreatic and periampullary tumors in order to identify elements associated with acceptance into high impact journals and a high likelihood of future citations.

METHODS

A comprehensive review of nine surgical journals was performed between 2010-19. Journals were grouped based on impact factor into high (>3), medium (1-3) and low (<1) impact categories. Each publication was annotated to identify study topic, methodology and statistical approach. Findings were compared according to journal impact and number of citations to identify predictors of success across these two domains.

RESULTS

A total of 1,044 out of 21,536 (4.8%) articles published in the index journals were related to pancreatic tumors. The most common focus of study was perioperative outcomes and complications (46.7%). There was significantly more number of authors, participating institutions, countries, and randomized clinical trials in higher impact journals as well as high-cited articles (p<0.05). Though advanced statistical analysis was used more commonly in high-impact journals (p<0.05), it did not translate to higher citations (p>0.05).

CONCLUSION

Pancreatic neoplasia continues to be extensively studied in surgical literature. Specific elements of study methodology and design were identified as potentially key attributes to acceptance in high impact journals and citation success.

Factors Associated with Low and High Article Citations in Four Prominent Orthopaedic Surgery Journals

BACKGROUND

Research is a cornerstone for the advancement of clinical practice and guidelines across all medical and surgical fields. To achieve significant contribution to the field, research must be circulated, read, and understood.

PURPOSE

The purpose of this paper is to investigate which factors are associated with higher and lower citation rates in orthopaedic surgery literature.

METHODS

A query was performed to identify all of the primary research articles published between 1998 and 2008 in four prominent orthopaedics journals. From there, the 50 most highly and lowly cited articles were identified and analyzed for various factors that distinguished the highly cited group from the lowly cited group. Various statistical tests were used depending on the type of variable being evaluated.

RESULTS

After data compilation and statistical analysis, 16 statistically significant factors were apparent that differed between the two groups. Seven non-statistically significant factors were also identified.

CONCLUSION

This study illustrates that certain statistically significant factors influence the citation rates of papers in orthopaedic surgery literature. If utilized appropriately, these factors could lead to increased consumption and circulation of future orthopaedic surgery literature.

Differences in Highly-Cited and Lowly-Cited Manuscripts in Plastic Surgery

BACKGROUND

There is a paucity of research comparing journal articles that accrue numerous citations with those that accrue few citations over time. Understanding differences between journal articles can help direct investigators in designing and conducting their research.

METHODS

Using advanced bibliometric tools, we queried four plastic surgery journals (Journal of Reconstructive Microsurgery, Annals of Plastic Surgery, Plastic and Reconstructive Surgery, and Microsurgery) for primary research articles published between 1998 and 2008 accruing zero or one citations with at least a 10-y lag time. Forty-seven articles were identified as low citation and were compared with an equal number of articles in the same journals that accrued the highest number of citations in the same period as high citation (HC). The data were analyzed using Student t-tests, Wilcoxon rank sum tests, chi-square tests, and Fisher exact tests. The level of significance was established at P < 0.05.

RESULTS

When compared with the HC cohort, the low citation articles were more likely to be nonclinical (P < 0.001), have no plastic surgery authors (P = 0.0026), and focus on the field of microsurgery (P = 0.003). The HC cohort was more likely to have higher sample sizes (P = 0.0339), focus on aesthetic/cosmetic surgery (P = 0.003), have a higher number of other disciplines included on authorship (P < 0.001), references (P = 0.0451), manuscript pages (P < 0.001), and words in the abstract (P < 0.001).

CONCLUSIONS

A small number of articles published in four plastic surgery journals were uncited during a 10-y period. There are qualitative and quantitative differences between highly and lowly cited articles in the plastic surgery literature. Investigators should consider these differences when designing and conducting studies.

Factors Associated With the Highest and Lowest Cited Research Articles in General Surgery Journals

BACKGROUND

Citation count is a common bibliometric tool used to determine the long-term impact and performance of journal articles. Many of the other potential factors associated with highly and lowly cited articles in the general surgery literature, however, remain unknown. The purpose of this study was to attempt to identify characteristics of articles that may predict or correlate with article citation counts and, consequently, article impact.

METHODS

We identified articles from Annals of Surgery, British Journal of Surgery, and Journal of the American College of Surgeons between 1998 and 2008 that had 0-5 total citations. We then matched these articles to an identical number of the highest cited articles from these same journals for comparison. Student's t-tests, Wilcoxon rank-sum tests, chi-squared tests, and Fisher's exact tests were used to determine the significance of difference between data sets at a predetermined level of significance set to P < 0.05.

RESULTS

Significant differences of article characteristics between the two cohorts included higher prevalence of clinical studies (P = 0.3919), multi-institutional (P = 0.0007) and multi-national (P = 0.0023) studies, surgical oncology (P < 0.0001) or hepatobiliary focus (P < 0.0001) and published in Annals of Surgery (P < 0.0001) for the highly cited cohort. Highly cited articles were also more likely to have larger sample sizes (P = 0.0009), more authors (P < 0.0001), presence of statistically significant results (P < 0.0001), more references (P < 0.0001), more tables (P < 0.0001), more figures (P = 0.0001), and higher word counts for manuscript (P < 0.0001), abstract (P < 0.0001), and title (P < 0.0001).

CONCLUSIONS

There are a relatively small number of articles with 0-5 citations after 10 y for these major general surgery journals. This indicates that journals are consistently able to select articles that will be impactful in aiding future research. Certain factors, however, are associated with being highly cited as opposed to lowly cited, and an understanding of these factors can aid researchers and journals in designing and reporting future studies.

Examining the Correlation Between Altmetric Score and Citations in the General Surgery Literature

BACKGROUND

With the emergence of social media platforms, new bibliometric profiles measuring impact and exposure of scientific research online have been introduced as an alternative to traditional bibliometric outcomes. The objective of this article is to evaluate relationships between Altmetric scores, journal impact factor, and citation counts among the surgical literature.

METHODS

We analyzed the top 10 highest cited articles for the 10 general surgery journals with the highest impact factors for 2013 and 2016 by noting citation counts and Altmetric scores for each article. We also identified the journal impact factor and age of journal associated Twitter accounts. Variables were assessed for correlation using Pearson's correlation testing via Microsoft Excel.

RESULTS

A total of 240 articles were analyzed. For 2013, Altmetrics score analysis demonstrated a significant, positive correlation with citation number (r = 0.462, P < 0.0001) and journal impact factor (r = 0.439, P < 0.0001). The 2016 cohort also demonstrated significant, positive correlations between Altmetric scores and citation count after the removal of one outlier (r = 0.182, P = 0.047) and journal impact factor when considering all articles (r = 0.425, P < 0.0001). From 2013 to 2016, the total number of citations for all articles decreased from 11,027 to 7661, but cumulative Altmetric scores increased from 1078 to 4782. Age of creation for a journal's Twitter account did not significantly affect Altmetric score or traditional bibliometric measures in either 2013 (r = 0.370, P = 0.293) or 2016 (r = 0.441, P = 0.202).

CONCLUSIONS

Altmetric scores, while significantly associated with citation count in the surgical literature, should not necessarily be used as a surrogate marker for evaluating research performance, impact, or exposure. It is possible, however, that as the use of social media for distributing and sharing scientific research continues to expand, that exposure on such platforms could impact future interest or studies.

Analyzing the relationship between Altmetric score and literature citations in the Implantology literature

BACKGROUND

The influence of research has long been studied using citations and impact factors (IFs). Electronic media is changing how people interact with the scientific literature. There are few investigations into these trends.

PURPOSE

To explore whether Altmetrics correlate with traditional bibliometrics in the Implantology literature.

MATERIALS AND METHODS

Five Implantology journals with the highest IF and the 10 most highly-cited articles within those journals from 2013 to 2016 were reviewed. Altmetric score, citation count, and media "mentions" were recorded. Comparisons were conducted between Altmetric score, citations, and IF by performing Pearson correlation coefficients and descriptive statistics. Twitter accounts were studied and compared to other metrics.

RESULTS

Analysis revealed no correlation between citations and Altmetrics (r = .096,P = .506) or IF and Altmetrics (r = .111,P = .443) in 2013. Altmetrics were also not significantly correlated with citations (r = 0.148,P = .305) or IF (r = .145,P = .315) in 2016. Total Altmetric scores were nine times higher in 2016 compared to 2013, with news outlets and Twitter seeing large increases in mentions. Twitter was the top medium receiving mentions across the two cohorts.

CONCLUSIONS

Compared to other fields, Implantology articles received lower Altmetric scores, noting an area of improvement. Altmetrics at this time are insufficient to replace traditional bibliometrics, but may provide helpful real-time information concerning article dissemination.

Examining the Correlation Between Altmetric Score and Citations in the Urology Literature

OBJECTIVE

To determine if article Altmetric scores correlate with journal impact factor and citation count in the urologic literature.

METHODS

We identified the top 10 most-cited articles for the 15 urology journals with the highest impact factor in 2013 and 2016. Citation count and Altmetric scores were recorded for each of the articles. The journal impact factor and date of Twitter account development were recorded for each of the journals. The variables were analyzed in Microsoft excel using Pearson's correlation testing.

RESULTS

A total of 300 articles were analyzed. In 2013, Altmetric scores and citation number showed a significant positive correlation (r = 0.164, P = .045), although Altmetric scores did not correlate with journal impact factor (r = 0.005, P = .957). In 2016, there was significant positive correlation between Altmetric scores and citation number (r = 0.268, P = .0009), as well as between Altmetric scores and journal impact factor (r = 0.201, P = .014). The total citation count decreased from 15,235 in 2013 to 8622 in 2016 while the total Altmetric score increased from 1135 in 2013 to 2563 in 2016. Older Twitter accounts were not associated with increasing correlations between Altmetric score and bibliometrics in either 2013 (r = 0.221, P = .54) or 2016 (r = 0.083, P = .819).

CONCLUSION

At this point in time, Altmetric score is only weakly correlated with citation counts in the urology literature. Altmetrics and traditional bibliometrics should be viewed as complements to one another rather than surrogates when determining research dissemination and impact.