The Relative Citation Ratio: An Impartial Assessment of Productivity in Academic Otolaryngologists

OBJECTIVES

Quantification of academic productivity relies on bibliometric measurements, such as the Hirsch index (h-index). The National Institutes of Health (NIH) recently developed the relative citation ratio (RCR), an article-level, citation-driven metric that compares researchers with others within their respective fields. Our study is the first to compare the usage of RCR in academic otolaryngology.

STUDY DESIGN

Retrospective Database Review.

METHODS

Academic otolaryngology residency programs were identified using the 2022 Fellowship and Residency Electronic Interactive Database. Demographic and training data were collected for surgeons using institutional websites. RCR was calculated using the NIH iCite tool, and h-index was calculated using Scopus. Mean RCR (m-RCR) is the average score of the author's articles. Weighted RCR (w-RCR) is the sum of all article scores. These derivatives are a measure of impact and output, respectively. The career duration of a physician was categorized into the following cohorts: 0-10, 11-20, 21-30, and 31+ years.

RESULTS

A total of 1949 academic otolaryngologists were identified. Men had higher h-indices and w-RCRs than women (both p less than 0.001). m-RCR was not different between genders (p = 0.083). There was a difference in h-index and w-RCR (both p less than 0.001) among the career duration cohorts, but there was no difference in m-RCR among the cohorts (p = 0.416). The faculty rank professor was the greatest for all metrics (p < 0.001).

CONCLUSION

Critics of the h-index argue that it is reflective of the time a researcher has spent in the field, instead of impact. The RCR may reduce historic bias against women and younger otolaryngologists.

LEVEL OF EVIDENCE

NA Laryngoscope, 134:592-599, 2024.

UK paediatric surgical academic output (2005-2020): A cause for concern?

BACKGROUND

The publication record can be regarded as a key metric of the academic output of a craft surgical speciality with an almost exponential increase in the number of such publications worldwide over the past 20 years (Ashfaq et al. J Surg Res 2018;229:10-11). We aimed to examine and explore if this was the experience within UK paediatric surgery centres.

METHODS

The academic search engine Scopus™ (Elsevier) was used to track every paediatric surgeon's (NHS or University) publication history between Jan. 2005 - Dec. 2020. This was validated by an algorithmic search of PubMed™. The h-index (citations/publication), considered a validated metric of career academic output, was also calculated for each individual surgeon. A Field-Weighted Citation Index (Scopus™) (FWCI) was used to assess impact of individual publications. Textbooks, book chapters, abstracts, duplications ("double dipping") and output attributed to UK BAPS-CASS national studies were excluded. Some output(s), not considered as relevant to "paediatric surgery", was edited. Data are quoted as median(range).

RESULTS

During this 16-year period, there were 3838 publications identified from 26 centres with a "top ten" listing of those paediatric surgical units contributing over half the output (n = 2189, 57%). To look for evidence of trend(s) we analysed the output from these surgical centres in two 5-year periods (2005-9 and 2015-19) and showed an overall fall in output(s) - [730 (53.4%) to 645 (46.4%)] with 6/10 (60%) ' top ten ' centres here recording a reduction in publications. The median h-index of the 232 contributing paediatric surgical consultants was 12 (range 1-56). The best performing publication from the "top ten" centres had 96.5(51-442) citations with the FWCI being 4.5 (2.2 - 30.2).

CONCLUSIONS

This study highlights current paediatric surgery publication output metrics in UK centres. There is evidence of a relative reduction in outputs overall which is a cause for concern for the future, although individual publications from the 10 most active units in the UK remain highly cited. These findings may serve purpose in several ways: (i) UK paediatric surgical centre rankings may be helpful for guiding residency / trainee application; (ii) surgical research funding for the top performing units may be better facilitated and finally (iii) UK centres showing a ' fertile ground ' for nurturing and training paediatric surgeons with academic aspirations could be useful for future workforce planning.

For the further training of individuals in neurosurgery II: the academic legacy of the William P. Van Wagenen Fellowship

OBJECTIVE

William P. Van Wagenen pursued a research fellowship in Europe early in his career under the recommendation of Harvey Cushing. Later, Van Wagenen would be instrumental in the establishment of the William P. Van Wagenen Fellowship, a postgraduate fellowship for neurosurgeons from the AANS that requires study outside of a fellow's country of residency training with plans to return to academic practice.

METHODS

Since 1968, 54 Van Wagenen Fellowships have been awarded, sending 54 fellows from 31 institutions to 13 different countries. The academic productivity of fellows was studied to determine the academic "return on investment" of the fellowship.

RESULTS

Almost all fellows have spent some time in academic neurosurgery (94%), with the vast majority remaining in academics for their entire career (87%); 52% of fellows have received NIH funding, and 55% have been promoted to professor. The numbers are even more striking for the first half of Van Wagenen Fellows (who received the fellowship from its inception in 1968 to 1994) with at least 25 years of career development who remained in academics: 65% received NIH funding, 86% were promoted to professor, and 62% became chairs of academic departments. The Hirsch index of fellows, defined as h papers from an individual with at least h citations, is higher than the national mean and median values for academic neurosurgeons at every academic rank. Fellows have served on national committees and as AANS and CNS presidents and have given back financially to the Neurosurgery Research and Education Foundation (NREF) to fund future research activities of neurosurgical residents and young faculty.

CONCLUSIONS

The Van Wagenen Fellowship will continue to provide young neurosurgeons with opportunities to pursue novel research and network with peers internationally and to motivate young neurosurgeons to transform neurosurgery. The legacy of Cushing and Van Wagenen continues today through the Van Wagenen Fellowship, a legacy that will only continue to grow.

A Systematic Review of Gender-Based Differences in Hirsch Index Among Academic Surgeons

BACKGROUND

The h-index is a commonly used bibliometric in academic medicine which enumerates the number of publications (h) that have been cited h times. Recent investigations have suggested that gender-based differences in h-index may exist among academic physicians. We systematically reviewed studies of academic surgeons' h-index, hypothesizing that a significant difference would exist between the h-index of men and women at all academic ranks.

METHODS

Peer-reviewed journal articles authored by academic surgeons of any subspecialization in the United States between January 1, 2006, and November 20, 2017, were reviewed. We excluded studies of trainees or gender-based differences in funding without mention of h-index. Two reviewers assessed article quality using the Newcastle-Ottawa criteria. Pooled estimates of standard mean differences (SMD) in h-index between genders were calculated using random-effects meta-analyses. A subgroup analysis based on the academic rank was performed. Heterogeneity was assessed using the I² statistic. Sensitivity analyses determined the effect of study on h-index. Meta-regression identified whether surgical specialty contributed to heterogeneity.

RESULTS

Twelve articles comparing h-index between genders were selected from 7950. Men possessed higher h-indices than women (SMD, 0.547; P < 0.001; I² = 89.5%). Men exhibited higher h-indices at the assistant rank (SMD, 0.12; 95% confidence interval [CI], 0.01-0.24; P = 0.039) but not at the associate (SMD, 0.14; 95% CI, -0.06 to 0.33; P = 0.165) or full professor (SMD, 0.12; 95% CI, -0.08 to -0.31; P = 0.25) ranks.

CONCLUSIONS

The h-index is higher for men than that for women in academic surgery overall but not at individual ranks. Further investigations are necessary to address limitations in h-index and to further characterize the relationship between h-index, gender, and promotion.

Stance and influence of Twitter users regarding the Brexit referendum

Social media are an important source of information about the political issues, reflecting, as well as influencing, public mood. We present an analysis of Twitter data, collected over 6 weeks before the Brexit referendum, held in the UK in June 2016. We address two questions: what is the relation between the Twitter mood and the referendum outcome, and who were the most influential Twitter users in the pro- and contra-Brexit camps? First, we construct a stance classification model by machine learning methods, and are then able to predict the stance of about one million UK-based Twitter users. The demography of Twitter users is, however, very different from the demography of the voters. By applying a simple age-adjusted mapping to the overall Twitter stance, the results show the prevalence of the pro-Brexit voters, something unexpected by most of the opinion polls. Second, we apply the Hirsch index to estimate the influence, and rank the Twitter users from both camps. We find that the most productive Twitter users are not the most influential, that the pro-Brexit camp was four times more influential, and had considerably larger impact on the campaign than the opponents. Third, we find that the top pro-Brexit communities are considerably more polarized than the contra-Brexit camp. These results show that social media provide a rich resource of data to be exploited, but accumulated knowledge and lessons learned from the opinion polls have to be adapted to the new data sources.

Academic productivity among fellowship associated adult total joint reconstruction surgeons

Background

The Hirsch index (h-index) is a measure that evaluates both research volume and quality-taking into consideration both publications and citations of a single author. No prior work has evaluated academic productivity and contributions to the literature of adult total joint replacement surgeons. This study uses h-index to benchmark the academic impact and identify characteristics associated with productivity of faculty members at joint replacement fellowships.

Methods

Adult reconstruction fellowship programs were obtained via the American Association of Hip and Knee Surgeons website. Via the San Francisco match and program-specific websites, program characteristics (Accreditation Council for Graduate Medical Education approval, academic affiliation, region, number of fellows, fellow research requirement), associated faculty members, and faculty-specific characteristics (gender, academic title, formal fellowship training, years in practice) were obtained. H-index and total faculty publications served as primary outcome measures. Multivariable linear regression determined statistical significance.

Results

Sixty-six adult total joint reconstruction fellowship programs were identified: 30% were Accreditation Council for Graduate Medical Education approved and 73% had an academic affiliation. At these institutions, 375 adult reconstruction surgeons were identified; 98.1% were men and 85.3% had formal arthroplasty fellowship training. Average number of publications per faculty member was 50.1 (standard deviation 76.8; range 0-588); mean h-index was 12.8 (standard deviation 13.8; range 0-67). Number of fellows, faculty academic title, years in practice, and formal fellowship training had a significant (P < .05) positive correlation with both h-index and total publications.

Conclusions

The statistical overview presented in this work can help total joint surgeons quantitatively benchmark their academic performance against that of their peers.