1
|
Revercomb L, Patel AM, Tripathi OB, Filimonov A. Factors Associated with Research Productivity and National Institutes of Health Funding in Academic Otology. Laryngoscope 2024; 134:3786-3794. [PMID: 38529707 DOI: 10.1002/lary.31408] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/25/2023] [Revised: 03/05/2024] [Accepted: 03/08/2024] [Indexed: 03/27/2024]
Abstract
OBJECTIVES Bibliometrics, such as the Hirsch index (h-index) and the more recently developed relative citation ratio (RCR), are utilized to evaluate research productivity. Our study evaluates demographics, research productivity, and National Institutes of Health (NIH) funding in academic otology. METHODS Academic otologists were identified, and their demographics were collected using institutional faculty profiles (N = 265). Funding data were obtained using the NIH Research Portfolio Online Reporting Tools Expenditures and Reports Database. The h-index was calculated using Scopus and mean (m-RCR) and weighted RCR (w-RCR) were calculated using the NIH iCite tool. RESULTS H-index (aOR 1.18, 95% CI 1.10-1.27, p < 0.001), but not m-RCR (aOR 1.50, 95% CI 0.97-2.31, p = 0.069) or w-RCR (aOR 1.00, 95% CI 0.99-1.00, p = 0.231), was associated with receiving NIH funding. Men had greater h-index (16 vs. 9, p < 0.001) and w-RCR (51.8 vs. 23.0, p < 0.001), but not m-RCR (1.3 vs. 1.3, p = 0.269) than women. Higher academic rank was associated with greater h-index and w-RCR (p < 0.001). Among assistant professors, men had greater h-index than women (9.0 vs. 8.0, p = 0.025). At career duration 11-20 years, men had greater h-index (14.0 vs. 8.0, p = 0.009) and w-RCR (52.7 vs. 25.8, p = 0.022) than women. CONCLUSION The h-index has a strong relationship with NIH funding in academic otology. Similar h-index, m-RCR, and w-RCR between men and women across most academic ranks and career durations suggests production of similarly impactful research. The m-RCR may correct some deficiencies of time-dependent bibliometrics and its consideration in academic promotion and research funding allocation may promote representation of women in otology. LEVEL OF EVIDENCE N/A Laryngoscope, 134:3786-3794, 2024.
Collapse
Affiliation(s)
- Lucy Revercomb
- Department of Otolaryngology-Head and Neck Surgery, Rutgers New Jersey Medical School, Newark, New Jersey, U.S.A
| | - Aman M Patel
- Department of Otolaryngology-Head and Neck Surgery, Rutgers New Jersey Medical School, Newark, New Jersey, U.S.A
| | - Om B Tripathi
- Department of Otolaryngology-Head and Neck Surgery, Rutgers New Jersey Medical School, Newark, New Jersey, U.S.A
| | - Andrey Filimonov
- Department of Otolaryngology-Head and Neck Surgery, Rutgers New Jersey Medical School, Newark, New Jersey, U.S.A
| |
Collapse
|
2
|
Patel PA, Keane CA, Akhter MF, Fang AH, Soto E, Boyd CJ. Examination of the Novel National Institutes of Health-Supported Relative Citation Ratio, a Measure of Research Productivity, Among Academic Plastic Surgeons. Ann Plast Surg 2023; 91:143-148. [PMID: 37347161 DOI: 10.1097/sap.0000000000003567] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 06/23/2023]
Abstract
BACKGROUND Considerable interest has been devoted to quantifying research productivity for the purposes of academic appointment and promotion in plastic surgery. A novel bibliometric, the relative citation ratio (RCR), integrates features unavailable in prior metrics, such as the h-index, including the ability to compare researchers in distinct fields. This investigation examines the RCR in relation to established measures of academic productivity and provides the benchmark data in plastic surgery. METHODS Online sources were queried to identify the characteristics of 955 academic plastic surgeons from 94 programs, ie, academic rank, gender, degrees, and fellowships. Bibliometric data were acquired using the iCite and Scopus databases. Comparative and correlational analyses of variables were performed. RESULTS Academic plastic surgeons were exceptionally productive, with a mean RCR of 1.20 (interquartile range, 0.79-1.67) and a weighted RCR of 17.68 (interquartile range, 5.14-52.48). Increased mean RCR was significantly associated with advanced academic rank and fellowship training. Increased weighted RCR was significantly associated with advanced academic rank, male gender, PhD acquisition, publication experience, and fellowship training. The h-index was weakly correlated with mean RCR but strongly correlated with weighted RCR and publication experience. CONCLUSIONS The RCR was associated with established markers of academic productivity, indicating its validity as a reliable field-normalized measure for the evaluation of plastic surgery faculty. Because scholarly output is a potential factor with respect to decisions of hiring, promotion, and allocation of funding, this modality of standardized comparison is paramount for plastic surgeons who exist within a larger general surgery department.
Collapse
Affiliation(s)
| | - Charles A Keane
- Central Michigan University College of Medicine, Mount Pleasant, MI
| | - Maheen F Akhter
- Central Michigan University College of Medicine, Mount Pleasant, MI
| | | | | | - Carter J Boyd
- NYU Langone Hansjörg Wyss Department of Plastic Surgery, NYU Langone Health, New York, NY
| |
Collapse
|
3
|
Conway J, Ricardo JW, Lipner SR. Dermatology Receives Fewer Grants Versus Other Specialties but Excels in Citation Impact in a Cross-sectional Analysis of R01 Grants 2000-2022. Dermatol Pract Concept 2023; 13:e2023162. [PMID: 37557156 PMCID: PMC10412025 DOI: 10.5826/dpc.1303a162] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 01/21/2023] [Indexed: 08/11/2023] Open
Affiliation(s)
- Jade Conway
- New York Medical College, Valhalla, New York, USA
| | - Jose W. Ricardo
- Department of Dermatology, Weill Cornell Medicine, New York, New York, USA
| | - Shari R. Lipner
- Department of Dermatology, Weill Cornell Medicine, New York, New York, USA
| |
Collapse
|
4
|
Levin G, Pareja R, Harrison R, Ramirez PT, Meyer R. Association of literature metrics in gynecologic oncology with country classification by income level. Int J Gynecol Cancer 2023:ijgc-2023-004380. [DOI: 10.1136/ijgc-2023-004380] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 04/03/2023] Open
Abstract
ObjectiveBibliometric literature in gynecologic oncology is limited. We aimed to study the association between the level of income of the country of authorship and citation metrics.MethodsA retrospective study including all articles and reviews published during 1977–2022 in theInternational Journal of Gynecological Cancer(IJGC) andGynecologic Oncologyjournals. Country of origin was defined as the corresponding author’s address. We classified articles into groups by level of income of the country of origin, as defined by the World Bank. The primary outcome measure was the median number of citations per year.ResultsA total of 9835 articles were included in the analysis (IJGCn=3786 (38.5%),Gynecologic Oncologyn=6049 (61.5%)). There were 8587 (87.3%) publications from high income countries, 1134 (11.5%) from upper-middle income countries, and 114 (1.2%) from lower-middle income countries. There were no publications from countries of low income. Most publications originated in the United States with 4089 (41.6%), followed by China (n=730, 7.4%), Italy (n=533, 5.4%), Canada (n=467, 4.7%), and Japan (n=461, 4.7%). Over the most recent 5 years there was a decrease in the representation of upper-middle income countries and lower-middle income countries; 16.3% (91/557) in 2018 versus 9.1% (38/417) in 2022 (p=0.005). In a multivariable regression analysis that included year of publication, open access publication model, study being supported by funding, publishing journal, review article, and level of income, all factors were associated with high citation per year score except the income classification of the article’s country of origin (adjusted OR 1.59–1.72, 95% CI 0.61 to 4.30).ConclusionHigh income countries have a disproportionate representation in gynecologic oncology publications. After adjusting for confounders, the country’s level of income was not independently associated with a high citation per year score. This implies that the number of citations per year is not compromised by the country’s level of income.
Collapse
|
5
|
Gupta A, Reddy V, Barpujari A, Lavadi RS, Agarwal P, Chang YF, Mooney J, Elsayed GA, Agarwal N. Current Trends in Subspecialty Fellowship Training for 1691 Academic Neurological Surgeons. World Neurosurg 2023; 171:e47-e56. [PMID: 36403934 DOI: 10.1016/j.wneu.2022.11.074] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/23/2022] [Accepted: 11/16/2022] [Indexed: 11/19/2022]
Abstract
OBJECTIVE Neurosurgical subspecialty fellowship training has become increasingly popular in recent decades. However, few studies have evaluated recent trends in postgraduate subspecialty education. This study aims to provide a detailed cross-sectional analysis of subspecialty fellowship training completion trends and demographics among U.S. academic neurosurgeons. METHODS Academic clinical faculty (M.D. or D.O.) teaching at accredited neurosurgery programs were included. Demographic, career, and fellowship data were collected from departmental physician profiles and the American Association of Neurological Surgeons (AANS) membership database. Relative citation ratio scores were retrieved using the National Institutes of Health iCite tool. RESULTS This study included 1691 surgeons (1756 fellowships) from 125 institutions. The majority (79.13%) reported fellowship training. Fellowship completion was more common among recent graduates (residency year >2000), as was training in multiple subspecialties (P < 0.0001). Spine was the most popular subspecialty (16.04%), followed by pediatrics (11.18%), and cerebrovascular (9.46%). The least common were trauma/critical care (2.52%) and peripheral nerve (1.26%). Spine, neuroradiology, and endovascular subspecialties grew in popularity over time. Pediatrics and spine were the most popular for females and males, respectively. Epilepsy and cerebrovascular had the most full professors, while endovascular and spine had the most assistant professors. Stereotactic/functional and epilepsy had the most Ph.Ds. Fellowship training correlated with higher weighted, but not mean, relative citation ratio scores among associate (P = 0.002) and full professors (P = 0.005). CONCLUSIONS There is an emerging proclivity for additional fellowship training among young neurosurgeons, often in multiple subspecialties. These findings are intended to help guide professional decision-making and optimize the delivery of postgraduate education.
Collapse
Affiliation(s)
- Arjun Gupta
- Department of Neurological Surgery, Rutgers New Jersey Medical School, Newark, New Jersey, USA
| | - Vamsi Reddy
- Department of Neurological Surgery, University of Texas Health San Antonio, San Antonio, Texas, USA
| | - Awinita Barpujari
- Department of Neurological Surgery, Drexel University College of Medicine, Philadelphia, Pennsylvania, USA
| | - Raj Swaroop Lavadi
- Department of Neurological Surgery, Washington University School of Medicine, St. Louis, Missouri, USA
| | - Prateek Agarwal
- Department of Neurological Surgery, University of Pittsburgh Medical Center, Pittsburgh, Pennsylvania, USA
| | - Yue-Fang Chang
- Department of Neurological Surgery, University of Pittsburgh Medical Center, Pittsburgh, Pennsylvania, USA
| | - James Mooney
- Department of Neurological Surgery, The University of Alabama at Birmingham, Birmingham, Alabama, USA
| | - Galal A Elsayed
- Department of Neurological Surgery, Washington University School of Medicine, St. Louis, Missouri, USA
| | - Nitin Agarwal
- Department of Neurological Surgery, Washington University School of Medicine, St. Louis, Missouri, USA.
| |
Collapse
|
6
|
Pizzo DM, Imam N, Sudah SY, Faccone RD, Akinrodoye M, Manzi JE, Menendez ME, Gesell MW. Evaluation of the National Institutes of Health-Supported Relative Citation Ratio Among Fellowship-Trained American Orthopaedic Joint Reconstruction Surgery Faculty: A New Bibliometric Measure of Scientific Influence. J Arthroplasty 2023; 38:165-170. [PMID: 35940351 DOI: 10.1016/j.arth.2022.07.022] [Citation(s) in RCA: 4] [Impact Index Per Article: 4.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 05/12/2022] [Revised: 07/21/2022] [Accepted: 07/25/2022] [Indexed: 02/02/2023] Open
Abstract
BACKGROUND The relative citation ratio (RCR), a novel National Institutes of Health-Supported measure of research productivity, allows for accurate interdisciplinary comparison of publication influence. This study evaluates the RCR of fellowship-trained adult reconstructive orthopaedic surgeons with the goal of analyzing potentially influential physician demographics. METHODS Adult Reconstruction Accreditation Council for Graduate Medical Education fellowship-trained faculty for orthopaedic residency programs were identified via departmental websites. The National Institutes of Health's iCite database was retrospectively reviewed for mean RCR, weighted RCR, and publication count by surgeon. Multivariate analyses were performed using the Wilcoxon rank-sum tests and analyses of variance testing to compare sex, career length, academic rank, and professional degrees in addition to an MD or DO. Significance was considered P < .05. RESULTS A total of 488 fellowship-trained adult reconstruction faculty from 144 programs were included in the analysis. Overall, the faculty recorded a median RCR of 1.65 (interquartile range: 1.01-2.28) and a median weighted RCR of 16.59 (interquartile range: 3.98-61.92). The weighted RCR and total number of publications were associated with academic rank and career longevity, while the mean RCR was associated with academic rank. The median RCR ranged from 1.12 to 1.87 for all subgroups. CONCLUSION Adult reconstruction faculty are exceptionally productive and generate highly impactful studies as evidenced by the high median RCR value relative to the National Institutes of Health standard value of 1.0. Our data have important implications in the assessment of grant outcomes, promotion, and continued evaluation of research influence within the hip and knee community.
Collapse
Affiliation(s)
- Dane M Pizzo
- Orthopaedic Surgery, Rutgers Robert Wood Johnson Medical School, New Brunswick, New Jersey
| | - Nareena Imam
- Orthopaedic Surgery, Rutgers Robert Wood Johnson Medical School, New Brunswick, New Jersey
| | - Suleiman Y Sudah
- Orthopaedic Surgery, Monmouth Medical Center, Long Branch, New Jersey
| | - Robert D Faccone
- Research, Alabama College of Osteopathic Medicine, Dothan, Alabama
| | - Micky Akinrodoye
- Orthopaedic Surgery, Monmouth Medical Center, Long Branch, New Jersey
| | | | | | - Mark W Gesell
- Orthopaedic Surgery, Monmouth Medical Center, Long Branch, New Jersey
| |
Collapse
|
7
|
Gupta A, Meeter A, Norin J, Ippolito JA, Beebe KS. The relative citation ratio (RCR) as a novel bibliometric among 2511 academic orthopedic surgeons. J Orthop Res 2022. [PMID: 36403125 DOI: 10.1002/jor.25490] [Citation(s) in RCA: 3] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/11/2022] [Revised: 11/07/2022] [Accepted: 11/15/2022] [Indexed: 11/21/2022]
Abstract
Objectively measuring research output is important for grant awards, promotion, and tenure, or self-evaluation of productivity. However, certain shortcomings limit common bibliometric indicators. The time- and field-independent relative citation ratio (RCR) was proposed to overcome these limitations. The objective of this study was to determine whether the RCR correlates with academic rank, gender, and PhD degree status among US academic orthopedic surgeons. Full-time faculty surgeons at Accreditation Council for Graduate Medical Education-accredited orthopedic surgery residency programs were included in this study. Mean (mRCR) and weighted (wRCR) RCR scores were collected from the National Institutes of Health iCite database to quantify scholarly "impact" and "production," respectively, and were compared by academic rank, gender, and PhD status. All information was collected from publicly available faculty listings on departmental websites. A total of 2511 orthopedic surgeons from 132 residency programs were assessed. Overall, the median (interquartile range) mRCR score was 1.56 (1.05-2.12) and the median wRCR score was 27.6 (6.97-88.44). Both metrics increased with each successive academic rank, except for department chairs. There was no difference in mRCR between male and female surgeons. Among assistant professors, males had higher wRCR scores. Both metrics were higher among surgeons with a PhD degree. The RCR offers key advantages over other indices, which are reflected in differences in score distributions compared with the widely used h-index. Nevertheless, implementation of the RCR should be preceded with careful consideration of its own limitations.
Collapse
Affiliation(s)
- Arjun Gupta
- Department of Orthopaedic Surgery, Rutgers New Jersey Medical School, Newark, New Jersey, USA
| | - Alexandra Meeter
- Department of Orthopaedic Surgery, Rutgers New Jersey Medical School, Newark, New Jersey, USA
| | - Jorden Norin
- Department of Orthopaedic Surgery, Rutgers New Jersey Medical School, Newark, New Jersey, USA
| | - Joseph A Ippolito
- Department of Orthopaedic Surgery, Rutgers New Jersey Medical School, Newark, New Jersey, USA
| | - Kathleen S Beebe
- Department of Orthopaedic Surgery, Rutgers New Jersey Medical School, Newark, New Jersey, USA
| |
Collapse
|
8
|
Co-citation Percentile Rank and JYUcite: a new network-standardized output-level citation influence metric and its implementation using Dimensions API. Scientometrics 2022. [DOI: 10.1007/s11192-022-04393-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/18/2022]
Abstract
AbstractJudging value of scholarly outputs quantitatively remains a difficult but unavoidable challenge. Most of the proposed solutions suffer from three fundamental shortcomings: they involve (i) the concept of journal, in one way or another, (ii) calculating arithmetic averages from extremely skewed distributions, and (iii) binning data by calendar year. Here, we introduce a new metric Co-citation Percentile Rank (CPR), that relates the current citation rate of the target output taken at resolution of days since first citable, to the distribution of current citation rates of outputs in its co-citation set, as its percentile rank in that set. We explore some of its properties with an example dataset of all scholarly outputs from University of Jyväskylä spanning multiple years and disciplines. We also demonstrate how CPR can be efficiently implemented with Dimensions database API, and provide a publicly available web resource JYUcite, allowing anyone to retrieve CPR value for any output that has a DOI and is indexed in the Dimensions database. Finally, we discuss how CPR remedies failures of the Relative Citation Ratio (RCR), and remaining issues in situations where CPR too could potentially lead to biased judgement of value.
Collapse
|
9
|
Grace ZT, Patel H, Omari AM, Sanders A, Imam N, Koerner JD. Evaluating the Academic Influence of Orthopedic Surgeons in Spinal Literature Through Relative Citation Ratio. Cureus 2022; 14:e25147. [PMID: 35733502 PMCID: PMC9205450 DOI: 10.7759/cureus.25147] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 05/19/2022] [Indexed: 11/05/2022] Open
|
10
|
Rogers JL, Barpujari A, Reddy VP. Letter to the Editor Regarding "Bibliometric Evaluation of U.S. Neurosurgery Subspecialties and Academic Rank Using RCR Index". World Neurosurg 2022; 159:244-245. [PMID: 35255623 DOI: 10.1016/j.wneu.2021.11.132] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/29/2021] [Accepted: 11/30/2021] [Indexed: 12/01/2022]
Affiliation(s)
- James L Rogers
- School of Science and Engineering, Tulane University, New Orleans, Louisiana, USA
| | - Awinita Barpujari
- Department of Neurosurgery, Drexel University College of Medicine, Philadelphia, Pennsylvania, USA
| | - Vamsi P Reddy
- Department of Neurological Surgery, The University of Texas Southwestern Medical Center, Dallas, Texas, USA.
| |
Collapse
|
11
|
Asfaw ZK, Kalagara R, Schupper AJ, Choudhri TF. In Reply to the Letter to the Editor Regarding “Bibliometric Evaluation of U.S. Neurosurgery Subspecialties and Academic Rank Using RCR Index”. World Neurosurg 2022; 159:246-247. [DOI: 10.1016/j.wneu.2021.12.120] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/30/2021] [Accepted: 12/31/2021] [Indexed: 10/18/2022]
|
12
|
Luo P, Li J, Li P, Wang G, Li W, Song Z, Sun X, Fu Z, Zhou H, Yi X, Zhu L, Zhu S. A bibliometric and visual analysis of obesity and polycystic ovary syndrome from 2012 to 2022. Front Endocrinol (Lausanne) 2022; 13:1011105. [PMID: 36407303 PMCID: PMC9666686 DOI: 10.3389/fendo.2022.1011105] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 08/03/2022] [Accepted: 10/17/2022] [Indexed: 12/04/2022] Open
Abstract
BACKGROUND Obesity is associated with polycystic ovary syndrome (PCOS). We aimed to elucidate the research status and explore research trends and future directions of research on obesity and PCOS. METHODS A bibliometric analysis of the published papers in the field of obesity and PCOS between 2012 and 2022 was conducted on the basis of the Web of Science Core Collection database. The collaboration networks, research trends, literature sources, citation analysis, co-citation analysis, and keywords analysis were statistically analyzed and visualized using the VOSviewer software. RESULTS We retrieved 2843 records from 681 journals by 12307 authors from 2942 institutes in 99 countries. The number of published papers and citations had a roughly increasing trend annually. The United States and China contributed the majority of the records. Monash University, Shanghai Jiaotong University, Aristotle University of Thessaloniki, Karolinska Institute, University of São Paulo, and Tehran University of Medical Sciences were the biggest nodes in their cluster of the collaboration network map, and Moran LJ, Teede HJ, Joham AE, Escobar-Morreale HF, and Macut D were prolific authors. Research trends and hotspots were identified and visualized in the field of obesity and PCOS. Research hotspots in this field focused on insulin resistance (IR), metabolic syndrome, metformin, and inflammation. Bariatric surgery, mitochondrial dysfunction, binding globulins, and comorbidities may be the frontiers of future research. CONCLUSIONS We concluded the research status and trends in the field of obesity and PCOS. A better understanding of collaboration patterns, research hotspots, and frontiers may be useful for researchers.
Collapse
Affiliation(s)
| | | | | | | | | | | | | | | | | | | | - Liyong Zhu
- *Correspondence: Liyong Zhu, ; Shaihong Zhu,
| | | |
Collapse
|
13
|
Structure, function, and productivity from the National Dental Practice-Based Research Network. J Clin Transl Sci 2022; 6:e87. [PMID: 35989860 PMCID: PMC9379935 DOI: 10.1017/cts.2022.421] [Citation(s) in RCA: 11] [Impact Index Per Article: 5.5] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/30/2022] [Revised: 06/14/2022] [Accepted: 06/16/2022] [Indexed: 11/06/2022] Open
Abstract
Abstract
Introduction:
Following inception in 2005 as a multiregional practice-based research network (PBRN), the “National Dental PBRN” expanded nationwide in 2012, and in 2019 implemented additional organizational changes. The objectives are to: (1) describe the new structure and function of the network; and (2) quantify its scientific productivity since 2005.
Methods:
A national Administrative and Resource Center is based in Alabama; regional and specialty nodes are based in Alabama, Florida, Illinois, Minnesota, Oregon, New York, and Texas. A Network Coordinating Center is based in Oregon. Studies are funded via investigator-initiated grants. Scientific productivity is assessed using specific metrics, including the Relative Citation Ratio.
Results:
To date, 58 studies have been completed or are in data collection or development. These studies have investigated a broad range of topics using a wide variety of study designs. Of the studies that have completed enrollment, 70,665 patients were enrolled, as were 19,827 practitioners (some participated in multiple studies), plus electronic records for 790,493 patients in two data-only studies. To date, these studies have led to 193 peer-reviewed scientific publications in 62 different journals. The mean (1.40) Relative Citation Ratio of Network publications connotes a greater-than-average influence in their fields.
Conclusions:
These metrics demonstrate that the PBRN research context can successfully engage practitioners and patients from diverse settings nationally with a high and sustained level of scientific productivity. This infrastructure has enabled clinical scientists in oral health and nonoral health topics and provided additional recruitment venues outside of the typical academic health center research context.
Collapse
|
14
|
Triggle CR, MacDonald R, Triggle DJ, Grierson D. Requiem for impact factors and high publication charges. Account Res 2021; 29:133-164. [PMID: 33787413 DOI: 10.1080/08989621.2021.1909481] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 01/08/2023]
Abstract
Journal impact factors, publication charges and assessment of quality and accuracy of scientific research are critical for researchers, managers, funders, policy makers, and society. Editors and publishers compete for impact factor rankings, to demonstrate how important their journals are, and researchers strive to publish in perceived top journals, despite high publication and access charges. This raises questions of how top journals are identified, whether assessments of impacts are accurate and whether high publication charges borne by the research community are justified, bearing in mind that they also collectively provide free peer-review to the publishers. Although traditional journals accelerated peer review and publication during the COVID-19 pandemic, preprint servers made a greater impact with over 30,000 open access articles becoming available and accelerating a trend already seen in other fields of research. We review and comment on the advantages and disadvantages of a range of assessment methods and the way in which they are used by researchers, managers, employers and publishers. We argue that new approaches to assessment are required to provide a realistic and comprehensive measure of the value of research and journals and we support open access publishing at a modest, affordable price to benefit research producers and consumers.
Collapse
Affiliation(s)
- Chris R Triggle
- Departments of Medical Education & Pharmacology, Weill Cornell Medicine-Qatar, Doha, Qatar
| | - Ross MacDonald
- Distributed eLibrary, Weill Cornell Medicine-Qatar, Doha, New York, Qatar
| | - David J Triggle
- School of Pharmacy and Pharmaceutical Sciences, State University of New York, Buffalo, New York, USA
| | - Donald Grierson
- School of Biosciences, University of Nottingham, Loughborough, UK
| |
Collapse
|
15
|
Živković Ž, Panić M. Development of science and education in the Western Balkan countries: competitiveness with the EU. Scientometrics 2020. [DOI: 10.1007/s11192-020-03554-x] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/30/2022]
|
16
|
Abstract
Most scientometricians reject the use of the journal impact factor for assessing individual articles and their authors. The well-known San Francisco Declaration on Research Assessment also strongly objects against this way of using the impact factor. Arguments against the use of the impact factor at the level of individual articles are often based on statistical considerations. The skewness of journal citation distributions typically plays a central role in these arguments. We present a theoretical analysis of statistical arguments against the use of the impact factor at the level of individual articles. Our analysis shows that these arguments do not support the conclusion that the impact factor should not be used for assessing individual articles. Using computer simulations, we demonstrate that under certain conditions the number of citations an article has received is a more accurate indicator of the value of the article than the impact factor. However, under other conditions, the impact factor is a more accurate indicator. It is important to critically discuss the dominant role of the impact factor in research evaluations, but the discussion should not be based on misplaced statistical arguments. Instead, the primary focus should be on the socio-technical implications of the use of the impact factor.
Collapse
Affiliation(s)
- Ludo Waltman
- Centre for Science and Technology Studies, Leiden University, Leiden, The Netherlands
| | - Vincent A. Traag
- Centre for Science and Technology Studies, Leiden University, Leiden, The Netherlands
| |
Collapse
|
17
|
Davis FM, Obi AT, Gallagher KA, Henke PK. Accessing the academic influence of vascular surgeons within the National Institutes of Health iCite database. J Vasc Surg 2020; 71:1741-1748.e2. [DOI: 10.1016/j.jvs.2019.09.036] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/27/2019] [Accepted: 09/07/2019] [Indexed: 10/25/2022]
|
18
|
Schanzer A. As vascular surgeons, we need to punch above our weight. J Vasc Surg 2020; 71:1749. [PMID: 32334732 DOI: 10.1016/j.jvs.2019.10.023] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/23/2019] [Accepted: 10/04/2019] [Indexed: 11/26/2022]
Affiliation(s)
- Andres Schanzer
- Division of Vascular and Endovascular Surgery, University of Massachusetts Medical School, Worcester, Mass
| |
Collapse
|
19
|
Abstract
Quantifying the interdisciplinarity of a research is a relevant problem in the evaluative bibliometrics. The concept of interdisciplinarity is ambiguous and multidimensional. Thus, different measures of interdisciplinarity have been propose in the literature. However, few studies have proposed interdisciplinary metrics without previously defining classification sets, and no one has used the co-citation network for this purpose. In this study we propose an interdisciplinary metric based on the co-citation network. This is a way to define the publication’s field without resorting to pre-defined classification sets. We present a characterization of a publication’s field and then we use this definition to propose a new metric of the interdisciplinarity degree for publications (papers) and journals as units of analysis. The proposed measure has an aggregative property that makes it scalable from a paper individually to a set of them (journal) without more than adding the numerators and denominators in the proportions that define this new indicator. Moreover, the aggregated value of two or more units is strictly among all the individual values.
Collapse
|
20
|
Garstka ME, Randolph GW, Haddad AB, Nathan CAO, Ibraheem K, Farag M, Deot N, Adib H, Hoof M, French K, Killackey MT, Kandil E. Gender disparities are present in academic rank and leadership positions despite overall equivalence in research productivity indices among senior members of American Head and Neck Society (AHNS) Fellowship Faculty. Head Neck 2019; 41:3818-3825. [PMID: 31418942 DOI: 10.1002/hed.25913] [Citation(s) in RCA: 17] [Impact Index Per Article: 3.4] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 02/11/2019] [Revised: 06/09/2019] [Accepted: 07/30/2019] [Indexed: 11/10/2022] Open
Abstract
BACKGROUND This study aims to examine potential disparities in scholarly performance based on sex, academic rank, leadership positions, and regional distribution of faculty in accredited Head and Neck Surgery fellowships in the United States. METHODS Online faculty listings for 37 accredited fellowships were organized according to academic rank, leadership position, sex, and institutional location. Academic productivity was measured with three bibliometric indices: h-index, m-index, and the weighted relative citation ratio. RESULTS A total of 732 faculty members were included, of which 153 (21%) were female. Fifty-eight males (89.2%) held leadership positions, compared to seven females (10.8%). There was no significant difference in overall productivity between male and female senior faculty. There were regional differences in productivity by sex. CONCLUSIONS Females are underrepresented in senior faculty and within three common leadership positions, although scholarly productivity for male and female senior faculty and for those in leadership positions is similar.
Collapse
Affiliation(s)
- Meghan E Garstka
- Department of Surgery, Tulane University School of Medicine, New Orleans, Louisiana
| | | | - Antoine B Haddad
- Department of Surgery, Tulane University School of Medicine, New Orleans, Louisiana
| | | | - Kareem Ibraheem
- Department of Surgery, Tulane University School of Medicine, New Orleans, Louisiana
| | - Mahmoud Farag
- Department of Surgery, Tulane University School of Medicine, New Orleans, Louisiana
| | - Neal Deot
- Department of Surgery, Tulane University School of Medicine, New Orleans, Louisiana
| | - Hania Adib
- Department of Surgery, Tulane University School of Medicine, New Orleans, Louisiana
| | - Marcus Hoof
- Department of Surgery, Tulane University School of Medicine, New Orleans, Louisiana
| | - Kaley French
- Department of Surgery, Tulane University School of Medicine, New Orleans, Louisiana
| | - Mary T Killackey
- Department of Surgery, Tulane University School of Medicine, New Orleans, Louisiana
| | - Emad Kandil
- Department of Surgery, Tulane University School of Medicine, New Orleans, Louisiana
| |
Collapse
|
21
|
Dunaiski M, Geldenhuys J, Visser W. Globalised vs averaged: Bias and ranking performance on the author level. J Informetr 2019. [DOI: 10.1016/j.joi.2019.01.006] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 10/27/2022]
|
22
|
Dunaiski M, Geldenhuys J, Visser W. On the interplay between normalisation, bias, and performance of paper impact metrics. J Informetr 2019. [DOI: 10.1016/j.joi.2019.01.003] [Citation(s) in RCA: 6] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/28/2022]
|
23
|
Surkis A, Spore S. The relative citation ratio: what is it and why should medical librarians care? J Med Libr Assoc 2018; 106:508-513. [PMID: 30271298 PMCID: PMC6148595 DOI: 10.5195/jmla.2018.499] [Citation(s) in RCA: 25] [Impact Index Per Article: 4.2] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 05/01/2018] [Accepted: 05/01/2018] [Indexed: 12/05/2022] Open
Abstract
Bibliometrics is becoming increasingly prominent in the world of medical libraries. The number of presentations related to research impact at the Medical Library Association (MLA) annual meeting has been increasing in past years. Medical centers have been using institutional dashboards to track clinical performance for over a decade, and more recently, these institutional dashboards have included measures of academic performance. This commentary reviews current practices and considers the role for a newer metric, the relative citation ratio.
Collapse
Affiliation(s)
- Alisa Surkis
- Assistant Director for Research Data and Metrics, Health Sciences Library, NYU, 577 1st Avenue, New York, NY 10016
| | - Stuart Spore
- Lead, Scholarly Output Assessment, Health Sciences Library, NYU, 577 1st Avenue, New York, NY 10016
| |
Collapse
|
24
|
Hicks DJ, Stahmer C, Smith M. Impacting Capabilities: A Conceptual Framework for the Social Value of Research. Front Res Metr Anal 2018. [DOI: 10.3389/frma.2018.00024] [Citation(s) in RCA: 7] [Impact Index Per Article: 1.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/13/2022] Open
|
25
|
Critical rationalism and the search for standard (field-normalized) indicators in bibliometrics. J Informetr 2018. [DOI: 10.1016/j.joi.2018.05.002] [Citation(s) in RCA: 19] [Impact Index Per Article: 3.2] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/19/2022]
|
26
|
|
27
|
Lindner MD, Torralba KD, Khan NA. Scientific productivity: An exploratory study of metrics and incentives. PLoS One 2018; 13:e0195321. [PMID: 29614101 PMCID: PMC5882165 DOI: 10.1371/journal.pone.0195321] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 11/06/2017] [Accepted: 03/20/2018] [Indexed: 12/15/2022] Open
Abstract
Competitive pressure to maximize the current bibliometric measures of productivity is jeopardizing the integrity of the scientific literature. Efforts are underway to address the ‘reproducibility crisis’ by encouraging the use of more rigorous, confirmatory methods. However, as long as productivity continues to be defined by the number of discoveries scientists publish, the impact factor of the journals they publish in and the number of times their papers are cited, they will be reluctant to accept high quality methods and consistently conduct and publish confirmatory/replication studies. This exploratory study examined a sample of rigorous Phase II-IV clinical trials, including unpublished studies, to determine if more appropriate metrics and incentives can be developed. The results suggest that rigorous procedures will help reduce false positives, but to the extent that higher quality methods are accepted as the standard of practice, the current bibliometric incentives will discourage innovative studies and encourage scientists to shift their research to less informative studies of subjects that are already being more actively investigated. However, the results also suggest that it is possible to develop a more appropriate system of rewards. In contrast to the current bibliometric incentives, evaluations of the quality of the methods and reproducibility of the results, innovation and diversity of thought, and amount of information produced may serve as measures and incentives that maintain the integrity of the scientific literature and maximize scientific progress.
Collapse
Affiliation(s)
- Mark D. Lindner
- Center for Scientific Review, National Institutes of Health, Bethesda, Maryland, United States of America
- * E-mail:
| | - Karina D. Torralba
- Division of Rheumatology, Loma Linda University, Loma Linda, California, United States of America
| | - Nasim A. Khan
- Division of Rheumatology, Department of Internal Medicine, University of Arkansas for Medical Sciences, Arkansas, United States of America
- Central Arkansas Veterans Healthcare System, Little Rock, Arkansas, United States of America
| |
Collapse
|
28
|
Moher D, Naudet F, Cristea IA, Miedema F, Ioannidis JPA, Goodman SN. Assessing scientists for hiring, promotion, and tenure. PLoS Biol 2018; 16:e2004089. [PMID: 29596415 PMCID: PMC5892914 DOI: 10.1371/journal.pbio.2004089] [Citation(s) in RCA: 160] [Impact Index Per Article: 26.7] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Revised: 04/10/2018] [Indexed: 11/24/2022] Open
Abstract
Assessment of researchers is necessary for decisions of hiring, promotion, and tenure. A burgeoning number of scientific leaders believe the current system of faculty incentives and rewards is misaligned with the needs of society and disconnected from the evidence about the causes of the reproducibility crisis and suboptimal quality of the scientific publication record. To address this issue, particularly for the clinical and life sciences, we convened a 22-member expert panel workshop in Washington, DC, in January 2017. Twenty-two academic leaders, funders, and scientists participated in the meeting. As background for the meeting, we completed a selective literature review of 22 key documents critiquing the current incentive system. From each document, we extracted how the authors perceived the problems of assessing science and scientists, the unintended consequences of maintaining the status quo for assessing scientists, and details of their proposed solutions. The resulting table was used as a seed for participant discussion. This resulted in six principles for assessing scientists and associated research and policy implications. We hope the content of this paper will serve as a basis for establishing best practices and redesigning the current approaches to assessing scientists by the many players involved in that process.
Collapse
Affiliation(s)
- David Moher
- Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California, United States of America
| | - Florian Naudet
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California, United States of America
- INSERM CIC-P 1414, Clinical Investigation Center, CHU Rennes, Rennes 1 University, Rennes, France
| | - Ioana A. Cristea
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California, United States of America
- Department of Clinical Psychology and Psychotherapy, Babeş-Bolyai University, Cluj-Napoca, Romania
| | - Frank Miedema
- Executive Board, UMC Utrecht, Utrecht University, Utrecht, the Netherlands
| | - John P. A. Ioannidis
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California, United States of America
- Department of Medicine, Stanford University, Stanford, California, United States of America
- Department of Health Research and Policy, Stanford University, Stanford, California, United States of America
- Department of Biomedical Data Science, Stanford University, Stanford, California, United States of America
- Department of Statistics, Stanford University, Stanford, California, United States of America
| | - Steven N. Goodman
- Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, California, United States of America
- Department of Medicine, Stanford University, Stanford, California, United States of America
- Department of Health Research and Policy, Stanford University, Stanford, California, United States of America
| |
Collapse
|
29
|
Additional support for RCR: A validated article-level measure of scientific influence. PLoS Biol 2017; 15:e2003552. [PMID: 28968381 PMCID: PMC5624567 DOI: 10.1371/journal.pbio.2003552] [Citation(s) in RCA: 16] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/07/2017] [Accepted: 08/29/2017] [Indexed: 12/16/2022] Open
|