Prevalence of Research Publication Misrepresentation Among Urology Residency Applicants and Its Effect on Match Success

Fate of Research Articles Reported by Ophthalmology Residency Applicants: What Happens to "Submitted" and "In Preparation" Articles?

OBJECTIVE

As coronavirus disease 2019 affects clinical training opportunities and with the transition of U.S. Medical Licensing Examination Step 1 to pass-fail, research may become increasingly important for evaluating ophthalmology residency applicants. Though publication misrepresentation has been studied among ophthalmology residency applicants, eventual publication rates of incomplete articles remain unknown. We aimed to determine publication rates for manuscripts listed as "submitted" or "in preparation" on ophthalmology residency applications and identify factors associated with unpublished manuscripts.

DESIGN

San Francisco Match applications to the Wilmer Eye Institute for the 2019 ophthalmology residency cycle were retrospectively reviewed. Each applicant's number of "published," "submitted," and "in preparation" manuscripts was recorded, then verified 1.5 years later through PubMed, Google Scholar, or journal websites. Unverifiable manuscripts were deemed "unpublished."

SETTING

Single academic institution (Wilmer Eye Institute, Baltimore, MD, USA) PARTICIPANTS: All 458 medical students who applied to the Wilmer Eye Institute for the 2019 ophthalmology residency cycle through the San Francisco Match.

RESULTS

A total of 458 applications were reviewed. Of 428 "submitted" publications, 126 (29.4%) remained unpublished after 1.5 years. Of 324 manuscripts "in preparation," 215 (66.4%) remained unpublished. In a multivariate model, AOA was associated with not having an unpublished manuscript compared to applicants without AOA (OR: 0.423, p = 0.0163). Gender, Step 1 score, additional degrees, and a research year had no association.

CONCLUSIONS

Nearly two-thirds of manuscripts listed as "in preparation" remained unpublished. Specific guidance from research mentors may help applicants better represent their publications in residency applications.

Trust but Verify: Misrepresentation of Publication Records Among Child Neurology Residency Applicants

INTRODUCTION

Numerous authors have reported that misrepresentation of publication records among residency applicants is not uncommon. We sought to determine the percentage of child neurology residency applicants who falsify information about publications in the documents submitted to the Electronic Residency Application Service.

METHODS

The records of applicants to our residency program between 2014 and 2020 (898 individuals) were analyzed. Publications were verified by searching online databases and with a research librarian's assistance. Searches were conducted using all available information-author name, article title, and journal name-alone or in combination. Articles in inaccessible publications were disregarded. Misrepresentation was recorded when one the following was identified: listing a nonexistent article, promoting the applicant's name on the authors' list, misattributing authorship, and listing a publication as "in-press" yet never published.

RESULTS

From the 7 residency match cycles reviewed, 514 applicants reported having had authored at least 1 publication. Of those, 47 (9.14%) listed 75 unquestionably misrepresented publications. In 33 instances, the paper did not exist, with some applicants going as far as submitting a nonexistent PubMed Identifier; in 21 instances, the author's name was promoted on the authors' list; in 15 instances, the applicant was not an author; and in 6 instances, the author reported the publication as "in-press," but the work has never been published.

CONCLUSION

As happens in other specialties, child neurology residency applicants sometimes submit fraudulent information about their publications. Program directors need to be aware of this possibility and carefully review the information provided by applicants.

Racial and Ethnic Differences in Medical Student Timing and Perceived Quality of Exposure to Urology

OBJECTIVE

To contextualize the low representation of Under-Represented in Medicine (URiM) in urology, we examine differences in timing and perceived quality of urology clinical and research exposures for medical students across race/ethnicity.

METHODS

A cross-sectional survey was distributed to all medical students at UCLA. Dependent variables were timing of urology exposure and perceived quality of urology exposure. Descriptive statistics and multivariate analyses were used to compare variables across race/ethnicity. Logistic regression was used to determine odds of early exposure to urology across race/ethnicity.

RESULTS

Black and Latinx students were significantly less likely to discover urology before MS3 (P<.001). Although URiM students were more likely to recall receiving a urology interest group invitation (Asian 46%, Black 53%, Latinx 67%, White 48%, P=.03), they were less likely to attend an event (Asian 23%, Black 4%, Latinx 3% and White 15%, P<.001) despite being more likely to be interested in urology (Asian 32%, Black 38%, Latinx 50%, White 28%, P=.01). Black students were more likely to gain exposure via family/friend with a urological diagnosis. Black and Latinx students were twice as dissatisfied with timing and method of medical school exposure to urology versus their peers. There were differences across race/ethnicity for whether or not a student had engaged in urology research (Asian 10%, Black 5%, Latinx 2%, White 2%, P=.01).

CONCLUSIONS

Racial/ethnic disparities exist in early exposure to urology, involvement in urology interest group, access to research, and satisfaction with exposure to urology. Interventions addressing the timing and quality of urology exposures may optimize recruitment of URiM students into urology.

Diversifying Graduate Medical Education & the Urology Workforce: Re-imagining our Structures, Policies, Practices, Norms, & Values

This article offers a framework for critically examining the structures, policies, norms, practices, and values that shape the Urology Match as a foundation for interventions to improve diversity, equity, inclusion, and justice in the workforce. Points of leverage for transformational change in the urology workforce diversification include modifying the structure of the urology application process, optimizing reviewer factors, addressing Under-Represented in Medicine applicant experience, providing resources to applicants, and evaluating selection criteria. To achieve an inclusive diverse urology workforce, we must change policy and practice, expand what we include in the norm, which will translate into increased value ascribed to a more varied cohort of applicants, leading to the establishment of structures that accommodate true diversity.

The Financial Burden of Applying to Urology Residency in 2020

OBJECTIVES

To determine contemporary costs of preparing for and applying for a urology residency position for the 2019-2020 American Urological Association Match.

METHODS

An electronic survey was emailed to all urology residency applicants who applied to Rutgers Robert Wood Johnson Medical School during the 2019-2020 application cycle; it was sent 2 weeks after the Match results were released. We collected information on applicant demographics, interview logistics, and estimated costs incurred applying to residency.

RESULTS

A total of 26% (64/242) of subjects responded, representing all 8 the American Urological Association sections, international schools, and schools without urology programs. 62% were male, 75% were single, and 52% attended public medical school. Applicants paid for the interview trail using loans (67%), family donations (50%), previous or current income (36%), and scholarships (16%). Subjects completed a median of 2 visiting student rotations (IQR 2-3), applied to 80 programs (IQR 66-99), and attended 16 interviews (IQR 13-18.75). The median cost per applicant for the 2019-2020 Match was $9725 (IQR $6134-12,564). This estimate included expenditures on application fees, visiting student rotations, interview trail travel and lodging, research, interview attire, and professional photos. Subjects who attended public medical school were likely to spend $3546.31 (95% confidence interval: 5630.71-1461.916; P < .001) more than those attending private schools.

CONCLUSION

Urology residency applicants spend almost $10,000 in pursuit of a residency position. These high costs not only contribute to student debt but also may deter applicants from entering the field of urology.

The neurosurgery applicant's "arms race": analysis of medical student publication in the Neurosurgery Residency Match

OBJECTIVE

Neurosurgery is consistently one of the most competitive specialties for resident applicants. The emphasis on research in neurosurgery has led to an increasing number of publications by applicants seeking a successful residency match. The authors sought to produce a comprehensive analysis of research produced by neurosurgical applicants and to establish baseline data of neurosurgery applicant research productivity given the increased emphasis on research output for successful residency match.

METHODS

A retrospective review of publication volume for all neurosurgery interns in 2009, 2011, 2014, 2016, and 2018 was performed using PubMed and Google Scholar. Missing data rates were 11% (2009), 9% (2011), and < 5% (all others). The National Resident Matching Program report "Charting Outcomes in the Match" (ChOM) was interrogated for total research products (i.e., abstracts, presentations, and publications). The publication rates of interns at top 40 programs, students from top 20 medical schools, MD/PhD applicants, and applicants based on location of residency program and medical school were compared statistically against all others.

RESULTS

Total publications per neurosurgery intern (mean ± SD) based on PubMed and Google Scholar were 5.5 ± 0.6 in 2018 (1.7 ± 0.3, 2009; 2.1 ± 0.3, 2011; 2.6 ± 0.4, 2014; 3.8 ± 0.4, 2016), compared to 18.3 research products based on ChOM. In 2018, the mean numbers of publications were as follows: neurosurgery-specific publications per intern, 4.3 ± 0.6; first/last author publications, 2.1 ± 0.3; neurosurgical first/last author publications, 1.6 ± 0.2; basic science publications, 1.5 ± 0.2; and clinical research publications, 4.0 ± 0.5. Mean publication numbers among interns at top 40 programs were significantly higher than those of all other programs in every category (p < 0.001). Except for mean number of basic science publications (p = 0.1), the mean number of publications was higher for interns who attended a top 20 medical school than for those who did not (p < 0.05). Applicants with PhD degrees produced statistically more research in all categories (p < 0.05) except neurosurgery-specific (p = 0.07) and clinical research (p = 0.3). While there was no statistical difference in publication volume based on the geographical location of the residency program, students from medical schools in the Western US produced more research than all other regions (p < 0.01). Finally, research productivity did not correlate with likelihood of medical students staying at their home institution for residency.

CONCLUSIONS

The authors found that the temporal trend toward increased total research products over time in neurosurgery applicants was driven mostly by increased nonindexed research (abstracts, presentations, chapters) rather than by increased peer-reviewed publications. While we also identified applicant-specific factors (MD/PhDs and applicants from the Western US) and an outcome (matching at research-focused institutions) associated with increased applicant publications, further work will be needed to determine the emphasis that programs and applicants will need to place on these publications.

Analysis of academic publishing output among 1634 successful applicants in the 2011-2018 neurosurgery residency match

BACKGROUND

Research productivity is a key criterion for applicant selection reported by residency program directors. Research volume reported on neurosurgery residency applications has risen steadily over the past decade.

OBJECTIVE

Perform retrospective bibliographic searches of successful applicants who matched into U.S. neurosurgery residency programs from 2011 to 2018, and assess the relationship between academic publishing and residency placement.

METHODS

Gender, MD/PhD status, U.S. News research ranking of medical school, and international medical graduate status (IMG) were determined for 1634 successful applicants from 2011 to 2018. Indexed publications before and after the start of residency were tabulated by Scopus®. Publication counts were stratified by first author, basic/clinical science, case reports, reviews, or other research. We then compared publishing trends across demographic variables and match cohorts.

RESULTS

Average pre-residency publications increased from 2.6 [1.7, 3.4] in 2011 to 6.5 [5.1, 7.9] in 2018. Men, PhD-holders, Top 20 and Top 40 U.S. medical school graduates, and IMGs had higher pre-residency publication counts overall. After stratifying by match cohort, however, there was no significant effect of gender on pre-residency publications. Applicants matching into residency programs with highly ranked affiliated hospitals had significantly higher pre-residency publications.

CONCLUSION

Publishing volume of successful neurosurgery applicants in the U.S. has risen recently and is associated with the stature of matched residency programs. Given the gap between verifiable and claimed research on residency applications, attention is needed to objectively evaluate research credentials in the selection process. The impending phase out of USMLE step 1 scores may increase emphasis on academic productivity.

Evaluating Urology Residency Applications: What Matters Most and What Comes Next?

PURPOSE OF REVIEW

In light of the announcement that the United States Medical Licensing Examination Step 1 exam will transition to pass/fail reporting, we reviewed recent literature on evaluating residency applicants with a focus on identifying objective measurements of applicant potential.

RECENT FINDINGS

References from attending urologists, Step 1 scores, overall academic performance, and research publications are among the most important criteria used to assess applicants. There has been a substantial increase in the average number of applications submitted per applicant, with both applicants and residency directors indicating support for a cap on the number of applications that may be submitted. Additionally, there are increasing efforts to promote diversity with the goal of improving care and representation in urology. Despite progress in standardizing interview protocols, inappropriate questioning remains an issue. Opportunities to improve residency application include promoting diversity, enforcing prohibitions of illegal practices, limiting application numbers, and finding more transparent and equitable screening measures to replace Step 1.

The Outcomes of "Submitted" Publications From Applicants to Orthopaedic Surgery Residency Programs: A Retrospective Review of 1303 Residency Applications

Purpose:

To evaluate research listed as “Submitted” on orthopaedic surgery residency applications for eventual publication rates and quality.

Significance:

As the orthopaedic surgery residency selection process becomes increasingly competitive, the number of research publications listed on applications continually increases. However, the utility of using publications listed as “Submitted” in the applicant evaluation process remains unknown.

Methods:

Demographic and publication data were retrospectively collected from 1303 applications to an orthopaedic surgery residency program. The PubMed database was used to verify “Submitted” publications for (1) publication fruition or (2) publication mismatch, defined as discordance between the listed journal of submission and the eventual journal of publication.

Results:

A total of 594 applications (45.6%) listed ≥1 publication as “Submitted.” Out of 1636 “Submitted” publications, 565 were unverifiable (32.5%). Of the 1071 verified publications, 362 (33.8%) experienced publication mismatch. Within this subgroup, a significant difference existed between the mean impact factors of the listed journal of submission and the eventual journal of publication (1.5 ± 2.7 versus 3.0 ± 2.5, P < 0.01). Demographic data were not predictive of having an unverified publication.

Conclusion:

Publications listed as “Submitted” in orthopaedic surgery residency applications frequently remain unpublished or are published in less impactful journals than originally intended.

Dishonesty and research misconduct within the medical profession

While there has been much discussion of how the scientific establishment’s culture can engender research misconduct and scientific irreproducibility, this has been discussed much less frequently with respect to the medical profession. Here the authors posit that a lack of self-criticism, an encouragement of novel scientific research generated by the recruitment policies of the UK Royal Training Colleges along with insufficient training in the sciences are core reasons as to why research misconduct and dishonesty prevail within the medical community. Furthermore, the UK General Medical Council’s own data demonstrates a historic inattentiveness to the ease with which doctors can engage in research misconduct. Suggestions are made as to how these issues can be investigated and alternative incentives for career advancement are adumbrated.

Low Misrepresentation Rates of Scholarly Work in Otolaryngology-Head and Neck Surgery Residency Applications

Objectives

To evaluate research trends, including rates of misrepresentation of scholarly work, in otolaryngology residency applications received by a single institution during the 2018-2019 residency application cycle.

Methods

After obtaining Institutional Review Board approval, all residency applications to the Department of Otolaryngology-Head and Neck Surgery at Indiana University School of Medicine, Indianapolis, IN for the 2018-2019 cycle were de-identified and analyzed. Demographic and research information including the number of listed peer-reviewed articles/abstracts, types of research projects, and misrepresentations were retrospectively evaluated.

Results

Our institution received 321 applications, which represented 69.5% of the entire 2018-2019 otolaryngology applicant pool. The average United States Medical Licensing Examination (USMLE) Step 1 score was 246 ±12.4. There were 203 (62.2%) applicants who reported 591 published citations with 20 (6.2%) applicants misrepresenting 26 items (4.4%). Applicants who misrepresented research output had lower average Step 1 scores (237.4 vs 246.4, p: <0.05). Self-promotion to higher authorship status was the most common form of misrepresentation (61.5%).

Conclusions

The role of scholarly work in stratifying applicants continues to expand. Although a competitive application climate motivates a minority of applicants to misrepresent scholarly work, rates of misrepresentation in otolaryngology applications are low and continue to decline. The level of evidence assigned to this study is III.

Update on Misrepresentation of Research Publications Among Orthopaedic Surgery Residency Applicants

BACKGROUND

Our 2 previous studies (1999, 2007) examining misrepresentation of research publications among orthopaedic residency applicants revealed rates of misrepresentation of 18.0% and 20.6%, respectively. As the residency selection process has become more competitive, the number of applicants who list publications has increased. The purpose of this study was to determine current rates of research misrepresentation by orthopaedic surgery applicants.

METHODS

We reviewed the publication listings and research section of the Common Application Form from the Electronic Residency Application Service (ERAS) for all applicants applying to 1 orthopaedic residency program. The PubMed-MEDLINE database was principally used to search for citations. The PubMed Identifier (PMID) number was used; if no PMID number was listed, a combination of authors or the title of the work was used. If the citations were not found through PubMed, a previously developed algorithm was followed to determine misrepresentation. Misrepresentation was defined as (1) nonauthorship of a published article in which authorship was claimed, (2) claimed authorship of a nonexistent article, or (3) self-promotion to a higher authorship status within a published article.

RESULTS

Five hundred and seventy-three applicants applied to our institution for residency in 2016 to 2017: 250 (43.6%) of 573 applicants did not list a publication, whereas 323 (56.4%) of 573 applicants listed ≥1 publication. We found 13 cases of misrepresentation among a total of 1,100 citations (1.18% in 2017 versus 18.0% in 1999 and 20.6% in 2007, p < 0.001). Ten cases of misrepresentation were self-promotion to a higher authorship status. There were 2 cases of claimed authorship of an article that could not be found. Only 1 applicant misrepresented more than once.

CONCLUSIONS

Based on our findings, orthopaedic surgery residency applicants are accurately representing their publication information. The incorporation of the PMID number on the ERAS application has streamlined the process for finding publications, and has possibly encouraged veracity on residency applications. Faculty involved in the resident selection process should be aware of the significant decline in the rate of misrepresentation by medical students applying for orthopaedic surgery residency versus the rate in our prior studies.

Pre-residency publication and its association with paediatric residency match outcome-a retrospective analysis of a national database

INTRODUCTION

Scholarly activity is considered valuable in the resident selection process by candidates and program directors alike, despite existing literature suggesting applicants with scholarly work do not perform better in the match. These studies, however, are limited in that they have only measured whether candidates have successfully matched or not. To try and reconcile the existing disconnect in the value of pre-residency scholarly activity, we sought to deepen the understanding by investigating whether pre-residency publication is associated with a higher rank-order list match achievement.

METHODS

Anonymized data were collected from the Canadian Residency Matching Service for individuals matched to paediatric programs from 2007-2012. The primary analysis was to identify whether documentation of ≥1 pre-residency publication was associated with achieving a first-choice match. Secondary analyses included evaluating for an association between multiple pre-residency publications, academic presentations or a graduate degree and match outcome.

RESULTS

Of a total of 843 matched individuals, 406 (48.2%) listed ≥1 pre-residency publication and 494 (58.6%) matched to their first-choice program. The possession of ≥1 pre-residency publications was not associated with matching to a candidate's first-choice program (odds ratio = 0.94 [95% confidence interval = 0.71-1.24], p = 0.66). Similarly, listing ≥2 publications, ≥3 publications, a graduate degree, or an academic presentation was not associated with achieving a first-choice match.

CONCLUSIONS

The results provide increased support for the notion that in aggregate, candidate scholarly activity does not influence match outcome. Accordingly, it is recommended that medical student research activities are fostered with the goal to improve their skills as scientists, and not simply to achieve a better residency match outcome.