1
|
Houston Thompson R, Lohse CM, Leibovich BC, Gettman MT, Husmann DA, Viers BR. Predictors of Excellent Urology Residents at the Time of the Urology Match. Urology 2024; 183:17-24. [PMID: 37866652 DOI: 10.1016/j.urology.2023.09.038] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 06/22/2023] [Revised: 08/10/2023] [Accepted: 09/12/2023] [Indexed: 10/24/2023]
Abstract
OBJECTIVE To update our experience and report on features predictive of high-quality urology residents at the time of the urology match, because data predicting which medical students will mature into excellent urology residents are sparse. METHODS We reviewed our experience with 84 urology residents who graduated from 2006 to 2023. Residents were independently scored 1-10 based on overall quality by the current and former Program Director. Discrepant scoring by >2 was resolved by an independent review. Associations of features from the medical student application with an excellent score (defined as 8-10) were evaluated with logistic regression. RESULTS Discrepant scoring >2 was noted in only 5 (6%) residents. Among the 84 residents, the median overall score was 7 (range 1-10) and 36 (43%) residents had an excellent score of 8-10. Univariably, higher USMLE step II score (P = .03), election to alpha omega alpha (P = .004), no negative interview comments (P = .002), honors in OB/Gyn (P = .048) and psychiatry clerkships (P = .04), and honors in all core clinical clerkships (P < .001) were significantly associated with an excellent score. In a multivariable model, no negative interview comments (P = .003) and honors in all core clinical clerkships (P = .001) were independently associated with an excellent score (c-index 0.76). There were several notable features (sex, letters of recommendation, USMLE step I, externship at our institution, surgery clerkship grade, and rank list) that were not significantly associated with excellent residents. CONCLUSION We demonstrate features associated with excellent urology residents, most notably no negative interview comments and an honors grade in all core clinical clerkships.
Collapse
Affiliation(s)
| | | | | | | | | | - Boyd R Viers
- Department of Urology, Mayo Clinic, Rochester, MN
| |
Collapse
|
2
|
Manstein SM, Laikhter E, Kazei DD, Comer CD, Shiah E, Lin SJ. The Upcoming Pass/Fail USMLE Step 1 Score Reporting: An Impact Assessment From Medical School Deans. Plast Surg (Oakv) 2023; 31:169-176. [PMID: 37188137 PMCID: PMC10170630 DOI: 10.1177/22925503211034838] [Citation(s) in RCA: 3] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/21/2021] [Accepted: 06/22/2021] [Indexed: 11/15/2022] Open
Abstract
Background: The US Medical Licensing Examination (USMLE) Step 1 change to pass/fail has been met with mixed reviews, and the impact on medical student education and residency match is unknown. We surveyed medical school student affairs deans regarding their thoughts on the upcoming transition of Step 1 to pass/fail. Methods: A questionnaire was emailed to medical school deans. Deans were asked to rank the importance of the following after the Step 1 reporting change: Step 2 Clinical Knowledge (Step 2 CK), clerkship grades, letters of recommendation, personal statement, medical school reputation, class rank, Medical Student Performance Evaluation, and research. They were asked how the score change will affect curriculum, learning, diversity, and student mental health. Deans were asked to select 5 specialties they thought would be most affected. Results: Regarding perceived importance of residency applications following the scoring change, the most frequent number 1 choice was Step 2 CK. The majority of deans (93.5%, n = 43) felt that the change to pass/fail would benefit medical student education/learning environment; however, most (68.2%, n = 30) did not believe their school curriculum would change. Students applying to dermatology, neurosurgery, orthopedic surgery, ENT, and plastic surgery were felt to be most affected by the scoring change; 58.7% (n = 27) felt it would not adequately address future diversity. Conclusion: The majority of deans feel the USMLE Step 1 change to pass/fail would benefit medical student education. Deans feel that students applying to traditionally more competitive specialties (ie, programs with fewer overall residency positions available) will be most affected.
Collapse
Affiliation(s)
- Samuel M. Manstein
- Division of Plastic Surgery, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - Elizabeth Laikhter
- Division of Plastic Surgery, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | | | - Carly D. Comer
- Division of Plastic Surgery, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - Eric Shiah
- Division of Plastic Surgery, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| | - Samuel J. Lin
- Division of Plastic Surgery, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, MA, USA
| |
Collapse
|
3
|
Choi P, Langenau E, Roberts M, Blalock TW. Perspectives of Dermatology Program Directors on the Impact of Step 1 Pass/Fail. Cureus 2023; 15:e35801. [PMID: 36895522 PMCID: PMC9990960 DOI: 10.7759/cureus.35801] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Accepted: 03/05/2023] [Indexed: 03/08/2023] Open
Abstract
INTRODUCTION The shift of Step 1 to Pass/Fail has generated several questions and concerns about obtaining residency positions among allopathic and osteopathic students alike. Determining the perspectives of Dermatology Program Directors in regards to post-Step 1 Pass/Fail is critical for students to better prepare for matching into dermatology. METHODS After receiving Institutional Review Board (IRB) exemption status, the program directors were chosen from 144 Accreditation Council for Graduate Medical Education (ACGME) and 27 American Osteopathic Association (AOA) Dermatology programs using contact information from their respective online website databases. An eight-item survey was constructed on a three-point Likert scale, one free text response, and four demographic questions. The anonymous survey was sent out over the course of three weeks with weekly individualized reminder requests for participation. RESULTS A total of 54.54% of responders had "Letters of Recommendation" in their top 3. Forty-five percent of responders had "Completed Audition Rotation at Program" in their top 3. And, 38.09% of responders had "USMLE Step 2 CK Scores" in their top 3. CONCLUSION Approximately 50% of responders agreed that all medical students will have more difficulty matching dermatology. Based on the survey study, Dermatology program directors want to focus more on letters of recommendation, audition rotations, and Step 2 CK scores. Because each field seems to prioritize different aspects of an application, students should attempt to gain as much exposure to different fields such as through research and shadowing to narrow down their ideal specialties. Consequently, the student will have more time to tailor their applications to what residency admissions are looking for.
Collapse
Affiliation(s)
- Peter Choi
- Osteopathic Medicine, Philadelphia College of Osteopathic Medicine, Suwanee, USA
| | - Erik Langenau
- Family Medicine, Philadelphia College of Osteopathic Medicine, Philadelphia, USA
| | - Michael Roberts
- Statistics, Philadelphia College of Osteopathic Medicine, Philadelphia, USA
| | | |
Collapse
|
4
|
Berk GA, Ho TD, Stack‐Pyle TJ, Zeatoun A, Kong KA, Chaskes MB, Thorp BD, Ebert CS, DeMason CE, Kimple AJ, Senior BA. The next step: Replacing step 1 as a metric for residency application. Laryngoscope Investig Otolaryngol 2022; 7:1756-1761. [PMID: 36544915 PMCID: PMC9764748 DOI: 10.1002/lio2.947] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/29/2022] [Accepted: 10/01/2022] [Indexed: 12/24/2022] Open
Abstract
Objective As of January 2022, USMLE Step 1 scores are reported as pass/fail. Historically, Step 1 scores have been a critical component of residency applications, representing one of the few metrics standardized across all applicants independent of the school they attended. In competitive specialties, such as otolaryngology, programs routinely get 100+ applicants for each residency spot and use Step 1 as a screening tool. This study seeks to assess quantifiable metrics in the application that highly competitive residency programs could use for screening in place of Step 1 scores. Methods Otolaryngology applications to an academic medical center for the 2019-20 and 2020-21 ERAS cycles were reviewed. Board scores and quantitative research data were extracted. The relationships between Step 1 score and the other metrics were examined by computing Pearson's correlation coefficients and building regression models. Similar analyses were done separately for three different score tiers defined by Step 1 cutoffs at 220 points and 250 points. Results Step 2 score was the only variable that had meaningful correlation with Step 1 score (R = .67, p < 2.2e-16). No other objective metric such as journal articles, posters, or oral presentations correlated with Step 1 scores. Conclusion Step 1 scores were moderately correlated with Step 2 scores; however, using a Step 2 cutoff instead of a Step 1 cutoff would identify a different cohort of applicants for interview. No other quantifiable application metric had a positive correlation. In future match cycles, highly competitive residency programs will need to adopt new methods to screen candidates.Level of Evidence: Level 3.
Collapse
Affiliation(s)
- Garrett A. Berk
- Department of Otolaryngology—Head & Neck Surgerythe University of North CarolinaChapel HillNorth CarolinaUSA
| | - Tiffany D. Ho
- Department of Otolaryngology—Head & Neck Surgerythe University of North CarolinaChapel HillNorth CarolinaUSA
| | - Taylor J. Stack‐Pyle
- Department of Otolaryngology—Head & Neck Surgerythe University of North CarolinaChapel HillNorth CarolinaUSA
| | - Abdullah Zeatoun
- Department of Otolaryngology—Head & Neck Surgerythe University of North CarolinaChapel HillNorth CarolinaUSA
| | - Keonho A. Kong
- Department of Otolaryngology—Head & Neck Surgerythe University of North CarolinaChapel HillNorth CarolinaUSA
| | - Mark B. Chaskes
- Department of Otolaryngology—Head & Neck Surgerythe University of North CarolinaChapel HillNorth CarolinaUSA
| | - Brian D. Thorp
- Department of Otolaryngology—Head & Neck Surgerythe University of North CarolinaChapel HillNorth CarolinaUSA
| | - Charles S. Ebert
- Department of Otolaryngology—Head & Neck Surgerythe University of North CarolinaChapel HillNorth CarolinaUSA
| | - Christine E. DeMason
- Department of Otolaryngology—Head & Neck Surgerythe University of North CarolinaChapel HillNorth CarolinaUSA
| | - Adam J. Kimple
- Department of Otolaryngology—Head & Neck Surgerythe University of North CarolinaChapel HillNorth CarolinaUSA
| | - Brent A. Senior
- Department of Otolaryngology—Head & Neck Surgerythe University of North CarolinaChapel HillNorth CarolinaUSA
| |
Collapse
|
5
|
Panda N, Bahdila D, Abdullah A, Ghosh AJ, Lee SY, Feldman WB. Association Between USMLE Step 1 Scores and In-Training Examination Performance: A Meta-Analysis. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2021; 96:1742-1754. [PMID: 34323860 DOI: 10.1097/acm.0000000000004227] [Citation(s) in RCA: 7] [Impact Index Per Article: 2.3] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
PURPOSE On February 12, 2020, the sponsors of the United States Medical Licensing Examination announced that Step 1 will transition to pass/fail scoring in 2022. Step 1 performance has historically carried substantial weight in the evaluation of residency applicants and as a predictor of subsequent subject-specific medical knowledge. Using a systematic review and meta-analysis, the authors sought to determine the association between Step 1 scores and in-training examination (ITE) performance, which is often used to assess knowledge acquisition during residency. METHOD The authors systematically searched Medline, EMBASE, and Web of Science for observational studies published from 1992 through May 10, 2020. Observational studies reporting associations between Step 1 and ITE scores, regardless of medical or surgical specialty, were eligible for inclusion. Pairs of researchers screened all studies, evaluated quality assessment using a modified Newcastle-Ottawa Scale, and extracted data in a standardized fashion. The primary endpoint was the correlation of Step 1 and ITE scores. RESULTS Of 1,432 observational studies identified, 49 were systematically reviewed and 37 were included in the meta-analysis. Overall study quality was low to moderate. The pooled estimate of the correlation coefficient was 0.42 (95% confidence interval [CI]: 0.36, 0.48; P < .001), suggesting a weak-to-moderate positive correlation between Step 1 and ITE scores. The random-effects meta-regression found the association between Step 1 and ITE scores was weaker for surgical (versus medical) specialties (beta -0.25 [95% CI: -0.41, -0.09; P = .003]) and fellowship (versus residency) training programs (beta -0.25 [95% CI: -0.47, -0.03; P = .030]). CONCLUSIONS The authors identified a weak-to-moderate positive correlation between Step 1 and ITE scores based on a meta-analysis of low-to-moderate quality observational data. With Step 1 scoring transitioning to pass/fail, the undergraduate and graduate medical education communities should continue to develop better tools for evaluating medical students.
Collapse
Affiliation(s)
- Nikhil Panda
- N. Panda is a clinical fellow of surgery, Massachusetts General Hospital and Harvard Medical School, and a postdoctoral researcher, Harvard T.H. Chan School of Public Health, Boston, Massachusetts
| | - Dania Bahdila
- D. Bahdila is a doctoral candidate, Department of Oral Health Policy and Epidemiology, Harvard School of Dental Medicine, Boston, Massachusetts, and Department of Preventive Dental Sciences, Faculty of Dentistry, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Abeer Abdullah
- A. Abdullah is a doctoral candidate, Department of Oral Health Policy and Epidemiology, Harvard School of Dental Medicine, Boston, Massachusetts, and Department of Preventive Dental Sciences, Faculty of Dentistry, King Abdulaziz University, Jeddah, Saudi Arabia
| | - Auyon J Ghosh
- A.J. Ghosh is a clinical fellow of medicine and postdoctoral researcher, Division of Pulmonary and Critical Care Medicine, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts
| | - Sun Yeop Lee
- S.Y. Lee is research assistant, Department of Epidemiology, Harvard T.H. Chan School of Public Health, Boston, Massachusetts
| | - William B Feldman
- W.B. Feldman is associate physician and research fellow, Division of Pulmonary and Critical Care Medicine and the Program On Regulation, Therapeutics, And Law (PORTAL), Division of Pharmacoepidemiology and Pharmacoeconomics, Department of Medicine, Brigham and Women's Hospital and Harvard Medical School, Boston, Massachusetts
| |
Collapse
|
6
|
Ost SR, Wells D, Goedecke PJ, Tolley EA, Kleinman M, Thompson NS. Relationship Between Standardized Test Scores and Board Certification Exams in a Combined Internal Medicine/Pediatrics Residency Program. Cureus 2021; 13:e13567. [PMID: 33815979 PMCID: PMC8008765 DOI: 10.7759/cureus.13567] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 12/04/2022] Open
Abstract
Background Combined Internal Medicine/Pediatrics (Med/Peds) residencies rely on categorical program data to predict pass rates for the American Board of Internal Medicine Certifying Exam (ABIM-CE) and the American Board of Pediatrics Certifying Exam (ABP-CE). There is insufficient literature describing what best predicts a Med/Peds resident passing board exams. In this study, we aimed to determine how standardized test scores predict performance on ABIM-CE and ABP-CE for Med/Peds residents. Methodology We analyzed prior exam scores for 91/96 (95%) residents in a Med/Peds program from 2008 to 2017. Scores from the United States Medical Licensing Examination (USMLE) Steps 1 and 2 Clinical Knowledge (CK) and In-Training Exams in Internal Medicine (ITE-IM) and Pediatrics (ITE-P) were analyzed with the corresponding ABIM-CE and ABP-CE first-time scores. Linear and logistic regression were applied to predict board scores/passage. Results USMLE 1 and 2 CK, ITE-IM, and ITE-P scores had a linear relationship with both ABIM-CE and ABP-CE scores. In the linear regression, adjusted R2 values showed low-to-moderate predictive ability (R2 = 0.11-0.35), with the highest predictor of ABIM-CE and ABP-CE being USMLE Step 1 (0.35) and Postgraduate Year 1 (PGY-1) ITE-IM (0.33), respectively. Logistic regression showed odds ratios of passing board certifications ranging from 1.05 to 1.53 per point increase on the prior exam score. The PGY-3 ITE-IM was the best predictor of passing both certifying exams. Conclusions In one Med/Peds program, USMLE Steps 1 and 2 and all ITE-IM and ITE-P scores predicted certifying exam scores and passage. This provides Med/Peds-specific data to allow individualized resident counseling and guide programmatic improvements targeted to board performance.
Collapse
Affiliation(s)
- Shelley R Ost
- General Internal Medicine, University of Tennessee Health Science Center College of Medicine, Memphis, USA
| | - Daniel Wells
- General Pediatrics, University of Tennessee Health Science Center College of Medicine, Memphis, USA
| | - Patricia J Goedecke
- Preventive Medicine, University of Tennessee Health Science Center College of Medicine, Memphis, USA
| | - Elizabeth A Tolley
- Preventive Medicine, University of Tennessee Health Science Center College of Medicine, Memphis, USA
| | - Michael Kleinman
- General Internal Medicine, University of Tennessee Health Science Center College of Medicine, Memphis, USA
| | - Natascha S Thompson
- General Internal Medicine, University of Tennessee Health Science Center College of Medicine, Memphis, USA
| |
Collapse
|
7
|
Filiberto AC, Cooper LA, Loftus TJ, Samant SS, Sarosi GA, Tan SA. Objective predictors of intern performance. BMC MEDICAL EDUCATION 2021; 21:77. [PMID: 33499857 PMCID: PMC7839184 DOI: 10.1186/s12909-021-02487-0] [Citation(s) in RCA: 2] [Impact Index Per Article: 0.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 09/21/2020] [Accepted: 12/14/2020] [Indexed: 05/15/2023]
Abstract
BACKGROUND Residency programs select medical students for interviews and employment using metrics such as the United States Medical Licensing Examination (USMLE) scores, grade-point average (GPA), and class rank/quartile. It is unclear whether these metrics predict performance as an intern. This study tested the hypothesis that performance on these metrics would predict intern performance. METHODS This single institution, retrospective cohort analysis included 244 graduates from four classes (2015-2018) who completed an Accreditation Council for Graduate Medical Education (ACGME) certified internship and were evaluated by program directors (PDs) at the end of the year. PDs provided a global assessment rating and ratings addressing ACGME competencies (response rate = 47%) with five response options: excellent = 5, very good = 4, acceptable = 3, marginal = 2, unacceptable = 1. PDs also classified interns as outstanding = 4, above average = 3, average = 2, and below average = 1 relative to other interns from the same residency program. Mean USMLE scores (Step 1 and Step 2CK), third-year GPA, class rank, and core competency ratings were compared using Welch's ANOVA and follow-up pairwise t-tests. RESULTS Better performance on PD evaluations at the end of intern year was associated with higher USMLE Step 1 (p = 0.006), Step 2CK (p = 0.030), medical school GPA (p = 0.020) and class rank (p = 0.016). Interns rated as average had lower USMLE scores, GPA, and class rank than those rated as above average or outstanding; there were no significant differences between above average and outstanding interns. Higher rating in each of the ACGME core competencies was associated with better intern performance (p < 0.01). CONCLUSIONS Better performance as an intern was associated with higher USMLE scores, medical school GPA and class rank. When USMLE Step 1 reporting changes from numeric scores to pass/fail, residency programs can use other metrics to select medical students for interviews and employment.
Collapse
Affiliation(s)
- Amanda C Filiberto
- Department of Surgery, University of Florida Health, 1600 SW Archer Ave, PO Box 100109, Gainesville, FL, 32610, USA
| | - Lou Ann Cooper
- Office for Educational Affairs, University of Florida College of Medicine, Gainesville, FL, USA
| | - Tyler J Loftus
- Department of Surgery, University of Florida Health, 1600 SW Archer Ave, PO Box 100109, Gainesville, FL, 32610, USA
| | - Sonja S Samant
- University of Florida College of Medicine, Gainesville, FL, USA
| | - George A Sarosi
- Department of Surgery, University of Florida Health, 1600 SW Archer Ave, PO Box 100109, Gainesville, FL, 32610, USA
| | - Sanda A Tan
- Department of Surgery, University of Florida Health, 1600 SW Archer Ave, PO Box 100109, Gainesville, FL, 32610, USA.
| |
Collapse
|
8
|
Tsunekawa K, Suzuki Y, Shioiri T. Identifying and supporting students at risk of failing the National Medical Licensure Examination in Japan using a predictive pass rate. BMC MEDICAL EDUCATION 2020; 20:419. [PMID: 33167945 PMCID: PMC7654142 DOI: 10.1186/s12909-020-02350-8] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 06/10/2020] [Accepted: 10/30/2020] [Indexed: 06/11/2023]
Abstract
BACKGROUND Students who fail to pass the National Medical Licensure Examination (NMLE) pose a huge problem from the educational standpoint of healthcare professionals. In the present study, we developed a formula of predictive pass rate (PPR)" which reliably predicts medical students who will fail the NMLE in Japan, and provides an adequate academic support for them. METHODS Six consecutive cohorts of 531 medical students between 2012 and 2017, Gifu University Graduate School of Medicine, were investigated. Using 7 variables before the admission to medical school and 10 variables after admission, we developed a prediction formula to obtain the PPR for the NMLE using logistic regression analysis. In a new cohort of 106 medical students in 2018, we applied the formula for PPR to them to confirm the capability of the PPR and predicted students who will have a strong likelihood of failing the NMLE. RESULTS Medical students who passed the NMLE had the following characteristics: younger age at admission, graduates of high schools located in the surrounding area, high scores in the graduation examination and in the comprehensive computer-based test provided by the Common Achievement Test Organization in Japan. However, total score of examination in pre-clinical medical sciences and Pre-CC OSCE score in the 4th year were not correlated with the PPR. Ninety-one out of 531 students had a strong likelihood of failing the NMLE between 2012 and 2017 and 33 of these 91 students failed NMLE. Using the PPR, we predicted 12 out of 106 students will have a strong likelihood of failing the NMLE. Actually, five of these 12 students failed NMLE. CONCLUSIONS The PPR can be used to predict medical students who have a higher probability of failing the NMLE. This prediction would enable focused support and guidance by faculty members. Prospective and longitudinal studies for larger and different cohorts would be necessary.
Collapse
Affiliation(s)
- Koji Tsunekawa
- Department of Institutional Research for Medical Education, Gifu University Graduate School of Medicine, Yanagito 1-1, Gifu City, 501-1194, Japan.
- Medical Educational Development Center, Gifu University, Gifu, Japan.
| | - Yasuyuki Suzuki
- Medical Educational Development Center, Gifu University, Gifu, Japan
| | - Toshiki Shioiri
- Department of Institutional Research for Medical Education, Gifu University Graduate School of Medicine, Yanagito 1-1, Gifu City, 501-1194, Japan
- Department of Psychiatry and Psychotherapy, Gifu University Graduate School of Medicine, Gifu, Japan
| |
Collapse
|
9
|
Goshtasbi K, Abouzari M, Tjoa T, Malekzadeh S, Bhandarkar ND. The Effects of Pass/Fail USMLE Step 1 Scoring on the Otolaryngology Residency Application Process. Laryngoscope 2020; 131:E738-E743. [PMID: 32880975 DOI: 10.1002/lary.29072] [Citation(s) in RCA: 42] [Impact Index Per Article: 10.5] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 07/06/2020] [Revised: 08/04/2020] [Accepted: 08/13/2020] [Indexed: 11/10/2022]
Abstract
OBJECTIVES To investigate how the decision to report United States Medical Licensing Examination (USMLE) Step 1 score as pass/fail will influence future otolaryngology residency application and match processes. STUDY DESIGN Survey study. METHODS An anonymous and voluntary survey approved by the Otolaryngology Program Directors Organization was administered to academic faculty members from April 24, 2020 through May 19, 2020. RESULTS Two hundred fifty-seven surveys were received from department chairs (17.5%), program directors (24.1%), associate program directors (12.5%), and department faculty (45.9%). USMLE Step 1 has been the most heavily weighted metric for offering interviews (44.0%), and it has correlated with residents' medical knowledge (77.0%) and in-service performance (79.8%) but not with surgical skills (57.6%) or patient care (47.1%). In total, 68.1% disagreed with the decision to make USMLE Step 1 pass/fail. This change is anticipated to lead to an increase in significance of USMLE Step 2 CK (89.1%), core clerkship grades (80.9%), elective rotation at the respective institutions (65.7%), Alpha Omega Alpha and other awards (64.6%), and letters of recommendation (63.8%). The new scoring is also anticipated to especially benefit students from top-ranked schools (70.8%), increase medical students' anxiety/uncertainty regarding obtaining interview invites (59.1%), and negatively affect international (51.4%), doctor of osteopathic medicine (45.9%), and underrepresented students (36.9%). Indication that USMLE Step 2 CK will significantly increase in weight varied according to department position (P = .049), geographic region (P = .047), years of practice (P < .001), and residency program size (P = .002). CONCLUSION Most academic otolaryngologists disagreed with changing USMLE Step 1 scoring to pass/fail and believe that it will increase other objective/subjective metrics' weight and put certain student populations at a disadvantage. LEVEL OF EVIDENCE N/A. Laryngoscope, 131:E738-E743, 2021.
Collapse
Affiliation(s)
- Khodayar Goshtasbi
- Department of Otolaryngology-Head and Neck Surgery, University of California, Irvine, California, U.S.A
| | - Mehdi Abouzari
- Department of Otolaryngology-Head and Neck Surgery, University of California, Irvine, California, U.S.A
| | - Tjoson Tjoa
- Department of Otolaryngology-Head and Neck Surgery, University of California, Irvine, California, U.S.A
| | - Sonya Malekzadeh
- Department of Otolaryngology-Head and Neck Surgery, Georgetown University Medical Center, Washington, District of Columbia, U.S.A
| | - Naveen D Bhandarkar
- Department of Otolaryngology-Head and Neck Surgery, University of California, Irvine, California, U.S.A
| |
Collapse
|
10
|
Peterson LE, Boulet JR, Clauser B. Associations Between Medical Education Assessments and American Board of Family Medicine Certification Examination Score and Failure to Obtain Certification. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2020; 95:1396-1403. [PMID: 32271228 DOI: 10.1097/acm.0000000000003344] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/11/2023]
Abstract
PURPOSE Family medicine residency programs can be cited for low pass or take rates on the American Board of Family Medicine (ABFM) certification examination, and the relationships among standardized medical education assessments and performance on board certification examinations and eventual board certification have not been comprehensively studied. The objective of this study was to evaluate the associations of all required standardized examinations in medical education with ABFM certification examination scores and eventual ABFM certification. METHOD All graduates of U.S. MD-granting family medicine residency programs from 2008 to 2012 were included. Data on ABFM certification examination score, ABFM certification status (as of December 31, 2014), Medical College Admission Test (MCAT) section scores, undergraduate grade point average, all United States Medical Licensing Examination (USMLE) Step scores, and all ABFM in-training examination scores were linked. Nested logistic and linear regression models, controlling for clustering by residency program, determined associations between assessments and both certification examination scores and board certification status. As many international medical graduates (IMGs) do not take the MCAT, separate models for U.S. medical graduates (USMG) and IMGs were run. RESULTS The study sample was 15,902 family medicine graduates, of whom 92.1% (14,648/15,902) obtained board certification. In models for both IMGs and USMGs, the addition of more recent assessments weakened the associations of earlier assessments. USMLE Step 2 Clinical Knowledge was predictive of certification examination scores and certification status in all models in which it was included. CONCLUSIONS For family medicine residents, more recent assessments generally have stronger associations with board certification score and status than earlier assessments. Solely using medical school admissions (grade point average and MCAT) and licensure (USMLE) scores for resident selection may not adequately predict ultimate board certification.
Collapse
Affiliation(s)
- Lars E Peterson
- L.E. Peterson is vice president of research, American Board of Family Medicine, and associate professor, Department of Family and Community Medicine, University of Kentucky, Lexington, Kentucky; ORCID: http://orcid.org/0000-0003-4853-3108
| | - John R Boulet
- J.R. Boulet is vice president, Research and Data Resources, Foundation for Advancement of International Medical Education and Research, Philadelphia, Pennsylvania
| | - Brian Clauser
- B. Clauser is vice president, Center for Advanced Assessment, National Board of Medical Examiners, Philadelphia, Pennsylvania
| |
Collapse
|
11
|
Isaq NA, Bowers S, Chen ST. Taking a "step" toward diversity in dermatology: De-emphasizing USMLE Step 1 scores in residency applications. Int J Womens Dermatol 2020; 6:209-210. [PMID: 32637547 PMCID: PMC7330438 DOI: 10.1016/j.ijwd.2020.02.008] [Citation(s) in RCA: 10] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Key Words] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 10/24/2019] [Revised: 01/28/2020] [Accepted: 02/27/2020] [Indexed: 10/31/2022] Open
Affiliation(s)
- Nasro A Isaq
- Indiana University School of Medicine, Indianapolis, IN, United States
| | - Sacharitha Bowers
- Southern Illinois University School of Medicine, Department of Medicine, Division of Dermatology, Springfield, IL, United States
| | - Steven T Chen
- Harvard Medical School, Boston, MA, United States.,Massachusetts General Hospital, Department of Dermatology, 50 Staniford St, 2nd Floor, Boston, MA, United States
| |
Collapse
|
12
|
Marshall AL, Gupta R, Grill D, Mann S, Freidline K, Nowakowski G, Thompson C, Hobday T. Identification of Factors Associated with Hematology-Oncology Fellow Academic Success and Career Choice. JOURNAL OF CANCER EDUCATION : THE OFFICIAL JOURNAL OF THE AMERICAN ASSOCIATION FOR CANCER EDUCATION 2019; 34:1219-1224. [PMID: 30267295 DOI: 10.1007/s13187-018-1432-7] [Citation(s) in RCA: 11] [Impact Index Per Article: 2.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/08/2023]
Abstract
Factors affecting hematology-oncology trainees' academic success and career choices have not been well characterized. We performed a retrospective study of 57 hematology-oncology fellows trained at Mayo Clinic between 2008 and 2017 in an attempt to identify factors associated with success during fellowship and with career choice (academic versus private). Sex, age, residency quality, and letters of recommendation indicating a "top" applicant were not associated with hematology or oncology in-training examination (ITE) scores, research productivity (abstracts/publications during fellowship), or career choice (academic versus private). Fellows with higher United States Medical Licensing Examination (USMLE) scores were more likely to perform well on ITE, but examination scores did not predict academic productivity or academic versus private career choice. More academically productive fellows were more likely to choose academic careers. Both ITE scores and productivity were associated with receipt of national and/or institutional awards. Finally, fellows who were non-US citizens and/or international medical graduates (IMG) had higher academic productivity both pre-fellowship and during fellowship and as per the observations above were more likely to choose academic careers. In conclusion, predictors of superior knowledge differ from predictors of academic productivity/career choice, and it is important to take multiple factors into account when selecting candidates most likely to succeed during fellowship.
Collapse
Affiliation(s)
- Ariela L Marshall
- Division of Hematology, Department of Internal Medicine, Mayo Clinic, Mayo Building 10th floor, 10-90E, 200 First Street SW, Rochester, MN, 55905, USA.
- Department of Laboratory Medicine and Pathology, Mayo Clinic, Rochester, MN, USA.
| | - Ruchi Gupta
- Division of Biomedical Statistics and Informatics, Mayo Clinic, Rochester, MN, USA
| | - Diane Grill
- Division of Biomedical Statistics and Informatics, Mayo Clinic, Rochester, MN, USA
| | - Susan Mann
- Division of Hematology, Department of Internal Medicine, Mayo Clinic, Mayo Building 10th floor, 10-90E, 200 First Street SW, Rochester, MN, 55905, USA
| | - Kimberly Freidline
- Division of Hematology, Department of Internal Medicine, Mayo Clinic, Mayo Building 10th floor, 10-90E, 200 First Street SW, Rochester, MN, 55905, USA
| | - Grzegorz Nowakowski
- Division of Hematology, Department of Internal Medicine, Mayo Clinic, Mayo Building 10th floor, 10-90E, 200 First Street SW, Rochester, MN, 55905, USA
| | - Carrie Thompson
- Division of Hematology, Department of Internal Medicine, Mayo Clinic, Mayo Building 10th floor, 10-90E, 200 First Street SW, Rochester, MN, 55905, USA
| | | |
Collapse
|
13
|
Rozenshtein A, Mullins ME, Marx MV. The USMLE Step 1 Pass/Fail Reporting Proposal: The APDR Position. Acad Radiol 2019; 26:1400-1402. [PMID: 31383545 DOI: 10.1016/j.acra.2019.06.004] [Citation(s) in RCA: 31] [Impact Index Per Article: 6.2] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 04/01/2019] [Revised: 06/07/2019] [Accepted: 06/10/2019] [Indexed: 11/17/2022]
Abstract
BACKGROUND The National Board of Medical Examiners (NBME) and the United States Medical Licensing Examination (USMLE) has convened a conference of "key stakeholders" on March 11-12, 2019 to consider reporting the results of the USMLE Step 1 as pass/fail. DISCUSSION While the original purpose of the USMLE Step 1 was to provide an objective basis for medical licensing, the score is increasingly used in residency applicant screening and selection because it is an objective, nationally recognized metric allowing comparison across medical schools in and outside the United States. Excessive reliance on the Step 1 score in the matching process has led to "Step 1 Culture" that drives medical schools to "teach to the test," increases medical student anxiety, and disadvantages minorities that have been shown to score lower on the USMLE Step 1 examination. The outsize role of the USMLE Step 1 score in resident selection is due to lack of standardization in medical school transcripts, grade inflation, and the lack of class standing in many summative assessments. Furthermore, the numeric score allows initial Electronic Residency Application Service filtering, commonly used by programs to limit the number of residency applications to review. CONCLUSION The Association of Program Directors in Radiology (APDR) is concerned that pass/fail reporting of the USMLE Step 1 score would take away an objective measure of medical student's knowledge and the incentive to acquire as much of it as possible. Although the APDR is not in favor of the Step 1 exam being used as a screening tool, in the absence of an equal or better metric for applicant comparison the APDR opposes the change in Step 1 reporting from the numeric score to pass/fail.
Collapse
Affiliation(s)
- Anna Rozenshtein
- Department of Radiology, Westchester Medical Center-New York Medical College, 100 Woods Road, Valhalla, NY 10595.
| | - Mark E Mullins
- Department of Radiology and Imaging Sciences, Emory University, Atlanta, Georgia
| | - M Victoria Marx
- Department of Radiology, Keck School of Medicine University of South California, Los Angeles, California
| |
Collapse
|
14
|
Thompson RH, Lohse CM, Husmann DA, Leibovich BC, Gettman MT. Predictors of Urology Resident Surgical Skills, Clinical Communication Skills, Common Sense and In-Service Scores. UROLOGY PRACTICE 2019. [DOI: 10.1016/j.urpr.2017.12.002] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Indexed: 11/26/2022]
|
15
|
Predictors of a Successful Urology Resident Using Medical Student Application Materials. Urology 2017; 108:22-28. [DOI: 10.1016/j.urology.2017.06.046] [Citation(s) in RCA: 21] [Impact Index Per Article: 3.0] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 03/31/2017] [Revised: 05/24/2017] [Accepted: 06/06/2017] [Indexed: 11/19/2022]
|
16
|
Nagasawa DT, Beckett JS, Lagman C, Chung LK, Schmidt B, Safaee M, Bergsneider M, Martin N, Gaonkar B, Macyszyn L, Yang I. United States Medical Licensing Examination Step 1 Scores Directly Correlate with American Board of Neurological Surgery Scores: A Single-Institution Experience. World Neurosurg 2017; 98:427-431. [DOI: 10.1016/j.wneu.2016.11.091] [Citation(s) in RCA: 19] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/03/2016] [Revised: 11/15/2016] [Accepted: 11/16/2016] [Indexed: 10/20/2022]
|
17
|
Gauer JL, Jackson JB. The association of USMLE Step 1 and Step 2 CK scores with residency match specialty and location. MEDICAL EDUCATION ONLINE 2017; 22:1358579. [PMID: 28762297 PMCID: PMC5653932 DOI: 10.1080/10872981.2017.1358579] [Citation(s) in RCA: 54] [Impact Index Per Article: 7.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/03/2017] [Accepted: 07/17/2017] [Indexed: 05/25/2023]
Abstract
BACKGROUND For future physicians, residency programs offer necessary extended training in specific medical specialties. Medical schools benefit from an understanding of factors that lead their students to match into certain residency specialties. One such factor, often used during the residency application process, is scores on the USA Medical Licensing Exam (USMLE). OBJECTIVES To determine the relationship between USMLE Step 1 and Step 2 Clinical Knowledge (CK) scores and students' residency specialty match, and the association between both USMLE scores and state of legal residency (Minnesota) at the time of admission with students staying in-state or leaving the state for residency program. DESIGN USMLE scores and residency match data were analyzed from five graduating classes of students at the University of Minnesota Medical School (N = 1054). RESULTS A MANOVA found significant differences (p < 0.001) between residency specialties and both USMLE Step 1 and Step 2 CK scores, as well as the combination of the two. Students who matched in Dermatology had the highest mean USMLE scores overall, while students who matched in Family Medicine had the lowest mean scores. Students who went out of state for residency had significantly higher Step 1 scores (p = 0.027) than students who stayed in-state for residency, while there was no significant difference between the groups for Step 2 scores. A significant positive association was found between a student who applied as a legal resident of Minnesota and whether the student stayed in Minnesota for their residency program. CONCLUSIONS Residency specialty match was significantly associated with USMLE Step 1 and USMLE Step 2 CK scores, as was staying in-state or leaving the state for residency. Students who were legal residents of the state at the time of application were more likely to stay in-state for residency, regardless of USMLE score. ABBREVIATIONS CK: Clinical knowledge; COMLEX: Comprehensive Osteopathic Medical Licensing Examination; GME: Graduate medical education; NRMP: National Resident Matching Program; UME: Undergraduate medical education; USMLE: United States Medical Licensing Examination.
Collapse
|
18
|
O'Neill TR, Peabody MR, Song H. The Predictive Validity of the National Board of Osteopathic Medical Examiners' COMLEX-USA Examinations With Regard to Outcomes on American Board of Family Medicine Examinations. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2016; 91:1568-1575. [PMID: 27254014 DOI: 10.1097/acm.0000000000001254] [Citation(s) in RCA: 5] [Impact Index Per Article: 0.6] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/05/2023]
Abstract
PURPOSE To examine the predictive validity of the National Board of Osteopathic Medical Examiners' Comprehensive Osteopathic Medical Licensing Examination of the United States of America (COMLEX-USA) series with regard to the American Board of Family Medicine's (ABFM's) In-Training Examination (ITE) and Maintenance of Certification for Family Physicians (MC-FP) Examination. METHOD A repeated-measures design was employed, using test scores across seven levels of training for 1,023 DOs who took the MC-FP for the first time between April 2012 and November 2014 and for whom the ABFM had ITE scores for each of their residency years. Pearson and disattenuated correlations were calculated; Fisher r to z transformation was performed; and sensitivity, specificity, and positive and negative predictive values for the COMLEX-USA Level 2-Cognitive Evaluation (CE) with regard to the MC-FP were computed. RESULTS The Pearson and disattenuated correlations ranged from 0.55 to 0.69 and from 0.61 to 0.80, respectively. For MC-FP scores, only the correlation increase from the COMLEX-USA Level 2-CE to Level 3 was statistically significant (for Pearson correlations: z = 2.41, P = .008; for disattenuated correlations: z = 3.16, P < .001). The sensitivity, specificity, and positive and negative predictive values of the COMLEX-USA Level 2-CE with the MC-FP were 0.90, 0.39, 0.96, and 0.19, respectively. CONCLUSIONS Evidence was found that the COMLEX-USA can assist family medicine residency program directors in predicting later resident performance on the ABFM's ITE and MC-FP, which is becoming increasingly important as graduate medical education accreditation moves toward a single aligned model.
Collapse
Affiliation(s)
- Thomas R O'Neill
- T.R. O'Neill is vice president of psychometric services, American Board of Family Medicine, Lexington, Kentucky. M.R. Peabody is a psychometrician, American Board of Family Medicine, Lexington, Kentucky. H. Song is senior director for psychometrics and research, National Board of Osteopathic Medical Examiners, Chicago, Illinois
| | | | | |
Collapse
|
19
|
Kimbrough TN, Heh V, Wijesooriya NR, Ryan MS. Family-centered rounds and medical student performance on the NBME pediatrics subject (shelf) examination: a retrospective cohort study. MEDICAL EDUCATION ONLINE 2016; 21:30919. [PMID: 27087016 PMCID: PMC4834362 DOI: 10.3402/meo.v21.30919] [Citation(s) in RCA: 3] [Impact Index Per Article: 0.4] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/06/2016] [Accepted: 02/22/2016] [Indexed: 06/02/2023]
Abstract
OBJECTIVE To determine the association between family-centered rounds (FCR) and medical student knowledge acquisition as assessed by the National Board of Medical Examiners (NBME) pediatric subject (shelf) exam. METHODS A retrospective cohort study was conducted of third-year medical students who graduated from Virginia Commonwealth University School of Medicine between 2009 and 2014. This timeframe represented the transition from 'traditional' rounds to FCR on the pediatric inpatient unit. Data collected included demographics, United States Medical Licensing Examination (USMLE) Step 1 and 2 scores, and NBME subject examinations in pediatrics (PSE), medicine (MSE), and surgery (SSE). RESULTS Eight hundred and sixteen participants were included in the analysis. Student performance on the PSE could not be statistically differentiated from performance on the MSE for any year except 2011 (z-score=-0.17, p=0.02). Average scores on PSE for years 2009, 2010, 2013, and 2014 were significantly higher than for SSE, but not significantly different for all other years. The PSE was highly correlated with USMLE Step 1 and Step 2 examinations (correlation range 0.56-0.77) for all years. CONCLUSIONS Our results showed no difference in PSE performance during a time in which our institution transitioned to FCR. These findings should be reassuring for students, attending physicians, and medical educators.
Collapse
Affiliation(s)
- Tiffany N Kimbrough
- Department of Pediatrics, Virginia Commonwealth University, Richmond, VA, USA;
| | - Victor Heh
- CORE Research Office, Heritage College of Osteopathic Medicine, Ohio University, Dublin, OH, USA
| | | | - Michael S Ryan
- Department of Pediatrics, Virginia Commonwealth University, Richmond, VA, USA
| |
Collapse
|
20
|
Yousem IJ, Liu L, Aygun N, Yousem DM. United States Medical Licensing Examination Step 1 and 2 Scores Predict Neuroradiology Fellowship Success. J Am Coll Radiol 2016; 13:438-44.e2. [DOI: 10.1016/j.jacr.2015.10.024] [Citation(s) in RCA: 13] [Impact Index Per Article: 1.6] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/11/2015] [Revised: 10/12/2015] [Accepted: 10/14/2015] [Indexed: 11/29/2022]
|
21
|
|
22
|
Abstract
From senior school through to consultancy, a plethora of assessments shape medical careers. Multiple methods of assessment are used to discriminate between applicants. Medical selection in the UK appears to be moving increasingly towards non-knowledge-based testing at all career stages. We review the evidence for non-knowledge-based tests and discuss their perceived benefits. We raise the question: is the current use of non-knowledge-based tests within the UK at risk of undermining more robust measures of medical school and postgraduate performance?
Collapse
Affiliation(s)
- Benjamin H L Harris
- John Radcliffe Hospital, Oxford, UK, and non-stipendiary lecturer, St Anne's College, University of Oxford, Oxford, UK
| | | | - Simon Lammy
- Nuffield Department of Surgical Sciences, University of Oxford, John Radcliffe Hospital, Oxford, UK, and ST1 neurosurgery, Department of Neurosurgery, Institute of Neurological Sciences, Southern General Hospital, Glasgow, UK
| |
Collapse
|
23
|
Kay C, Jackson JL, Frank M. The relationship between internal medicine residency graduate performance on the ABIM certifying examination, yearly in-service training examinations, and the USMLE Step 1 examination. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2015; 90:100-104. [PMID: 25271892 DOI: 10.1097/acm.0000000000000500] [Citation(s) in RCA: 47] [Impact Index Per Article: 5.2] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/03/2023]
Abstract
PURPOSE To explore the relationship between United States Medical Licensing Examination (USMLE) Step 1 scores, yearly in-service training exam (ITE) scores, and passing the American Board of Internal Medicine certifying examination (ABIM-CE). METHOD The authors conducted a retrospective database review of internal medicine residents from the Medical College of Wisconsin from 2004 through 2012. Residents' USMLE Step 1, ITE, and ABIM-CE scores were extracted. Pearson rho, chi-square, and logistic regression were used to determine whether relationships existed between the scores and if Step 1 and ITE scores correlate with passing the ABIM-CE. RESULTS There were 241 residents, who participated in 728 annual ITEs. There were Step 1 scores for 195 (81%) residents and ABIM-CE scores for 183 (76%). Step 1 and ABIM-CE scores had a modest correlation (rho: 0.59), as did ITE and ABIM-CE scores (rho: 0.48-0.67). Failing Step 1 or being in the bottom ITE quartile during any year of testing markedly increased likelihood of failing the boards (Step 1: relative risk [RR]: 2.4; 95% CI: 1.0-5.9; first-year residents' RR: 1.3; 95% CI: 1.0-1.6; second-year residents' RR: 1.3; 95% CI: 1.1-1.5; third-year residents' RR: 1.3; 95% CI: 1.1-1.5). CONCLUSIONS USMLE Step 1 and ITE scores have a modest correlation with board scores. Failing Step 1 or scoring in the bottom quartile of the ITE increased the risk of failing the boards. What effective intervention, if any, program directors may use with at-risk residents is a question deserving further research.
Collapse
Affiliation(s)
- Cynthia Kay
- Dr. Kay is instructor/fellow, Department of Medicine, Medical College of Wisconsin, Milwaukee, Wisconsin. Dr. Jackson is professor of medicine, Medical College of Wisconsin, and chief, General Internal Medicine Section, Zablocki VA Medical Center, Milwaukee, Wisconsin. Dr. Frank is professor of medicine and program director, Internal Medicine Residency Program, Medical College of Wisconsin, Milwaukee, Wisconsin
| | | | | |
Collapse
|
24
|
Dermatology residency selection criteria with an emphasis on program characteristics: a national program director survey. Dermatol Res Pract 2014; 2014:692760. [PMID: 24772165 PMCID: PMC3977115 DOI: 10.1155/2014/692760] [Citation(s) in RCA: 27] [Impact Index Per Article: 2.7] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Subscribe] [Scholar Register] [Received: 12/28/2013] [Accepted: 02/06/2014] [Indexed: 11/18/2022] Open
Abstract
Background. Dermatology residency programs are relatively diverse in their resident selection process. The authors investigated the importance of 25 dermatology residency selection criteria focusing on differences in program directors' (PDs') perception based on specific program demographics. Methods. This cross-sectional nationwide observational survey utilized a 41-item questionnaire that was developed by literature search, brainstorming sessions, and online expert reviews. The data were analyzed utilizing the reliability test, two-step clustering, and K-means methods as well as other methods. The main purpose of this study was to investigate the differences in PDs' perception regarding the importance of the selection criteria based on program demographics. Results. Ninety-five out of 114 PDs (83.3%) responded to the survey. The top five criteria for dermatology residency selection were interview, letters of recommendation, United States Medical Licensing Examination Step I scores, medical school transcripts, and clinical rotations. The following criteria were preferentially ranked based on different program characteristics: “advanced degrees,” “interest in academics,” “reputation of undergraduate and medical school,” “prior unsuccessful attempts to match,” and “number of publications.” Conclusions. Our survey provides up-to-date factual data on dermatology PDs' perception in this regard. Dermatology residency programs may find the reported data useful in further optimizing their residency selection process.
Collapse
|
25
|
Kenny S, McInnes M, Singh V. Associations between residency selection strategies and doctor performance: a meta-analysis. MEDICAL EDUCATION 2013; 47:790-800. [PMID: 23837425 DOI: 10.1111/medu.12234] [Citation(s) in RCA: 50] [Impact Index Per Article: 4.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 11/27/2012] [Revised: 02/04/2013] [Accepted: 03/21/2013] [Indexed: 05/17/2023]
Abstract
OBJECTIVES The purpose of this study was to use meta-analysis to establish which of the information available to the resident selection committee is associated with resident or doctor performance. METHODS Multiple electronic databases were searched to 4 September 2012. Two reviewers independently selected studies that met the present inclusion criteria and extracted data in duplicate; disagreement was resolved by consensus. Risk for bias was assessed using a customised bias assessment tool. Measures of association were converted to a common effect size (Hedges' g). Meta-analysis was performed using the random-effects model for each selection strategy and all outcomes without pooling. Sensitivity analysis for each selection strategy-outcome pair was performed with pooling of effect size. RESULTS Eighty studies involving a total of 41 704 participants were included in the meta-analysis. Seventeen different selection strategies and 17 outcomes were assessed across these studies. The strongest positive associations referred to examination-based selection strategies, such as the US Medical Licensing Examination (USMLE) Step 1, and examination-based outcomes, such as scores on in-training examinations. Moderate positive associations were present for medical school marks and both examination-based and subjective outcomes. Minimal or no associations were seen for the selection tools represented by interviews, reference letters and deans' letters. CONCLUSIONS Standardised examination performance and medical school grades show the strongest associations with current measures of doctor performance. Deans' letters, reference letters and interviews all show a lower than expected strength of association given the relative value often assigned to them during resident doctor selection. Objective selection strategies are potentially the most useful to residency selection committees based on current evaluative methods. However, reports in the literature of validated long-term doctor performance outcomes are scant.
Collapse
Affiliation(s)
- Stephanie Kenny
- Department of Medical Imaging, Ottawa Hospital, University of Ottawa, Ottawa, Ontario, Canada
| | | | | |
Collapse
|
26
|
|
27
|
Grewal SG, Yeung LS, Brandes SB. Predictors of success in a urology residency program. JOURNAL OF SURGICAL EDUCATION 2013; 70:138-143. [PMID: 23337683 DOI: 10.1016/j.jsurg.2012.06.015] [Citation(s) in RCA: 43] [Impact Index Per Article: 3.9] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 02/26/2012] [Revised: 05/18/2012] [Accepted: 06/19/2012] [Indexed: 05/27/2023]
Abstract
OBJECTIVES In selecting a medical student for a urology residency, a set of preconceived criteria as to what will predict a successful resident are generally applied. To determine what factors predict an "excellent" clinical resident and a successful in-service test taker, we analyzed 10 years of urology resident files. PARTICIPANTS AND STUDY DESIGN: Retrospective chart review of 29 urology residents at Washington University graduating from July 2000 to July 2009. Medical student applications and interview evaluations were compared with future performance as a general surgical intern and then as a urology resident, in terms of clinical performance and in-service examination scores. RESULTS Of 29 residents, based on clinical evaluations over 4 years of urology residency, 12 were "excellent," 17 "average and needing improvement." "Excellent" residents had higher applicant rank submitted to the "match" (7.2 vs. 12.1, p = 0.04) and better letters of recommendation (3.0 vs. 2.5, 0.018). "Excellent" residents also had better evaluations as an intern (3.9 vs 2.7, p < 0.001). "Good" urology in-service examination test takers compared with "below average" test takers noted higher rank on the match list (7.8 vs 12.1, p = 0.04), better quality med school (2.6 vs 2.0; p = 0.002), higher USMLE scores (92.5 vs 86.6% tile, p = 0.02), American Board of Surgery in-training examination (ABSITE) score (58.6 vs 37.2% tile, p = 0.04), and were more likely to pass the board examination (100% vs 76.9%, p = 0.03). Residents with higher clinical evaluations were also more likely to go into fellowships (83.3% vs 16.2%, OR = 23.3) and academic careers (41.6 vs 11.1%, OR = 5.71). CONCLUSIONS Performance as a surgery intern predicts future performance as a GU Resident. "Good" test takers as medical students and as interns continue to test well as GU residents. Early identification, intervention, and mentoring while still an intern are essential. Selection criteria we currently use to select GU residents are surprisingly predictive.
Collapse
Affiliation(s)
- Shaun G Grewal
- Department of Surgery, Washington University, St. Louis, Missouri, USA.
| | | | | |
Collapse
|
28
|
Impact of a Six-Year Integrated Thoracic Surgery Training Program at the Medical College of Wisconsin. Ann Thorac Surg 2012; 93:592-5; discussion 596-7. [DOI: 10.1016/j.athoracsur.2011.11.005] [Citation(s) in RCA: 18] [Impact Index Per Article: 1.5] [Reference Citation Analysis] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 01/26/2011] [Revised: 10/21/2011] [Accepted: 11/02/2011] [Indexed: 11/20/2022]
|