Mini Surgical Simulation, Role Play, and Group and Behavioral Interviews in Resident Selection

Evidence-Based Practices for Interviewing Graduate Medical Education Applicants: A Systematic Review

Background Although the selection interview is a standard admission practice for graduate medical education (GME) programs in the United States, there is a dearth of recent reviews on optimizing the trainee interview process, which has low reliability, high cost, and major risk of bias. Objective To investigate the evidence base for different selection interview practices in GME. Methods We searched 4 literature databases from inception through September 2022. Two investigators independently conducted title/abstract screening, full-text review, data extraction, and quality assessment. Disagreements were mediated by discussion. We used backward reference searching of included articles to identify additional studies. We included studies of different interview methods and excluded literature reviews, non-GME related publications, and studies comparing different applicant populations. We examined study characteristics, applicant and interviewer preferences, and interview format. We evaluated study quality using the Medical Education Research Study Quality Instrument (MERSQI). Results Of 2192 studies, 39 (2%) met our inclusion criteria. The evidence base was rated as moderately low quality using MERSQI criteria. Applicants reported preferences for several one-on-one interviews lasting 15 to 20 minutes, interviews by current trainees, and interviews including social events with only trainees. Applicants had mixed perceptions of virtual versus in-person interviews and reported that virtual interviews saved costs. The multiple mini interview (MMI) required more applicant and interviewer time than individual interviews but demonstrated construct and predictive validity and was preferred by applicants and interviewers. Conclusions Based on moderately low-quality evidence, using the MMI, training interviewers, and providing applicants with basic program information in advance should be considered for GME selection interviews.

Gender Associations with Selection into Australian Orthopaedic Surgical Training: 2007-2019

BACKGROUND

Communities have better health outcomes when their clinicians reflect the diversity of the communities they serve. More than 50% of Australian medical school graduates are female, yet women represent less than 5% of Australian orthopaedic surgeons. Selection into orthopaedic surgical training in Australia is an annual, nation-wide process, based on curriculum vitae (CV), referee reports and performance in multiple mini-interviews (MMI). The influence of applicant gender on these selection scores was examined.

METHODS

The CV, referee reports and MMI scores used for selection for each year from 2007 to 2019 were analysed from the perspective of the applicant's gender.

RESULTS

Over the years of the study, male applicants had higher CV scores and referee report scores, which determined the gender proportions invited to interview. By contrast, the interview process and selection from interview did not demonstrate a gender association.

CONCLUSION

We describe the impact of selection tools, utilized over the past 13 years, on the gender diversity of trainees commencing orthopaedic surgery training in Australia. Leaders in postgraduate training should examine commonly used selection procedures to identify and reduce the unconscious biases that may affect their performance and value.

Utilization of a Novel Program-Specific Evaluation Tool Results in a Decidedly Different Interview Pool Than Traditional Application Review

BACKGROUND

There are almost twice as many applicants as there are general surgery internships, each utilizing a common application with standard components. These elements are frequently not useful in determining affinity for a program or overall ability, and resultant poor fit may be partially responsible for program attrition. Alternative evaluation instruments would be beneficial to both programs and applicants.

METHODS

An application review committee comprised of resident representatives, faculty representing all program-affiliated institutions, and program leadership completed a written evaluation developed by a third party (SurgWise Consulting) that specializes in industrial and organizational psychology. The responses were compiled to create a standardized assessment tool. This assessment was sent to applicants who were subsequently ranked according to fit with our program. The pool of applicants was separately evaluated using our traditional application review. Two residents independently graded each applicant on a 5-point Likert scale to evaluate common application elements; applicants were subsequently assigned an overall score.

RESULTS

The assessment was completed by 507 (99%) of 512 qualifying applicants. Separately, 378 applications were reviewed by the traditional method for a total of 756 reviews. Of the 96 applicants identified by the assessment tool to invite for interviews, 22 (23%) qualified for interview invitations according to the traditional review method. The assessment produced 74 applicants that otherwise would not have been interviewed.

CONCLUSION

Traditional application review strategies have many shortcomings. A competency-based assessment tool in the residency application selection process identifies a pool of applicants not identified by traditional review methods.

Career Interest and Psychomotor Aptitude Among Medical Students

OBJECTIVE

The primary objective is to assess psychomotor aptitude of medical students interested in pursuing a procedural career. Secondary objectives include exploring the relationship between actual and perceived aptitude, and identifying predictors of superior aptitude.

DESIGN

This is a cross-sectional, multisite study in which participants completed a paper survey, four visuospatial aptitude assessments, and a laparoscopic simulation modeled after the Fundamentals of Laparoscopic Surgery (FLS) peg transfer test (used as a proxy for psychomotor aptitude).

SETTING

Johns Hopkins University School of Medicine and Mayo Clinic School of Medicine.

PARTICIPANTS

All second-year medical students who had not yet initiated clinical rotations were eligible. Sixty-four students participated.

RESULTS

Students interested in a procedural career exhibited superior psychomotor aptitude (faster FLS task completion time), and a majority of these students correctly identified themselves as having above-average aptitude compared with peers. However, over one quarter of all students, regardless of career interest, incorrectly over- or under-rated their psychomotor aptitude. Upon completing their preclinical curriculum, a minority of students felt prepared to participate or assist in their surgical clinical rotations.

CONCLUSIONS

Prior to embarking on their clinical rotations, over one quarter of medical students lack awareness of their psychomotor aptitude and many do not feel prepared to participate in the next phase of their training. Early aptitude testing and introduction to laparoscopic training may assist in career selection, preparedness, and success.

Interview Techniques Utilized in Radiology Resident Selection-A Survey of the APDR

RATIONALE AND OBJECTIVES

To evaluate interview techniques currently used in the selection of diagnostic radiology resident candidates and to identify factors influencing the use of alternative interview techniques.

MATERIALS AND METHODS

An anonymous 25 question e-mail survey was provided to 319 active members of the Association of Program Directors in Radiology. The survey included questions on residency demographics, organization of resident applicant interviews, types of interview techniques utilized, scoring and ranking of applicants, and facets of the interview/application felt most important to the selection process. Statistical analysis was performed to identify factors associated with the use of alternative interview techniques.

RESULTS

93.7% of responding programs use traditional interview techniques, with 92% using unblinded, unstructured interviews, 8% blinded, unstructured interviews. Structured interview questions were incorporated in 22%. Few programs used alternative techniques like the multiple mini-interview. None of the programs used written prompts during the interview, 3% used casual visual cognitive testing, 10% used panel interview techniques, and none used formal personality testing. For ranking candidates in the match, the most important facets considered were USMLE Step scores, performance on the interview, clinical course grades, and letters of reference. Factors associated with use of alternative techniques were domains associated with program size and number of faculty.

CONCLUSION

The majority of radiology training programs still rely upon the traditional unblinded interview technique. There is an opportunity for training programs to examine alternative techniques that reduce bias and may provide better insight into other aspects of the candidate that may not be as readily highlighted with the traditional, unblinded interview.

An Analysis of Obstetrics-Gynecology Residency Interview Methods in a Single Institution

OBJECTIVE

Interviews for residency are important for resident selection, yet how to best perform the interviews remains uncertain. Multiple approaches have been described with a variety of results. Our purpose of this study was to evaluate our Obstetrics-Gynecology residency program's interview structure to determine which interview components have association with performance evaluations completed by physicians and nurses during the residents' years of training at our program.

DESIGN

Using only data from our matched residents, existing interview scores from our standard interview process were compared to performance evaluation scores completed by faculty and nursing throughout the training years. Our standard interview process consisted of 4 interviews with individual faculty, 1 structured behavioral-based interview by a blinded faculty member, and 1 interview with a pair of current residents.

SETTING

A single, multisite, Obstetrics-Gynecology residency program.

PARTICIPANTS

Interviewees of the Obstetrics-Gynecology residency program that ultimately matched with our program.

RESULTS

Interview scores from 44 residents were compared to their performance evaluation scores. Positive associations were seen between performance evaluation scores and both resident teams' interviews and unstructured faculty interviews, with the resident teams' score showing a stronger degree of association. The behavioral-based interview total score did not have association with the performance evaluation scores, nor did any of the individual questions.

CONCLUSIONS

Resident teams' interview scores of applicants show the strongest association with eventual performance evaluations completed by faculty and nursing during residency. This demonstrates that current residents should have a role in the resident selection process during interviews. This does not provide data to abandon behavior-based interview techniques, but rather encourage each program to carefully apply their use during the resident selection process.

Personal domains assessed in multiple mini interviews (MMIs) for healthcare student selection: A narrative synthesis systematic review

OBJECTIVES

To examine the personal domains multiple mini interviews (MMIs) are being designed to assess, explore how they were determined and contextualise such domains in current and future healthcare student selection processes DESIGN: A systematic review of empirical research reporting on MMI model design was conducted from database inception to November 2017.

DATA SOURCES

Twelve electronic bibliographic databases.

REVIEW METHODS

Evidence was extracted from original studies, and integrated in a narrative synthesis guided by the PRISMA statement for reporting systematic reviews. Personal domains were clustered into themes using a modified Delphi technique.

RESULTS

A total of 584 articles were screened. 65 unique studies (80 articles) matched our inclusion criteria of which seven were conducted within nursing/midwifery faculties. Six in 10 studies featured applicants to medical school. Across selection processes, we identified 32 personal domains assessed by MMIs, the most frequent being: communication skills (84%), teamwork/collaboration (70%), and ethical/moral judgement (65%). Domains capturing ability to cope with stressful situations (14%), make decisions (14%), and resolve conflict in the workplace (13%) featured in fewer than ten studies overall. Intra- and inter-disciplinary inconsistencies in domain profiles were noted, as well as differences by entry level. MMIs deployed in nursing and midwifery assessed compassion and decision-making more frequently than in all other disciplines. Own programme philosophy and professional body guidance were most frequently cited (~50%) as sources for personal domains; a blueprinting process was reported in only 8% of studies.

CONCLUSIONS

Nursing, midwifery and allied healthcare professionals should develop their theoretical frameworks for MMIs to ensure they are evidence-based and fit-for-purpose. We suggest a re-evaluation of domain priorities to ensure that students who are selected, not only have the capacity to offer the highest standards of care provision, but are able to maintain these standards when facing clinical practice and organisational pressures.