Best practices for interviewing applicants for medical school admissions: a systematic review

Insights into undergraduate medical student selection tools: a systematic review and meta-analysis

PURPOSE

Evaluating medical school selection tools is vital for evidence-based student selection. With previous reviews revealing knowledge gaps, this meta-analysis offers insights into the effectiveness of these selection tools.

METHODS

A systematic review and meta-analysis were conducted applying the following criteria: peer-reviewed articles available in English, published from 2010 and which include empirical data linking performance in selection tools with assessment and dropout outcomes of undergraduate entry medical programs. Systematic reviews, meta-analyses, general opinion pieces, or commentaries were excluded. Effect sizes (ESs) of the predictability of academic and clinical performance within and by the end of the medicine program were extracted, and the pooled ESs were presented.

RESULTS

Sixty-seven out of 2,212 articles were included, which yielded 236 ESs. Previous academic achievement predicted medical program academic performance (Cohen’s d=0.697 in early program; 0.619 in end of program) and clinical exams (0.545 in end of program). Overall aptitude tests predicted academic achievement in both the early and last years (0.550 & 0.371, respectively). Within aptitude tests, verbal reasoning and quantitative reasoning best predicted academic achievement in the early program (0.704 & 0.643, respectively). Neither panel interviews, multiple mini-interviews, nor situational judgement tests (SJT) yielded statistically significant pooled ES.

CONCLUSION

Current evidence suggests that learning outcomes are predicted by previous academic achievement and aptitude tests. The predictive value of SJT and topics such as selection algorithms, features of interview (e.g., content of the questions) and the way the interviewers’ reports are used, warrant further research.

Evidence-Based Practices for Interviewing Graduate Medical Education Applicants: A Systematic Review

Background Although the selection interview is a standard admission practice for graduate medical education (GME) programs in the United States, there is a dearth of recent reviews on optimizing the trainee interview process, which has low reliability, high cost, and major risk of bias. Objective To investigate the evidence base for different selection interview practices in GME. Methods We searched 4 literature databases from inception through September 2022. Two investigators independently conducted title/abstract screening, full-text review, data extraction, and quality assessment. Disagreements were mediated by discussion. We used backward reference searching of included articles to identify additional studies. We included studies of different interview methods and excluded literature reviews, non-GME related publications, and studies comparing different applicant populations. We examined study characteristics, applicant and interviewer preferences, and interview format. We evaluated study quality using the Medical Education Research Study Quality Instrument (MERSQI). Results Of 2192 studies, 39 (2%) met our inclusion criteria. The evidence base was rated as moderately low quality using MERSQI criteria. Applicants reported preferences for several one-on-one interviews lasting 15 to 20 minutes, interviews by current trainees, and interviews including social events with only trainees. Applicants had mixed perceptions of virtual versus in-person interviews and reported that virtual interviews saved costs. The multiple mini interview (MMI) required more applicant and interviewer time than individual interviews but demonstrated construct and predictive validity and was preferred by applicants and interviewers. Conclusions Based on moderately low-quality evidence, using the MMI, training interviewers, and providing applicants with basic program information in advance should be considered for GME selection interviews.

Pulmonary and Critical Care Fellowship Program Directors' Perspective on the Virtual Interview Process

Background

Since the start of the coronavirus disease (COVID-19) pandemic, the residency and fellowship recruitment process has changed significantly with the use of virtual interview (VI) platforms. Pulmonary and critical care medicine (PCCM) candidates reported in a survey that VIs hindered their ability to evaluate their fit within the program. However, the program directors' (PDs') opinion of this process remains unknown.

Objective

We aim to provide insight into the PCCM fellowship PDs' perspective regarding the virtual recruitment process since the first class of fellows undergoing this process has now completed 1 year of training.

Methods

An anonymous survey was sent to the PDs of PCCM programs participating in the National Resident Matching Program match process in 2020 and 2021. The survey consisted of five sections and 26 closed-ended questions and was distributed via email using the SurveyMonkey platform. The survey was conducted for a total of 6 weeks. A follow-up email to nonrespondents was sent every week. The collected responses were divided into two categories: favoring VIs versus not favoring VIs. A multivariable logistic regression analysis was performed to determine the factors associated with favoring VIs.

Results

The survey was sent to 190 email accounts from the Fellowship and Residency Electronic Interactive Database Access System website. Over the course of 6 weeks, 64 respondents participated in the survey, with a response rate of 33.68%. Of 64 respondents, 56 (87.5%) fully completed the survey and 8 (12.5%) partially completed the survey. The final sample size was 59. Thirty-six (61.02%) of the PDs favored VIs for future recruitment, and 23 (38.98%) did not (P < 0.001). Ninety-seven percent of PDs who favored VIs versus 72.73% of those who did not perceived the current fellows to fit well in the program (P = 0.007). The multivariable logistic regression analysis showed a trend toward higher odds of favoring VIs if PDs believed they were able to represent the program well virtually (adjusted odds ratio, 6.99; 95% confidence interval, 0.79 - 61.58) and if they found that the current fellows fit the program well (adjusted odds ratio, 7.15; 95% confidence interval, 0.76-66.52); however, these results were not statistically significant.

Conclusion

In this survey research, we found that the majority of the PCCM fellowship PDs favored a virtual process for future recruitment.

The impact of the medical school admissions interview: a systematic review

Background

Interviews are considered an important part of the medical school admissions process but have been critiqued based on bias and reliability concerns since the 1950s. To determine the impact of the interview, this systematic review investigated the characteristics and outcomes of medical students admitted with and without interviews.

Methods

We searched four literature databases from inception through August 2022; all studies comparing medical students admitted with and without interviews were included. We excluded studies from outside the medical school setting and non-research reports. We reviewed interview type, study design, quality, and outcomes.

Results

Eight studies from five institutions across five countries were included. Six reported no demographic differences between students admitted with and without interviews; one found that more men were admitted without than with semi-structured interviews, and both cohorts had similar academic and clinical performance. Structured interviews admitted students who scored higher on clinical exams and social competence and lower on academic exams. Cohorts admitted with and without structured interviews had similar mental health issues by their final year of medical school.

Discussion

This review suggests that students admitted with and without unstructured and semi-structured interviews were similar demographically, academically, and clinically. Moreover, structured interviews selected more socially competent students who performed better clinically but worse academically. Further research is needed to determine the impact of the selection interview in medical school admissions.

Evaluating non-cognitive skills in medical school applicants

Medical school candidates must have both the cognitive and professional competencies required to become good physicians. In this commentary, we outline the evidence and outcomes associated with the implementation of these selection methodologies and evaluate their ability to assess non-cognitive skills.

Transforming the Medical School Admissions Process: Prioritizing Team-Based Skills and Patient-Centered Values

This article examines the need for reform in medical school admissions to better align with the collaborative and patient-centered nature of contemporary healthcare. Traditional admissions processes prioritize academic excellence, often neglecting essential interpersonal and team-based skills. We advocate for several strategies to address this gap. Firstly, diversifying admissions committees to include healthcare professionals such as nurses and pharmacists, as well as patients can provide insights into candidates' teamwork abilities. Secondly, incorporating group interviews and exercises can better assess applicants' interpersonal skills and collaboration potential. We also propose leveraging the "Voice of the Consumer" (VOC) concept, using patient and family feedback to guide the selection of future physicians. This aligns with the shift toward shared decision-making in patient care. The paper underscores the importance of interprofessional education in addressing communication challenges within healthcare. By integrating these reforms early in medical education, we can cultivate physicians who excel in both evidence-based and patient-centered care. This perspective calls for a holistic evaluation process in medical school admissions, prioritizing patient-centered values and effective teamwork to develop healthcare professionals capable of navigating modern healthcare complexities.

Commentary on robust, defensible, and fair: The AMEE guide to selection into medical school: AMEE Guide No. 153

The AMEE Guide to Selection for medical school is a welcome addition that provides much sound advice and guidance. It employs a comprehensive framework and a number of innovations, international case studies, for instance. There are also some omissions that a future revision could usefully address. The key ones concern the evidence base for assessment of personal attributes by questionnaire or interview; conflation of two separable stages in selection, meeting minimum requirements for suitability, and discriminating between suitable candidates; how best to provide feedback to candidates; and the question of what counts as a fair, equitable approach to selection. Nevertheless, the new AMEE Guide (No 153) is well-aligned with the most recent Ottawa consensus statement on selection, and will make a good contribution to the development or revision of selection systems in medical schools.

Validity evidence for a virtual multiple mini interview at a pharmacy program

BACKGROUND

Numerous health professions schools have transitioned to virtual admissions interviews in recent years. While some research suggests that virtual multiple mini-interviews (vMMIs) are feasible, acceptable, and more affordable, there is a paucity of research concerning the validity of this approach. The purpose of this study was to examine the validity and reliability of vMMIs and explore differences in performance between vMMI and in-person MMIs.

METHODS

Data were collected for two years of in-person MMIs and two years of vMMIs at a pharmacy program/school in the United States. An exploratory factor analysis (principal components analysis) with varimax rotation and Kaiser rule (i.e. retaining factors with eigenvalue > 1.0) was used to explore the construct validity of the vMMI data. Pearson correlation was used to examine correlations between vMMI stations and Cronbach alpha was used to determine the internal consistency of each station. Independent t-tests were used to examine differences between in-person MMI and vMMI scores. Cohen's d was used to determine effect sizes.

RESULTS

Four hundred and thirty-eight (42.69%) candidates completed an in-person MMI and 588 (57.31%) completed a vMMI. Factor analysis indicated that each vMMI station formed a single factor with loads ranging from 0.86 to 0.96. The vMMI stations accounted for most of the total variance, demonstrated weak to negligible intercorrelations, and high internal consistency. Significant differences between in-person and vMMI scores were found for the teamwork-giving, teamwork-receiving, and integrity stations. Medium effect sizes were found for teamwork-giving and teamwork-receiving and a small effect size was found for integrity.

CONCLUSIONS

Initial evidence suggests that the vMMI is a valid and reliable alternative to in-person MMIs. Additional research is needed to examine sources of differences in rating patterns between the two approaches and identify strategies that align with institutional priorities for recruitment and admissions.

"I'd rather see action": Application and recruitment experiences of underrepresented in emergency medicine trainees

Objectives

The emergency medicine (EM) physician workforce is largely composed of white men. Despite recruitment efforts over the past decade, there has not been a significant increase of trainees with underrepresented racial and ethnic identities in EM (URM). Prior studies have focused on institutional strategies to improve diversity, equity, and inclusion (DEI) in EM residency recruitment but have been limited in describing URM trainees' perspectives. We sought to assess URM trainees' perspectives on DEI in the EM residency application and selection process.

Methods

This study was conducted at an urban academic medical center in the United States from November 2021 to March 2022. Junior residents were invited to participate in individual semistructured interviews. We used a combined deductive-inductive approach to categorize responses in predetermined areas of interest then elicit dominant themes within each category through consensus discussions. Thematic saturation was reached after eight interviews, indicating adequate sample size.

Results

Ten residents participated in semistructured interviews. All identified as racial or ethnic minorities. Three dominant themes emerged relating to authenticity, representation, and being treated as a learner first. Participants assessed the authenticity of a program's DEI efforts by evaluating the time frame and scope of DEI efforts. Participants reported a desire for representation of other URM colleagues in a residency program and training environment. While participants wanted their lived experience as URM trainees acknowledged, they were wary of being viewed solely through the lens of future DEI leaders rather than as learners first.

Conclusions

URM residents value multifaceted commitment to DEI efforts, representation, and being seen as learners first when assessing residency programs. Programs seeking to recruit URM residents should develop a department-wide, multipronged, comprehensive DEI plan and showcase how their program will contribute to an applicant's professional development.

Exploring entry pathways towards nurse practitioner program admissions: a rapid review

OBJECTIVES

This rapid review provides an overview of the current landscape of, and the criteria used for admission into nurse practitioner programs.

METHODS

A series of application and admission criteria were collected from nurse practitioner programs in Canada, the United States of America (USA) and Australia. Key descriptive statistics and indicators were analyzed.

RESULTS

Most programs in the USA and Canada required a minimum grade point average (GPA) between 3.00 and 3.24, while Australian programs did not identify minimum GPA requirements. Contrastingly, Australian programs required the highest minimum clinical practice hours. Many North American programs required writing samples, while Australian programs did not.

CONCLUSIONS

Despite role similarity, nurse practitioner admission criteria differ substantially between Canada, the USA and Australia, hindering standardization and integration of the role globally.