The Multiple Mini-Interview (MMI) for student selection in health professions training - a systematic review

Dear admission committee…: Which moves in application essays predict student master grades?

Application essays are a commonly used admission instrument for students entering higher education. The quality of the essay is usually scored, but this score is often subjective and has poor interrater reliability due to the unstructured format of the essays. This results in mixed findings on the validity of application essays as an admission instrument. We propose a more objective method of using application essays, using Latent Dirichlet Allocation (LDA), a text mining method, to distinguish seven moves occurring in application essays written by students who apply to a master degree program. We use the probability that these moves occur in the essay to predict study success in the master. Thereby we answer the following research question: What is the effect of discussing different moves in students' application essays on the student grades in a master program? From the seven different moves (functional unit of text) we distinguished, five of which have a significant effect on student grades. The moves we labeled as 'master specific' and 'interest to learn' have a positive effect on student grades, and the moves we labeled as 'research skills', 'societal impact' and 'city and university' have a negative effect. Our interpretation of this finding is that topics related to intrinsic motivation and specific knowledge, as opposed to generic knowledge, are positively related with study success. We thereby demonstrate that application essays can be a valid predictor of study success. This contributes to justifying their use as admission instruments.

Investigating the internal structure of multiple mini interviews-A perspective from Pakistan

BACKGROUND

Healthcare professionals require many personal attributes in addition to cognitive abilities and psychomotor skills for competent practice. Multiple Mini- Interviews are being employed globally to assess personality attributes of candidates for selection in health professions education at all level of entry; these attributes are namely, communication skills, critical thinking, honesty, responsibility, health advocacy, empathy and sanctity of life. Considering the high stakes involved for students, faculty, institutions and the society, rigorous quality assurance mechanisms similar to those used for student assessment must be employed for student selection, throughout the continuum of medical education. It is a difficult undertaking as these psychological constructs are difficult to define and measure. Though considered to yield reliable and valid scores, studies providing multiple evidences of internal structure especially dimensionality of Multiple Mini-Interviews are sparse giving rise to questions if they are measuring a single or multiple constructs and even if they are measuring what they are purported to be measuring.

OBJECTIVE

The main objective is to provide statistical support of the multi-dimensional nature of our Multiple Mini Interviews, hypothesized a-priori, through CFA. Another objective is to provide multiple evidences for the internal structure. Our study highlights the link between content and internal structure evidences of the constructs, thus establishing that our Multiple Mini Interviews measure what they were intended to measure.

METHOD

After securing permission from the Institutional review board, an a-priori seven factor-model was hypothesized based on the attributes considered most essential for the graduating student of the institution. After operationally defining the attributes through extensive literature search, scenarios were constructed to assess them. A 5-point rating scale was used to rate each item on the station. A total 259 students participated in the multiple mini interviews over a period of three days. A training workshop had been arranged for the participating faculty.

RESULTS

The reliability coefficient using Cronbach's alpha were calculated (range from 0.73 to 0.94), Standard Error of Measurement (ranged from 0.80 to1.64), and item to station-total correlation ranged from 0.43-0.50 to 0.75-0.83. Inter-station correlation was also determined. Confirmatory factor analysis endorsed the results of Exploratory factor analysis in the study revealing a seven model fit with multiple indices of Goodness-of-fit statistics such as Root mean square error of approximation (RMSEA) value 0.05, Standardized root mean square residual (SRMR) value with less than 0.08. All these indices showed that model fit is good. The Confirmatory factor analysis confirmed the multi-dimensional nature of our MMIs and also confirmed that our stations measured the attributes that they were supposed to measure.

CONCLUSION

This study adds to the validity evidence of Multiple Mini-Interviews, in selection of candidates, with required personality traits for healthcare profession. It provides the evidence for the multi-dimensional structure of Multiple Mini interviews administered with multiple evidences for its internal structure and demonstrates the independence of different constructs being measured.

How a Team Effectiveness Approach to Health Systems Science Can Illuminate Undergraduate Medical Education Outcomes

ABSTRACT

Health care delivery requires physicians to operate in teams to successfully navigate complexity in caring for patients and communities. The importance of training physicians early in core concepts of working in teams (i.e., "teaming") has long been established. Over the past decade, however, little evidence of team effectiveness training for medical students has been available. The recent introduction of health systems science as a third pillar of medical education provides an opportunity to teach and prepare students to work in teams and achieve related core competencies across the medical education continuum and health care delivery settings. Although educators and health care system leaders have emphasized the teaching and learning of team-based care, conceptual models and evidence that inform effective teaming within all aspects of undergraduate medical education (including classroom, clinical, and community settings) are needed to advance the science regarding learning and working in teams. Anchoring teaming through the core foundational theory of team effectiveness and its operational components could catalyze the empirical study of medical student teams, uncover modifiable factors that lead to the evidence for improved student learning, and improve the link among competency-based assessments between undergraduate medical education and graduate medical education. In this article, authors articulate several implications for medical schools through 5 conceptual areas: admissions, the design and teaching of team effectiveness in health systems science curricula, the related competency-based assessments, and course and program evaluations. The authors then discuss the relevance of the measurable components and intended outcomes to team effectiveness in undergraduate medical education as critical to successfully prepare students for teaming in clerkships and eventually residency and clinical practice.

Relationship between admission selection tools and student attrition in the early years of medical school

Objectives

Placement in medical schools is highly sought after worldwide with fierce competition among applicants. However, some of the best students withdraw after being accepted to medical school. The aim of this study was to investigate early student attrition within the first 2 years of medical school and determine its relationship to admission selection tools.

Methods

Quantitative research was conducted at the College of Medicine and Health Sciences from 2016 until 2020, during which time routine admission data and students' examination results for the first 2 years were collected and analyzed.

Results

The attrition rate during the study period was 31.7%. High school and college written examination scores were significantly related to completing the premedical program (p = 0.001 and p = 0.002, respectively). Female students scored significantly higher in multiple mini interviews (MMIs) compared with male counterparts (p < 0.001). However, the difference in MMI score was not related to student attrition (p = 0.148).

Conclusion

The cause of early attrition is complex and cannot be attributed to a single factor.Undergraduate high school score and written admission examination results were statistically significant factors in relation to student attrition rate and low academic performance. The results of this study showed that the female students scored significantly higher in the multiple MMI tests compared to male students. However, MMI score alone was not significantly related to student attrition.

How professional and academic pre-qualifications relate to success in medical education: Results of a multicentre study in Germany

OBJECTIVE

Every year, many applicants want to study medicine. Appropriate selection procedures are needed to identify suitable candidates for the demanding curriculum. Although research on medical school admissions has shown good predictive validity for cognitive selection methods (undergraduate GPA, aptitude tests), the literature on applicants with professional and/or academic experience prior to entering medical school remains slim. In our study, we therefore aimed to examine the association between academic success in medical school and having previously completed vocational training in the medical field, voluntary service (≥11 months) or an academic degree.

METHODS

Data were collected in a multicentre, cross-sectional study at five medical schools in Germany (Baden-Wuerttemberg) from students during medical school (i.e. 3rd-, 6th-, and 10th-semester and final-year students). Academic success was assessed according to scores on the first and second state examinations, the total number of examinations repeated and the number of semesters beyond the standard period of study. For the analysis we calculated ordinal logistic regression models for each outcome variable of academic success.

RESULTS

A total of N = 2,370 participants (response rate: RR = 47%) participated in the study. Having completed vocational training was associated with a higher amount of repeated examinations (small effect), while having an academic degree was associated with worse scores on the second state examination (medium effect). No significant association emerged between voluntary service and academic success.

CONCLUSION

The results indicate that professional and academic pre-qualifications pose no advantage for academic success. Possible associations with the financing of study and living conditions of students with pre-qualifications were analysed and discussed in an exploratory manner. However, the operationalisation of academic success from objective and cognitive data should be critically discussed, as the benefits of prior experience may be captured by personal qualities rather than examination results.

The impact of the medical school admissions interview: a systematic review

Background

Interviews are considered an important part of the medical school admissions process but have been critiqued based on bias and reliability concerns since the 1950s. To determine the impact of the interview, this systematic review investigated the characteristics and outcomes of medical students admitted with and without interviews.

Methods

We searched four literature databases from inception through August 2022; all studies comparing medical students admitted with and without interviews were included. We excluded studies from outside the medical school setting and non-research reports. We reviewed interview type, study design, quality, and outcomes.

Results

Eight studies from five institutions across five countries were included. Six reported no demographic differences between students admitted with and without interviews; one found that more men were admitted without than with semi-structured interviews, and both cohorts had similar academic and clinical performance. Structured interviews admitted students who scored higher on clinical exams and social competence and lower on academic exams. Cohorts admitted with and without structured interviews had similar mental health issues by their final year of medical school.

Discussion

This review suggests that students admitted with and without unstructured and semi-structured interviews were similar demographically, academically, and clinically. Moreover, structured interviews selected more socially competent students who performed better clinically but worse academically. Further research is needed to determine the impact of the selection interview in medical school admissions.

Evaluating non-cognitive skills in medical school applicants

Medical school candidates must have both the cognitive and professional competencies required to become good physicians. In this commentary, we outline the evidence and outcomes associated with the implementation of these selection methodologies and evaluate their ability to assess non-cognitive skills.

'Click, I Guess I'm Done': Applicants' and Assessors' Experiences Transitioning to a Virtual Multiple Mini Interview Format

Introduction

During the COVID-19 pandemic, medical schools were forced to suspend in-person interviews and transition to a virtual Multiple Mini Interview (vMMI) format. MMIs typically comprise multiple short assessments overseen by assessors, with the aim of measuring a wide range of non-cognitive competencies. The adaptation to vMMI required medical schools to make swift changes to their MMI structure and delivery. In this paper, we focus on two specific groups greatly impacted by the decision to transition to vMMIs: medical school applicants and MMI assessors.

Methods

We conducted an interpretive qualitative study to explore medical school applicants' and assessors' experiences transitioning to an asynchronous vMMI format. Ten assessors and five medical students from one Canadian medical school participated in semi-structured interviews. Data was analyzed using a thematic analysis framework.

Results

Both applicants and assessors shared a mutual feeling of longing and nostalgia for an interview experience that, due to the pandemic, was understandably adapted. The most obvious forms of loss experienced - albeit in different ways - were: 1) human connection and 2) missed opportunity. Applicants and assessors described several factors that amplified their grief/loss response. These were: 1) resource availability, 2) technological concerns, and 3) the virtual interview environment.

Discussion

While virtual interviewing has obvious advantages, we cannot overlook that asynchronous vMMIs do not lend themselves to the same caliber of interaction and camaraderie as experienced in in-person interviews. We outline several recommendations medical schools can implement to enhance the vMMI experience for applicants and assessors.

Has a national recruitment scheme created a positive intervention for Black, Asian or other Minority Ethnic pharmacy trainees?

OBJECTIVES

A National Recruitment Scheme (NRS) for Pharmacy trainees was introduced in England and Wales in 2017, standardising recruitment processes on behalf of employers and with the aim of reducing bias for candidates applying to training posts within the National Health Service (NHS). This research attempted to identify whether the introduction of the NRS had an impact on the recruitment of Black, Asian, or other Minority Ethnic applicants into the most sought-after posts within the Scheme (hospital posts).

METHODS

An observational study was undertaken. Anonymised pharmacist trainee recruitment data between the cohort intakes of 2015-16 and 2020-21 was obtained from the pharmacy regulator the General Pharmaceutical Council and a comparison of proportional representations of ethnicities was undertaken, to ascertain whether a greater proportion of applicants from minority backgrounds attained the most sought-after posts in the NHS after the NRS was introduced. A robust generalised linear model was then used to analyse the data using binomial as the variance function and logit as a link function, where the proportion of hospital recruitment was an outcome with a two-way interaction between intervention and ethnicity after adjusting for overall proportion.

KEY FINDINGS

The statistical analysis of 18 283 pharmacy trainees in total, of whom 4446 were in hospital, shows a significant overall impact of intervention, with a significant positive change in the proportions of Asian-Pakistani applicants (P-value < 0.001) and Black-African applicants (P-value < 0.001) recruited to hospital posts.

CONCLUSIONS

Since the introduction of the NRS there has been a statistically significant impact on the correlation between the overall number of Black, Asian or other Minority Ethnic applicants and their proportion in hospital. That is, not only is the makeup of the hospital cohort increasingly reflecting the diversity of the overall cohort, but also a larger percentage of each ethnic cohort is attaining hospital training places.

Front row seat: The role MMI assessors play in widening access to medical school

BACKGROUND

While many medical schools utilize the Multiple Mini-Interview (MMI) to help select a diverse student body, we know little about MMI assessors' roles. Do MMI assessors carry unique insights on widening access (WA) to medical school? Herein we discuss the hidden expertise and insights that assessors contribute to the conversation around WA.

METHODS

Ten MMI assessors (1-10 years' experience) participated in semi-structured interviews exploring factors influencing equitable medical school recruitment. Given their thoughtfulness during initial interviews, we invited them for follow-up interviews to gain further insight into their perceived role in WA. Fourteen interviews were conducted and analyzed using a thematic analysis approach.

RESULTS

Assessors expressed concerns with diversity in medicine; dissatisfaction with the status quo fueled their contributions to the selection process. Assessors advocated for greater diversity among the assessor pool, citing benefits for all students, not only those from underrepresented groups. They noted that good intentions were not enough and that medical schools can do more to include underrepresented groups' perspectives in the admissions process.

CONCLUSION

Our analysis reveals that MMI assessors are committed to WA and make thoughtful contributions to the selection process. A medical school selection process, inclusive of assessors' expertise is an important step in WA.

Effects of candidates' demographics and evaluation of the virtual Multiple Mini Interview (vMMI) as a tool for selection into paediatric training in Queensland

INTRODUCTION

The Queensland Basic Paediatric Training Network (QBPTN) is responsible for the selection of candidates into paediatric training in Queensland. The COVID-19 pandemic necessitated interviews to be conducted 'virtually' as virtual Multiple-Mini-Interviews (vMMI). The study aimed to describe the demographic characteristics of candidates applying for selection into paediatric training in Queensland, and to explore their perspectives and experiences with the vMMI selection tool.

METHODOLOGY

The demographic characteristics of candidates and their vMMI outcomes were collected and analysed with a mixed methods approach. The qualitative component was comprised of seven semi-structured interviews with consenting candidates.

RESULTS

Seventy-one shortlisted candidates took part in vMMI and 41 were offered training positions. The demographic characteristics of candidates at various stages of selection were similar. The mean vMMI scores were not statistically different between candidates from the Modified Monash Model 1 (MMM1) location and others [mean (SD): 43.5 (5.1) versus 41.7 (6.7), respectively, p = 0.26]. However, there was a statistically significant difference (p value 0.03) between being offered and not offered a training position for candidates from MMM2 and above. The analysis of the semi-structured interviews suggested that candidate experiences of the vMMI were influenced by the quality of the management of the technology used. Flexibility, convenience, and reduced stress were the main factors that influenced candidates' acceptance of vMMI. Perceptions of the vMMI process focused on the need to build rapport and facilitate communication with the interviewers.

DISCUSSION

vMMI is a viable alternative to face-to-face (FTF) MMI. The vMMI experience can be improved by facilitating enhanced interviewer training, by making provision for adequate candidate preparation and by having contingency plans in place for unexpected technical challenges. Given government priorities in Australia, the impact of candidates' geographical location on the vMMI outcome for candidates from MMM >1 location needs to be further explored.

Validity evidence for a virtual multiple mini interview at a pharmacy program

BACKGROUND

Numerous health professions schools have transitioned to virtual admissions interviews in recent years. While some research suggests that virtual multiple mini-interviews (vMMIs) are feasible, acceptable, and more affordable, there is a paucity of research concerning the validity of this approach. The purpose of this study was to examine the validity and reliability of vMMIs and explore differences in performance between vMMI and in-person MMIs.

METHODS

Data were collected for two years of in-person MMIs and two years of vMMIs at a pharmacy program/school in the United States. An exploratory factor analysis (principal components analysis) with varimax rotation and Kaiser rule (i.e. retaining factors with eigenvalue > 1.0) was used to explore the construct validity of the vMMI data. Pearson correlation was used to examine correlations between vMMI stations and Cronbach alpha was used to determine the internal consistency of each station. Independent t-tests were used to examine differences between in-person MMI and vMMI scores. Cohen's d was used to determine effect sizes.

RESULTS

Four hundred and thirty-eight (42.69%) candidates completed an in-person MMI and 588 (57.31%) completed a vMMI. Factor analysis indicated that each vMMI station formed a single factor with loads ranging from 0.86 to 0.96. The vMMI stations accounted for most of the total variance, demonstrated weak to negligible intercorrelations, and high internal consistency. Significant differences between in-person and vMMI scores were found for the teamwork-giving, teamwork-receiving, and integrity stations. Medium effect sizes were found for teamwork-giving and teamwork-receiving and a small effect size was found for integrity.

CONCLUSIONS

Initial evidence suggests that the vMMI is a valid and reliable alternative to in-person MMIs. Additional research is needed to examine sources of differences in rating patterns between the two approaches and identify strategies that align with institutional priorities for recruitment and admissions.

Health professions school applicant experiences of discrimination during interviews

BACKGROUND

Bias pervades every aspect of healthcare including admissions, perpetuating the lack of diversity in the healthcare workforce. Admissions interviews may be a time when applicants to health profession education programs experience discrimination.

METHODS

Between January and June 2021 we invited US and Canadian applicants to health profession education programs to complete a survey including the Everyday Discrimination Scale, adapted to ascertain experiences of discrimination during admissions interviews. We used chi-square tests and multivariable logistic regression to determine associations between identity factors and positive responses.

RESULTS

Of 1115 respondents, 281 (25.2%) reported discrimination in the interview process. Individuals with lower socioeconomic status (OR: 1.78, 95% CI [1.26, 2.52], p = 0.001) and non-native English speakers (OR: 1.76, 95% CI [1.08, 2.87], p = 0.02) were significantly more likely to experience discrimination. Half of those experiencing discrimination (139, or 49.6%) did nothing in response, though 44 (15.7%) reported the incident anonymously and 10 (3.6%) reported directly to the institution where it happened.

CONCLUSIONS

Reports of discrimination are common among HPE applicants. Reforms at the interviewer- (e.g. avoiding questions about family planning) and institution-level (e.g. presenting institutional efforts to promote health equity) are needed to decrease the incidence and mitigate the impact of such events.

Interviewer bias when using multiple mini-interviews in selecting student nurses in a Chinese setting

BACKGROUND

Interviewer effects may cause unfairness in assessments in multiple mini-interviews (MMIs). Due to cultural differences, the bias factors of interviewers may vary between the East and the West. MMIs are a relatively new type of assessment setting in China and few studies have been conducted to examine the interviewer effects of MMIs in this context.

OBJECTIVES

We adopted a multi-faceted Rasch measurement (MFRM) to measure interviewer effects in assessments in Hong Kong.

METHODS

Data were collected from a nursing school in Hong Kong. There were 431 candidates and 12 interviewers engaged in a six-station MMI setting. The scores collected from the interviews were analyzed in terms of 1) interviewer stringency/leniency, 2) candidate gender, 3) interview time, and 4) rating category in the station. The Student's t-statistic values were calculated to investigate the marking tendencies of individual interviewers.

RESULTS

The research findings suggest that interviewers differ in their degree of stringency/leniency, but the number of candidates examined by each interviewer does not affect interviewer stringency/leniency in terms of the interviewer's assessment. There is not sufficient evidence indicating that candidate gender and interview time are bias factors affecting assessment score in this study. Among the six rating categories examined, honesty/integrity is the most stringent category, while self-awareness is the most lenient category. Interview bias from individuals was identified. When we consider the interview scores given by individual interviewers, it is evident that some interviewers may have been biased toward a certain gender or rating categories.

CONCLUSIONS

MMIs are useful when selecting student nurses in a Chinese setting. However, interviewer bias may exist. We used an MFRM to better understand interviewer bias across various dimensions. The present study contributes to the development and use of MMIs in non-Western countries and can be used as a reference to extend this research to other locations.

Can Different Admissions to Medical School Predict Performance of Non-Technical Skill Performance in Simulated Clinical Settings?

Non-technical skills (NTS) in medical care are essential to ensure patient safety. Focussing on applicants' NTS during medical school admission could be a promising approach to ensure that future physicians master NTS at a high level. Next to pre-university educational attainment, many selection tests have been developed worldwide to facilitate and standardise the selection process of medical students. The predictive validity of these tests regarding NTS performance in clinical settings has not been investigated (yet). Therefore, we explored the predictive validities and prognosis of the Hamburg MMI (HAM-Int), HAM-Nat, PEA, and waiting as well as other quota (as example) designated by the Federal Armed Forces) for NTS performance in clinical emergency medicine training of medical students. During 2017 and 2020, N = 729 second, third, and fourth year students were enrolled within the study. The mean age of participants was 26.68 years (SD 3.96) and 49% were female students. NTS of these students were assessed during simulation scenarios of emergency training with a validated rating tool. Students admitted via waiting quota and designated by the Armed Forces performed significantly better than students admitted by excellent PEA (p = 0.026). Non-EU students performed significantly inferior (p = 0.003). Our findings provide further insight to explain how and if admission to medical school could predict NTS performance of further physicians.

A Semistructured Interview for Surgical Residency Targeting Nontechnical Skills

OBJECTIVE

We review the development, implementation, and initial outcomes of a semistructured interview process to assess the nontechnical skills of surgical residency applicants.

DESIGN

In 2018, we restructured our residency selection interview process. Through semistructured faculty interviews, we sought to evaluate candidates along seven nontechnical skills (grit, ownership, rigor, teamwork, presence, impact, and organizational alignment). We plotted each candidate's scores on a radar plot for graphical representation and calculated the plot area of each candidate. We retrospectively evaluated 3 years of data, comparing the nontechnical skill scores of matriculants into the training program to those of nonmatriculants.

SETTING

Tenwek Hospital is a 361-bed tertiary teaching and referral hospital in rural western Kenya with a 5-year general surgery residency program.

PARTICIPANTS

Thirty-one applicants were interviewed over 3 years. Thirteen matriculated into the program.

RESULTS

Scores for grit, (4.8 vs 3.9; p = 0.0004), impact (4.2 vs 3.5; p = 0.014), ownership (4.2 vs 3.6; p = 0.01), and organizational alignment (4.3 vs 3.8; p = 0.008) were significantly higher in matriculants.

CONCLUSIONS

This semistructured interview process provides a robust and beneficial mechanism for assessing applicants' nontechnical skills, which may allow for the matriculation of more well-rounded candidates into surgical residency and, ultimately, surgical practice.

Best practices for interviewing applicants for medical school admissions: a systematic review

INTRODUCTION

Interviews are commonly used to select applicants for medical school, residency, and fellowship. However, interview techniques vary in acceptability, feasibility, reliability, and validity. This systematic review investigated the effectiveness of different interview methods in selecting the best qualified applicants for admission to medical school and developed a logic model to implement best practices for interviewing.

METHODS

Five electronic literature databases were searched for comparative studies related to interviewing in medical schools from inception through February 1, 2021. Inclusion criteria included publications in English that compared different methods of conducting a selection interview in medical schools with a controlled trial design. General study characteristics, measurement methodologies, and outcomes were reviewed. Quality appraisal was performed using the Medical Education Research Study Quality Instrument (MERSQI) and the Oxford Risk of Bias Scale. Based on these findings, a logic model was constructed using content analysis.

RESULTS

Thirteen studies were included. The multiple mini-interview (MMI) was reliable, unbiased, and predicted clinical and academic performance; the virtual MMI increased reliability and lowered costs. For unstructured interviews, blinding interviewers to academic scores reduced bias towards higher scorers; student and faculty interviewers rated applicants similarly. Applicants preferred structured over unstructured interviews. Study quality was above average per the MERSQI, risk of bias was high per the Oxford scale, and between-study heterogeneity was substantial.

DISCUSSION

There were few high-quality studies on interviewing applicants for admission to medical school; the MMI appears to offer a reliable method of interviewing. A logic model can provide a conceptual framework for conducting evidence-based admissions interviews.

Effect of station format on the psychometric properties of Multiple Mini Interviews

BACKGROUND

Given the widespread use of Multiple Mini Interviews (MMIs), their impact on the selection of candidates and the considerable resources invested in preparing and administering them, it is essential to ensure their quality. Given the variety of station formats used and the degree to which that factor resides in the control of training programmes that we know so little about, format's effect on MMI quality is a considerable oversight. This study assessed the effect of two popular station formats (interview vs. role-play) on the psychometric properties of MMIs.

METHODS

We analysed candidate data from the first 8 years of the Integrated French MMIs (IF-MMI) (2010-2017, n = 11 761 applicants), an MMI organised yearly by three francophone universities and administered at four testing sites located in two Canadian provinces. There were 84 role-play and 96 interview stations administered, totalling 180 stations. Mixed design analyses of variance (ANOVAs) were used to test the effect of station format on candidates' scores and stations' discrimination. Cronbach's alpha coefficients for interview and role-play stations were also compared. Predictive validity of both station formats was estimated with a mixed multiple linear regression model testing the relation between interview and role-play scores with average clerkship performance for those who gained entry to medical school (n = 462).

RESULTS

Role-play stations (M = 20.67, standard deviation [SD] = 3.38) had a slightly lower mean score than interview stations (M = 21.36, SD = 3.08), p < 0.01, Cohen's d = 0.2. The correlation between role-play and interview stations scores was r = 0.5 (p < 0.01). Discrimination coefficients, Cronbach's alpha and predictive validity statistics did not vary by station format.

CONCLUSION

Interview and role-play stations have comparable psychometric properties, suggesting format to be interchangeable. Programmes should select station format based on match to the personal qualities for which they are trying to select.

Investigating the Relationship between Multiple Mini-Interview Communication Skills Outcomes and First-Year Communication Skills Performance and Reflections in Students at the Ontario Veterinary College

An important outcome for veterinary education is ensuring that graduates can provide an appropriate level of care to patients and clients by demonstrating core competencies such as communication skills. In addition, accreditation requirements dictate the need to assess learning outcomes and may drive the motivation to incorporate relevant and appropriate methods of entry assessments for incoming students. Predicting the success of Doctor of Veterinary Medicine (DVM) students based on entry assessment performance has been scantly investigated and can be challenging. Specifically, no research presently exists on predicting DVM students' first-year performance in relation to communication skills at the time of program entry. Objectives of this exploratory study were to investigate (a) the relationship between communication skills outcomes from multiple mini-interview (MMI) data and first-year academic performance related to communication and (b) the relationship between communication skills outcomes from MMI data and self-reported first-year communication reflections. A retrospective single-class study was conducted. Data were analyzed using descriptive statistics, correlation statistics, regression models, and paired t-tests to identify relationships among variables. Paired t-tests showed that students felt more prepared to meet second-year expectations over first-year expectations. Spearman's correlation revealed an association between MMI communication scores and one pre-year 1 survey question related to professionalism. Noo relationships were observed between MMI communication scores and marks from a self-reflection assignment in a communications course, or grades from a clinical medicine course that included clinical communication. The merit for further exploration of the relationship between communication competencies and student performance is discussed.

Recruitment and retention of volunteer multiple mini interview interviewers: Understanding their motivations

BACKGROUND

Without volunteer interviewers, many universities would not be able to run multiple mini interviews (MMIs) due to the prohibitive cost of paying interviewers. Despite the opportunity cost borne by volunteers, many interviewers participate in multiple MMI sessions per year and volunteer year after year. There is surprisingly little research into what motivates interviewers to volunteer as MMI interviewers. This research aims to explore both what motivates individuals to volunteer to interview in MMIs for undergraduate medical selection and what adds and detracts value from their participation.

METHODS

We applied a qualitative sequential two-phase design consisting of open-ended survey questions, followed by semi-structured interviews to explore interviewers' motivators in more depth. The survey data on motivators and the six functions from the volunteer functions inventory (VFI) informed interview data collection and provided a lens through which to examine MMI interviewer motivations. Content analysis was used to analyse the survey data. Framework analysis was used to analyse the interview data.

RESULTS

The survey was completed by 108 interviewers (50% response rate), and 19 semi-structured interviews were conducted (54% response rate). From the content analysis, the time commitment of involvement was the biggest detractor identified by participants. Through the framework analysis, five overarching motivators were developed: (i) acting on values, (ii) gaining understanding, (iii) gaining personal satisfaction and gratification, (iv) shaping the future workforce and (v) having social interaction. These mirrored five of the six functions proposed in the VFI.

CONCLUSIONS

There are a range of motivating factors that influenced the participants' decision to volunteer as an interviewer for MMIs. Some motivations were for the benefit of others, some were self-serving, and some a combination of both. Universities should utilise these motivating factors to aid in targeted recruitment of volunteer interviewers.

The current dental school applicant: an overview of the admission process for UK dental schools and the sociodemographic status of applicants

Aims To determine the current processes used to assess dental school admissions in the UK as well as compare the applicants' demography.

Methods All 16 dental schools in the UK were invited to complete a questionnaire analysing the admissions protocols between 2018-2019 and 2019-2020. These data were combined with the admission process information available online. Retrospective data from the University Clinical Aptitude Test including the sociodemographic status of dental applicants were collected. For the six dental schools that did not reply, data presented were collected from information available to the public.

Results The majority of applicants were women (63.2% in 2019-2020), white (27.9%) and are sixth form attendees or attend a further education college (40.6%). Of those who apply to study dentistry, 15.5% are graduates who hold a first degree. For each undergraduate dental place available, there were 12.6 applicants and 9.4 applicants to each post-qualified dental undergraduate place.

Conclusion Further advancements are required to widen participation and broaden the sociodemographic status of dental applicants. This area would benefit from a long-term prospective study about recruitment methods and its correlation with performance at dental school. COVID-19 is impacting the application process, the full extent of which is yet to be determined.

Provides an overview of the current admission processes for dental schools within the UK.

Encourages critical analysis of the admission processes used for undergraduate and postgraduate dental students.

Explores how the admission processes may change as a result of the COVID-19 pandemic and gives an indication as to the sociodemographics of the future dental career force.

Implementation and evaluation of virtual multiple mini interviews as a selection tool for entry into paediatric postgraduate training: A Queensland experience

INTRODUCTION

The Queensland Basic Paediatric Training Network (QBPTN) is the centralised pathway for entry into paediatric training in Queensland, Australia. In response to COVID-19 travel and social distancing restrictions imposed in 2020, QBPTN successfully adopted a Virtual Multiple Mini Interviews (vMMIs) model for the selection of candidates for entry into paediatric training. The authors describe the planning, implementation, challenges, and evaluation of candidates' and interviewers' experiences of vMMIs, including the differences between candidates from two geographical areas.

METHODS

The contents of six vMMI stations were similar to face-to-face MMI. Implementation required the identification of ZOOM^TM as a preferred online platform, securing venues, communication, development of contingency plans and central coordination by the network. Candidates' experiences with vMMI were explored through thematic analysis of the qualitative data from focus groups and free text responses, and descriptive analysis of SurveyMonkey© questionnaire responses. Experiences between 'metropolitan' and 'regional and interstate' candidates were compared.

RESULTS

5-minute stations with 2-minute pre-reading were used. 78 candidates and 14 interviewers participated in the selection process. All candidates attended the focus group. 58.7% of candidates responded to post vMMI questionnaire. 93% of survey responders were happy to undertake vMMI in the future, with 23% feeling they would have performed better in face-to-face. Experiences between 'metropolitan' and 'other' groups were similar. Positive experiences of participants were related to the user-friendly IT platform, successful pre-interview communications, preparation, convenience, time, and cost savings. Stress related IT failures and difficulties establishing rapport with interviewers were reported as the main negative experiences.

CONCLUSION

'vMMI' is a feasible and acceptable method of selection into paediatric training. vMMI has many benefits and can be implemented relatively quickly by addressing key logistical requirements. The model under discussion could be adapted by other centres based on local needs.

How rural immersion training influences rural work orientation of medical students: Theory building through realist evaluation

AIM

To develop theory about how contexts and mechanisms interact to contribute to openness to future rural practice by medical students undertaking immersive rural training.

METHODS

A realist evaluation based on RAMESES II protocol. We interviewed 23 students exploring Contexts (C) which were external (place-based) and internal (the student's characteristics), Mechanisms (M) (that drive a response) and Outcomes (O) (openness to rural work).

RESULTS

'Openness to rural work' related to: a desire to live rurally, work in rural medicine, or consider this as a possibility. This was triggered by responses to experience in rural places of an aspirational, intellectual and emotional nature (mechanisms). Students most affected were those with a strong motivation to help others and who value teamwork. Students with clearly envisaged career paths suited to metropolitan areas, or those retaining/prioritising strong social and community ties in metropolitan areas were less likely to commit to future rural work.

CONCLUSION

Our theory indicates multi-level stimuli activates openness. Implications are that rural immersion programs could select students with an orientation towards teamwork, without pre-set professional ideation, and with a strong commitment to helping others. Experiencing rural immersion will trigger aspirational, intellectual and emotional responses leading to rural work openness for such students.

An investigation of the predictive validity of selection tools on performance in physiotherapy training in Australia

OBJECTIVES

Despite a large body of research on selection in medical education, very little is conducted in other health professions. This study investigated the predictive validity of multiple selection tools on academic and clinical performance outcomes of undergraduate physiotherapy students.

DESIGN

A retrospective observational study.

SETTING

Undergraduate physiotherapy program in Australia.

PARTICIPANTS

497 undergraduate physiotherapy students across seven entry cohorts. Including students directly from secondary school (n=381) and with prior tertiary study (n=116).

MAIN OUTCOME MEASURES

Academic performance as measured by written examinations. Clinical performance, measured by Objective Structured Clinical Examinations (OSCEs) during on-campus units and the Assessment of Physiotherapy Practice (APP) for off-campus clinical placements. Predictor variables included selection tools (academic achievement, interview, aptitude test) and demographic variables (age, gender).

RESULTS

Selection interview was a positive predictor of OSCEs and final year clinical performance in direct school leaver participants. Academic achievement scores from selection positively predicted written examinations scores.

CONCLUSION

Clinical and academic performance were predicted by tools measuring different domains at selection. Assessing broadly across academic and non-academic domains at selection can be valuable in identifying applicants who will be able to meet the range of outcomes for course completion and subsequent registration in the physiotherapy profession.

Difference in demographics and motivation to study medicine with respect to medical students' channel of admission: A national study

INTRODUCTION

Medical schools employ various tools to select suitable medical students (MS). This study investigated whether MS who were admitted through multiple mini-interviews (MMI) and MS who were admitted through Taiwan's Joint College Entrance Written Test (JCEWT) differed in their characteristics.

METHODS AND SUBJECTS

First-year MS from seven medical schools completed a semi-structured questionnaire that inquired into their channel of admission (MMI or JCEWT), gender, location (metropolitan or rural), high school type (public or private), parents' socioeconomic status (SES), and motivations to study medicine.

RESULTS

In total, 513 MS participated, 493 (96%) returned valid questionnaires, and 397 were enrolled in the study, (MMI group: 205 MS; JCEWT group: 192 MS). Irrespective of channel of admission, most MS came from metropolitan areas (80%-86%), belonged to high-SES families (73%-76%), and had mixed motivations (51%-96%). Female applicants, private school leavers, and those who were less motivated by the physician's SES were more likely to be selected through the MMI channel than the JCEWT channel.

CONCLUSION

Irrespective of the channels of entry, MS had similar demographics and motivations for studying medicine. MS selected through MMI had different characteristics than those selected through a JCEWT.

A transparent and defensible process for applicant selection within a Canadian emergency medicine residency program

OBJECTIVES

The Canadian Resident Matching Service (CaRMS) selection process has come under scrutiny due to the increasing number of unmatched medical graduates. In response, we outline our residency program's selection process including how we have incorporated best practices and novel techniques.

METHODS

We selected file reviewers and interviewers to mitigate gender bias and increase diversity. Four residents and two attending physicians rated each file using a standardized, cloud-based file review template to allow simultaneous rating. We interviewed applicants using four standardized stations with two or three interviewers per station. We used heat maps to review rating discrepancies and eliminated rating variance using Z-scores. The number of person-hours that we required to conduct our selection process was quantified and the process outcomes were described statistically and graphically.

RESULTS

We received between 75 and 90 CaRMS applications during each application cycle between 2017 and 2019. Our overall process required 320 person-hours annually, excluding attendance at the social events and administrative assistant duties. Our preliminary interview and rank lists were developed using weighted Z-scores and modified through an organized discussion informed by heat mapped data. The difference between the Z-scores of applicants surrounding the interview invitation threshold was 0.18-0.3 standard deviations. Interview performance significantly impacted the final rank list.

CONCLUSIONS

We describe a rigorous resident selection process for our emergency medicine training program which incorporated simultaneous cloud-based rating, Z-scores, and heat maps. This standardized approach could inform other programs looking to adopt a rigorous selection process while providing applicants guidance and reassurance of a fair assessment.

Implementation of Virtual Multiple Mini-Interviews for Fellowship Recruitment

Introduction

The SARS-CoV Disease (COVID-19) pandemic has upended health care systems and one of the casualties has been the trainee recruitment process since social distancing and travel restrictions make an in-person experience improbable. At the University of California, Davis (UCD), our Pain Division transitioned our internally validated multiple mini-interview (MMI) process to a virtual environment

Methods

Applicants signed a confidentiality agreement prior to their interviews and were invited to watch a series of videos orienting them to the process and to the program itself. All faculty raters interviewed candidates using a total of 6 non-medical MMI scenarios with corresponding questions and scoring rubrics through the Zoom platform. Applicants were then welcomed to voluntary informal conversations with the current fellow trainees and faculty. An optional survey was sent to the applicants post-interview to assess their overall satisfaction with the virtual process.

Results

The survey analyzed the following using a 5 point Likert scale: Overall Satisfaction, Video Overview Program, Interview Day Details, Video Tour, Web-Based Interviews, Process was Fair, and finally a question on Informed Decision regarding receiving sufficient information to formulate their rank list. All respondents (80% response rate) reported being either satisfied or very satisfied with each of the aspects of the interview process detailed above.

Conclusions

While technical difficulties and confidentiality issues are of concern when offering an entirely web based recruitment, our group was able to transition traditional in-person MMI to a virtual platform using a similar structure which was well-received by applicants

Beyond a Hashtag: Strategies to Move Toward a More Inclusive Physiatry Workforce

ABSTRACT

The field of physical medicine and rehabilitation should strive for a physician workforce that is ethnically/racially, sex, and ability diverse. Considering the recent realities of disparities in health outcomes related to COVID-19 and in racial injustice in the United States, we are called to be champions for antiracism and equity. The specialty of physical medicine and rehabilitation should be the leaders in fostering a culture of inclusion and pay special attention to the population of applicants who are underrepresented in medicine. The specialty needs tools to start addressing these disparities. This article aims to provide strategic and intentional evidence-based recommendations for programs to follow. Holistic review, implicit bias training, structured interviews, and targeted outreach for those underrepresented in medicine are some of the tools that will help students enter and become successful in our specialty. Furthermore, this article provides novel guidance and considerations for virtual interviews during the COVID-19 pandemic.

Online interviews for the selection of applicants for admission into an entry to practice program in pharmacy: Relationship to performance and student perspectives

INTRODUCTION

Admissions processes for entry-to-practice pharmacy programs across North America vary in the types of information used to decide which students to admit. The use of online interviews emerged at our program as an option to assess basic communication skills and personal traits to supplement traditional achievement measures such as prerequisite course grade point average (GPA) and admissions letters.

METHODS

Student cumulative grade, year-by-year grade, and some grades from selected classes were correlated with the interview score for three years of consecutive cohorts. Linear regression was used to explore relationships. A survey was completed by students to understand their views on the use of the online interview process.

RESULTS

There was no relationship seen between the compiled GPA and the interview score. The survey data revealed a strong preference for online compared to face-to-face interviews, with high student agreement that the interviews provide a good opportunity to demonstrate general communication skills. An assessment of the strength of the relationships of interview scores with a set of program GPA outcomes showed limited predictive utility. Scores weakly correlated with performance in communications and skills courses.

CONCLUSIONS

Online interviews were preferred over in-person interviews by most students. The lack of strong significant correlations between interview scores and grades suggests that use be judiciously applied in the overall admissions decision-making process.

Student perspectives of preparedness characteristics for clinical learning within a fully distributed veterinary teaching model

The transition into the clinical environment is challenging and associated with significant stress and anxiety. This study aimed to examine the perspectives of students on the characteristics important for preparedness for clinical learning and the influence of gender, age, and graduate status on those perspectives. This descriptive, questionnaire-based study of 62 characteristics categorised into six themes was conducted within the Surrey School of Veterinary Medicine completed by 139 students commencing their final clinical year. The Friedman test and post-hoc Wilcoxon signed rank sum test explored for differences in ranking across the themes. Ordinal logistic regression and Mann-Whitney U pairwise comparisons were utilised to investigate for effects of gender, age, and graduate status on theme ranking. There was a significant difference (P <0.05) between medians for themes of preparedness characteristics with comparisons revealing willingness and communication and interaction as the most highly rated characteristics. Knowledge and understanding were viewed as the least important characteristic. Regression and pairwise Mann-Whitney U comparisons confirmed no significant effects (P >0.05) of gender, age or graduate status on student rating of preparedness characteristics. Integrating learning opportunities of those preparedness characteristics in the pre-clinical curriculum may improve students' preparedness for the clinical environment.

An exploration of the predictive validity of selection criteria on progress outcomes for pre-registration nursing programmes-A systematic review

AIM

To identify the selection methods currently being used for pre-registration nursing programmes and to assess the predictive power that these methods have on students' success.

BACKGROUND

Research into selection methods in nursing education is beginning to emerge, yet it is unclear which methods are most predictive of students' success.

DESIGN

A systematic review of the literature.

METHODS

A systematic search of ten electronic databases: CINAHL, MEDLINE Ovid, EMBASE, PROQUEST Health and Medical, PROQUEST Education, COCHRANE Library, Web of Science, ASSIA, SCOPUS and PROSPERO was conducted. The results were expanded by the handsearching of journals, reference lists and grey literature. The PRISMA statement guided the review. Studies published in English between January 2008-March 2020 were eligible for inclusion, and quality assessment was undertaken using the CASP Checklist for Cohort Studies.

RESULTS

Twenty-five studies met the criteria for inclusion. A range of selection methods was identified including prior academic achievement, admissions tests, interviews, emotional intelligence tests, personal statements and previous healthcare experience. Prior academic achievement and admissions tests appear to be the selection methods most predictive of student success. The evidence surrounding other selection methods such as interviews and personal statements is less conclusive.

CONCLUSION

Selecting individuals with the appropriate knowledge, interpersonal skills and personal qualities needed to complete an undergraduate nursing programme is an important part of the role of nurse educators. This review shows that a wide variety of selection methods are used across different institutions, some of which are more effective than others in predicting student outcomes.

RELEVANCE TO CLINICAL PRACTICE

Further research is required to justify the continued use of some commonly used selection methods for undergraduate nursing programmes. Selection models that combine various types of selection criteria with predictive power appear to increase the probability of selecting students that will have successful outcomes.

Accuracy of rating scale interval values used in multiple mini-interviews: a mixed methods study

When determining the score given to candidates in multiple mini-interview (MMI) stations, raters have to translate a narrative judgment to an ordinal rating scale. When adding individual scores to calculate final ranking, it is generally presumed that the values of possible scores on the evaluation grid are separated by constant intervals, following a linear function, although this assumption is seldom validated with raters themselves. Inaccurate interval values could lead to systemic bias that could potentially distort candidates' final cumulative scores. The aim of this study was to establish rating scale values based on rater's intent, to validate these with an independent quantitative method, to explore their impact on final score, and to appraise their meaning according to experienced MMI interviewers. A 4-round consensus-group exercise was independently conducted with 42 MMI interviewers who were asked to determine relative values for the 6-point rating scale (from A to F) used in the Canadian integrated French MMI (IFMMI). In parallel, relative values were also calculated for each option of the scale by comparing the average scores concurrently given to the same individual in other stations every time that option was selected during three consecutive IFMMI years. Data from the same three cohorts was used to simulate the impact of using new score values on final rankings. Comments from the consensus group exercise were reviewed independently by two authors to explore raters' rationale for choosing specific values. Relative to the maximum (A = 100%) and minimum (F = 0%), experienced raters concluded to values of 86.7% (95% CI 86.3-87.1), 69.5% (68.9-70.1), 51.2% (50.6-51.8), and 29.3% (28.1-30.5), for scores of B, C, D and E respectively. The concurrent score approach was based on 43,412 IFMMI stations performed by 4345 medical school applicants. It provided quasi-identical values of 87.1% (82.4-91.5), 70.4% (66.1-74.7), 51.2% (47.1-55.3) and 31.8% (27.9-35.7), respectively. Qualitative analysis explained that while high scores are usually based on minor details of relatively low importance, low scores are usually attributed for more serious offenses and were assumed by the raters to carry more weight in the final score. Individual drop or increase in final MMI ranking with the use of new scale values ranged from - 21 to + 5 percentiles, with the average candidate changing by ± 1.4 percentiles. Consulting with experienced interviewers is a simple and effective approach to establish rating scale values that truly reflects raters' intent in MMI, thus improving the accuracy of the instrument and contributing to the general fairness of the process.

The carbon footprint of a UK University during the COVID-19 lockdown

The COVID-19 pandemic has provided a unique opportunity to compare the carbon intensity of higher education delivered on- and off-campus. This is attributed to governmental lockdown orders that have forced Universities to close their campuses, ban business travel and move all teaching and learning activities online. This study represents the first known attempt to compare the carbon footprint of a mid-sized UK University produced during the COVID-19 lockdown (April-June 2020) against that generated within the respective time period in previous years. Although the overall carbon footprint of the University decreased by almost 30% during the lockdown, the carbon intensity of online teaching and learning was found to be substantial and almost equal to that of staff and student commute in the pre-lockdown period. The study contributed to an emerging academic discourse on the carbon (dis)benefits of different models of higher education provision in the UK and beyond. The study suggested that policy and management decisions on transferring education online should carefully consider the carbon implications of this transfer.

Benefits and Feasibility of Using Videos to Assess Medical School Applicants' Empathetic Abilities in Multiple Mini Interviews

PURPOSE

We sought to evaluate the feasibility and benefits of using video-based scenarios in Multiple Mini Interviews (MMIs) to assess candidate's empathic abilities by investigating candidate perceptions and the acceptability, fairness, reliability, and validity of the test.

METHODS

The study sample was candidates for admission interviews held in the MMI format at a medical school in South Korea. In this six-station MMI, one station included a 2-min video clip of a patient-doctor communication scenario to assess candidate emphatic abilities, whereas paper-based scenarios were used in the other stations. Candidate's perceptions and acceptability of using the video-based scenario in the empathy station were examined using a 41-item post-MMI questionnaire. Fairness of the test was assessed by means of differences in candidate perceptions and performance across different demographics or backgrounds. Construct validity was assessed by examining the relationship of candidate performances in the empathy station with those in other stations. The G-coefficient was analyzed to estimate the reliability of the test.

RESULTS

Eighty-two questionnaires were returned, a 97.6% response rate. Candidates showed overall positive perceptions of the video-based scenario and they found it authentic and interesting. The test was fair as there were no differences in candidates' perceptions of the patient-doctor relationship presented in the video clip and neither in their performance nor in their perceived difficulty of the station across demographics or backgrounds. Construct validity was established as candidate performance in the empathy station was not associated with that of any other stations. The G-coefficient was 0.74.

CONCLUSIONS

The present study demonstrates that the video-based scenario is a feasible tool to assess candidate's empathy in the MMI.

Case Report: Defining Applicant Attributes to Be Prioritized in the Selection of Child and Adolescent Psychiatry Subspecialty Residents at the University of Toronto

Background/Objectives: The child and adolescent psychiatry (CAP) subspecialty training program at the University of Toronto was among the first fully accredited CAP programs in Canada. As one of Canada's largest CAP subspecialty programs, we attract many excellent applicants annually. While objectivity and transparency in the selection of candidates have been valued, it was unclear which applicant attributes should be prioritized. This quality improvement project was undertaken to identify the key applicant attributes that should be prioritized for admission to the program. Materials/Methods: An initial list of attributes was compiled by project team members and feedback solicited. Through iterative design, this list was categorized into "end products," "branding attributes" and "generic attributes." The "end products" were removed as these represented outputs of training rather than attributes on which applicant selection should be based. Subsequent steps involved only the "branding" and "generic" attributes. A consensus-building exercise led to the creation of two short-lists of five attributes within each category. Finally, a paired-comparison forced choice methodology was used to determine the ranking of these attributes in order of importance when assessing applicants. Results: The final lists of "generic" and "branding" attributes developed through a consensus-building exercise are presented in rank order based on the paired-comparison methodology. The overall response rate for the forced choice electronic survey was 49% of faculty and learners. Conclusions/Discussion: This project used an iterative process of consensus building & pairwise comparison to prioritize key attributes for assessing trainee selection to the program. Going forward, these attributes will be incorporated into the file review and interview portions of our admissions process. In addition to emphasizing these priority attributes in admissions, there are implications for other aspects of the program including curriculum and faculty development, as well as guiding the overall mission and vision for the Division. A similar process could be undertaken by other training programs seeking to identify priority attributes for admission to their programs.

Semantic fluency including task switching predicts academic success in medical school

Objectives

The future state treaty on the admission of students to German medical schools calls for a variety of selection criteria among which at least two are required to be independent of school leaving grades. Against this background, the present study investigated achievement motivation and executive functions as predictors of academic success in medical school.

Material and methods

Second year medical students were assessed for executive functioning by using the Tower of London Test (ToL), a German version of the Controlled Oral Word Association Test (COWAT), the Trail Making Test (TMT-A) and for motivation by using the Achievement Motivation Inventory (AMI). Academic success was evaluated twofold, i) whether the first state exam (M1) was passed at the earliest possible, after completion of the second year and ii) via the grades obtained.

Results

81 out of 226 students enrolled participated in our study. Passing the M1 was best explained by semantic fluency including task switching. Moreover, academically successful students achieved significantly higher levels in the AMI-facets "compensatory effort" and "engagement". All students scored above average in the TMT-A and average in the ToL.

Conclusion

Alternating semantic fluency—requiring simultaneously inhibition, updating and task shifting—turned out highly predictive of academic success in medical school. Moreover, higher levels in "compensatory effort" and “engagement” suggested that both, increased energy expenditure as response to fear of failure and elevated readiness to exert effort also impacted positively on success.

A Critical Disconnect: Residency Selection Factors Lack Correlation With Intern Performance

BACKGROUND

Emergency medicine (EM) residency programs want to employ a selection process that will rank best possible applicants for admission into the specialty.

OBJECTIVE

We tested if application data are associated with resident performance using EM milestone assessments. We hypothesized that a weak correlation would exist between some selection factors and milestone outcomes.

METHODS

Utilizing data from 5 collaborating residency programs, a secondary analysis was performed on residents trained from 2013 to 2018. Factors in the model were gender, underrepresented in medicine status, United States Medical Licensing Examination Step 1 and 2 Clinical Knowledge (CK), Alpha Omega Alpha (AOA), grades (EM, medicine, surgery, pediatrics), advanced degree, Standardized Letter of Evaluation global assessment, rank list position, and controls for year assessed and program. The primary outcomes were milestone level achieved in the core competencies. Multivariate linear regression models were fitted for each of the 23 competencies with comparisons made between each model's results.

RESULTS

For the most part, academic performance in medical school (Step 1, 2 CK, grades, AOA) was not associated with residency clinical performance on milestones. Isolated correlations were found between specific milestones (eg, higher surgical grade increased wound care score), but most had no correlation with residency performance.

CONCLUSIONS

Our study did not find consistent, meaningful correlations between the most common selection factors and milestones at any point in training. This may indicate our current selection process cannot consistently identify the medical students who are most likely to be high performers as residents.

A comparative study on the psychological health status of pre-clinical medical students enrolled through different admission tests

Objective

Methods

Results

Conclusions

Adaptation of internet-based multiple mini-interviews in a limited-resource medical school during the coronavirus disease 2019 pandemic

The coronavirus disease 2019 pandemic has affected many aspects of life, including medical education. The selection of students using multiple mini-interviews (MMI) should be adapted promptly as internet-based MMI (iMMI) in the current setting. The current transition from MMI to iMMI is supported by experience in conducting conventional MMI in previous years; hence, the station and assessor preparation could be completed without significant issues. The greater challenges to the use of iMMI are due to technical issues, including poor internet connections and adaptations necessary for assessing candidates' nonverbal expressions. This paper aims to describe the experience of conducting iMMI in the current pandemic situation, with further highlights on contextual challenges in a limited-resource setting.

Comparison between Personality, Emotional, and Educational Outcomes of Multiple Mini Interviews and Personal Interview

Objectives

This study investigated the outcomes of multiple mini interviews and personal interview on personality traits, emotional intelligence, perceived educational environment, and stressors.

Methods

This is a comparative cross-sectional study on two cohorts of pre-clinical medical students who were selected by multiple mini interviews and personal interview, respectively. Their personality traits, emotional intelligence, perceived educational environment, and perceived stressors were measured using different measurement tools.

Results

Multiple mini interviews and personal interview demonstrated a similar ability to recruit medical students with a high level of emotional intelligence. The main advantage of personal interviews over multiple mini interviews in terms of personality traits is that it recruited candidates who had a higher level of conscientiousness trait. The main advantage of multiple mini interviews over personal interview on the educational environment is that medical students chosen by multiple mini interviews had a higher level of satisfaction with social aspects of medical training. Regardless of admission processes, the medical students were equally vulnerable to psychological distress due to various stressful events throughout medical training particularly related to academic loads.

Conclusions

This study provided evidence to support the outcomes that multiple mini interviews and personal interview have on medical students’ emotional intelligence, personality traits, perceived educational environment, and perceived stressors during the pre-clinical medical training. Interestingly, personal interview had a better outcome on conscientiousness while multiple mini interviews had a better outcome on the social aspect.

Transition to multiple mini interview (MMI) interviewing for medical school admissions

INTRODUCTION

The multiple mini interview (MMI) has been incorporated into the holistic review process in the selection of students to US medical schools. The MMI has been used to evaluate interpersonal and intrapersonal attributes which are deemed as necessary for future physicians. We hypothesized that there would be little difference in overall MMI evaluation data compared with traditional interview ratings.

METHODS

The University of North Carolina School of Medicine developed an interview process that included a traditional interview and MMI format during the 2019 admissions cycle. Evaluation data along with key demographic variables for 608 MD program applicants were analyzed using descriptive and inferential statistical analyses.

RESULTS

The MMI format slightly favored female over male applicants (p = 0.002) but did not select for or against applicants based on age, race/ethnicity, underserved/rural area upbringing, or indicators of disadvantage. Out of 608 applicants, 356 (59%) completed a post-interview survey in which the experience was positively rated.

DISCUSSION

Based on our experience, the use of a hybrid model of traditional interviews complemented with MMI stations provided greater details in the assessment of medical school applicants while obtaining equivalent data and acceptability amongst applicants.

COVID 19: Designing and conducting an online mini-multiple interview (MMI) in a dynamic landscape

Introduction: The COVID-19 pandemic presented numerous, significant challenges for medical schools, including how to select the best candidates from a pool of applicants when social distancing and other measures prevented "business as usual" admissions processes. However, selection into medical school is the gateway to medicine in many countries, and it is critical to use processes which are evidence-based, valid and reliable even under challenging circumstances. Our challenge was to plan and conduct a multiple-mini interview (MMI) in a dynamic and stringent safe distancing context.Methods: This paper reports a case study of how to plan, re-plan and conduct MMIs in an environment where substantially tighter safe distancing measures were introduced just before the MMI was due to be delivered.Results: We report on how to design and implement a fully remote, online MMI which ensured the safety of candidates and assessors.Discussion: We discuss the challenges of this approach and also reflect on broader issues associated with selection into medical school during a pandemic. The aim of the paper is to provide broadly generalizable guidance to other medical schools faced with the challenge of selecting future students under difficult conditions.

Exam performance of different admission quotas in the first part of the state examination in medicine: a cross-sectional study

BACKGROUND

Most medical students in Germany are admitted via selection procedures, which are adjusted to the demands of the universities. At Lübeck medical school, scores from interviews that measure non-academic skills and pre-university GPAs are summed to arrive at an admission decision. This article seeks to illuminate the effectiveness of this selection procedure in comparison to other non-selected student groups.

METHODS

Quota information and exam results from the first federal exam were linked for students admitted to Lübeck medical school between 2012 and 2015 (N = 655). Five different student groups (university-specific selection quota, pre-university GPA quota, waiting time quota, ex-ante quota and foreign students) were compared regarding exam attempts, written and oral grades, temporal continuity and examination success in the standard study period.

RESULTS

While the pre-university GPA quota outperformed all other quotas regarding written and oral grades, it did not differ from the selection quota regarding exam attempts, temporal continuity and examination success in the standard study period. Students in the waiting time and ex-ante quotas performed inferior by comparison. The results of foreign students were the most problematic.

CONCLUSION

Students selected by the university show high temporal continuity and examination success. These results, and possible advantages in physician eligibility, argue for the utilisation of non-academic skills for admission.

The predictive validity of Multiple Mini Interviews (MMIs) in nursing and midwifery programmes: Year three findings from a cross-discipline cohort study

BACKGROUND

Education literature worldwide is replete with studies evaluating the effectiveness of Multiple Mini Interviews (MMIs) in admissions to medicine but <1% of published studies have been conducted in selection to nursing and midwifery programmes.

OBJECTIVES

To examine the predictive validity of MMIs using end of programme clinical and academic performance indicators of pre-registration adult, child, and mental health nursing and midwifery students.

DESIGN AND SETTING

A cross-sectional cohort study at one university in the United Kingdom.

PARTICIPANTS

A non-probability consecutive sampling strategy whereby all applicants to the September 2015 pre-registration adult, child, mental health nursing and midwifery programmes were invited to participate. Of the 354 students who commenced year one, 225 (64%) completed their three-year programme and agreed to take part (adult 120, child 32, mental health nursing 30 and midwifery 43).

METHODS

All applicants were interviewed using MMIs with six and seven station, four-minute models deployed in nursing and midwifery student selection respectively. Associations between MMI scores and the cross-discipline programme performance indicators available for each student at this university at the end of year three: clinical practice (assessed by mentors) and academic attainment (dissertation mark) were explored using multiple linear regression adjusting for applicant age, academic entry level, discipline and number of MMI stations.

RESULTS

In the adjusted models, students with higher admissions MMI score (at six and seven stations) performed better in clinical practice (p < 0.001) but not in academic attainment (p = 0.122) at the end of their three-year programme.

CONCLUSION

These findings provide the first report of the predictive validity of MMIs for performance in clinical practice using six and seven station models in nursing and midwifery programmes. Further evidence is required from both clinical and academic perspectives from larger, multi-site evaluations.

Need for cognitive closure, tolerance for ambiguity, and perfectionism in medical school applicants

BACKGROUND

Physicians have to deal with uncertainty on a daily basis, which requires high tolerance for ambiguity. When medical decisions have to be made in ambiguous situations, low levels of need for cognitive closure and high levels of adaptive perfectionism are beneficial. It might be useful to measure such personality traits during medical school selection processes. In our study, we explored the expression of need for cognitive closure, tolerance for ambiguity, and perfectionism in medical school applicants who participated in a multiple mini-interview selection process with respect to the final decision of admission or rejection.

METHODS

After participating in the multiple mini-interview procedure (HAM-Int) at Hamburg Medical School in August 2019, 189 medical school applicants filled out a questionnaire including the Multidimensional Perfectionism Scale by Hewitt and Flett (MPS-H), the Multidimensional Perfectionism Scale by Frost (MPS-F), the Tolerance for Ambiguity Scale (TAS), the 16-Need for Cognitive Closure Scale (16-NCCS), and sociodemographic data. After the final admission decision, the scores of need for cognitive closure, tolerance for ambiguity, and perfectionism of admitted and rejected applicants were compared. We also assessed the predictive power of need for cognitive closure and age for the admission decision in a binary logistic regression model.

RESULTS

Compared to the admitted applicants, the rejected applicants showed a significantly higher need for cognitive closure (p = .009). A high need for cognitive closure correlated significantly positively with maladaptive perfectionism (p < .001) and significantly negatively with tolerance for ambiguity (p < .001). Low need for cognitive closure and older age were associated with a positive admission decision.

CONCLUSIONS

Regarding the personality traits need for cognitive closure, tolerance for ambiguity, and perfectionism we identified interesting differences and correlations of relevance for physicians' daily work in medical school applicants who were admitted or rejected after participating in a multiple mini-interview selection procedure. Further studies are needed to investigate these characteristics and their development longitudinally in medical students and to correlate them with students' medical performance.

Predictive validity of multiple mini interview scores for future academic and clinical placement performance in physiotherapy, occupational therapy and speech and language therapy programmes

Background/aims

Limited research exists on the predictive validity of multiple mini interviews when used during the selection process for physiotherapy, occupational therapy and speech and language therapy programmes. Traditional interviews were replaced by multiple mini interviews in the selection process for these three undergraduate programmes in one UK university. The purpose of secondary data analysis was to determine whether multiple mini interviews can predict academic and clinical placement performance during the first and second year of study.

Methods

Secondary data analysis was performed using the admissions data from 169 students. Data analysed comprised predictors (entry tariff, age and multiple mini interview scores) and outcomes (academic and placement achievement in years 1 and 2 over a period of 2 years from 2014 to 2016).

Results

Multiple standard regression analyses revealed that, overall, multiple mini interview score was not a significant predictor of academic achievement or clinical placement performance in years 1 and 2 for any of the three professions. Pearson's correlations indicated that age and entry tariff were frequently and significantly correlated with academic achievements for all cohorts.

Conclusions

Performance in the multiple mini interview is not a predictor of academic achievement or clinical placement performance in this specific cohort of students. Entry tariff showed a significant correlation with academic achievement. Additional studies with larger cohorts are recommended.

The Multiple Mini Interview as a Dental School Admission Tool: Can It Predict Noncognitive Traits Associated with Professional Behaviors?

The aim of this study was to investigate the reliability and validity of the Multiple Mini Interview (MMI) process and to assess its use as an admission tool to predict noncognitive traits associated with professional behavior during patient care in one cohort of dental students at a single U.S. dental school. Data were analyzed for the 95 candidates who matriculated and graduated as part of the 2017 graduating cohort at the University of Michigan School of Dentistry. All MMI interviewees for one cycle of admissions rotated through ten stations: two traditional interview question stations and eight scenario stations measuring domains that included four questions scored on a five-point Likert scale. Generalizability theory analysis showed the MMI to have good reliability (G Coefficient of 0.74). Station reliabilities (Cronbach's alpha) ranged from 0.88 to 0.92. MMI scores showed a positive significant correlation with students' scores on the Dental Admission Test and Perceptual Ability Test, D1 cumulative GPA, and D4 Patient Management grade. MMI scores positively correlated with professional behaviors relating to three domains-openness, conscientiousness, and emotional stability-thus demonstrating good predictive validity for measuring noncognitive traits associated with professionalism. This study found that the MMI was a reliable and valid tool that predicted key behavioral traits associated with professionalism in dental students.

Multiple Speed Assessments

Abstract. This paper presents Multiple Speed Assessments as an umbrella term to encompass a variety of approaches that include multiple (e.g., 20), short (e.g., 3 min), and often integrated interpersonal simulations to elicit overt behavior in a standardized way across participants. Multiple Speed Assessments can be used to get insight into the behavioral repertoire of a target person in situations sampled from a predefined target domain and their intraindividual variability across these situations. This paper outlines the characteristics and theoretical basis of Multiple Speed Assessments. We also discuss various already existing examples of Multiple Speed Assessments (Objective Structured Clinical Examinations, Multiple Mini-Interviews, and constructed response multimedia tests) and provide an overview of design variations. Finally, we present current research evidence and future research directions related to Multiple Speed Assessments. Although we present Multiple Speed Assessments in the context of personnel selection, it can also be used for assessment in the educational, personality, or clinical psychology field.

Construct validity of multiple mini interviews - Investigating the role of stations, skills, and raters using Bayesian G-theory

Background: One popular procedure in the medical student selection process are multiple mini-interviews (MMIs), which are designed to assess social skills (e.g., empathy) by means of brief interview and role-play stations. However, it remains unclear whether MMIs reliably measure desired social skills or rather general performance differences that do not depend on specific social skills. Here, we provide a detailed investigation into the construct validity of MMIs, including the identification and quantification of performance facets (social skill-specific performance, station-specific performance, general performance) and their relations with other selection measures.Methods: We used data from three MMI samples (N = 376 applicants, 144 raters) that included six interview and role-play stations and multiple assessed social skills.Results: Bayesian generalizability analyses show that, the largest amount of reliable MMI variance was accounted for by station-specific and general performance differences between applicants. Furthermore, there were low or no correlations with other selection measures.Discussion: Our findings suggest that MMI ratings are less social skill-specific than originally conceptualized and are due more to general performance differences (across and within-stations). Future research should focus on the development of skill-specific MMI stations and on behavioral analyses on the extents to which performance differences are based on desirable skills versus undesired aspects.