What does the multiple mini interview have to offer over the panel interview?

Scoping review of current challenges and circumstances impacting Indigenous applications to Canadian medical schools

Introduction

Considering the relevant 2015 Truth and Reconciliation Commission recommendations, this paper reviews the current state of Canadian medical schools' Indigenous admissions processes and explores continued barriers faced by Indigenous applicants.

Methods

A summary of literature illustrating disadvantages for Indigenous applicants of current admissions tools is presented. A grey literature search of current admissions requirements, interview processes, and other relevant data from each medical school was performed. Tables comparing differences in their approaches are included. A calculation of Indigenous access to medical school seats compared to the broader Canadian population was conducted. Gaps in execution are explored, culminating in a table of recommendations.

Results

Despite formal commitments to reduce barriers, Indigenous applicants to medical school in Canada still face barriers that non-Indigenous applicants do not. Most programs use tools for admission known to disadvantage Indigenous applicants. Indigenous applicants do not have equitable access to medical school seats. Facilitated Indigenous stream processes first ensure Indigenous applicants meet all minimum requirements of Canadian students, and then require further work.

Discussion

Seven years after the Truth and Reconciliation Commission called on Canadian universities and governments to train more Indigenous health care providers, there has been limited progress to reduce the structural disadvantages Indigenous students face when applying to medical school. Based on best practices observed in Canada and coupled with relevant Indigenous-focused literature, recommendations are made for multiple stakeholders.

Conclusions

The study was limited by the data available on numbers of Indigenous applicants and matriculants. Where available, data are not encouraging as to equitable access to medical school for Indigenous populations in Canada. These findings were presented at the International Congress of Academic Medicine 2023 Conference, April 2023, Quebec City, Canada.

Evaluating non-cognitive skills in medical school applicants

Medical school candidates must have both the cognitive and professional competencies required to become good physicians. In this commentary, we outline the evidence and outcomes associated with the implementation of these selection methodologies and evaluate their ability to assess non-cognitive skills.

Effects of candidates' demographics and evaluation of the virtual Multiple Mini Interview (vMMI) as a tool for selection into paediatric training in Queensland

INTRODUCTION

The Queensland Basic Paediatric Training Network (QBPTN) is responsible for the selection of candidates into paediatric training in Queensland. The COVID-19 pandemic necessitated interviews to be conducted 'virtually' as virtual Multiple-Mini-Interviews (vMMI). The study aimed to describe the demographic characteristics of candidates applying for selection into paediatric training in Queensland, and to explore their perspectives and experiences with the vMMI selection tool.

METHODOLOGY

The demographic characteristics of candidates and their vMMI outcomes were collected and analysed with a mixed methods approach. The qualitative component was comprised of seven semi-structured interviews with consenting candidates.

RESULTS

Seventy-one shortlisted candidates took part in vMMI and 41 were offered training positions. The demographic characteristics of candidates at various stages of selection were similar. The mean vMMI scores were not statistically different between candidates from the Modified Monash Model 1 (MMM1) location and others [mean (SD): 43.5 (5.1) versus 41.7 (6.7), respectively, p = 0.26]. However, there was a statistically significant difference (p value 0.03) between being offered and not offered a training position for candidates from MMM2 and above. The analysis of the semi-structured interviews suggested that candidate experiences of the vMMI were influenced by the quality of the management of the technology used. Flexibility, convenience, and reduced stress were the main factors that influenced candidates' acceptance of vMMI. Perceptions of the vMMI process focused on the need to build rapport and facilitate communication with the interviewers.

DISCUSSION

vMMI is a viable alternative to face-to-face (FTF) MMI. The vMMI experience can be improved by facilitating enhanced interviewer training, by making provision for adequate candidate preparation and by having contingency plans in place for unexpected technical challenges. Given government priorities in Australia, the impact of candidates' geographical location on the vMMI outcome for candidates from MMM >1 location needs to be further explored.

Experiences of Rural and Metropolitan Background Applicants in Preparing for and Completing a Regionally Focused Multiple Mini-interview for Admission into a Regional Medical Program

BACKGROUND

To better target rural background and rurally interested applicants during medical school admission, it is increasingly common for rural medical programs to include multiple mini-interview (MMI) scenarios designed to screen for rural interest. It remains unclear whether the inclusion of regionally/rurally focused MMI scenarios positively impacts the selection of rural background applicants and evidence is limited regarding why rural background applicants may perform worse on the MMI. Therefore, this study explored how rural and metropolitan applicants prepare for and perceive the MMI for admission to a regional medical pathway.

METHODS

A mixed-methods survey was sent to provisional entry regional pathway medical school applicants who had completed an MMI. The survey was distributed before any offers of admission had been released.

RESULTS

Rural applicants spent less time and money preparing for the MMI and felt less prepared (P < 0.05). However, time and money spent, and resources used to prepare were not associated with feeling more prepared (all P > 0.05). Respondents mostly felt that the MMI process aligned with their expectations (83%), is fair (64%), and helps a rural program select the most suitable applicants (61%). Rural applicants generally felt that they had an advantage over other applicants (61%) while most metropolitan applicants did not (23%; P = 0.002).

DISCUSSION

Applicants to a regional medical pathway are generally supportive of the MMI process. It appears valuable for applicants to prepare for the MMI by understanding the format and requirements; however, investing substantial time and money does not underpin feeling better prepared. MMI scenarios which include a regional focus are perceived to advantage rural applicants.

Comparison of the Multiple Mini-Interview and the Traditional Interview in Medical School Admissions: Lessons Learned Using a Hybrid Model at One Institution

PURPOSE

Medical school admissions interviews are a critical form of assessment; however, the most effective interview strategy is debated. This study compares the traditional interview (TI) and multiple mini-interview (MMI) within a hybrid TI-MMI model at one medical school to determine whether the interview approaches reveal different information about applicants and whether a hybrid model results in a more diversified applicant pool.

METHOD

Admissions data from 3 application cycles at the Donald and Barbara Zucker School of Medicine at Hofstra/Northwell were used. The TI was used in 2017-2018 and the hybrid TI-MMI model in 2018-2019 and 2019-2020. Applicants were scored on a 5-point scale and referred to a voting committee for acceptance consideration if interview scores met threshold criteria. Changes in the number of students referred to the committee using the TI vs the TI-MMI score criteria were analyzed.

RESULTS

In 2017-2018 (TI only), 683 applicants were interviewed; in 2018-2019 (TI-MMI), 844 applicants were interviewed; and in 2019-2020 (TI-MMI), 805 applicants were interviewed. Medium correlations were found between total MMI and TI scores in 2018-2019 ( ρ = 0.37, P < .001) and 2019-2020 ( ρ = 0.33, P < .001). No differences were found in TI scores between 2017-2018 and 2018-2019 ( P = .30), but TI scores were significantly lower in 2019-2020 vs 2017-2018 ( P < .001) and 2018-2019 ( P = .002). Overall, a 10% to 18% increase was found in the number of applicants referred to the voting committee when using hybrid criteria, with a 19% to 27% increase in underrepresented in medicine applicants.

CONCLUSIONS

The TI-MMI model may allow for a more holistic interview approach and an expanded pool of applicants, particularly underrepresented in medicine applicants, considered for acceptance.

A Model for Online Delivery of Multiple Mini Interviews

Multiple mini interviews (MMIs) have become the mainstay of medical school admission interviews in the United Kingdom. During the COVID-19 pandemic, Government imposed restrictions on the meeting of people indoors precipitated a move towards conducting interviews online. Thus, the development of methodologies to conduct robust MMI style interviews remotely was required. In this article, a validated method for conducting remote MMIs is described. This method of delivery produced comparable candidate scores compared with pre-pandemic in-person interviews and maintained reliability.

Evaluating a Global Assessment Measure Created by Standardized Patients for the Multiple Mini Interview in Medical School Admissions: Mixed Methods Study

BACKGROUND

Standardized patients (SPs) are essential stakeholders in the multiple mini interviews (MMIs) that are increasingly used to assess medical school applicants' interpersonal skills. However, there is little evidence for their inclusion in the development of instruments.

OBJECTIVE

This study aimed to describe the process and evaluate the impact of having SPs co-design and cocreate a global measurement question that assesses medical school applicants' readiness for medical school and acceptance status.

METHODS

This study used an exploratory, sequential, and mixed methods study design. First, we evaluated the initial MMI program and determined the next quality improvement steps. Second, we held a collaborative workshop with SPs to codevelop the assessment question and response options. Third, we evaluated the created question and the additional MMI rubric items through statistical tests based on 1084 applicants' data from 3 cohorts of applicants starting in the 2018-2019 academic year. The internal reliability of the MMI was measured using a Cronbach α test, and its prediction of admission status was tested using a forward stepwise binary logistic regression.

RESULTS

Program evaluation indicated the need for an additional quantitative question to assess applicant readiness for medical school. In total, 3 simulation specialists, 2 researchers, and 21 SPs participated in a workshop leading to a final global assessment question and responses. The Cronbach α's were >0.8 overall and in each cohort year. The final stepwise logistic model for all cohorts combined was statistically significant (P<.001), explained 9.2% (R2) of the variance in acceptance status, and correctly classified 65.5% (637/972) of cases. The final model consisted of 3 variables: empathy, rank of readiness, and opening the encounter.

CONCLUSIONS

The collaborative nature of this project between stakeholders, including nonacademics and researchers, was vital for the success of this project. The SP-created question had a significant impact on the final model predicting acceptance to medical school. This finding indicates that SPs bring a critical perspective that can improve the process of evaluating medical school applicants.

Association of Preadmission Factors With Physician Assistant Student Diversity

PURPOSE

A new physician assistant (PA) program accreditation standard requires institutional support for PA student diversity. Therefore, preadmission factors associated with diversity should be identified to inform decision-making. The purpose of this study was to explore associations between preadmission factors of interest and percentages of underrepresented minority (URM) students in PA programs.

METHODS

Physician Assistant Education Association (PAEA) survey responses from 169 PA programs were analyzed by using simultaneous multiple regression. The variables included the standardized testing requirement of the Graduate Record Examinations (GRE), undergraduate preprofessional phase, public or private institution, writing sample requirement, mission statement inclusive of care for underserved populations, and local racial or ethnic demographics.

RESULTS

GRE score requirements were found to be negatively associated with percentages of African American and total non-White, non-Asian students (p < .05). Local URM racial and ethnic population demographics were shown to be positively associated with African American, Hispanic, and total non-White, non-Asian students (p < .05). Undergraduate preprofessional phase, type of institution, writing sample requirement, and mission statement supportive of underserved populations did not show significant associations with URM diversity.

CONCLUSIONS

Faculty and administration members who are involved in decision-making in PA admissions processes should consider the potentially negative impact of a GRE requirement as it relates to the percentages of African American and total non-White, non-Asian matriculants. Educational institutions might also consider exploring new PA program development in geographic areas with supportive local demographics as an approach to achieving more student diversity.

The current dental school applicant: an overview of the admission process for UK dental schools and the sociodemographic status of applicants

Aims To determine the current processes used to assess dental school admissions in the UK as well as compare the applicants' demography.

Methods All 16 dental schools in the UK were invited to complete a questionnaire analysing the admissions protocols between 2018-2019 and 2019-2020. These data were combined with the admission process information available online. Retrospective data from the University Clinical Aptitude Test including the sociodemographic status of dental applicants were collected. For the six dental schools that did not reply, data presented were collected from information available to the public.

Results The majority of applicants were women (63.2% in 2019-2020), white (27.9%) and are sixth form attendees or attend a further education college (40.6%). Of those who apply to study dentistry, 15.5% are graduates who hold a first degree. For each undergraduate dental place available, there were 12.6 applicants and 9.4 applicants to each post-qualified dental undergraduate place.

Conclusion Further advancements are required to widen participation and broaden the sociodemographic status of dental applicants. This area would benefit from a long-term prospective study about recruitment methods and its correlation with performance at dental school. COVID-19 is impacting the application process, the full extent of which is yet to be determined.

Provides an overview of the current admission processes for dental schools within the UK.

Encourages critical analysis of the admission processes used for undergraduate and postgraduate dental students.

Explores how the admission processes may change as a result of the COVID-19 pandemic and gives an indication as to the sociodemographics of the future dental career force.

Implementation and evaluation of virtual multiple mini interviews as a selection tool for entry into paediatric postgraduate training: A Queensland experience

INTRODUCTION

The Queensland Basic Paediatric Training Network (QBPTN) is the centralised pathway for entry into paediatric training in Queensland, Australia. In response to COVID-19 travel and social distancing restrictions imposed in 2020, QBPTN successfully adopted a Virtual Multiple Mini Interviews (vMMIs) model for the selection of candidates for entry into paediatric training. The authors describe the planning, implementation, challenges, and evaluation of candidates' and interviewers' experiences of vMMIs, including the differences between candidates from two geographical areas.

METHODS

The contents of six vMMI stations were similar to face-to-face MMI. Implementation required the identification of ZOOM^TM as a preferred online platform, securing venues, communication, development of contingency plans and central coordination by the network. Candidates' experiences with vMMI were explored through thematic analysis of the qualitative data from focus groups and free text responses, and descriptive analysis of SurveyMonkey© questionnaire responses. Experiences between 'metropolitan' and 'regional and interstate' candidates were compared.

RESULTS

5-minute stations with 2-minute pre-reading were used. 78 candidates and 14 interviewers participated in the selection process. All candidates attended the focus group. 58.7% of candidates responded to post vMMI questionnaire. 93% of survey responders were happy to undertake vMMI in the future, with 23% feeling they would have performed better in face-to-face. Experiences between 'metropolitan' and 'other' groups were similar. Positive experiences of participants were related to the user-friendly IT platform, successful pre-interview communications, preparation, convenience, time, and cost savings. Stress related IT failures and difficulties establishing rapport with interviewers were reported as the main negative experiences.

CONCLUSION

'vMMI' is a feasible and acceptable method of selection into paediatric training. vMMI has many benefits and can be implemented relatively quickly by addressing key logistical requirements. The model under discussion could be adapted by other centres based on local needs.

Non-cognitive testing in medical student selection: the development and evaluation of a novel MMI station

BACKGROUND

Multiple mini-interviews (MMIs) are a well-established method of assessing non-cognitive skills in medical student selection in the United Kingdom; however, there are concerns that as MMIs become more predictable, candidates can be coached through the process. An innovative station was introduced at a new medical school to assess less commonly evaluated non-cognitive skills in the context of a skills task.

OBJECTIVE

Our study aimed to evaluate candidate scores on the novel station and explore how they compared with scores on a traditional questions-based station and to the overall interview score.

METHODS

Using a laparoscopic skills trainer, candidates were asked to pass a string through as many hoops as possible, only viewing the image on a screen. All candidates were assessed on decision making, managing stressful situations and creativity using several psychological parameters. Scores were then compared with scores on a traditional values-based question and the overall interview score.

RESULTS

Two hundred eighty-seven candidates completed the station. The average score on the novel station was significantly higher than the average scores on the other stations (paired Student's t-test, p < 0.05). There was no correlation between scores on the novel station and the traditional values-based question, suggesting different skills were being assessed in each station.

DISCUSSION

This station showed it is possible to test less commonly tested non-cognitive skills, in a realistic way. Unlike traditional stations, there is no reliance on candidate recall to award marks, therefore making it more difficult to coach. However, further work is required to refine and improve the testing environment.

Conducting multiple mini-interviews in the midst of COVID-19 pandemic

Multiple mini-interview (MMI) is a 'multiple sample-based' approach comprising multiple focused encounters intended to access and assess a range of attributes in order to gain more objectively multiple impressions of an applicant's interpersonal skills, thoughtfulness and general demeanour. It is designed to focus on four domains that are not considered to be comprehensive, but are considered to be vital for a successful career in the health sciences: critical thinking, ethical decision making, communication and knowledge of the healthcare system. Traditionally, the MMI is conducted face-to-face, but with COVID-19 pandemic and the implementation of social distancing measures, no onsite or campus teaching, banning of mass gatherings and cancellation of face-to-face interviews, Pengiran Anak Puteri Rashidah Sa'adatul Bolkiah Institute of Health Sciences at Universiti Brunei Darussalam explored the feasibility of conducting MMI through virtual means. This report provides an account of our experience in conducting internet-MMI for the selection of new applicants into the August 2020 cohort of the Medicine programme. We also aimed to determine whether the scores derived from internet-MMI were reliable and equivalent to the scores derived from traditional MMI.

Implementation of Virtual Multiple Mini-Interviews for Fellowship Recruitment

Introduction

The SARS-CoV Disease (COVID-19) pandemic has upended health care systems and one of the casualties has been the trainee recruitment process since social distancing and travel restrictions make an in-person experience improbable. At the University of California, Davis (UCD), our Pain Division transitioned our internally validated multiple mini-interview (MMI) process to a virtual environment

Methods

Applicants signed a confidentiality agreement prior to their interviews and were invited to watch a series of videos orienting them to the process and to the program itself. All faculty raters interviewed candidates using a total of 6 non-medical MMI scenarios with corresponding questions and scoring rubrics through the Zoom platform. Applicants were then welcomed to voluntary informal conversations with the current fellow trainees and faculty. An optional survey was sent to the applicants post-interview to assess their overall satisfaction with the virtual process.

Results

The survey analyzed the following using a 5 point Likert scale: Overall Satisfaction, Video Overview Program, Interview Day Details, Video Tour, Web-Based Interviews, Process was Fair, and finally a question on Informed Decision regarding receiving sufficient information to formulate their rank list. All respondents (80% response rate) reported being either satisfied or very satisfied with each of the aspects of the interview process detailed above.

Conclusions

While technical difficulties and confidentiality issues are of concern when offering an entirely web based recruitment, our group was able to transition traditional in-person MMI to a virtual platform using a similar structure which was well-received by applicants

Benefits and Feasibility of Using Videos to Assess Medical School Applicants' Empathetic Abilities in Multiple Mini Interviews

PURPOSE

We sought to evaluate the feasibility and benefits of using video-based scenarios in Multiple Mini Interviews (MMIs) to assess candidate's empathic abilities by investigating candidate perceptions and the acceptability, fairness, reliability, and validity of the test.

METHODS

The study sample was candidates for admission interviews held in the MMI format at a medical school in South Korea. In this six-station MMI, one station included a 2-min video clip of a patient-doctor communication scenario to assess candidate emphatic abilities, whereas paper-based scenarios were used in the other stations. Candidate's perceptions and acceptability of using the video-based scenario in the empathy station were examined using a 41-item post-MMI questionnaire. Fairness of the test was assessed by means of differences in candidate perceptions and performance across different demographics or backgrounds. Construct validity was assessed by examining the relationship of candidate performances in the empathy station with those in other stations. The G-coefficient was analyzed to estimate the reliability of the test.

RESULTS

Eighty-two questionnaires were returned, a 97.6% response rate. Candidates showed overall positive perceptions of the video-based scenario and they found it authentic and interesting. The test was fair as there were no differences in candidates' perceptions of the patient-doctor relationship presented in the video clip and neither in their performance nor in their perceived difficulty of the station across demographics or backgrounds. Construct validity was established as candidate performance in the empathy station was not associated with that of any other stations. The G-coefficient was 0.74.

CONCLUSIONS

The present study demonstrates that the video-based scenario is a feasible tool to assess candidate's empathy in the MMI.

More Than Their Test Scores: Redefining Success with Multiple Mini-Interviews

INTRODUCTION

Medical schools are shifting to a holistic approach for admissions. As non-cognitive abilities become more valued in the medical field and admissions become more competitive, multiple mini-interviews (MMI) are becoming increasingly common in selection criteria. The purpose of this article is to examine the relationship among admissions criteria to create a predictive model of acceptance to medical school. This article also aims to examine, among matriculated students, the relationship between MMI and traditional academic success, and the relationship between MMI scores and clinical competence and academic accolades.

METHODS

This longitudinal study followed a cohort of students from the MMI process through the first two pre-clinical years at a medical school in the southeastern USA. Multiple logistic regression with backward elimination variable selection was used to examine the relationship between admission criteria and acceptance. A multivariate multinomial ordered probit model was used to assess the relationship between MMI and traditional academic success among matriculated students. Simple linear regression models were used to assess relationships between MMI and Objective Structured Clinical Examination (OSCE) scores and honors nomination.

RESULTS

MMI are among eight significant predictors of acceptance. Among matriculated students, there were weak negative associations between MMI and traditional academic success; however, all but one relationship was statistically non-significant. There was not a significant association between MMI and OSCE scores or academic accolades.

CONCLUSIONS

While MMI can identify students with non-cognitive skills needed to be a good physician, continued assessment needs to be incorporated in their education.

Interviewer perceptions during the implementation of the multiple mini-interview model at a school of pharmacy

INTRODUCTION

Studies reveal positive interviewer perceptions of multiple mini-interview (MMI) upon MMI completion. No studies evaluate change in interviewer perceptions during MMI implementation. The objective was to evaluate the change in interviewer perceptions during the implementation of the MMI model at the University of Toledo College of Pharmacy and Pharmaceutical Sciences.

METHODS

Interviewers (faculty volunteers, preceptors, student pharmacists) were eligible for inclusion in the prospective cohort. Consenting individuals (1) completed a pre-MMI training survey regarding perceptions of MMI, (2) participated in a 90-minute MMI training program (PowerPoint presentation and review of videos demonstrating MMI practices), (3) completed a post-MMI training survey, and (4) after interviews, completed a post-interview survey. The six Likert-scale MMI perception questions were independently analyzed for changes in the rank response across the three survey time points using Friedman's nonparametric repeated-measures analysis. Each question was evaluated for all respondents together, and for nine different respondent subgroups. The overall criteria for significance was α = 0.05 for each question, with Bonferroni correction for the ten overall comparisons made for each question.

RESULTS

Thirty-two interviewers participated (20 faculty members, five preceptors, and seven student pharmacists). From the pre-MMI training survey through the post-interview survey, interviewers gained confidence in their ability to explain the rationale behind the MMI model, were more likely to agree that six minutes was adequate time to assess an applicant and believed MMI provides a fair assessment of an applicant's noncognitive attributes.

CONCLUSIONS

After interviewers received training and gained experience with MMI, perceptions of MMI improved.

Effect of Applicant Gender on Multiple Mini-Interview Admissions Score

PURPOSE

Admissions criteria for physician assistant (PA) schools vary considerably, but they often involve a combination of academic measures, noncognitive "humanistic" qualities, and mission-related items. To address noncognitive attributes, some PA schools have implemented the multiple mini-interview (MMI) format developed at McMaster University in Canada. This research project looked at differences in interview scores by gender over 3 consecutive admissions cycles at one PA program.

METHODS

Three years of pre-existing de-identified data gathered as part of the routine admissions process were analyzed retrospectively using SPSS-v25.

RESULTS

Data were available for all interviewees for 2015, 2016, and 2017 (N = 350 total). Between-group differences were not statistically significant by gender. Reliability (Cronbach's α) was 0.865 for academic scoring and 0.694 for MMI scoring.

DISCUSSION

Analysis of 3 years of admissions data from a single program did not show evidence for gender bias in MMI scores. Although this result is reassuring, it requires continued monitoring and replication.

Multiple Mini-Interviews: Current Perspectives on Utility and Limitations

The growing role of healthcare professionals urged admissions committees to restructure their selection process and assess key personal attributes rather than academic achievements only. Multiple mini interviews (MMIs) were designed in 2002 to assess such domains in prospective healthcare professions. Being a high-stake assessment, the utility and limitations of MMI need to be explored. The purpose of this article is to review the available evidence to establish its utility. The claim of the reliability is verified by the studies assessing the effect of number of stations, duration of stations, format and scoring systems of stations and number of raters assessing the applicants. Similarly, by gathering evidence concerning its content validity, convergent/divergent correlation and predictive ability, validity is ensured. Finally, its acceptability and feasibility along with limitations is discussed. This article concludes by providing recommendations for further work required to deal with the limitations and enhance its utility.

Using the Multiple Mini Interview for selection into vocational general practice training

INTRODUCTION Interviews for selection into post graduate training courses are an accepted method of selection. There is the presumption that such interview processes are fair to both candidates and the training scheme. AIM Due to concerns over unconscious bias and a desire to move to best practice, the Royal New Zealand College of General Practitioners introduced the Mini Multiple Interview (MMI) process as the selection process for doctors wishing to enter vocational training in general practice. METHODS Aspects assessed during the interviews were developed through wide consultation and included: reason for wanting to undertake training, managing poor compliance, addressing issues of equity, managing complaints, insight and understanding the role of general practitioners in chronic care. There were 218 applicants who took the MMI. Demographic data as well as scores were collected. RESULTS The MMI process has good reliability and performs well in several aspects of validity. All three interview venues had similar results. There was no gender difference in overall result or scores. New Zealand graduates scored higher than overseas graduates. Of the 218 candidates, 12 were considered not yet ready to enter training. DISCUSSION The MMI process appears to have acceptable reliability and good validity. The structure of the MMI is likely to have reduced unconscious bias. Further research will study the predictive validity of the MMI for this cohort of candidates.

Assessing noncognitive domains of respiratory therapy applicants: Messick's framework appraisal of the multiple mini-interview

Educators who assess incoming applicants into a health professional training program are looking for a wide array of cognitive and noncognitive skills that best predict success in the program and as a future practicing professional. While aptitude tests generally measure cognitive skills, noncognitive constructs are more difficult to measure appropriately. The traditional method of measuring noncognitive constructs has been the panel interview. Panel interviews have been described as inconsistent in measuring noncognitive domains and consistently reported as unreliable and susceptible to bias. An alternate interview method used in many health professions schools is the multiple mini-interview (MMI) that was specifically designed to assess noncognitive domains in health professions education. This paper discusses the purpose of using the MMI, how the MMI is conducted, specific domains of focus for the MMI, and the feasibility of creating an MMI. Finally, the paper uses Messick's framework on validity to guide the consideration of the MMI.

Does the sequence of rotations in Multiple Mini Interview stations influence the candidates' performance?

In Multiple Mini Interviews (MMIs), the candidates start out with different stations depending on the sequence of rotation they are assigned; thus, their perceived difficulty level and nervousness with their first station may differ. We examined whether such differences influenced the candidates' overall performance in the MMI. A 32-item questionnaire was developed and administered to candidates for admission interviews at a medical school regarding their perceptions of the MMI. Candidates' interview scores were also obtained and were compared across groups who differed in the sequence of rotation of MMI stations. Candidates felt nervous when they encountered the first station, which did not differ regardless of which station was their first encounter. Candidates were neutral regarding whether their first station was more difficult than the others and about whether its difficulty level influenced their overall performance in the test. Although candidates' perceived station difficulty differed across stations, there were no differences in their performance in each station between those it was their first station and those it was not. Candidates' total interview scores also did not differ across groups of different sequence of rotations. The MMI is a fair process as it does not give disadvantage to those who encounter a more difficult station than others for their first station.

Multiple mini interviews: revealing similarities across institutions

BACKGROUND

Across the globe multiple mini interviews (MMIs) have rapidly replaced the use of panel interviews in the selection of medical students and other health professionals. MMIs typically demonstrate better reliability and validity than panel interviews but there is limited research on whether these different types of interview process measure the same or different constructs. Our research aims to ascertain if MMIs are multidimensional or unidimensional, and whether MMIs conducted at different institutions assess the same or different constructs to each other or to panel interviews.

METHODS

Participants were applicants to medical degrees who were shortlisted for interviews at three different institutions in 2013 (n = 165) and 2014 (n = 128). Two institutions used a bespoke MMI developed independently from each other and the third used a panel interview. Stations scores and overall (mean) interview scores were examined.

RESULTS

Exploratory principal components analysis and confirmatory factor analysis showed similar results in both years' data, supporting a unidimensional model. The two overall MMI scores were more strongly correlated to each other (r = .56 and .64 in 2013 and 2014 respectively) than either were to the panel interview scores (r = .07 and .15 in 2013; .39 and .48 in 2014).

CONCLUSIONS

It appears that both MMIs panel interviews tap a single latent construct, but not the same construct. We suggest that the MMI methodology might allow the measurement of an emergent construct such as adaptability.

Vive la Différence: The Freedom and Inherent Responsibilities When Designing and Implementing Multiple Mini-Interviews

The literature on multiple mini-interviews (MMIs) is replete with heterogeneous study results related to the constructs measured, correlations with other measures, and demographic relationships. Rather than view these results as contradictory, the authors ask, What if all of the results are correct? They point out that the MMI is not an assessment tool but, rather, an assessment method. Design and implementation of locally conducted MMIs in medical school admissions processes should reflect local needs. As with other local assessments, MMIs should be considered separate from nationally conducted assessments that reflect more universal competencies. With the freedom to exercise unique values in locally constructed MMIs, individual institutions, or small bands of like-minded institutions, in parallel carry the responsibility to ensure local assessment tool validity.

Personal domains assessed in multiple mini interviews (MMIs) for healthcare student selection: A narrative synthesis systematic review

OBJECTIVES

To examine the personal domains multiple mini interviews (MMIs) are being designed to assess, explore how they were determined and contextualise such domains in current and future healthcare student selection processes DESIGN: A systematic review of empirical research reporting on MMI model design was conducted from database inception to November 2017.

DATA SOURCES

Twelve electronic bibliographic databases.

REVIEW METHODS

Evidence was extracted from original studies, and integrated in a narrative synthesis guided by the PRISMA statement for reporting systematic reviews. Personal domains were clustered into themes using a modified Delphi technique.

RESULTS

A total of 584 articles were screened. 65 unique studies (80 articles) matched our inclusion criteria of which seven were conducted within nursing/midwifery faculties. Six in 10 studies featured applicants to medical school. Across selection processes, we identified 32 personal domains assessed by MMIs, the most frequent being: communication skills (84%), teamwork/collaboration (70%), and ethical/moral judgement (65%). Domains capturing ability to cope with stressful situations (14%), make decisions (14%), and resolve conflict in the workplace (13%) featured in fewer than ten studies overall. Intra- and inter-disciplinary inconsistencies in domain profiles were noted, as well as differences by entry level. MMIs deployed in nursing and midwifery assessed compassion and decision-making more frequently than in all other disciplines. Own programme philosophy and professional body guidance were most frequently cited (~50%) as sources for personal domains; a blueprinting process was reported in only 8% of studies.

CONCLUSIONS

Nursing, midwifery and allied healthcare professionals should develop their theoretical frameworks for MMIs to ensure they are evidence-based and fit-for-purpose. We suggest a re-evaluation of domain priorities to ensure that students who are selected, not only have the capacity to offer the highest standards of care provision, but are able to maintain these standards when facing clinical practice and organisational pressures.

Multiple mini-interview as a predictor of performance in the objective structured clinical examination among Physician Associates in the United Kingdom: a cohort study

INTRODUCTION

Patient satisfaction and health care outcomes are directly linked to useful communication skills. Therefore, excellent interpersonal skills are imperative for health care professionals. Multiple mini-interview (MMI) is designed as a selection tool to assess the communication skills of applicants in medical schools during the admission process. However, objective structured clinical examination (OSCE) assesses students' communication and clinical skills at the end of their academic terms. Recently, Anglia Ruskin University, Chelmsford, UK, adopted MMI in the selection process for the first cohort of MSc Physician Associate trainees for the academic year 2015-2016. This study aimed to determine the likelihood of MMI as a predictor of future performance of communication skills in the OSCE.

MATERIALS AND METHODS

The anonymous data of the average scores of communication skills attained in MMI and OSCE at the end of year 1 were collected for 30 students from the Physician Associate program team. Subsequently, Pearson's correlation was computed to determine the relationship between the average scores of communication skills attained in MMI, and OSCE during trimester 2 and trimester 3 by the Physician Associate trainees.

RESULTS

The study showed positive correlation between the scores of communication skills attained in MMI and OSCE during trimester 2 (r=0.956, n=30, p<0.001) and trimester 3 (r=0.966, n=30, p<0.001).

CONCLUSION

The study provides empirical evidence for the validity of MMI as a predictor of future performance of Physician Associate trainees' communication skills during subsequent OSCEs.

Predicting performance in health professions education programs from admissions information - Comparisons of other health professions with pharmacy

BACKGROUND

The goal of the present review was to assess the state of performance prediction in healthcare programs generally, versus performance prediction in pharmacy schools, using didactic and non-didactic admissions measures. This is important because clinical success represents a combination of skills that are not fully predicted by either type of measure alone.

METHODS

PubMed searches were conducted focusing on work published from 2000 onwards, since it is during this period that non-didactic admissions measures have come to be incorporated into the applicant evaluation process. Relevant free full text papers available were used. When these papers were not available by direct import into EndNote, we went directly to the journal to try to retrieve the paper.

RESULTS

We acknowledge that health professions programs have been successful in recruiting excellent candidates into their schools. However, based on the modest amount of healthcare program performance accounted for by didactic measures, admissions committees should consider expanding their holistic evaluation of applicants. Schools would benefit from using two-step screening phases in the application process - perhaps evaluating didactic potential in phase 1 and experiential in phase 2. Using combination measures throughout the admission process should help ensure admission of students more likely to be successful throughout their healthcare practice.

IMPLICATIONS

Future investigations of the prediction of healthcare program performance by formal combinations of didactic and non-didactic admissions measures are imperative. In addition, it is likely that combination admission measures will incorporate more metrics of critical thinking than do simpler approaches. Furthermore, systematic evaluation of the usefulness of the two-step screening approaches to admissions used by most competitive health professions programs also needs to be done.

The utility of multiple mini-interviews: experience of a medical school

PURPOSE

This paper aims to introduce the design of multiple mini-interviews (MMIs) as a tool to assess medical school applicants' attributes in alignment with the school's educational goals and to evaluate its utility.

METHODS

In this MMI, candidates rotated through six stations (10 minutes per station), in which specific interview topics were drawn by mapping the school's educational goals with the core competencies for entering medical students. We conducted post-MMI surveys of all of the interviewers and candidates to investigate their experiences of MMIs. The G-coefficient and interclass correlation were analyzed to investigate the reliability of this test. Additionally, the candidates' MMI scores were compared across different backgrounds and a univariate analysis was used to estimate correlations between their MMI scores and prior academic achievements.

RESULTS

A total of 164 candidates (a 98.8% response rate) and 19 interviewers (a 100% response rate) completed the surveys in the years 2014 and 2015. Both candidates and assessors showed positive responses to MMIs. The G-coefficient of MMI scores was 0.88 and the interclass correlation coefficients ranged from 0.58 to 0.75. The participants' total MMI scores did not differ across genders or undergraduate backgrounds and were not associated with age, undergraduate graduate point averages, nor the Korean medical school admission test (Medical Education Eligibility Test) scores.

CONCLUSION

Our study illustrates the utility of MMIs that utilize the institution's educational goals to identify attributes to be pursued in the admission interviews in alignment with the institution's core values. Future research is warranted of the predictive validity of this MMI.