The validity of a behavioural multiple-mini-interview within an assessment centre for selection into specialty training

'Selection into training will always be an inexact process': A survey of Directors of Physician Education on selection into Basic Physician Training in Australia and New Zealand

BACKGROUND

Despite being one of the largest medical specialty training programmes in Australasia, there is no standardised method for selection into Basic Physician Training (BPT), and limited data exist regarding current practices.

AIMS

To address existing knowledge gaps, we aimed to create a 'snapshot' of current BPT selection practices and explore the perspectives of Directors of Physician Education (DPEs) regarding trainee selection.

METHODS

An electronic survey of DPEs from adult and paediatric medicine BPT sites in Australia and New Zealand was undertaken in January-February 2022. A combination of free text, multiple-choice and yes/no answers were analysed using descriptive statistics and qualitative content analysis.

RESULTS

A total of 70 responses were received, achieving a response rate of 35% (70/198). Selection practices were found to be heterogenous across BPT sites. Respondents had varying opinions regarding the utility of selection tools and desirable candidate attributes. A heavy reliance upon interviews and the reported use of subjective assessments raise concerns for selection process bias.

CONCLUSION

BPT sites should critically evaluate their selection methods, and more research in this field is needed to establish best practice.

Different approaches to selection of surgical trainees in the European Union

BACKGROUND

There is an increasing global interest in selection processes for candidates to surgical training. The aim of the present study is to compare selection processes to specialist surgeon training in the European Union (EU). A secondary goal is to provide guidance for evidence-based methods by a proposed minimum standard that would align countries within the EU.

METHODS

Publications and grey literature describing selection strategies were sought. Correspondence with Union Européenne des Médecins Specialists (UEMS) Section of Surgery delegates was undertaken to solicit current information on national selection processes. Content analysis of 13 semi-structured interviews with experienced Swedish surgeons on the selection process. Two field trips to Ireland, a country with a centralized selection process were conducted. Based on collated information typical cases of selection in a centralized and decentralized setting, Ireland and Sweden, are described and compared.

RESULTS

A multitude of methods for selection to surgical training programs were documented in the 27 investigated countries, ranging from locally run processes with unstructured interviews to national systems for selection of trainees with elaborate structured interviews, and non-technical and technical skills assessments. Associated with the difference between centralized and decentralized selection systems is whether surgical training is primarily governed by an employment or educational logic. Ireland had the most centralized and elaborate system, conducting a double selection process using evidence-based methods along an educational logic. On the opposite end of the scale Sweden has a decentralized, local selection process with a paucity of evidence-based methods, no national guidelines and operates along an employment logic, and Spain that rely solely on examination tests to rank candidates.

CONCLUSION

The studied European countries all have different processes for selection of surgical trainees and the use of evidence-based methods for selection is variable despite similar educational systems. Selection in decentralized systems is currently often conducted non-transparent and subjectively. A suggested improvement towards an evidence-based framework for selection applicable in centralized and decentralized systems as well as educational and employer logics is suggested.

Role of formative assessment in predicting academic success among GP registrars: a retrospective longitudinal study

OBJECTIVES

The James Cook University General Practice Training (JCU GPT) programme's internal formative exams were compared with the Royal Australian College of General Practitioners (RACGP) pre-entry exams to determine ability to predict final performance in the RACGP fellowship exams.

DESIGN

A retrospective longitudinal study.

SETTING

General Practice (GP) trainees enrolled between 2016 and 2019 at a Registered Training Organisation in regional Queensland, Australia.

PARTICIPANTS

376 GP trainees enrolled in the training programme.

EXPOSURE MEASURES

The pre-entry exams were Multiple-Mini Interviews (MMI), Situational Judgement Test (SJT) and Candidate Assessment and Applied Knowledge Test. The internal formative exams comprised multiple choice questions (MCQ1 and MCQ2), short answer questions, clinical skills and clinical reasoning.

PRIMARY OUTCOME MEASURE

The college exams were Applied Knowledge Test (AKT), Key Feature Problems (KFP) and Objective Structured Clinical Examination (OSCE).

RESULTS

Correlations (r), coefficients of determination (R²) and OR were used as parameters for estimating strength of relationship and precision of predictive accuracy. SJT and MMI were moderately (r=0.13 to 0.31) and MCQ1 and MCQ2 highly (r=0.37 to 0.53) correlated with all college exams (p<0.05 to p<0.01), with R² ranging from 0.070 to 0.376. MCQ1 was predictive of failure in all college exams (AKT: OR=2.32, KFP: OR=3.99; OSCE: OR=3.46); while MCQ2 predicted failure in AKT (OR=2.83) and KFP (OR=3.15).

CONCLUSION

We conclude that the internal MCQ formative exams predict performance in the RACGP fellowship exams. We propose that our formative assessment tools could be used as academic markers for early identification of potentially struggling trainees.

[Situational judgement test as teaching method for the critical discussion on scientific practice and misconduct]

Wissenschaftskompetenz ist eine Schlüsselqualifikation für jede ärztliche Tätigkeit und sollte ebenso wie die Auseinandersetzung mit Entscheidungsprozessen von Beginn an ins Medizinstudium integriert werden. Ziel der Studie war, die Themen der guten wissenschaftlichen Praxis und des wissenschaftlichen Fehlverhaltens zu vermitteln. Ferner wurde durch die methodische Intervention „Gruppendiskussion“ eine Reflexion im Kontext der wissenschaftlichen Angemessenheit herbeigeführt. Hierfür wurde der Situational Judgement Test (SJT) von den Studierenden (N = 743) (individuell und in der Gruppe) bearbeitet, und dessen Resultate wurden mit den Antworten von Expert/innen/en (N = 23) verglichen. Nach der Gruppendiskussion näherten sich die Ergebnisse in der Verteilung und Reihenfolge den Antwortmöglichkeiten der Expert/innen/en an. Jedoch tendierten die Studierenden signifikant häufiger zu jenen Antworten, die hilfesuchende, passive und verantwortungsübertragende Optionen bedeuteten. Insgesamt hat sich der SJT als didaktische Intervention bewährt. Die Studierenden setzten sich aktiv mit den Themen auseinander, eine Diskussion konnte angeregt und das eigene Verhalten kritisch reflektiert werden.

Situational judgement test validity for selection: A systematic review and meta-analysis

CONTEXT

Situational judgement tests (SJTs) are widely used to evaluate 'non-academic' abilities in medical applicants. However, there is a lack of understanding of how their predictive validity may vary across contexts. We conducted a systematic review and meta-analysis to synthesise existing evidence relating to the validity of such tools for predicting outcomes relevant to interpersonal workplace performance.

METHODS

Searches were conducted in relevant databases to June 2019. Study quality and risk of bias were assessed using the Quality In Prognosis Studies (QUIPS) tool. Results were pooled using random effects meta-analysis and meta-regressions.

RESULTS

Initially, 470 articles were identified, 218 title or abstracts were reviewed, and 44 full text articles were assessed with 30 studies meeting the final inclusion criteria and were judged, overall, to be at moderate risk of bias. Of these, 26 reported correlation coefficients relating to validity, with a pooled estimate of 0.32 (95% confidence interval 0.26 to 0.39, P < .0001). Considerable heterogeneity was observed (I²  = 96.5%) with the largest validity coefficients tending to be observed for postgraduate, rather than undergraduate, selection studies (β = 0.23, 0.11 to 0.36, P < .001). The correction of validity coefficients for attenuation was also independently associated with larger effects (β = 0.13, 0.03 to 0.23, P = .01). No significant associations with test medium (video vs text format), cross-sectional study design, or period of assessment (one-off vs longer-term) were observed. Where reported, the scores generally demonstrated incremental predictive validity, over and above tests of knowledge and cognitive ability.

CONCLUSIONS

The use of SJTs in medical selection is supported by the evidence. The observed trend relating to training stage requires investigation. Further research should focus on developing robust criterion-relevant outcome measures that, ideally, capture interpersonal aspects of typical workplace performance. This will facilitate additional work identifying the optimal place of SJTs within particular selection contexts and further enhancing their effectiveness.

Employing Multiple Mini Interviews in Selection Processes for Psychology Professional Training Programs: Ten Tips for Effective Implementation

This article was migrated. The article was marked as recommended. In Australia and internationally, entry into many professional psychology training programs is highly competitive, and as a duty of care to the public, training institutions must seek to offer places to those best-suited to training to become a psychologist. Typically, part of this selection process involves an interview. While panel interviews have been widely utilised, recently, in a range of health disciplines, these have been substituted for multiple mini interviews (MMIs) with evidence for their acceptability, validity and reliability. There is limited research on the use of MMIs in psychology professional training selection processes. We have used this approach for three years to select approximately 100 trainees for Clinical, Health and Organisational and Human Factors postgraduate psychology training programs. Based on our experience and feedback from applicants, we provide information that suggests this selection method is well-received by applicants. We also provide ten tips on how to effectively implement this approach to determine those most suitable for further training.

Development and piloting of a Situational Judgement Test for emotion-handling skills using the Verona Coding Definitions of Emotional Sequences (VR-CoDES)

OBJECTIVE

Emotion-handling skills are key components for interpersonal communication by medical professionals. The Verona Coding Definitions of Emotional Sequences (VR-CoDES) appears useful to develop a Situational Judgment Test (SJT) for assessing emotion-handling skills.

METHODS

In phase 1 we used a multi-stage process with expert panels (n_panel1 = 16; n_panel2 = 8; n_panel3 = 20) to develop 12 case vignettes. Each vignette includes (1) video representing a critical incident containing concern(s) and/or cue(s), (2) standardized lead-in-question, (3) five response alternatives. In phase 2 we piloted the SJT to assess validity via an experimental study with medical students (n = 88).

RESULTS

Experts and students rated most of the 'Reduce space' responses as inappropriate and preferred 'Explicit' responses. Women scored higher than men and there was no decline of empathy according to students' year of study. There were medium correlations with self-assessment instruments. The students' acceptance of the SJT was high.

CONCLUSION

The use of VR-CoDES, authentic vignettes, videos and expert panels contributed to the development and validity of the SJT.

PRACTICE IMPLICATIONS

Development costs were high but could be made up over time. The agreement on a proper score and the implementation of an adequate feedback structure seem to be useful.

The predictive validity of Multiple Mini Interviews (MMIs) in nursing and midwifery programmes: Year three findings from a cross-discipline cohort study

BACKGROUND

Education literature worldwide is replete with studies evaluating the effectiveness of Multiple Mini Interviews (MMIs) in admissions to medicine but <1% of published studies have been conducted in selection to nursing and midwifery programmes.

OBJECTIVES

To examine the predictive validity of MMIs using end of programme clinical and academic performance indicators of pre-registration adult, child, and mental health nursing and midwifery students.

DESIGN AND SETTING

A cross-sectional cohort study at one university in the United Kingdom.

PARTICIPANTS

A non-probability consecutive sampling strategy whereby all applicants to the September 2015 pre-registration adult, child, mental health nursing and midwifery programmes were invited to participate. Of the 354 students who commenced year one, 225 (64%) completed their three-year programme and agreed to take part (adult 120, child 32, mental health nursing 30 and midwifery 43).

METHODS

All applicants were interviewed using MMIs with six and seven station, four-minute models deployed in nursing and midwifery student selection respectively. Associations between MMI scores and the cross-discipline programme performance indicators available for each student at this university at the end of year three: clinical practice (assessed by mentors) and academic attainment (dissertation mark) were explored using multiple linear regression adjusting for applicant age, academic entry level, discipline and number of MMI stations.

RESULTS

In the adjusted models, students with higher admissions MMI score (at six and seven stations) performed better in clinical practice (p < 0.001) but not in academic attainment (p = 0.122) at the end of their three-year programme.

CONCLUSION

These findings provide the first report of the predictive validity of MMIs for performance in clinical practice using six and seven station models in nursing and midwifery programmes. Further evidence is required from both clinical and academic perspectives from larger, multi-site evaluations.

Multiple Speed Assessments

Abstract. This paper presents Multiple Speed Assessments as an umbrella term to encompass a variety of approaches that include multiple (e.g., 20), short (e.g., 3 min), and often integrated interpersonal simulations to elicit overt behavior in a standardized way across participants. Multiple Speed Assessments can be used to get insight into the behavioral repertoire of a target person in situations sampled from a predefined target domain and their intraindividual variability across these situations. This paper outlines the characteristics and theoretical basis of Multiple Speed Assessments. We also discuss various already existing examples of Multiple Speed Assessments (Objective Structured Clinical Examinations, Multiple Mini-Interviews, and constructed response multimedia tests) and provide an overview of design variations. Finally, we present current research evidence and future research directions related to Multiple Speed Assessments. Although we present Multiple Speed Assessments in the context of personnel selection, it can also be used for assessment in the educational, personality, or clinical psychology field.

Rethinking the Admissions Interview: Piloting Multiple Mini-Interviews in a Graduate Psychology Program

Health profession programs routinely utilize traditional interviews in admissions as a means of assessing important non-academic characteristics (e.g., critical thinking, interpersonal skills, judgment) of candidates. However, the reliability and validity of traditional interviews is highly questionable. Given this, multiple health profession programs (e.g., medicine, nursing, pharmacy, physical therapy) have implemented multiple mini-interviews as an alternative for assessing non-academic characteristics. This paper describes the development and implementation of multiple mini-interviews in the admissions process for a doctoral clinical psychology program, one of the health professions yet to use multiple mini-interviews. This paper also examines the feasibility and acceptability of the multiple mini-interviews in this program. Results of a mixed-method survey of all 120 candidates who participated in admissions days are presented along with discussion of factors associated with satisfaction and dissatisfaction. Recommendations for program refinement and application to other graduate psychology programs for improved admissions processes are discussed.

Multiple Mini-Interviews: Current Perspectives on Utility and Limitations

The growing role of healthcare professionals urged admissions committees to restructure their selection process and assess key personal attributes rather than academic achievements only. Multiple mini interviews (MMIs) were designed in 2002 to assess such domains in prospective healthcare professions. Being a high-stake assessment, the utility and limitations of MMI need to be explored. The purpose of this article is to review the available evidence to establish its utility. The claim of the reliability is verified by the studies assessing the effect of number of stations, duration of stations, format and scoring systems of stations and number of raters assessing the applicants. Similarly, by gathering evidence concerning its content validity, convergent/divergent correlation and predictive ability, validity is ensured. Finally, its acceptability and feasibility along with limitations is discussed. This article concludes by providing recommendations for further work required to deal with the limitations and enhance its utility.

The 'values journey' of nursing and midwifery students selected using multiple mini interviews: Evaluations from a longitudinal study

Values-based practice is deemed essential for healthcare provision worldwide. In England, values-based recruitment methods, such as multiple mini interviews (MMIs), are employed to ensure that healthcare students' personal values align with the values of the National Health Service (NHS), which focus on compassion and patient-centeredness. However, values cannot be seen as static constructs. They can be positively and negatively influenced by learning and socialisation. We have conceptualised students' perceptions of their values over the duration of their education programme as a 'values journey'. The aim of this hermeneutic longitudinal focus group study was to explore the 'values journey' of student nurses and midwives, recruited through MMIs, across the 3 years of their education programme. The study commenced in 2016, with 42 nursing and midwifery students, originally recruited onto their programmes through multiple mini interviews. At the third and final point of data collection, 25 participants remained. Findings indicate that students' confidence, courage and sense of accountability increased over the 3 years. However, their values were also shaped by time constraints, emotional experiences and racial discrimination. We argue that adequate psychological support is necessary as healthcare students embark on and progress through their values journey, and propose a framework for this.

Multiple Mini Interview as an admission tool in higher education: Insights from a systematic review

Objectives

Multiple Mini Interviews (MMI) have been conducted across the globe in the student selection process, particularly in health profession education. This paper reported the validity evidence of MMI in various educational settings.

Methods

A literature search was carried out through Scopus, Science Direct, Google Scholar, PubMed, and EBSCOhost databases based on specific search terms. Each article was appraised based on title, abstract, and full text. The selected articles were critically appraised, and relevant information to support the validity of MMI in various educational settings was synthesized. This paper followed the PRISMA guideline to ensure consistency in reporting systematic review results.

Results

A majority of the studies were from Canada, with 41.54%, followed by the United Kingdom (25.39%), the United States (13.85%), and Australia (9.23%). The rest (9.24%) were from Germany, Ireland, the United Arab Emirates, Japan, Pakistan, Taiwan, and Malaysia. Moreover, most MMI stations ranged from seven to 12 with a duration of 10 min per station (including a 2-min gap between stations).

Conclusion

The results suggest that the content, response process, and internal structure of MMI were well supported by evidence; however, the relation and consequences of MMI to important outcome variables were inconsistently supported. The evidence shows that MMI is a non-biased, practical, feasible, reliable, and content-valid admission tool. However, further research on its impact on non-cognitive outcomes is required.

Using the Multiple Mini Interview for selection into vocational general practice training

INTRODUCTION Interviews for selection into post graduate training courses are an accepted method of selection. There is the presumption that such interview processes are fair to both candidates and the training scheme. AIM Due to concerns over unconscious bias and a desire to move to best practice, the Royal New Zealand College of General Practitioners introduced the Mini Multiple Interview (MMI) process as the selection process for doctors wishing to enter vocational training in general practice. METHODS Aspects assessed during the interviews were developed through wide consultation and included: reason for wanting to undertake training, managing poor compliance, addressing issues of equity, managing complaints, insight and understanding the role of general practitioners in chronic care. There were 218 applicants who took the MMI. Demographic data as well as scores were collected. RESULTS The MMI process has good reliability and performs well in several aspects of validity. All three interview venues had similar results. There was no gender difference in overall result or scores. New Zealand graduates scored higher than overseas graduates. Of the 218 candidates, 12 were considered not yet ready to enter training. DISCUSSION The MMI process appears to have acceptable reliability and good validity. The structure of the MMI is likely to have reduced unconscious bias. Further research will study the predictive validity of the MMI for this cohort of candidates.

Twelve tips for constructing a multiple mini-interview

Health professions the world over value various competencies in their practitioners that are not easily captured by academic measures of performance. As a result, many programs have begun using multiple mini-interviews (MMIs) to facilitate the selection of candidates who are most likely to demonstrate and further develop such qualities. In this twelve-tips article, the authors offer evidence- and experience-based advice regarding how to construct an MMI that is fit for purpose. The tips are provided chronologically, offering guidance regarding how one might conceptualize their goals for creating an MMI, how to establish a database of stations that are context appropriate, and how to prepare both candidates and examiners for their task. While MMIs have been shown to have utility in many instances, the authors urge caution against over-generalization by stressing the importance of post-MMI considerations including data monitoring and integration between one's admissions philosophy and one's curricular efforts.

Do changing medical admissions practices in the UK impact on who is admitted? An interrupted time series analysis

INTRODUCTION

Medical admissions must balance two potentially competing missions: to select those who will be successful medical students and clinicians and to increase the diversity of the medical school population and workforce. Many countries address this dilemma by reducing the heavy reliance on prior educational attainment, complementing this with other selection tools. However, evidence to what extent this shift in practice has actually widened access is conflicting.

AIM

To examine if changes in medical school selection processes significantly impact on the composition of the student population.

DESIGN AND SETTING

Observational study of medical students from 18 UK 5-year medical programmes who took the UK Clinical Aptitude Test from 2007 to 2014; detailed analysis on four schools.

PRIMARY OUTCOME

Proportion of admissions to medical school for four target groups (lower socioeconomic classes, non-selective schooling, non-white and male).

DATA ANALYSIS

Interrupted time-series framework with segmented regression was used to identify the impact of changes in selection practices in relation to invitation to interview to medical school. Four case study medical schools were used looking at admissions within for the four target groups.

RESULTS

There were no obvious changes in the overall proportion of admissions from each target group over the 8-year period, averaging at 3.3% lower socioeconomic group, 51.5% non-selective school, 30.5% non-white and 43.8% male. Each case study school changed their selection practice in decision making for invite to interview during 2007-2014. Yet, this within-school variation made little difference locally, and changes in admission practices did not lead to any discernible change in the demography of those accepted into medical school.

CONCLUSION

Although our case schools changed their selection procedures, these changes did not lead to any observable differences in their student populations. Increasing the diversity of medical students, and hence the medical profession, may require different, perhaps more radical, approaches to selection.

Personal domains assessed in multiple mini interviews (MMIs) for healthcare student selection: A narrative synthesis systematic review

OBJECTIVES

To examine the personal domains multiple mini interviews (MMIs) are being designed to assess, explore how they were determined and contextualise such domains in current and future healthcare student selection processes DESIGN: A systematic review of empirical research reporting on MMI model design was conducted from database inception to November 2017.

DATA SOURCES

Twelve electronic bibliographic databases.

REVIEW METHODS

Evidence was extracted from original studies, and integrated in a narrative synthesis guided by the PRISMA statement for reporting systematic reviews. Personal domains were clustered into themes using a modified Delphi technique.

RESULTS

A total of 584 articles were screened. 65 unique studies (80 articles) matched our inclusion criteria of which seven were conducted within nursing/midwifery faculties. Six in 10 studies featured applicants to medical school. Across selection processes, we identified 32 personal domains assessed by MMIs, the most frequent being: communication skills (84%), teamwork/collaboration (70%), and ethical/moral judgement (65%). Domains capturing ability to cope with stressful situations (14%), make decisions (14%), and resolve conflict in the workplace (13%) featured in fewer than ten studies overall. Intra- and inter-disciplinary inconsistencies in domain profiles were noted, as well as differences by entry level. MMIs deployed in nursing and midwifery assessed compassion and decision-making more frequently than in all other disciplines. Own programme philosophy and professional body guidance were most frequently cited (~50%) as sources for personal domains; a blueprinting process was reported in only 8% of studies.

CONCLUSIONS

Nursing, midwifery and allied healthcare professionals should develop their theoretical frameworks for MMIs to ensure they are evidence-based and fit-for-purpose. We suggest a re-evaluation of domain priorities to ensure that students who are selected, not only have the capacity to offer the highest standards of care provision, but are able to maintain these standards when facing clinical practice and organisational pressures.

Utility of selection methods for specialist medical training: A BEME (best evidence medical education) systematic review: BEME guide no. 45

BACKGROUND

Selection into specialty training is a high-stakes and resource-intensive process. While substantial literature exists on selection into medical schools, and there are individual studies in postgraduate settings, there seems to be paucity of evidence concerning selection systems and the utility of selection tools in postgraduate training environments.

AIM

To explore, analyze and synthesize the evidence related to selection into postgraduate medical specialty training.

METHOD

Core bibliographic databases including PubMed; Ovid Medline; Embase, CINAHL; ERIC and PsycINFO were searched, and a total of 2640 abstracts were retrieved. After removing duplicates and screening against the inclusion criteria, 202 full papers were coded, of which 116 were included.

RESULTS

Gaps in underlying selection frameworks were illuminated. Frameworks defined by locally derived selection criteria, and heavily weighed on academic parameters seem to be giving way to the evidencing of competency-based selection approaches in some settings. Regarding selection tools, we found favorable psychometric evidence for multiple mini-interviews, situational judgment tests and clinical problem-solving tests, although the bulk of evidence was mostly limited to the United Kingdom. The evidence around the robustness of curriculum vitae, letters of recommendation and personal statements was equivocal. The findings on the predictors of past performance were limited to academic criteria with paucity of long-term evaluations. The evidence around nonacademic criteria was inadequate to make an informed judgment.

CONCLUSIONS

While much has been gained in understanding the utility of individual selection methods, though the evidence around many of them is equivocal, the underlying theoretical and conceptual frameworks for designing holistic and equitable selection systems are yet to be developed.

Factors related to doctors' choice of rural pathway in general practice specialty training

OBJECTIVE

To investigate the factors eligible applicants consider in electing for a rural pathway into specialty training.

DESIGN

Cohort study.

SETTING

Australia.

PARTICIPANTS

Applicants to the Australian General Practice Training program.

MAIN OUTCOME MEASURES

Applicants' initial preference of either a general or rural pathway to undertake specialty training.

RESULTS

Of the 2,221 applicants, 45% were Australian Medical Graduates (AMGs), 27% Foreign Graduates of Accredited Medical Schools (FGAMS) and 29% International Medical Graduates (IMGs). Through government regulation, two thirds (70%) were eligible to train on both general and rural pathways and a third (30%) were required to train rurally. For applicants eligible for general pathway (n = 1552), those with rural background [Odds Ratio (OR) = 3.7, 95% CI 2.7-5.2] and rural clinical school experience (OR = 2.0, 95% CI 1.5-2.8) were more likely to choose the rural pathway. In addition, FGAMS who were eligible for the general pathway were less likely to choose a rural pathway when compared with IMGs (OR = 0.33, 95%CI 0.1 = 0.7). In applicants who changed their training pathway from their initial to revised preference, lower Multiple-Mini-Interview (OR = 0.54, 95% CI 0.43-0.66) and Situational Judgement Test z-scores (OR = 0.68, 95% CI 0.56-0.83) were associated with a higher probability of changing from a general to rural pathway preference.

CONCLUSION

For those eligible for a general or rural pathway, rural background and rural clinical school experience are associated with the decision to elect for rural training. Targeted support for international and foreign graduates of Australia/New Zealand schools may influence them to train rurally.

Correlation between MMI performance and OSCE performance - a pilot study

OBJECTIVE

The multiple mini-interview (MMI) has been shown to have a positive correlation with early medical school performance, clerkship evaluations, and national licensing examinations. There is limited data on its predictive validity at the postgraduate level.

METHODS

Six hundred and nineteen internship candidates were interviewed using the MMI format by the internal medicine residency program of The Reading Health System, between September 2011 and February 2014. Fifty-two interns were recruited. Each intern participated in an objective structured clinical examination (OSCE) 3-4 months after the start of the program. The OSCE score of each intern was used as the independent variable to test the relationship with both the MMI interpersonal score and the MMI overall score.

RESULTS

There was a moderate positive correlation between the average MMI interpersonal score and the communication score on the OSCE, r=0.384, n=52, p=0.005, and a negligible relationship between the average MMI overall score and the communication score on the OSCE, r=0.175, n=52, p=0.214.

CONCLUSION

The MMI is a useful tool for residency programs to assess interpersonal and communication skills prior to matriculation into residency training. This study provides evidence for its validity in assessing these competencies.

The social validity of a national assessment centre for selection into general practice training

BACKGROUND

Internationally, recruiting the best candidates is central to the success of postgraduate training programs and the quality of the medical workforce. So far there has been little theoretically informed research considering selection systems from the perspective of the candidates. We explored candidates' perception of the fairness of a National Assessment Centre (NAC) approach for selection into Australian general practice training, where candidates were assessed by a Multiple Mini Interview (MMI) and a written Situational Judgment Test (SJT), for suitability to undertake general practice (GP) training.

METHODS

In 2013, 1,930 medical practitioners, who were eligible to work in Australia attended one of 14 NACs in each of 5 states and 2 territories. A survey was distributed to each candidate at the conclusion of their assessment, which included open-ended questions aimed at eliciting candidates' perceived benefits and challenges of the selection process. A framework analysis was informed by the theoretical lens of Social Validity Theory.

RESULTS

Qualitative data was available from 46% (n = 886/1,930) of candidates, who found the NAC experience fair and informative for their training and career goals, but wanted to be provided with more information in preparation. Candidates valued being able to communicate their skills during the MMI, but found some difficulty in interpreting the questions. A significant minority had concerns that a lack of relevant GP experience may inhibit their performance. Candidates also expressed concerns about the time limits within the written paper, particularly if English was not their first language. They also expressed a desire for formative feedback during the interview process.

CONCLUSION

During any job selection process, not only is the organisation assessing the candidates, but the candidates are also assessing the organisation. However, a focus on the candidate experience throughout an organisation's selection process may provide benefits to both candidates and the organisation, regardless of whether or not candidates secured the job. Social Validity Theory is a useful addition to the methods for demonstrating the reasonableness of any selection system.