A generalizability analysis of a veterinary school Multiple Mini Interview: effect of number of interviewers, type of interviewers, and number of stations

Part 2: Assessment of a virtual vs. onsite interview experience from the interviewer perspective

INTRODUCTION

The University of Texas at Austin College of Pharmacy transitioned from onsite interviews to virtual interviews in fall 2020. There is limited literature on whether the virtual format impacts an interviewer's assessment of a candidate. This study examined interviewer ability to assess candidates and barriers to participation.

METHODS

During the virtual interview process, interviewers utilized a modified multiple mini interview (mMMI) format to evaluate prospective college of pharmacy students. An 18-item survey was emailed to 62 interviewers from the 2020-2021 cycle. Virtual mMMI scores were compared to the previous year's onsite MMI scores. Descriptive statistics and thematic analysis were used to assess the data.

RESULTS

The response rate to the survey was 53% (33/62), and 59% of interviewers preferred virtual interviews to in-person. Interviewers stated that there were fewer barriers to participation, increased comfort, and more time with applicants during virtual interviews. For six of the nine attributes evaluated, ≥ 90% of interviewers reported that they were able to assess applicants as well as they did in person. When comparing virtual and onsite MMI scores, seven of nine attributes were statistically significantly higher in the virtual cohort than onsite.

CONCLUSIONS

From the perspective of interviewers, the virtual interview decreased barriers to participation and still allowed the ability to assess the candidates. While offering a choice of interview setting to interviewers could increase accessibility, the statistically significant difference in MMI scores between virtual and onsite formats suggests that additional standardization is required to offer both formats simultaneously.

Construct validity of multiple mini interviews - Investigating the role of stations, skills, and raters using Bayesian G-theory

Background: One popular procedure in the medical student selection process are multiple mini-interviews (MMIs), which are designed to assess social skills (e.g., empathy) by means of brief interview and role-play stations. However, it remains unclear whether MMIs reliably measure desired social skills or rather general performance differences that do not depend on specific social skills. Here, we provide a detailed investigation into the construct validity of MMIs, including the identification and quantification of performance facets (social skill-specific performance, station-specific performance, general performance) and their relations with other selection measures.Methods: We used data from three MMI samples (N = 376 applicants, 144 raters) that included six interview and role-play stations and multiple assessed social skills.Results: Bayesian generalizability analyses show that, the largest amount of reliable MMI variance was accounted for by station-specific and general performance differences between applicants. Furthermore, there were low or no correlations with other selection measures.Discussion: Our findings suggest that MMI ratings are less social skill-specific than originally conceptualized and are due more to general performance differences (across and within-stations). Future research should focus on the development of skill-specific MMI stations and on behavioral analyses on the extents to which performance differences are based on desirable skills versus undesired aspects.

Multiple Mini-Interviews: Current Perspectives on Utility and Limitations

The growing role of healthcare professionals urged admissions committees to restructure their selection process and assess key personal attributes rather than academic achievements only. Multiple mini interviews (MMIs) were designed in 2002 to assess such domains in prospective healthcare professions. Being a high-stake assessment, the utility and limitations of MMI need to be explored. The purpose of this article is to review the available evidence to establish its utility. The claim of the reliability is verified by the studies assessing the effect of number of stations, duration of stations, format and scoring systems of stations and number of raters assessing the applicants. Similarly, by gathering evidence concerning its content validity, convergent/divergent correlation and predictive ability, validity is ensured. Finally, its acceptability and feasibility along with limitations is discussed. This article concludes by providing recommendations for further work required to deal with the limitations and enhance its utility.

An Exploration of the Relationships Between Multiple Mini-Interview Scores and Personality Traits

Objective. To explore the relationship between pharmacy students' social and behavioral competencies assessed using multiple mini-interviews (MMIs) and their personality traits. Methods. A retrospective study was conducted using data from two Doctor of Pharmacy (PharmD) cohorts. Students' self-reported scores from an externally validated tool completed during their first professional year that assessed five personality traits (openness to experiences, conscientiousness, extraversion, agreeableness, and emotionality) were collected. Students' scores on empathy, integrity, adaptability, critical thinking, and collaboration were collected during MMIs conducted during the admissions process. Analysis of variance was used to test the homogeneity of the sample, and correlation analyses were used to determine the strength of relationships between variables. Results. When examining for associations between students' personality traits and their social and behavioral competencies measured during the MMIs, we found a few significant but very weak correlations with extraversion. Conclusion. Our results revealed weak to negligible correlations between scores on competencies measured during the MMIs and personality traits, potentially because the assessments measured different constructs.

Twelve tips for constructing a multiple mini-interview

Health professions the world over value various competencies in their practitioners that are not easily captured by academic measures of performance. As a result, many programs have begun using multiple mini-interviews (MMIs) to facilitate the selection of candidates who are most likely to demonstrate and further develop such qualities. In this twelve-tips article, the authors offer evidence- and experience-based advice regarding how to construct an MMI that is fit for purpose. The tips are provided chronologically, offering guidance regarding how one might conceptualize their goals for creating an MMI, how to establish a database of stations that are context appropriate, and how to prepare both candidates and examiners for their task. While MMIs have been shown to have utility in many instances, the authors urge caution against over-generalization by stressing the importance of post-MMI considerations including data monitoring and integration between one's admissions philosophy and one's curricular efforts.

Is a veterinary student's performance on multiple-mini interviews affected by personality preferences?

OBJECTIVES

The objectives of this study were to evaluate the association between a student's Myers-Briggs Type Indicator (MBTI) preference pairs and resulting types and his or her multiple-mini interview (MMI) scores upon admission, and to determine the proportions of types among veterinary classes over five years.

METHODS

A cross-sectional study was conducted for data collected from 706 students admitted into the University of California Davis School of Veterinary Medicine (UCDSVM) program beginning in the fall of 2013 and ending in the fall of 2018. Data consisted of a candidate's MBTI preference pairs and types which were collected during the first week of enrollment and multiple-mini interview scores from his or her admission data.

RESULTS

A total of 706 students from 5 classes completed the MBTI survey. Multivariate analysis showed no significant association between the MBTI preference pairs of extroversion and introversion (F(_{1, 697)} = 3.30, p=0.0959), sensing and intuition (F_{(1, 697)} = 0.40, p=0.4395), thinking and feeling (F_{(1, 697)} = 3.59, p=0.0591), or judging and perceiving (F_{(1, 697)} = 0.38, p = 0.5657) and MMI score. Analysis showed no trends (χ² (60, N=706) =76.51, p=0.074) in the student's MBTI types over the 5-year period.

CONCLUSIONS

The MMI score of a candidate admitted into the UCDSVM is unlikely to be affected by personality preferences. Therefore, it is unlikely that multiple-mini interview scores included in the admission process will affect the personality diversity of candidates admitted into a veterinary class.

Personal domains assessed in multiple mini interviews (MMIs) for healthcare student selection: A narrative synthesis systematic review

OBJECTIVES

To examine the personal domains multiple mini interviews (MMIs) are being designed to assess, explore how they were determined and contextualise such domains in current and future healthcare student selection processes DESIGN: A systematic review of empirical research reporting on MMI model design was conducted from database inception to November 2017.

DATA SOURCES

Twelve electronic bibliographic databases.

REVIEW METHODS

Evidence was extracted from original studies, and integrated in a narrative synthesis guided by the PRISMA statement for reporting systematic reviews. Personal domains were clustered into themes using a modified Delphi technique.

RESULTS

A total of 584 articles were screened. 65 unique studies (80 articles) matched our inclusion criteria of which seven were conducted within nursing/midwifery faculties. Six in 10 studies featured applicants to medical school. Across selection processes, we identified 32 personal domains assessed by MMIs, the most frequent being: communication skills (84%), teamwork/collaboration (70%), and ethical/moral judgement (65%). Domains capturing ability to cope with stressful situations (14%), make decisions (14%), and resolve conflict in the workplace (13%) featured in fewer than ten studies overall. Intra- and inter-disciplinary inconsistencies in domain profiles were noted, as well as differences by entry level. MMIs deployed in nursing and midwifery assessed compassion and decision-making more frequently than in all other disciplines. Own programme philosophy and professional body guidance were most frequently cited (~50%) as sources for personal domains; a blueprinting process was reported in only 8% of studies.

CONCLUSIONS

Nursing, midwifery and allied healthcare professionals should develop their theoretical frameworks for MMIs to ensure they are evidence-based and fit-for-purpose. We suggest a re-evaluation of domain priorities to ensure that students who are selected, not only have the capacity to offer the highest standards of care provision, but are able to maintain these standards when facing clinical practice and organisational pressures.

The effect of personality traits on undergraduate dental students' performance in multiple mini interviews

BACKGROUND

Multiple mini interviews (MMI) is a structured student selection method where applicants rotate around a series of stations that do not require previously learned knowledge. The Big Five Inventory (BFI) is a tool often used to measure personality traits commonly used in applicant selection.

OBJECTIVE

The aim of the current study was to assess the effect of personality traits using the BFI on undergraduate dental students' performance in MMI.

METHODS

This research was conducted at The Faculty of Dentistry, King Abdulaziz University, Jeddah, Saudi Arabia. All undergraduates applying to the dental program at King Abdulaziz University, Faculty of Dentistry (KAUFD) in two academic years [(2014-2015) and (2015-2016)] undertook the MMI held at KAUFD and completed the BFI inventory with a total number of 350 students. The MMI consisted of five stations while the BFI contained forty-four items. Data were analysed by SPSS version 20, using independent-samples t-test, Mann-Whitney U test, Spearman's rho correlation coefficient and Chi-square test.

RESULTS

female students scored higher than their male counterparts but scores were only significant in the empathy station (p<0.001) in the first year, and in both empathetic skills and team work abilities (p<0.05) in the second. BFI results indicate that male students scored higher on agreeableness (p<0.003) and lower on neuroticism (p<0.001) in the first year while female students scored higher on agreeableness and conscientiousness (p<0.001) in the second year. Students of private schools had higher total MMI scores compared to those of public schools (p<0.05).

CONCLUSION

MMI combined with the Big Five Inventory can be a useful tool in the admission process.

Candidate Evaluation Using Targeted Construct Assessment in the Multiple Mini-Interview: A Multifaceted Rasch Model Analysis

Construct: A 7-station multiple mini-interview (MMI) circuit was implemented and assessed for 214 candidates rated by 37 interviewers (N = 1,498 ratings). The MMI stations were designed to assess 6 specific constructs (adaptability, empathy, integrity, critical thinking, teamwork [receiving instruction], teamwork [giving instruction]) and one open station about the candidate's interest in the school.

BACKGROUND

Despite the apparent benefits of the MMI, construct-irrelevant variance continues to be a topic of study. Refining the MMI to more effectively measure candidate ability is critical to improving our ability to identify and select candidates that are equipped for success within health professions education and the workforce.

APPROACH

Each station assessed a single construct and was rated by a single interviewer who was provided only the name of the candidate and no additional information about the candidate's background, application, or prior academic performance. All interviewers received online and in-person training in the fall prior to the MMI and the morning of the MMI. A 3-facet multifaceted Rasch measurement analysis was completed to determine interviewer severity, candidate ability, and MMI station difficulty and examine how the model performed overall (e.g., rating scale).

RESULTS

Altogether, the Rasch measures explained 62.84% of the variance in the ratings. Differences in candidate ability explained 45.28% of the variance in the data, whereas differences in interviewer severity explained 16.09% of the variance in the data. None of the interviewers had Infit or Outfit mean-square scores greater than 1.7, and only 2 (5.4%) had mean-square scores less than 0.5.

CONCLUSIONS

The data demonstrated acceptable fit to the multifaceted Rasch measurement model. This work is the first of its kind in pharmacy and provides insight into the development of an MMI that provides useful and meaningful candidate assessment ratings for institutional decision making.

The Multiple Mini-Interview as an Admission Tool for a PharmD Program Satellite Campus

Objective. To assess the multiple mini-interview (MMI) as an admission tool for a satellite campus. Methods. In 2013, the MMI was implemented as part of a new admissions model at the UNC Eshelman School of Pharmacy. From fall 2013 to spring 2015, 73 candidates were interviewed by 15 raters on the satellite campus in Asheville, North Carolina. A many-facet Rasch measurement (MFRM) with three facets was used to determine the variance in candidate ratings attributable to rater severity, candidate ability, and station difficulty. Candidates were surveyed to explore their perceptions of the MMI. Results. Rasch measures accounted for 48.3% of total variance in candidate scores. Rater severity accounted for 9.1% of the variance, and candidate ability accounted for 36.2% of the variance. Eighty percent of survey respondents (strongly) agreed that interviewers got to know them based on questions they answered.Conclusion. This study suggests that the MMI is a useful and valid tool for candidate selection at a satellite campus.

Evidence regarding the utility of multiple mini-interview (MMI) for selection to undergraduate health programs: A BEME systematic review: BEME Guide No. 37

BACKGROUND

In the 11 years since its development at McMaster University Medical School, the multiple mini-interview (MMI) has become a popular selection tool. We aimed to systematically explore, analyze and synthesize the evidence regarding MMIs for selection to undergraduate health programs.

METHODS

The review protocol was peer-reviewed and prospectively registered with the Best Evidence Medical Education (BEME) collaboration. Thirteen databases were searched through 34 terms and their Boolean combinations. Seven key journals were hand-searched since 2004. The reference sections of all included studies were screened. Studies meeting the inclusion criteria were coded independently by two reviewers using a modified BEME coding sheet. Extracted data were synthesized through narrative synthesis.

RESULTS

A total of 4338 citations were identified and screened, resulting in 41 papers that met inclusion criteria. Thirty-two studies report data for selection to medicine, six for dentistry, three for veterinary medicine, one for pharmacy, one for nursing, one for rehabilitation, and one for health science. Five studies investigated selection to more than one profession. MMIs used for selection to undergraduate health programs appear to have reasonable feasibility, acceptability, validity, and reliability. Reliability is optimized by including 7-12 stations, each with one examiner. The evidence is stronger for face validity, with more research needed to explore content validity and predictive validity. In published studies, MMIs do not appear biased against applicants on the basis of age, gender, or socio-economic status. However, applicants of certain ethnic and social backgrounds did less well in a very small number of published studies. Performance on MMIs does not correlate strongly with other measures of noncognitive attributes, such as personality inventories and measures of emotional intelligence.

DISCUSSION

MMI does not automatically mean a more reliable selection process but it can do, if carefully designed. Effective MMIs require careful identification of the noncognitive attributes sought by the program and institution. Attention needs to be given to the number of stations, the blueprint and examiner training.

CONCLUSION

More work is required on MMIs as they may disadvantage groups of certain ethnic or social backgrounds. There is a compelling argument for multi-institutional studies to investigate areas such as the relationship of MMI content to curriculum domains, graduate outcomes, and social missions; relationships of applicants' performance on different MMIs; bias in selecting applicants of minority groups; and the long-term outcomes appropriate for studies of predictive validity.

Development and Assessment of the Multiple Mini-Interview in a School of Pharmacy Admissions Model

OBJECTIVE

To describe the development, implementation, and evaluation of the multiple mini-interview (MMI) within a doctor of pharmacy (PharmD) admissions model.

METHODS

Demographic data and academic indicators were collected for all candidates who participated in Candidates' Day (n=253), along with the score for each MMI station criteria (7 stations). A survey was administered to all candidates who completed the MMI, and another survey was administered to all interviewers to examine perceptions of the MMI.

RESULTS

Analyses suggest that MMI stations assessed different attributes as designed, with Cronbach alpha for each station ranging from 0.90 to 0.95. All correlations between MMI station scores and academic indicators were negligible. No significant differences in average station scores were found based on age, gender, or race.

CONCLUSION

This study provides additional support for the use of the MMI as an admissions tool in pharmacy education.

Multiple mini-interviews: same concept, different approaches

OBJECTIVES

Increasing numbers of educational institutions in the medical field choose to replace their conventional admissions interviews with a multiple mini-interview (MMI) format because the latter has superior reliability values and reduces interviewer bias. As the MMI format can be adapted to the conditions of each institution, the question of under which circumstances an MMI is most expedient remains unresolved. This article systematically reviews the existing MMI literature to identify the aspects of MMI design that have impact on the reliability, validity and cost-efficiency of the format.

METHODS

Three electronic databases (OVID, PubMed, Web of Science) were searched for any publications in which MMIs and related approaches were discussed. Sixty-six publications were included in the analysis.

RESULTS

Forty studies reported reliability values. Generally, raising the number of stations has more impact on reliability than raising the number of raters per station. Other factors with positive influence include the exclusion of stations that are too easy, and the use of normative anchored rating scales or skills-based rater training. Data on criterion-related validities and analyses of dimensionality were found in 31 studies. Irrespective of design differences, the relationship between MMI results and academic measures is small to zero. The McMaster University MMI predicts in-programme and licensing examination performance. Construct validity analyses are mostly exploratory and their results are inconclusive. Seven publications gave information on required resources or provided suggestions on how to save costs. The most relevant cost factors that are additional to those of conventional interviews are the costs of station development and actor payments.

CONCLUSIONS

The MMI literature provides useful recommendations for reliable and cost-efficient MMI designs, but some important aspects have not yet been fully explored. More theory-driven research is needed concerning dimensionality and construct validity, the predictive validity of MMIs other than those of McMaster University, the comparison of station types, and a cost-efficient station development process.

Validating MMI scores: are we measuring multiple attributes?

The multiple mini-interview (MMI) used in health professional schools' admission processes is reported to assess multiple non-cognitive constructs such as ethical reasoning, oral communication, or problem evaluation. Though validation studies have been performed with total MMI scores, there is a paucity of information regarding how well MMI scores differentiate the constructs being measured, the relationship between MMI scores (construct or total) and personality characteristics, and how well MMI scores (construct or total) predict future performance in practice. Results from these studies could assist with MMI station development, rater training, score interpretation, and resource allocation. The purpose of this study was to investigate the validity of MMI construct scores (oral communication and problem evaluation), and their relationship to personality measures (emotionality and extraversion) and specific scores from standardized clinical communications interviews (building the relationship and explaining and planning). Confirmatory factor analysis results support a two factor MMI model, however the correlation between these factors was .87. Oral communication MMI scores significantly correlated with extraversion (r c = .25, p < .05), but MMI scores were not related to emotionality. Scores for building a relationship were significantly related to MMI oral communication scores, (r c = .46, p < .001) and problem evaluation scores (r c = .43, p < .001); scores for explaining and planning were significantly related to MMI problem evaluation scores (r c = .36, p < .01). The results provide validity evidence for assessing multiple non-cognitive attributes during the MMI process and reinforce the importance of developing MMI stations and scoring rubrics for attributes identified as important for future success in school and practice.

New workforce development in dementia care: screening for "caring": preliminary data

The United States has a significant shortage of trained geriatricians and of nurses, social workers, and paraprofessionals educated to care for elderly adults. As the aging population continues to grow, providing high-quality care will require new models that better address the many needs of aging individuals and their caregivers, using cost-effective strategies. Responding to this need, the Indiana University Center for Aging Research implementation scientists developed, tested, and are now scaling up a successful collaborative care coordination model for older adults with dementia, depression, or both: the Aging Brain Care program. This model now includes a newly created frontline care provider position, the Care Coordinator Assistant. The Care Coordinator Assistant works with individuals and caregivers to monitor biopsychosocial needs and deliver evidence-based and individualized care protocols, with close supervision from the registered nurse Care Coordinator. Recognizing that current hiring practices for frontline providers were insufficient to screen for critical abilities expected in this new position, including the ability to express "caring" and empathy, a new screening process was created building on the Multiple Mini Interview (MMI) format. The Care Coordinator Assistant MMI comprised six stations, each created to simulate challenging scenarios that will be frequently encountered and to assess important candidate abilities. Overall, the six-station MMI, with two to three items per station, provided factorially valid measures and good predictive ability. The process did not appear to be overly burdensome for candidates, and interviewers noted that it was helpful in discriminating between candidates.

The Multiple Mini-Interview (MMI) for student selection in health professions training - a systematic review

BACKGROUND

The Multiple Mini-Interview (MMI) has been used increasingly for selection of students to health professions programmes.

OBJECTIVES

This paper reports on the evidence base for the feasibility, acceptability, reliability and validity of the MMI.

DATA SOURCES

CINAHL and MEDLINE STUDY ELIGIBILITY CRITERIA: All studies testing the MMI on applicants to health professions training.

STUDY APPRAISAL AND SYNTHESIS METHODS

Each paper was appraised by two reviewers. Narrative summary findings on feasibility, acceptability, reliability and validity are presented.

RESULTS

Of the 64 citations identified, 30 were selected for review. The modal MMI consisted of 10 stations, each lasting eight minutes and assessed by one interviewer. The MMI was feasible, i.e. did not require more examiners, did not cost more, and interviews were completed over a short period of time. It was acceptable, i.e. fair, transparent, free from gender, cultural and socio-economic bias, and did not favour applicants with previous coaching. Its reliability was reported to be moderate to high, with Cronbach's alpha = 0.69-0.98 and G = 0.55-0.72. MMI scores did not correlate to traditional admission tools scores, were not associated with pre-entry academic qualifications, were the best predictor for OSCE performance and statistically predictive of subsequent performance at medical council examinations.

CONCLUSIONS

The MMI is reliable, acceptable and feasible. The evidence base for its validity against future medical council exams is growing with reports from longitudinal investigations. However, further research is needed for its acceptability in different cultural context and validity against future clinical behaviours.

The influence of interview on decision making and selection of prospective veterinary undergraduate students

Interviews form part of the selection process in many veterinary programmes worldwide. This study aimed to evaluate the influence of the interview on veterinary selection decisions. An admissions cycle of interviews of 181 shortlisted school-leaving applicants to veterinary medicine was investigated. Selection panel pairs (from cohorts of 5 chairs and 11 cointerviewers) independently evaluated applicants before interview on the basis of their written application. Applicants were then interviewed and selectors repeated their evaluations, both independently and after joint discussion. Results of the preinterview and postinterview evaluations of the applicants were analysed statistically. There was slight to moderate agreement between the chair and cointerviewer on selection decisions/rankings taken before interview, but substantial agreement after interview. Agreement between postinterview decisions/rankings and consensus decisions/rankings postinterview was extremely high. Applicant attributes that had the most influence on selection decisions were: 'communication skills', 'overall knowledge gained from work experience' and 'ability to think on their feet'. There was significantly more agreement possible between interviewers with regard to selection decisions based on interview assessment compared with preinterview assessment of written applications. This study suggests that interviews may be a useful and important aid in decision making for selecting the candidates perceived as most suitable for this veterinary programme.

Utility of a writing station in the multiple mini-interview

The multiple mini-interview (MMI) is a reliable and valid method of selecting applicants for admission to health professional schools on the basis of non-cognitive traits. Because the MMI is a series of short interview stations that applicants rotate through in coordinated sequence, it can potentially be resource intensive. However, the MMI design has room for innovation and efficiency. At the University of Manitoba Faculty of Medicine, a 10-minute unsupervised writing station (WS) was incorporated into the MMI to obtain a writing sample from each applicant, to increase the number of independent scores per applicant, and to increase the number of applicants interviewed per circuit without increasing interviewer numbers. One assessor evaluated all the writing samples and assigned a score ranging from 1 to 7. With the inclusion of a WS into an 11-station MMI, the faculty's capacity to interview applicants increased by 9% (from 297 to 324) without substantially increasing interviewer hours needed per day. For 1,257 applicants interviewed in 2008-2011, the mean WS score was 4.03 (SD=1.36), whereas applicants' mean of 10 oral station (OS) scores was 4.62 (SD=0.69). Correlations between WS score and mean OS score ranged from .16 to .27 (p<.01) over the four years. Because inter-station correlations for OS ranged from .01 to .37, the correlation of .21 between WS and mean OS scores for all four years combined appears reasonable. Institutions that want to effectively increase the capacity of their MMI process might consider adding a WS.

What should we be selecting for? A systematic approach for determining which personal characteristics to assess for during admissions

BACKGROUND

Admission committees are responsible for creating fair, defensible, reliable, and valid processes that assess those attributes considered important for professional success. There is evidence for the continuing use of academic ability as a selection criterion for health professional schools; however, there is little evidence for the reliability and validity of measures currently in place to assess personal characteristics. The Ontario Veterinary College (OVC) initiated a review of its admissions criteria in order to implement an evidence-based method to determine which characteristics veterinary stakeholders consider important to assess for admission.

METHODS

Eleven characteristics were identified by the OVC Admissions Committee and a survey was sent to all licensed veterinarians in Ontario (n=4,068), OVC students (n=450), and OVC faculty, interns and residents (n=192). A paired comparison method was used to identify the relative rank order of the characteristics, and multivariate analysis of variance with post hoc analyses was used to determine between group differences in the returned survey data.

RESULTS

Surveys were returned from 1,312 participants (27.86% response rate; female 59.70%). The relative rank of the characteristics was reasonably consistent among participant groups, with ethical behaviour, sound judgment, communication, and critical and creative thinking being ranked as the top four. However, the importance of certain characteristics like communication and empathy were perceived differently by groups. For instance, females scored communication (F(1, 1289) = 20.24, p < .001, d = .26) and empathy (F(1, 1289) = 55.41, p < .001, d = 0.42) significantly higher than males, while males scored knowledge of profession (F(1, 1289) = 12.81, p < .001, d = 0.20), leadership (F(1, 1289) = 10.28, p = .001, d = 0.18), and sound judgment (F(1, 1289) = 13.56, p < .001, d = 0.21) significantly higher than females.

CONCLUSIONS

The data from the paired comparison method provide convergent evidence for the characteristics participant groups identify as most important in determining who should be admitted to a veterinary program. The between group analyses provides important information regarding characteristics most important to various subgroups; this has implications for what characteristics are selected for at admission as well as on who is selecting for them.