1
|
Abraham Z, Melro C, Burm S. 'Click, I Guess I'm Done': Applicants' and Assessors' Experiences Transitioning to a Virtual Multiple Mini Interview Format. PERSPECTIVES ON MEDICAL EDUCATION 2023; 12:594-602. [PMID: 38163050 PMCID: PMC10756158 DOI: 10.5334/pme.1035] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [MESH Headings] [Grants] [Track Full Text] [Subscribe] [Scholar Register] [Received: 04/25/2023] [Accepted: 11/29/2023] [Indexed: 01/03/2024]
Abstract
Introduction During the COVID-19 pandemic, medical schools were forced to suspend in-person interviews and transition to a virtual Multiple Mini Interview (vMMI) format. MMIs typically comprise multiple short assessments overseen by assessors, with the aim of measuring a wide range of non-cognitive competencies. The adaptation to vMMI required medical schools to make swift changes to their MMI structure and delivery. In this paper, we focus on two specific groups greatly impacted by the decision to transition to vMMIs: medical school applicants and MMI assessors. Methods We conducted an interpretive qualitative study to explore medical school applicants' and assessors' experiences transitioning to an asynchronous vMMI format. Ten assessors and five medical students from one Canadian medical school participated in semi-structured interviews. Data was analyzed using a thematic analysis framework. Results Both applicants and assessors shared a mutual feeling of longing and nostalgia for an interview experience that, due to the pandemic, was understandably adapted. The most obvious forms of loss experienced - albeit in different ways - were: 1) human connection and 2) missed opportunity. Applicants and assessors described several factors that amplified their grief/loss response. These were: 1) resource availability, 2) technological concerns, and 3) the virtual interview environment. Discussion While virtual interviewing has obvious advantages, we cannot overlook that asynchronous vMMIs do not lend themselves to the same caliber of interaction and camaraderie as experienced in in-person interviews. We outline several recommendations medical schools can implement to enhance the vMMI experience for applicants and assessors.
Collapse
Affiliation(s)
- Zoe Abraham
- Dalhousie University, Halifax, Nova Scotia, Canada
| | | | - Sarah Burm
- Department of Continuing Professional Development and Division of Medical Education at Dalhousie University, Halifax, Nova Scotia, Canada
| |
Collapse
|
2
|
Callwood A, Harris J, Gillam L, Roberts S, Kubacki A, Christidis A, Tiffin PA. Cross-sectional evaluation of an asynchronous multiple mini-interview (MMI) in selection to health professions training programmes with 10 principles for fairness built-in. BMJ Open 2023; 13:e074440. [PMID: 37907294 PMCID: PMC10618971 DOI: 10.1136/bmjopen-2023-074440] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 04/06/2023] [Accepted: 09/18/2023] [Indexed: 11/02/2023] Open
Abstract
OBJECTIVES We aimed to explore the psychometric properties of the first known online asynchronous multiple mini-interview (MMI) designed for fairness with subgroup analyses by key characteristics, usability and acceptability. DESIGN Cross-discipline multimethod evaluation. SETTING One UK University. PARTICIPANTS Applicants to nursing, midwifery and paramedic science undergraduate programmes during 2021-2022. PRIMARY, SECONDARY OUTCOME MEASURES Psychometric properties (internal consistency, construct validity, dimensionality) were assessed using Cronbach's alpha (α), parallel analysis (PA), Schmid-Leiman transformation and ordinal confirmatory factor analysis (CFA). Usability and acceptability were evaluated using descriptive statistics and conventional content analysis. METHODS The system was configured in a seven question 4 min MMI. Applicants' videorecorded their answers which were later assessed by interviewers and scores summed. Applicants and interviewers completed online evaluation questionnaires. RESULTS Performance data from 712 applicants determined good-excellent reliability for the asynchronous MMI (mean α 0.72) with similar results across subgroups (gender, age, disability/support needs, UK/non-UK). PA and factor analysis results suggested there were seven factors relating to the MMI questions with an underlying general factor that explained the variance in observed candidate responses. A CFA testing a seven-factor hierarchical model showed an excellent fit to the data (Confirmatory Fit Index=0.99), Tucker Lewis Index=0.99, root mean square error (RMSE) =0.034). Applicants (n=210) viewed the flexibility, relaxed environment and cost savings advantageous. Interviewers (n=65) reported the system to be intuitive, flexible with >70% time saved compared with face-to-face interviews. Reduced personal communication was cited as the principal disadvantage. CONCLUSIONS We found that the asynchronous MMI was reliable, time-efficient, fair and acceptable and building fairness in was lost-cost. These novel, insights are applicable across health professions selection internationally informing the future configuration of online interviews to ensure workforces represent the societies they serve.
Collapse
Affiliation(s)
- Alison Callwood
- Faculty of Health and Medical Sciences, University of Surrey, Guildford, UK
| | - Jenny Harris
- Faculty of Health and Medical Sciences, University of Surrey, Guildford, UK
| | - Lee Gillam
- Faculty of Engineering and Physical Sciences, University of Surrey, Guildford, UK
| | - Sarah Roberts
- Faculty of Health and Medical Sciences, University of Surrey, Guildford, UK
| | | | - Angelos Christidis
- Faculty of Engineering and Physical Sciences, University of Surrey, Guildford, UK
| | | |
Collapse
|
3
|
Cleland J, Blitz J, Cleutjens KBJM, Oude Egbrink MGA, Schreurs S, Patterson F. Robust, defensible, and fair: The AMEE guide to selection into medical school: AMEE Guide No. 153. MEDICAL TEACHER 2023; 45:1071-1084. [PMID: 36708606 DOI: 10.1080/0142159x.2023.2168529] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/05/2023]
Abstract
Selection is the first assessment of medical education and training. Medical schools must select from a pool of academically successful applicants and ensure that the way in which they choose future clinicians is robust, defensible, fair to all who apply and cost-effective. However, there is no comprehensive and evidence-informed guide to help those tasked with setting up or rejuvenating their local selection process. To address this gap, our guide draws on the latest research, international case studies and consideration of common dilemmas to provide practical guidance for designing, implementing and evaluating an effective medical school selection system. We draw on a model from the field of instructional design to frame the many different activities involved in doing so: the ADDIE model. ADDIE provides a systematic framework of Analysis (of the outcomes to be achieved by the selection process, and the barriers and facilitators to achieving these), Design (what tools and content are needed so the goals of selection are achieved), Development (what materials and resources are needed and available), Implementation (plan [including piloting], do study and adjust) and Evaluation (quality assurance is embedded throughout but the last step involves extensive evaluation of the entire process and its outcomes).HIGHLIGHTSRobust, defensible and fair selection into medical school is essential. This guide systematically covers the processes required to achieve this, from needs analysis through design, development and implementation, to evaluation of the success of a selection process.
Collapse
Affiliation(s)
- J Cleland
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore
| | - J Blitz
- Centre for Health Professions Education, Faculty of Medicine and Health Sciences, Stellenbosch University, Stellenbosch, South Africa
| | - K B J M Cleutjens
- School of Health Professions Education, Maastricht University, the Netherlands
| | - M G A Oude Egbrink
- School of Health Professions Education, Maastricht University, the Netherlands
| | - S Schreurs
- School of Health Professions Education, Maastricht University, the Netherlands
- Centrum for Evidence Based Education, University of Utrecht, the Netherlands
| | | |
Collapse
|
4
|
Hammond S, McLaughlin JE, Cox WC. Validity evidence for a virtual multiple mini interview at a pharmacy program. BMC MEDICAL EDUCATION 2023; 23:551. [PMID: 37537588 PMCID: PMC10401851 DOI: 10.1186/s12909-023-04521-9] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 03/16/2023] [Accepted: 07/18/2023] [Indexed: 08/05/2023]
Abstract
BACKGROUND Numerous health professions schools have transitioned to virtual admissions interviews in recent years. While some research suggests that virtual multiple mini-interviews (vMMIs) are feasible, acceptable, and more affordable, there is a paucity of research concerning the validity of this approach. The purpose of this study was to examine the validity and reliability of vMMIs and explore differences in performance between vMMI and in-person MMIs. METHODS Data were collected for two years of in-person MMIs and two years of vMMIs at a pharmacy program/school in the United States. An exploratory factor analysis (principal components analysis) with varimax rotation and Kaiser rule (i.e. retaining factors with eigenvalue > 1.0) was used to explore the construct validity of the vMMI data. Pearson correlation was used to examine correlations between vMMI stations and Cronbach alpha was used to determine the internal consistency of each station. Independent t-tests were used to examine differences between in-person MMI and vMMI scores. Cohen's d was used to determine effect sizes. RESULTS Four hundred and thirty-eight (42.69%) candidates completed an in-person MMI and 588 (57.31%) completed a vMMI. Factor analysis indicated that each vMMI station formed a single factor with loads ranging from 0.86 to 0.96. The vMMI stations accounted for most of the total variance, demonstrated weak to negligible intercorrelations, and high internal consistency. Significant differences between in-person and vMMI scores were found for the teamwork-giving, teamwork-receiving, and integrity stations. Medium effect sizes were found for teamwork-giving and teamwork-receiving and a small effect size was found for integrity. CONCLUSIONS Initial evidence suggests that the vMMI is a valid and reliable alternative to in-person MMIs. Additional research is needed to examine sources of differences in rating patterns between the two approaches and identify strategies that align with institutional priorities for recruitment and admissions.
Collapse
Affiliation(s)
- Sarah Hammond
- School of Social Work, UNC Chapel Hill, Chapel Hill, NC, USA
| | - Jacqueline E McLaughlin
- Division of Practice Advancement and Clinical Education, Director, Center for Innovative Pharmacy Education and Research, UNC Eshelman School of Pharmacy, UNC Chapel Hill, Chapel Hill, NC, USA
| | - Wendy C Cox
- Division of Practice Advancement and Clinical Education, Associate Dean for Admissions and Accreditation, UNC Eshelman School of Pharmacy, UNC Chapel Hill, Chapel Hill, NC, USA.
- UNC Eshelman School of Pharmacy, University of North Carolina at Chapel Hill, Chapel Hill, NC, 27599, USA.
| |
Collapse
|
5
|
Alam F, Lim YC, Chaw LL, Idris F, Kok KYY. Multiple mini-interviews is a predictor of students' academic achievements in early undergraduate medical years: a retrospective study. BMC MEDICAL EDUCATION 2023; 23:187. [PMID: 36973779 PMCID: PMC10044430 DOI: 10.1186/s12909-023-04183-7] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Received: 11/29/2022] [Accepted: 03/21/2023] [Indexed: 06/18/2023]
Abstract
BACKGROUND Our study determined Multiple Mini-Interview (MMI) effectiveness in evaluating specific skill sets based on medical students' performances during the undergraduate years and compared the academic performances of medical students who appeared for onsite/online MMI. METHODS A retrospective study of 140 undergraduate medical students between 2016 and 2020 included data on age, gender, pre-university results, MMI scores, and examination results. Appropriate non-parametric tests were applied to compare the students' MMI and academic performances. RESULTS Ninety-eight students from cohorts 12 to 15 had an overall MMI score of 69.0(IQR: 65.0-73.2)/100 and an overall Cumulative Grade Point Average(GPA) of 3.64 (3.42-3.78)/5.0. Spearman's correlation revealed a significantly positive relationship between MMI and cGPA (rho = 0.23) and GPA from the first 2 semesters (GPA1 rho = 0.25, GPA2 rho = 0.27). This observation was similar to that for station A in the first year (cGPA rho = 0.28, GPA1 rho = 0.34, GPA2 rho = 0.24), and in station B (GPA4 rho = 0.25) and D (GPA3 rho = 0.28, GPA4 rho = 0.24) in the second year. Of twenty-nine cohort16 students, 17(58.6%) underwent online and 12(41.4%) offline modes of MMI assessment, respectively. The overall median MMI score was 66.6(IQR: 58.6-71.6)/100, and the overall median cGPA was 3.45 (3.23-3.58)/5.0. When comparing the median marks of cohort16 groups, the online group scored significantly higher marks for station D than the offline group (p = 0.040). CONCLUSION Correspondence between MMI scores and cGPA predicted MMI scoring during student selection and entry process might ensure the success of their academic performance in medical school.
Collapse
Affiliation(s)
- Faiza Alam
- PAPRSB Institute of Health Sciences, Universiti Brunei Darussalam, Jalan Tungku Link, Gadong, Bandar Seri Begawan, BE1410, Negara Brunei Darussalam.
| | - Ya Chee Lim
- PAPRSB Institute of Health Sciences, Universiti Brunei Darussalam, Jalan Tungku Link, Gadong, Bandar Seri Begawan, BE1410, Negara Brunei Darussalam
| | - Li Ling Chaw
- PAPRSB Institute of Health Sciences, Universiti Brunei Darussalam, Jalan Tungku Link, Gadong, Bandar Seri Begawan, BE1410, Negara Brunei Darussalam
| | - Fazean Idris
- PAPRSB Institute of Health Sciences, Universiti Brunei Darussalam, Jalan Tungku Link, Gadong, Bandar Seri Begawan, BE1410, Negara Brunei Darussalam
| | - Kenneth Yuh Yen Kok
- PAPRSB Institute of Health Sciences, Universiti Brunei Darussalam, Jalan Tungku Link, Gadong, Bandar Seri Begawan, BE1410, Negara Brunei Darussalam
| |
Collapse
|
6
|
Latiolais CA, Dang E, Adler S, Held H, Alquaisi Y, Lawson KA, Litten K. Part 1: Assessment of a virtual vs. onsite interview experience from the interviewee perspective. CURRENTS IN PHARMACY TEACHING & LEARNING 2023; 15:19-25. [PMID: 36925363 DOI: 10.1016/j.cptl.2023.02.003] [Citation(s) in RCA: 1] [Impact Index Per Article: 1.0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/02/2022] [Revised: 09/23/2022] [Accepted: 02/23/2023] [Indexed: 06/18/2023]
Abstract
INTRODUCTION The University of Texas at Austin College of Pharmacy transitioned the prospective student interview process for the incoming Class of 2025 from an onsite to a virtual Zoom interview. Differences between the two processes were assessed to determine utility of virtual interviews in the future. The objective was to compare preference, impact, and barriers to onsite and virtual interview experiences for prospective students. METHODS A survey to assess interviewees' opinions regarding the interview process, preference, and barriers to participation was emailed to candidates following the 2020-2021 interviews. Responses were evaluated using descriptive statistics, chi-square, Mann-Whitney U tests, and constant comparison thematic analysis. RESULTS The survey response rate was 40%. Of these, 54% preferred virtual interviews. Travel, lodging, and time were identified as barriers, with 80.5% of interviewees reporting at least one of these barriers. Respondents who chose time or had more barriers were more likely to prefer virtual interviews. Hosting a pre-interview day helped candidates prepare. Having a pharmacy student in the breakout room helped reduce stress. Interviewees were able to engage, showcase their personality, and learn the culture of the college despite the virtual nature. CONCLUSIONS From an interviewee perspective, the virtual interview process is a viable method to continue. Virtual interviews decrease barriers to access for candidates unable to attend onsite interviews while still allowing candidates to feel engaged, learn about the program, and have a positive experience. Pharmacy institutions may consider virtual interviews as an alternative or supplement to onsite interviews.
Collapse
Affiliation(s)
- Claire A Latiolais
- The University of Texas at Austin College of Pharmacy, 2409 University Ave, Mail Code A1900, Austin, TX 78712, United States; The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX 77030, United States.
| | - Elena Dang
- The University of Texas at Austin College of Pharmacy, 2409 University Ave, Mail Code A1900, Austin, TX 78712, United States.
| | - Stephanie Adler
- The University of Texas at Austin College of Pharmacy, 2409 University Ave, Mail Code A1900, Austin, TX 78712, United States.
| | - Hailey Held
- The University of Texas at Austin College of Pharmacy, 2409 University Ave, Mail Code A1900, Austin, TX 78712, United States.
| | - Yasmeen Alquaisi
- The University of Texas at Austin College of Pharmacy, 2409 University Ave, Mail Code A1900, Austin, TX 78712, United States.
| | - Kenneth A Lawson
- The University of Texas at Austin College of Pharmacy, 2409 University Ave, Mail Code A1900, Austin, TX 78712, United States.
| | - Kathryn Litten
- The University of Texas at Austin College of Pharmacy, 2409 University Ave, Mail Code A1900, Austin, TX 78712, United States.
| |
Collapse
|
7
|
Litten K, Dang E, Lawson KA, Latiolais CA. Part 2: Assessment of a virtual vs. onsite interview experience from the interviewer perspective. CURRENTS IN PHARMACY TEACHING & LEARNING 2023; 15:26-33. [PMID: 36898892 DOI: 10.1016/j.cptl.2023.02.004] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 06/02/2022] [Revised: 09/01/2022] [Accepted: 02/23/2023] [Indexed: 06/18/2023]
Abstract
INTRODUCTION The University of Texas at Austin College of Pharmacy transitioned from onsite interviews to virtual interviews in fall 2020. There is limited literature on whether the virtual format impacts an interviewer's assessment of a candidate. This study examined interviewer ability to assess candidates and barriers to participation. METHODS During the virtual interview process, interviewers utilized a modified multiple mini interview (mMMI) format to evaluate prospective college of pharmacy students. An 18-item survey was emailed to 62 interviewers from the 2020-2021 cycle. Virtual mMMI scores were compared to the previous year's onsite MMI scores. Descriptive statistics and thematic analysis were used to assess the data. RESULTS The response rate to the survey was 53% (33/62), and 59% of interviewers preferred virtual interviews to in-person. Interviewers stated that there were fewer barriers to participation, increased comfort, and more time with applicants during virtual interviews. For six of the nine attributes evaluated, ≥ 90% of interviewers reported that they were able to assess applicants as well as they did in person. When comparing virtual and onsite MMI scores, seven of nine attributes were statistically significantly higher in the virtual cohort than onsite. CONCLUSIONS From the perspective of interviewers, the virtual interview decreased barriers to participation and still allowed the ability to assess the candidates. While offering a choice of interview setting to interviewers could increase accessibility, the statistically significant difference in MMI scores between virtual and onsite formats suggests that additional standardization is required to offer both formats simultaneously.
Collapse
Affiliation(s)
- Kathryn Litten
- The University of Texas at Austin College of Pharmacy, 2409 University Ave, Mail Code A1900, Austin, TX 78712, United States.
| | - Elena Dang
- The University of Texas at Austin College of Pharmacy, 2409 University Ave, Mail Code A1900, Austin, TX 78712, United States.
| | - Kenneth A Lawson
- The University of Texas at Austin College of Pharmacy, 2409 University Ave, Mail Code A1900, Austin, TX 78712, United States.
| | - Claire A Latiolais
- The University of Texas MD Anderson Cancer Center, 1515 Holcombe Blvd, Houston, TX 77030, United States; The University of Texas at Austin College of Pharmacy, 2409 University Ave, Mail Code A1900, Austin, TX 78712, United States.
| |
Collapse
|
8
|
Melro CM, Abraham Z, Burm S. Seven ways to get a grip on preparing for and executing an inclusive virtual multiple mini interview. CANADIAN MEDICAL EDUCATION JOURNAL 2022; 13:77-81. [PMID: 35572017 PMCID: PMC9099167 DOI: 10.36834/cmej.73671] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Indexed: 06/15/2023]
Abstract
COVID-19 restrictions have prompted many medical schools to shift to virtual interview methods for medical school applicant selection. While extensive reflection has been documented around both the process and benefits of transitioning to a virtual Multiple Mini Interview (V-MMI) format, less attention has been given to examining the unintended consequences of this adaptation on increasing representation from underrepresented groups. In this Black Ice article, we consider the equity implications of taking a virtual approach to conducting MMIs and present some practical tips to ensure medical schools are giving attention to and addressing equity issues that may affect applicant and assessor engagement and success. The following seven recommendations include actionable steps medical schools can take immediately to optimize the interview process. This guide can be adapted to residency matching services and other health professions education programs that utilize the MMI.
Collapse
Affiliation(s)
- Carolyn M Melro
- Faculty of Medicine, Dalhousie University, Nova Scotia, Canada
| | - Zoe Abraham
- Faculty of Medicine, Dalhousie University, Nova Scotia, Canada
| | - Sarah Burm
- Faculty of Medicine, Dalhousie University, Nova Scotia, Canada
| |
Collapse
|
9
|
Carey C, Gale TCE, Evans CR. Recruitment to anaesthesia training posts during the
COVID
pandemic and beyond. Anaesthesia 2022; 77:514-518. [PMID: 35279823 PMCID: PMC9111468 DOI: 10.1111/anae.15710] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Track Full Text] [Download PDF] [Journal Information] [Subscribe] [Scholar Register] [Received: 01/26/2022] [Revised: 02/18/2022] [Accepted: 02/21/2022] [Indexed: 11/28/2022]
Affiliation(s)
- C. Carey
- Department of Anaesthesia University Hospitals Sussex NHS Trust Sussex UK
| | - T. C. E. Gale
- Department of Anaesthesia University Hospitals Plymouth NHS Trust Plymouth UK
- Peninsula Medical School University of Plymouth UK
| | - C. R. Evans
- Department of Anaesthesia Cardiff and Vale University Health Board Cardiff UK
| |
Collapse
|
10
|
Hayashi M, Saiki T, Kanter SL, Ho MJ. Leaders' perspectives and actions to manage challenges in medical education presented by the COVID-19 pandemic: a nationwide survey of Japanese medical colleges. BMC MEDICAL EDUCATION 2022; 22:144. [PMID: 35246125 PMCID: PMC8894829 DOI: 10.1186/s12909-022-03193-1] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Received: 01/31/2021] [Accepted: 02/21/2022] [Indexed: 06/14/2023]
Abstract
BACKGROUND The COVID-19 pandemic has caused medical colleges worldwide to suspend in-person classes and clinical clerkships. This fluid situation urgently required educators and learners to make a paradigm shift from traditional medical education. However, descriptions of how leaders manage policy decisions, especially considering cultural contexts, are limited. This study explores how the deans of medical colleges in Japan addressed the situations in which face-to-face contact is difficult and interacted with various stakeholders during the COVID-19 pandemic. METHODS The study employed a nationwide online survey by sending individual e-mails to the director of medical education at each of the 82 medical colleges in Japan. Responses were collected between May 26 and June 12, 2020 from the deans or directors of medical education. The survey questions were developed based on a literature review and consultations with international research collaborators. The survey asked what difficulties and opportunities were encountered through curriculum adjustments during the COVID-19 pandemic and what lessons could be shared with medical educators worldwide. Survey responses were analyzed using thematic analysis. The themes were categorized by stakeholder and then analyzed using the domains of sensemaking theory. RESULTS A total of 48 medical colleges in Japan completed the survey, yielding a response rate of 58.5%. The levels of participation in the study were 42.9%, 77.8%, and 74.2% among national, public, and private medical colleges, respectively, with responses from public and private medical colleges tending to be higher than those from national medical colleges. Japanese deans' decisions for actions in adapting to COVID-19 involve perceiving cues from multiple stakeholder groups, including medical students, parents of medical students, medical faculties, and government officials. Thematic analysis of survey data reveals that Japanese deans' actions in adapting to COVID-19 reflect characteristics of Japanese culture, with Japanese deans tending to emphasize in-depth introspection and collaboration with diverse stakeholders. CONCLUSIONS Despite a lack of clear national guidelines for decision making, Japanese deans adapted to COVID-19 challenges by learning from one another and seeking the perspectives of a diverse group of stakeholders, aligned with local cultural context. Their approach offers important lessons for global medical educators.
Collapse
Affiliation(s)
- Mikio Hayashi
- Center for Medical Education, Kansai Medical University, Osaka, Japan
| | - Takuya Saiki
- Medical Education Development Center, Gifu University, Gifu, Japan
| | | | - Ming-Jung Ho
- Center for Innovation and Leadership in Education, Georgetown University Medical Center, 20057, Washington DC, USA.
| |
Collapse
|
11
|
Callwood A, Gillam L, Christidis A, Doulton J, Harris J, Piano M, Kubacki A, Tiffin PA, Roberts K, Tarmey D, Dalton D, Valentin VL. Feasibility of an automated interview grounded in multiple mini interview (MMI) methodology for selection into the health professions: an international multimethod evaluation. BMJ Open 2022; 12:e050394. [PMID: 35140144 PMCID: PMC8830226 DOI: 10.1136/bmjopen-2021-050394] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Grants] [Track Full Text] [Download PDF] [Figures] [Journal Information] [Submit a Manuscript] [Subscribe] [Scholar Register] [Received: 02/18/2021] [Accepted: 12/20/2021] [Indexed: 11/16/2022] Open
Abstract
OBJECTIVES Global, COVID-driven restrictions around face-to-face interviews for healthcare student selection have forced admission staff to rapidly adopt adapted online systems before supporting evidence is available. We have developed, what we believe is, the first automated interview grounded in multiple mini-interview (MMI) methodology. This study aimed to explore test-retest reliability, acceptability and usability of the system. DESIGN, SETTING AND PARTICIPANTS Multimethod feasibility study in Physician Associate programmes from two UK and one US university during 2019-2020. PRIMARY, SECONDARY OUTCOMES Feasibility measures (test-retest reliability, acceptability and usability) were assessed using intraclass correlation (ICC), descriptive statistics, thematic and content analysis. METHODS Volunteers took (T1), then repeated (T2), the automated MMI, with a 7-day interval (±2) then completed an evaluation questionnaire. Admission staff participated in focus group discussions. RESULTS Sixty-two students and seven admission staff participated; 34 students and 4 staff from UK and 28 students and 3 staff from US universities. Good-excellent test-retest reliability was observed at two sites (US and UK2) with T1 and T2 ICC between 0.65 and 0.81 (p<0.001) when assessed by individual total scores (range 80.6-119), station total scores 0.6-0.91, p<0.005 and individual site (≥0.79 p<0.001). Mean test re-test ICC across all three sites was 0.82 p<0.001 (95% CI 0.7 to 0.9). Admission staff reported potential to reduce resource costs and bias through a more objective screening tool for preselection or to replace some MMI stations in a 'hybrid model'. Maintaining human interaction through 'touch points' was considered essential. Users positively evaluated the system, stating it was intuitive with an accessible interface. Concepts chosen for dynamic probing needed to be appropriately tailored. CONCLUSION These preliminary findings suggest that the system is reliable, generating consistent scores for candidates and is acceptable to end users provided human touchpoints are maintained. Thus, there is evidence for the potential of such an automated system to augment healthcare student selection.
Collapse
Affiliation(s)
- Alison Callwood
- Faculty of Health and Medical Sciences, University of Surrey, Guildford, Surrey, UK
| | - Lee Gillam
- Faculty of Engineering and Physical Sciences, University of Surrey, Guildford, Surrey, UK
| | - Angelos Christidis
- Faculty of Engineering and Physical Sciences, University of Surrey, Guildford, Surrey, UK
| | - Jia Doulton
- Faculty of Health and Medical Sciences, University of Surrey, Guildford, Surrey, UK
| | - Jenny Harris
- Faculty of Health and Medical Sciences, University of Surrey, Guildford, Surrey, UK
| | - Marianne Piano
- Melbourne School of Health Sciences, The University of Melbourne, Melbourne, Victoria, Australia
| | | | - Paul A Tiffin
- Department of Health Sciences, University of York, York, North Yorkshire, UK
| | - Karen Roberts
- Brighton and Sussex Medical School, Brighton, Brighton and Hove, UK
| | - Drew Tarmey
- The University of Manchester School of Medical Sciences, Manchester, Manchester, UK
| | - Doris Dalton
- Department of Family and Preventive Medicine, The University of Utah, Salt Lake City, Utah, USA
| | - Virginia L Valentin
- Department of Family and Preventive Medicine, The University of Utah, Salt Lake City, Utah, USA
| |
Collapse
|
12
|
Kulasegaram K, Baxan V, Giannone E, Latter D, Hanson MD. Adapting the Admissions Interview During COVID-19: A Comparison of In-Person and Video-Based Interview Validity Evidence. ACADEMIC MEDICINE : JOURNAL OF THE ASSOCIATION OF AMERICAN MEDICAL COLLEGES 2022; 97:200-206. [PMID: 34348379 PMCID: PMC8779599 DOI: 10.1097/acm.0000000000004331] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.5] [Reference Citation Analysis] [Abstract] [MESH Headings] [Track Full Text] [Figures] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
COVID-19 physical distancing limited many medical schools' abilities to conduct in-person interviews for the 2020 admissions cycle. The University of Toronto (U of T) Temerty Faculty of Medicine was already in the midst of its interview process, with two-thirds of applicants having completed the in-person modified personal interview (MPI). As the university and surrounding region were shut down, the shift was made in the middle of the application cycle to a semisynchronous video-based MPI interview (vMPI) approach. U of T undertook the development, deployment, and evaluation of the 2 approaches mid-admissions cycle. Existing resources and tools were used to create a tailored interview process with the assistance of applicants. The vMPI was similar in content and process to the MPI: a 4-station interview with each station mapped to attributes relevant to medical school success. Instead of live interviews, applicants recorded 5-minute responses to questions for each station using their own hardware. These responses were later assessed by raters asynchronously. Out of 627 applicants, 232 applicants completed the vMPI. Validity evidence was generated for the vMPI and compared with the MPI on the internal structure, relationship to other variables, and consequential validity, including applicant and interviewer acceptability. Overall, the vMPI demonstrated similar reliability and factor structure to the MPI. As with the MPI, applicant performance was predicted by nonacademic screening tools but not academic measures. Applicants' acceptance of the vMPI was positive. Most interviewers found the vMPI to be acceptable and reported confidence in their ratings. Continuing physical distancing concerns will require multiple options for admissions committees to select medical students. The vMPI is an example of a customized approach that schools can implement and may have advantages for selection beyond the COVID-19 pandemic. Future evaluation will examine additional validity evidence for the tool.
Collapse
Affiliation(s)
- Kulamakan Kulasegaram
- K. Kulasegaram is associate professor, Department of Family and Community Medicine, and scientist, MD Program and The Wilson Centre, Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Victorina Baxan
- V. Baxan is associate registrar for admissions, MD Program, Temerty Faculty of Medicine, and lecturer, Ontario Institute for Studies in Education, University of Toronto, Toronto, Ontario, Canada
| | - Elicia Giannone
- E. Giannone is enrolment coordinator, Enrolment Services—Undergraduate Medical Education, MD Program and The Wilson Centre, Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - David Latter
- D. Latter is professor of surgery and director, MD Admissions and Student Finances, Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada
| | - Mark D. Hanson
- M.D. Hanson is professor of psychiatry, Temerty Faculty of Medicine, University of Toronto, Toronto, Ontario, Canada; ORCID: https://orcid.org/0000-0002-0820-4521
| |
Collapse
|
13
|
Sabesan V, Kapur N, Zemanek K, Levitt D, Vu T, Van Erp A. Implementation and evaluation of virtual multiple mini interviews as a selection tool for entry into paediatric postgraduate training: A Queensland experience. MEDICAL TEACHER 2022; 44:87-94. [PMID: 34460339 DOI: 10.1080/0142159x.2021.1967906] [Citation(s) in RCA: 5] [Impact Index Per Article: 2.5] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 06/13/2023]
Abstract
INTRODUCTION The Queensland Basic Paediatric Training Network (QBPTN) is the centralised pathway for entry into paediatric training in Queensland, Australia. In response to COVID-19 travel and social distancing restrictions imposed in 2020, QBPTN successfully adopted a Virtual Multiple Mini Interviews (vMMIs) model for the selection of candidates for entry into paediatric training. The authors describe the planning, implementation, challenges, and evaluation of candidates' and interviewers' experiences of vMMIs, including the differences between candidates from two geographical areas. METHODS The contents of six vMMI stations were similar to face-to-face MMI. Implementation required the identification of ZOOMTM as a preferred online platform, securing venues, communication, development of contingency plans and central coordination by the network. Candidates' experiences with vMMI were explored through thematic analysis of the qualitative data from focus groups and free text responses, and descriptive analysis of SurveyMonkey© questionnaire responses. Experiences between 'metropolitan' and 'regional and interstate' candidates were compared. RESULTS 5-minute stations with 2-minute pre-reading were used. 78 candidates and 14 interviewers participated in the selection process. All candidates attended the focus group. 58.7% of candidates responded to post vMMI questionnaire. 93% of survey responders were happy to undertake vMMI in the future, with 23% feeling they would have performed better in face-to-face. Experiences between 'metropolitan' and 'other' groups were similar. Positive experiences of participants were related to the user-friendly IT platform, successful pre-interview communications, preparation, convenience, time, and cost savings. Stress related IT failures and difficulties establishing rapport with interviewers were reported as the main negative experiences. CONCLUSION 'vMMI' is a feasible and acceptable method of selection into paediatric training. vMMI has many benefits and can be implemented relatively quickly by addressing key logistical requirements. The model under discussion could be adapted by other centres based on local needs.
Collapse
Affiliation(s)
- Vanaja Sabesan
- Queensland Paediatric Training Network, Queensland Rural Medical Service (Darling Downs Health, Queensland Health), Brisbane, Queensland, Australia
| | - Nitin Kapur
- Department of Respiratory and Sleep Medicine, Queensland Children's Hospital, South Brisbane, Australia
- Faculty of Medicine, University of Queensland, Saint Lucia, Queensland, Australia
| | - Kylie Zemanek
- Queensland Paediatric Training Network, Queensland Rural Medical Service (Darling Downs Health, Queensland Health), Brisbane, Queensland, Australia
| | - David Levitt
- Department of Paediatric Medicine and Dermatology, Department of Paediatric Education, Queensland Children's Hospital, South Brisbane, Australia
| | - Tung Vu
- Department of Paediatrics, Gold Coast University Hospital, Southport, Australia
| | - Ansmarie Van Erp
- Queensland Rural Medical Services (Darling Downs Health, Queensland Health), Brisbane, Australia
| |
Collapse
|
14
|
Groene OR, Knorr M, Vogel D, Hild C, Hampe W. Reliability and validity of new online selection tests for midwifery students. Midwifery 2021; 106:103245. [PMID: 34999513 DOI: 10.1016/j.midw.2021.103245] [Citation(s) in RCA: 0] [Impact Index Per Article: 0] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Journal Information] [Subscribe] [Scholar Register] [Received: 09/13/2021] [Revised: 12/10/2021] [Accepted: 12/23/2021] [Indexed: 10/19/2022]
Abstract
BACKGROUND AND OBJECTIVE Newly created midwifery bachelor programmes attract an increasing number of applicants. Hence, educators have moved away from traditional non-standardised selection methods and started developing new tools for the admission of students. Researchers have expressed concerns around the validity of such selection measures and their ability to identify the appropriate candidates. The objective of this study was to develop and implement selection procedures for a new midwifery Bachelor of Science degree in Hamburg, Germany, and assess their psychometric properties. DESIGN This is a mixed-methods longitudinal study on the reliability, validity and acceptability of newly developed selection criteria. SETTING The newly established midwifery bachelor of science in Hamburg, Germany. PARTICIPANTS Upon completion of their online application, all midwifery applicants were invited by email to participate in our research project and thereby informed of its purpose and voluntary nature of their participation. The total number of candidates who took the test was 366, of which 309 agreed to take part in this study. Sixty-five applicants were eventually admitted to the midwifery Bachelor programme, of which 59 were included in the study. MEASUREMENTS AND FINDINGS We developed two tests for cognitive ability (HAM-Mid I and II) and adapted one test for personal and professional characteristics (Casper - Computer-based Assessment for Sampling Personal Characteristics). Due to the Covid pandemic, we implemented the tests online instead of on site as initially planned. HAM-Mid I had the lowest (Cronbach's Alpha = 0.38) whereas Casper had the highest internal consistency (Cronbach's Alpha = 0.77) of all three tests. Age, previous academic achievement and native tongue were significantly associated with applicants' performance on HAM-Mid II and Casper admission tests. HAM-Mid II was associated with students' performance in the first year midwifery exam (r = 0.31, p < 0.05). Overall evaluation of HAM-Mid II and Casper was good while HAM-Mid I received a more negative feedback. KEY CONCLUSIONS Evidence on the reliability and predictive validity of the newly developed tests suggests that the applied admission criteria are appropriate for an objective selection of bachelor midwifery students. IMPLICATIONS FOR PRACTICE The interests of university and clinical stakeholders of a midwifery bachelor degree can be represented through the development of tests for the assessment of cognitive ability and personal and professional characteristics.
Collapse
Affiliation(s)
- Oana R Groene
- Institute of Biochemistry and Molecular Cell Biology, Center for Experimental Medicine, University Medical Center Hamburg-Eppendorf, 52 Martinistraße, 20246 Hamburg, Germany.
| | - Mirjana Knorr
- Institute of Biochemistry and Molecular Cell Biology, Center for Experimental Medicine, University Medical Center Hamburg-Eppendorf, 52 Martinistraße, 20246 Hamburg, Germany
| | - Daniela Vogel
- Academy for Education and Career, University Medical Center Hamburg-Eppendorf, 52 Martinistraße, 20246 Hamburg, Germany
| | - Carmen Hild
- Academy for Education and Career, University Medical Center Hamburg-Eppendorf, 52 Martinistraße, 20246 Hamburg, Germany
| | - Wolfgang Hampe
- Institute of Biochemistry and Molecular Cell Biology, Center for Experimental Medicine, University Medical Center Hamburg-Eppendorf, 52 Martinistraße, 20246 Hamburg, Germany
| |
Collapse
|
15
|
Anand SV, Shuy YK, Lee PSS, Lee ES. One Year on: An Overview of Singapore's Response to COVID-19-What We Did, How We Fared, How We Can Move Forward. INTERNATIONAL JOURNAL OF ENVIRONMENTAL RESEARCH AND PUBLIC HEALTH 2021; 18:ijerph18179125. [PMID: 34501718 PMCID: PMC8431401 DOI: 10.3390/ijerph18179125] [Citation(s) in RCA: 4] [Impact Index Per Article: 1.3] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Download PDF] [Figures] [Subscribe] [Scholar Register] [Received: 07/04/2021] [Revised: 07/30/2021] [Accepted: 08/20/2021] [Indexed: 01/28/2023]
Abstract
Background—One year has passed since the first COVID-19 case in Singapore. This scoping review commemorates Singaporean researchers that have expanded the knowledge on this novel virus. We aim to provide an overview of healthcare-related articles published in peer-reviewed journals, authored by the Singapore research community about COVID-19 during the first year of the pandemic. Methods—This was reported using the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) protocol. It included healthcare-related articles about COVID-19 published between 23 January 2020 and 22 January 2021 with a Singapore-affiliated author. MEDLINE, Embase, Scopus, Web of Science, CINAHL, PsycINFO, Google Scholar, and local journals were searched. The articles were screened independently by two reviewers. Results—The review included 504 articles. Most of the articles narrated the changes to hospital practice (210), while articles on COVID-19 pathology (94) formed most of the non-narrative papers. Publications on public health (61) and the indirect impacts to clinical outcomes (45) were other major themes explored by the research community. The remaining articles detailed the psychological impact of the pandemic (35), adaptations of medical education (30), and narratives of events (14). Conclusion—Amidst a resurgence of community cases involving variant COVID-19 strains, the resources from the research community will provide valuable guidance to navigate these uncertain times.
Collapse
Affiliation(s)
- S Vivek Anand
- Ministry of Health Holdings, Singapore 099253, Singapore;
| | - Yao Kang Shuy
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore 308207, Singapore;
| | - Poay Sian Sabrina Lee
- Clinical Research Unit, National Healthcare Group Polyclinics, Singapore 138543, Singapore;
| | - Eng Sing Lee
- Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore 308207, Singapore;
- Clinical Research Unit, National Healthcare Group Polyclinics, Singapore 138543, Singapore;
- Correspondence:
| |
Collapse
|
16
|
Consorti F, Kanter SL, Basili S, Ho MJ. A SWOT analysis of Italian medical curricular adaptations to the COVID-19 pandemic: A nationwide survey of medical school leaders. MEDICAL TEACHER 2021; 43:546-553. [PMID: 33556296 DOI: 10.1080/0142159x.2021.1877266] [Citation(s) in RCA: 5] [Impact Index Per Article: 1.7] [Reference Citation Analysis] [Abstract] [Key Words] [MESH Headings] [Track Full Text] [Subscribe] [Scholar Register] [Indexed: 05/16/2023]
Abstract
BACKGROUND There is a growing literature on how medical education adapts to the COVID-19 pandemic. However, there is a need to examine the facilitators and barriers of these adaptations. This study explores the strengths, weaknesses, opportunities, and threats (SWOT) of how Italian medical schools adapted their curricula to the COVID -19 pandemic. METHODS The authors conducted an online survey of directors of medical curricula in Italy. Free-text responses to open-ended questions about curricular adaptations and reflections on these adaptations were analysed using qualitative thematic analysis. RESULTS Twenty out of 60 Italian medical school directors completed the survey. Strengths identified were rapid responses and a spirit of cooperation. Weaknesses included dependency on clinical facilities, teachers' limited skills to use technology, and lack of mental health support for staff. Opportunities highlighted were clear government rules, new ways of teaching and a renewed focus on underrepresented topics. Threats expressed included impaired relationships, difficulties related to online assessment, lack of IT access, and legal and insurance issues. CONCLUSIONS This study, in documenting the curricular adaptations of Italian medical schools during an active global pandemic, and recording the perspectives of medical education leaders, offers important lessons for the future.
Collapse
Affiliation(s)
- Fabrizio Consorti
- Faculty of Medicine and Dentistry, University Sapienza of Rome, Rome, Italy
| | | | - Stefania Basili
- Faculty of Medicine and Dentistry, University Sapienza of Rome, Rome, Italy
| | | |
Collapse
|
17
|
Abdollahi A, Labbaf A, Khabaz Mafinejad M, Sotoudeh-Anvari M, Azmoudeh-Ardalan F. Online Assessment for Pathology Residents during the COVID-19 Pandemic: Report of an Experience. IRANIAN JOURNAL OF PATHOLOGY 2020; 16:75-78. [PMID: 33391384 PMCID: PMC7691715 DOI: 10.30699/ijp.2020.129558.2425] [Citation(s) in RCA: 1] [Impact Index Per Article: 0.3] [Reference Citation Analysis] [Abstract] [Key Words] [Track Full Text] [Download PDF] [Subscribe] [Scholar Register] [Received: 06/20/2020] [Accepted: 07/21/2020] [Indexed: 11/30/2022]
Abstract
BACKGROUND & OBJECTIVE The world is facing COVID-19 pandemic, and medical education system and consequently the evaluation of students at different levels have been overshadowed. Residency students are among those affected. In the present study, we aim to share our experiences regarding holding exams for pathology residents before and during the pandemic. METHODS This cross-sectional study was carried out in Tehran University of Medical Sciences. The online exam, which consisted of 30 multiple-choice questions, was designed and held in April 2020 to evaluate pathology residents. To assess the quality of the exam, indices such as the number of questions, highest and lowest scores, the average score, the standard deviation, the variance, Cronbach's alpha reliability coefficient, standard error of measurement, discrimination index, difficulty index, number/percentage of questions on three difficulty levels of easy, normal, hard were reviewed and analyzed. RESULTS AND DISCUSSION The average score of the participants in the January exam which was held in the university's exam center was 16.23 (±5.03), while the average score in the online exam which was held after the onset of the pandemic was 20.86 (±5.18). The average discrimination indices in the first and second exams were 0.36 and 0.38, respectively, and the average difficulty indices in the first and second exams were 0.54 and 0.70, respectively. We found the administration of this online examination would be a positive experience. By sharing it, we hope to pave the way for similar ventures in the other departments.
Collapse
Affiliation(s)
- Alireza Abdollahi
- Department of Pathology, Imam Hospital Complex, School of Medicine, Tehran University of Medical Sciences, Tehran, Iran.
| | - Ali Labbaf
- Department of Emergency Medicine, Imam Khomeini Hospital Complex, Tehran University of Medical Sciences, Tehran, Iran.
| | - Mahboobeh Khabaz Mafinejad
- Education Development Center, Health Professions Education Research Center, Tehran University of Medical Sciences, Tehran, Iran.
| | - Maryam Sotoudeh-Anvari
- Department of Pathology, Children's Medical Center, School of Medicine, Tehran University of Medical Sciences, Tehran, Iran
| | - Farid Azmoudeh-Ardalan
- Department of Pathology, Imam Hospital Complex, School of Medicine, Tehran University of Medical Sciences, Tehran, Iran.
| |
Collapse
|
18
|
Affiliation(s)
- Trevor Gibbs
- Association for Medical Education in Europe (AMEE), AMEE Secretariat, Dundee
| |
Collapse
|