COVID 19: Designing and conducting an online mini-multiple interview (MMI) in a dynamic landscape

'Click, I Guess I'm Done': Applicants' and Assessors' Experiences Transitioning to a Virtual Multiple Mini Interview Format

Introduction

During the COVID-19 pandemic, medical schools were forced to suspend in-person interviews and transition to a virtual Multiple Mini Interview (vMMI) format. MMIs typically comprise multiple short assessments overseen by assessors, with the aim of measuring a wide range of non-cognitive competencies. The adaptation to vMMI required medical schools to make swift changes to their MMI structure and delivery. In this paper, we focus on two specific groups greatly impacted by the decision to transition to vMMIs: medical school applicants and MMI assessors.

Methods

We conducted an interpretive qualitative study to explore medical school applicants' and assessors' experiences transitioning to an asynchronous vMMI format. Ten assessors and five medical students from one Canadian medical school participated in semi-structured interviews. Data was analyzed using a thematic analysis framework.

Results

Both applicants and assessors shared a mutual feeling of longing and nostalgia for an interview experience that, due to the pandemic, was understandably adapted. The most obvious forms of loss experienced - albeit in different ways - were: 1) human connection and 2) missed opportunity. Applicants and assessors described several factors that amplified their grief/loss response. These were: 1) resource availability, 2) technological concerns, and 3) the virtual interview environment.

Discussion

While virtual interviewing has obvious advantages, we cannot overlook that asynchronous vMMIs do not lend themselves to the same caliber of interaction and camaraderie as experienced in in-person interviews. We outline several recommendations medical schools can implement to enhance the vMMI experience for applicants and assessors.

Cross-sectional evaluation of an asynchronous multiple mini-interview (MMI) in selection to health professions training programmes with 10 principles for fairness built-in

OBJECTIVES

We aimed to explore the psychometric properties of the first known online asynchronous multiple mini-interview (MMI) designed for fairness with subgroup analyses by key characteristics, usability and acceptability.

DESIGN

Cross-discipline multimethod evaluation.

SETTING

One UK University.

PARTICIPANTS

Applicants to nursing, midwifery and paramedic science undergraduate programmes during 2021-2022.

PRIMARY, SECONDARY OUTCOME MEASURES

Psychometric properties (internal consistency, construct validity, dimensionality) were assessed using Cronbach's alpha (α), parallel analysis (PA), Schmid-Leiman transformation and ordinal confirmatory factor analysis (CFA). Usability and acceptability were evaluated using descriptive statistics and conventional content analysis.

METHODS

The system was configured in a seven question 4 min MMI. Applicants' videorecorded their answers which were later assessed by interviewers and scores summed. Applicants and interviewers completed online evaluation questionnaires.

RESULTS

Performance data from 712 applicants determined good-excellent reliability for the asynchronous MMI (mean α 0.72) with similar results across subgroups (gender, age, disability/support needs, UK/non-UK). PA and factor analysis results suggested there were seven factors relating to the MMI questions with an underlying general factor that explained the variance in observed candidate responses. A CFA testing a seven-factor hierarchical model showed an excellent fit to the data (Confirmatory Fit Index=0.99), Tucker Lewis Index=0.99, root mean square error (RMSE) =0.034). Applicants (n=210) viewed the flexibility, relaxed environment and cost savings advantageous. Interviewers (n=65) reported the system to be intuitive, flexible with >70% time saved compared with face-to-face interviews. Reduced personal communication was cited as the principal disadvantage.

CONCLUSIONS

We found that the asynchronous MMI was reliable, time-efficient, fair and acceptable and building fairness in was lost-cost. These novel, insights are applicable across health professions selection internationally informing the future configuration of online interviews to ensure workforces represent the societies they serve.

Robust, defensible, and fair: The AMEE guide to selection into medical school: AMEE Guide No. 153

Selection is the first assessment of medical education and training. Medical schools must select from a pool of academically successful applicants and ensure that the way in which they choose future clinicians is robust, defensible, fair to all who apply and cost-effective. However, there is no comprehensive and evidence-informed guide to help those tasked with setting up or rejuvenating their local selection process. To address this gap, our guide draws on the latest research, international case studies and consideration of common dilemmas to provide practical guidance for designing, implementing and evaluating an effective medical school selection system. We draw on a model from the field of instructional design to frame the many different activities involved in doing so: the ADDIE model. ADDIE provides a systematic framework of Analysis (of the outcomes to be achieved by the selection process, and the barriers and facilitators to achieving these), Design (what tools and content are needed so the goals of selection are achieved), Development (what materials and resources are needed and available), Implementation (plan [including piloting], do study and adjust) and Evaluation (quality assurance is embedded throughout but the last step involves extensive evaluation of the entire process and its outcomes).HIGHLIGHTSRobust, defensible and fair selection into medical school is essential. This guide systematically covers the processes required to achieve this, from needs analysis through design, development and implementation, to evaluation of the success of a selection process.

Validity evidence for a virtual multiple mini interview at a pharmacy program

BACKGROUND

Numerous health professions schools have transitioned to virtual admissions interviews in recent years. While some research suggests that virtual multiple mini-interviews (vMMIs) are feasible, acceptable, and more affordable, there is a paucity of research concerning the validity of this approach. The purpose of this study was to examine the validity and reliability of vMMIs and explore differences in performance between vMMI and in-person MMIs.

METHODS

Data were collected for two years of in-person MMIs and two years of vMMIs at a pharmacy program/school in the United States. An exploratory factor analysis (principal components analysis) with varimax rotation and Kaiser rule (i.e. retaining factors with eigenvalue > 1.0) was used to explore the construct validity of the vMMI data. Pearson correlation was used to examine correlations between vMMI stations and Cronbach alpha was used to determine the internal consistency of each station. Independent t-tests were used to examine differences between in-person MMI and vMMI scores. Cohen's d was used to determine effect sizes.

RESULTS

Four hundred and thirty-eight (42.69%) candidates completed an in-person MMI and 588 (57.31%) completed a vMMI. Factor analysis indicated that each vMMI station formed a single factor with loads ranging from 0.86 to 0.96. The vMMI stations accounted for most of the total variance, demonstrated weak to negligible intercorrelations, and high internal consistency. Significant differences between in-person and vMMI scores were found for the teamwork-giving, teamwork-receiving, and integrity stations. Medium effect sizes were found for teamwork-giving and teamwork-receiving and a small effect size was found for integrity.

CONCLUSIONS

Initial evidence suggests that the vMMI is a valid and reliable alternative to in-person MMIs. Additional research is needed to examine sources of differences in rating patterns between the two approaches and identify strategies that align with institutional priorities for recruitment and admissions.

Multiple mini-interviews is a predictor of students' academic achievements in early undergraduate medical years: a retrospective study

BACKGROUND

Our study determined Multiple Mini-Interview (MMI) effectiveness in evaluating specific skill sets based on medical students' performances during the undergraduate years and compared the academic performances of medical students who appeared for onsite/online MMI.

METHODS

A retrospective study of 140 undergraduate medical students between 2016 and 2020 included data on age, gender, pre-university results, MMI scores, and examination results. Appropriate non-parametric tests were applied to compare the students' MMI and academic performances.

RESULTS

Ninety-eight students from cohorts 12 to 15 had an overall MMI score of 69.0(IQR: 65.0-73.2)/100 and an overall Cumulative Grade Point Average(GPA) of 3.64 (3.42-3.78)/5.0. Spearman's correlation revealed a significantly positive relationship between MMI and cGPA (rho = 0.23) and GPA from the first 2 semesters (GPA1 rho = 0.25, GPA2 rho = 0.27). This observation was similar to that for station A in the first year (cGPA rho = 0.28, GPA1 rho = 0.34, GPA2 rho = 0.24), and in station B (GPA4 rho = 0.25) and D (GPA3 rho = 0.28, GPA4 rho = 0.24) in the second year. Of twenty-nine cohort16 students, 17(58.6%) underwent online and 12(41.4%) offline modes of MMI assessment, respectively. The overall median MMI score was 66.6(IQR: 58.6-71.6)/100, and the overall median cGPA was 3.45 (3.23-3.58)/5.0. When comparing the median marks of cohort16 groups, the online group scored significantly higher marks for station D than the offline group (p = 0.040).

CONCLUSION

Correspondence between MMI scores and cGPA predicted MMI scoring during student selection and entry process might ensure the success of their academic performance in medical school.

Part 1: Assessment of a virtual vs. onsite interview experience from the interviewee perspective

INTRODUCTION

The University of Texas at Austin College of Pharmacy transitioned the prospective student interview process for the incoming Class of 2025 from an onsite to a virtual Zoom interview. Differences between the two processes were assessed to determine utility of virtual interviews in the future. The objective was to compare preference, impact, and barriers to onsite and virtual interview experiences for prospective students.

METHODS

A survey to assess interviewees' opinions regarding the interview process, preference, and barriers to participation was emailed to candidates following the 2020-2021 interviews. Responses were evaluated using descriptive statistics, chi-square, Mann-Whitney U tests, and constant comparison thematic analysis.

RESULTS

The survey response rate was 40%. Of these, 54% preferred virtual interviews. Travel, lodging, and time were identified as barriers, with 80.5% of interviewees reporting at least one of these barriers. Respondents who chose time or had more barriers were more likely to prefer virtual interviews. Hosting a pre-interview day helped candidates prepare. Having a pharmacy student in the breakout room helped reduce stress. Interviewees were able to engage, showcase their personality, and learn the culture of the college despite the virtual nature.

CONCLUSIONS

From an interviewee perspective, the virtual interview process is a viable method to continue. Virtual interviews decrease barriers to access for candidates unable to attend onsite interviews while still allowing candidates to feel engaged, learn about the program, and have a positive experience. Pharmacy institutions may consider virtual interviews as an alternative or supplement to onsite interviews.

Part 2: Assessment of a virtual vs. onsite interview experience from the interviewer perspective

INTRODUCTION

The University of Texas at Austin College of Pharmacy transitioned from onsite interviews to virtual interviews in fall 2020. There is limited literature on whether the virtual format impacts an interviewer's assessment of a candidate. This study examined interviewer ability to assess candidates and barriers to participation.

METHODS

During the virtual interview process, interviewers utilized a modified multiple mini interview (mMMI) format to evaluate prospective college of pharmacy students. An 18-item survey was emailed to 62 interviewers from the 2020-2021 cycle. Virtual mMMI scores were compared to the previous year's onsite MMI scores. Descriptive statistics and thematic analysis were used to assess the data.

RESULTS

The response rate to the survey was 53% (33/62), and 59% of interviewers preferred virtual interviews to in-person. Interviewers stated that there were fewer barriers to participation, increased comfort, and more time with applicants during virtual interviews. For six of the nine attributes evaluated, ≥ 90% of interviewers reported that they were able to assess applicants as well as they did in person. When comparing virtual and onsite MMI scores, seven of nine attributes were statistically significantly higher in the virtual cohort than onsite.

CONCLUSIONS

From the perspective of interviewers, the virtual interview decreased barriers to participation and still allowed the ability to assess the candidates. While offering a choice of interview setting to interviewers could increase accessibility, the statistically significant difference in MMI scores between virtual and onsite formats suggests that additional standardization is required to offer both formats simultaneously.

Seven ways to get a grip on preparing for and executing an inclusive virtual multiple mini interview

COVID-19 restrictions have prompted many medical schools to shift to virtual interview methods for medical school applicant selection. While extensive reflection has been documented around both the process and benefits of transitioning to a virtual Multiple Mini Interview (V-MMI) format, less attention has been given to examining the unintended consequences of this adaptation on increasing representation from underrepresented groups. In this Black Ice article, we consider the equity implications of taking a virtual approach to conducting MMIs and present some practical tips to ensure medical schools are giving attention to and addressing equity issues that may affect applicant and assessor engagement and success. The following seven recommendations include actionable steps medical schools can take immediately to optimize the interview process. This guide can be adapted to residency matching services and other health professions education programs that utilize the MMI.

Leaders' perspectives and actions to manage challenges in medical education presented by the COVID-19 pandemic: a nationwide survey of Japanese medical colleges

BACKGROUND

The COVID-19 pandemic has caused medical colleges worldwide to suspend in-person classes and clinical clerkships. This fluid situation urgently required educators and learners to make a paradigm shift from traditional medical education. However, descriptions of how leaders manage policy decisions, especially considering cultural contexts, are limited. This study explores how the deans of medical colleges in Japan addressed the situations in which face-to-face contact is difficult and interacted with various stakeholders during the COVID-19 pandemic.

METHODS

The study employed a nationwide online survey by sending individual e-mails to the director of medical education at each of the 82 medical colleges in Japan. Responses were collected between May 26 and June 12, 2020 from the deans or directors of medical education. The survey questions were developed based on a literature review and consultations with international research collaborators. The survey asked what difficulties and opportunities were encountered through curriculum adjustments during the COVID-19 pandemic and what lessons could be shared with medical educators worldwide. Survey responses were analyzed using thematic analysis. The themes were categorized by stakeholder and then analyzed using the domains of sensemaking theory.

RESULTS

A total of 48 medical colleges in Japan completed the survey, yielding a response rate of 58.5%. The levels of participation in the study were 42.9%, 77.8%, and 74.2% among national, public, and private medical colleges, respectively, with responses from public and private medical colleges tending to be higher than those from national medical colleges. Japanese deans' decisions for actions in adapting to COVID-19 involve perceiving cues from multiple stakeholder groups, including medical students, parents of medical students, medical faculties, and government officials. Thematic analysis of survey data reveals that Japanese deans' actions in adapting to COVID-19 reflect characteristics of Japanese culture, with Japanese deans tending to emphasize in-depth introspection and collaboration with diverse stakeholders.

CONCLUSIONS

Despite a lack of clear national guidelines for decision making, Japanese deans adapted to COVID-19 challenges by learning from one another and seeking the perspectives of a diverse group of stakeholders, aligned with local cultural context. Their approach offers important lessons for global medical educators.

Feasibility of an automated interview grounded in multiple mini interview (MMI) methodology for selection into the health professions: an international multimethod evaluation

OBJECTIVES

Global, COVID-driven restrictions around face-to-face interviews for healthcare student selection have forced admission staff to rapidly adopt adapted online systems before supporting evidence is available. We have developed, what we believe is, the first automated interview grounded in multiple mini-interview (MMI) methodology. This study aimed to explore test-retest reliability, acceptability and usability of the system.

DESIGN, SETTING AND PARTICIPANTS

Multimethod feasibility study in Physician Associate programmes from two UK and one US university during 2019-2020.

PRIMARY, SECONDARY OUTCOMES

Feasibility measures (test-retest reliability, acceptability and usability) were assessed using intraclass correlation (ICC), descriptive statistics, thematic and content analysis.

METHODS

Volunteers took (T1), then repeated (T2), the automated MMI, with a 7-day interval (±2) then completed an evaluation questionnaire. Admission staff participated in focus group discussions.

RESULTS

Sixty-two students and seven admission staff participated; 34 students and 4 staff from UK and 28 students and 3 staff from US universities. Good-excellent test-retest reliability was observed at two sites (US and UK2) with T1 and T2 ICC between 0.65 and 0.81 (p<0.001) when assessed by individual total scores (range 80.6-119), station total scores 0.6-0.91, p<0.005 and individual site (≥0.79 p<0.001). Mean test re-test ICC across all three sites was 0.82 p<0.001 (95% CI 0.7 to 0.9). Admission staff reported potential to reduce resource costs and bias through a more objective screening tool for preselection or to replace some MMI stations in a 'hybrid model'. Maintaining human interaction through 'touch points' was considered essential. Users positively evaluated the system, stating it was intuitive with an accessible interface. Concepts chosen for dynamic probing needed to be appropriately tailored.

CONCLUSION

These preliminary findings suggest that the system is reliable, generating consistent scores for candidates and is acceptable to end users provided human touchpoints are maintained. Thus, there is evidence for the potential of such an automated system to augment healthcare student selection.

Adapting the Admissions Interview During COVID-19: A Comparison of In-Person and Video-Based Interview Validity Evidence

COVID-19 physical distancing limited many medical schools' abilities to conduct in-person interviews for the 2020 admissions cycle. The University of Toronto (U of T) Temerty Faculty of Medicine was already in the midst of its interview process, with two-thirds of applicants having completed the in-person modified personal interview (MPI). As the university and surrounding region were shut down, the shift was made in the middle of the application cycle to a semisynchronous video-based MPI interview (vMPI) approach. U of T undertook the development, deployment, and evaluation of the 2 approaches mid-admissions cycle. Existing resources and tools were used to create a tailored interview process with the assistance of applicants. The vMPI was similar in content and process to the MPI: a 4-station interview with each station mapped to attributes relevant to medical school success. Instead of live interviews, applicants recorded 5-minute responses to questions for each station using their own hardware. These responses were later assessed by raters asynchronously. Out of 627 applicants, 232 applicants completed the vMPI. Validity evidence was generated for the vMPI and compared with the MPI on the internal structure, relationship to other variables, and consequential validity, including applicant and interviewer acceptability. Overall, the vMPI demonstrated similar reliability and factor structure to the MPI. As with the MPI, applicant performance was predicted by nonacademic screening tools but not academic measures. Applicants' acceptance of the vMPI was positive. Most interviewers found the vMPI to be acceptable and reported confidence in their ratings. Continuing physical distancing concerns will require multiple options for admissions committees to select medical students. The vMPI is an example of a customized approach that schools can implement and may have advantages for selection beyond the COVID-19 pandemic. Future evaluation will examine additional validity evidence for the tool.

Implementation and evaluation of virtual multiple mini interviews as a selection tool for entry into paediatric postgraduate training: A Queensland experience

INTRODUCTION

The Queensland Basic Paediatric Training Network (QBPTN) is the centralised pathway for entry into paediatric training in Queensland, Australia. In response to COVID-19 travel and social distancing restrictions imposed in 2020, QBPTN successfully adopted a Virtual Multiple Mini Interviews (vMMIs) model for the selection of candidates for entry into paediatric training. The authors describe the planning, implementation, challenges, and evaluation of candidates' and interviewers' experiences of vMMIs, including the differences between candidates from two geographical areas.

METHODS

The contents of six vMMI stations were similar to face-to-face MMI. Implementation required the identification of ZOOM^TM as a preferred online platform, securing venues, communication, development of contingency plans and central coordination by the network. Candidates' experiences with vMMI were explored through thematic analysis of the qualitative data from focus groups and free text responses, and descriptive analysis of SurveyMonkey© questionnaire responses. Experiences between 'metropolitan' and 'regional and interstate' candidates were compared.

RESULTS

5-minute stations with 2-minute pre-reading were used. 78 candidates and 14 interviewers participated in the selection process. All candidates attended the focus group. 58.7% of candidates responded to post vMMI questionnaire. 93% of survey responders were happy to undertake vMMI in the future, with 23% feeling they would have performed better in face-to-face. Experiences between 'metropolitan' and 'other' groups were similar. Positive experiences of participants were related to the user-friendly IT platform, successful pre-interview communications, preparation, convenience, time, and cost savings. Stress related IT failures and difficulties establishing rapport with interviewers were reported as the main negative experiences.

CONCLUSION

'vMMI' is a feasible and acceptable method of selection into paediatric training. vMMI has many benefits and can be implemented relatively quickly by addressing key logistical requirements. The model under discussion could be adapted by other centres based on local needs.

Reliability and validity of new online selection tests for midwifery students

BACKGROUND AND OBJECTIVE

Newly created midwifery bachelor programmes attract an increasing number of applicants. Hence, educators have moved away from traditional non-standardised selection methods and started developing new tools for the admission of students. Researchers have expressed concerns around the validity of such selection measures and their ability to identify the appropriate candidates. The objective of this study was to develop and implement selection procedures for a new midwifery Bachelor of Science degree in Hamburg, Germany, and assess their psychometric properties.

DESIGN

This is a mixed-methods longitudinal study on the reliability, validity and acceptability of newly developed selection criteria.

SETTING

The newly established midwifery bachelor of science in Hamburg, Germany.

PARTICIPANTS

Upon completion of their online application, all midwifery applicants were invited by email to participate in our research project and thereby informed of its purpose and voluntary nature of their participation. The total number of candidates who took the test was 366, of which 309 agreed to take part in this study. Sixty-five applicants were eventually admitted to the midwifery Bachelor programme, of which 59 were included in the study.

MEASUREMENTS AND FINDINGS

We developed two tests for cognitive ability (HAM-Mid I and II) and adapted one test for personal and professional characteristics (Casper - Computer-based Assessment for Sampling Personal Characteristics). Due to the Covid pandemic, we implemented the tests online instead of on site as initially planned. HAM-Mid I had the lowest (Cronbach's Alpha = 0.38) whereas Casper had the highest internal consistency (Cronbach's Alpha = 0.77) of all three tests. Age, previous academic achievement and native tongue were significantly associated with applicants' performance on HAM-Mid II and Casper admission tests. HAM-Mid II was associated with students' performance in the first year midwifery exam (r = 0.31, p < 0.05). Overall evaluation of HAM-Mid II and Casper was good while HAM-Mid I received a more negative feedback.

KEY CONCLUSIONS

Evidence on the reliability and predictive validity of the newly developed tests suggests that the applied admission criteria are appropriate for an objective selection of bachelor midwifery students.

IMPLICATIONS FOR PRACTICE

The interests of university and clinical stakeholders of a midwifery bachelor degree can be represented through the development of tests for the assessment of cognitive ability and personal and professional characteristics.

One Year on: An Overview of Singapore's Response to COVID-19-What We Did, How We Fared, How We Can Move Forward

Background—One year has passed since the first COVID-19 case in Singapore. This scoping review commemorates Singaporean researchers that have expanded the knowledge on this novel virus. We aim to provide an overview of healthcare-related articles published in peer-reviewed journals, authored by the Singapore research community about COVID-19 during the first year of the pandemic. Methods—This was reported using the Preferred Reporting Items for Systematic reviews and Meta-Analyses extension for Scoping Reviews (PRISMA-ScR) protocol. It included healthcare-related articles about COVID-19 published between 23 January 2020 and 22 January 2021 with a Singapore-affiliated author. MEDLINE, Embase, Scopus, Web of Science, CINAHL, PsycINFO, Google Scholar, and local journals were searched. The articles were screened independently by two reviewers. Results—The review included 504 articles. Most of the articles narrated the changes to hospital practice (210), while articles on COVID-19 pathology (94) formed most of the non-narrative papers. Publications on public health (61) and the indirect impacts to clinical outcomes (45) were other major themes explored by the research community. The remaining articles detailed the psychological impact of the pandemic (35), adaptations of medical education (30), and narratives of events (14). Conclusion—Amidst a resurgence of community cases involving variant COVID-19 strains, the resources from the research community will provide valuable guidance to navigate these uncertain times.

A SWOT analysis of Italian medical curricular adaptations to the COVID-19 pandemic: A nationwide survey of medical school leaders

BACKGROUND

There is a growing literature on how medical education adapts to the COVID-19 pandemic. However, there is a need to examine the facilitators and barriers of these adaptations. This study explores the strengths, weaknesses, opportunities, and threats (SWOT) of how Italian medical schools adapted their curricula to the COVID -19 pandemic.

METHODS

The authors conducted an online survey of directors of medical curricula in Italy. Free-text responses to open-ended questions about curricular adaptations and reflections on these adaptations were analysed using qualitative thematic analysis.

RESULTS

Twenty out of 60 Italian medical school directors completed the survey. Strengths identified were rapid responses and a spirit of cooperation. Weaknesses included dependency on clinical facilities, teachers' limited skills to use technology, and lack of mental health support for staff. Opportunities highlighted were clear government rules, new ways of teaching and a renewed focus on underrepresented topics. Threats expressed included impaired relationships, difficulties related to online assessment, lack of IT access, and legal and insurance issues.

CONCLUSIONS

This study, in documenting the curricular adaptations of Italian medical schools during an active global pandemic, and recording the perspectives of medical education leaders, offers important lessons for the future.

Online Assessment for Pathology Residents during the COVID-19 Pandemic: Report of an Experience

BACKGROUND & OBJECTIVE

The world is facing COVID-19 pandemic, and medical education system and consequently the evaluation of students at different levels have been overshadowed. Residency students are among those affected. In the present study, we aim to share our experiences regarding holding exams for pathology residents before and during the pandemic.

METHODS

This cross-sectional study was carried out in Tehran University of Medical Sciences. The online exam, which consisted of 30 multiple-choice questions, was designed and held in April 2020 to evaluate pathology residents. To assess the quality of the exam, indices such as the number of questions, highest and lowest scores, the average score, the standard deviation, the variance, Cronbach's alpha reliability coefficient, standard error of measurement, discrimination index, difficulty index, number/percentage of questions on three difficulty levels of easy, normal, hard were reviewed and analyzed.

RESULTS AND DISCUSSION

The average score of the participants in the January exam which was held in the university's exam center was 16.23 (±5.03), while the average score in the online exam which was held after the onset of the pandemic was 20.86 (±5.18). The average discrimination indices in the first and second exams were 0.36 and 0.38, respectively, and the average difficulty indices in the first and second exams were 0.54 and 0.70, respectively. We found the administration of this online examination would be a positive experience. By sharing it, we hope to pave the way for similar ventures in the other departments.