An equivalence study of interview platform: Does videoconference technology impact medical school acceptance rates of different groups?

Virtual and In-Person Multiple Mini-interviews: A Comparison of Two Modalities in Regard to Bias

Purpose

To examine the characteristics between virtual multiple mini-interview (vMMI) and in-person interviews (ipMMI) in regard to difference in performance between applicant-reported gender identity and racial groups.

Methods

Retrospective multiple mini-interview (MMI) data from two vMMI interview cycles (2021 and 2022) consisting of 627 applicants and four ipMMI cycles (2017-2020) consisting of 2248 applicants. Comparisons were made between applicant subgroups including reported gender (male and female) and minority status (URiM and non-URiM). A three-way analysis of variance (ANOVA) was conducted to examine the effects of gender, URiM status, and interview modality (in-person vs. virtual) on MMI scores.

Results

There were no overall significant differences between annual ipMMI and vMMI scores. A significant main effect of gender was observed, with females scoring higher than males overall. An interaction between gender and URiM status was also found. Although not statistically significant, when the MMI was virtual, URiM applicants on average scored higher than non-URiM applicants. In both the ipMMI and vMMI, URiM males tended to score lower than their non-URiM counterparts, though this difference was not statistically significant. URiM females tended to score higher than non-URiM females during the vMMI, and this difference was statistically significant.

Conclusions

The switch to vMMI shows that there are no overall significant differences between the in-person and virtual formats; however, the finding that female URiM's better performance in the virtual setting is novel. The cause of this finding is unknown but most likely reflects the complex interaction between race and gender. This insight requires future study and builds on the evidence that the MMI is an admissions tool to mitigate bias.

Some Perceptions Differ, Match Outcomes Do Not: A Multisite Retrospective Cross-Sectional Comparison of Virtual vs. In-Person Recruitment

BACKGROUND

Virtual interviewing for residency provides considerable savings. Its impact on match outcomes remains unclear.

OBJECTIVE

Evaluate the impact of virtual residency recruitment on program and applicant assessment and match outcomes.

DESIGN

Cross-sectional survey, September 2020-July 2021 PARTICIPANTS: Faculty interviewers and 2019 and 2020 PGY-1 classes at three academic internal medicine residencies.

MAIN MEASURES

Survey items rating effectiveness of interview format, preference for future interview format, and perceived impact on diversity.

KEY RESULTS

A total of 247/436 faculty (57%) interviewers responded. Faculty perceived that in-person interviews enhanced applicant assessment (3.23 ± 0.38, p < 0.01) and recruitment of the most qualified applicants (p < 0.01) but did not impact recruitment of a racially or gender diverse class (3.03 ± 0.99, p = 0.95 and 3.09 ± 0.76, p = 0.14 respectively). They also did not demonstrate a preference for future interview formats. A total of 259/364 matched applicants responded, corresponding to a 76% response rate in the in-person cohort and a 66% response rate for virtual. Trainees were equally likely to match at their top choice when interviewing virtually vs. in-person (p = 0.56), and racial/ethnic and gender composition of the incoming class also did not differ (p = 0.81 and p = 0.19 respectively). Trainees perceived many aspects of the institution were better assessed in-person, though the impact varied according to assessment domain. Trainees who interviewed in-person preferred in-person formats. Of those who interviewed virtually, 47% preferred virtual and 54% preferred in-person. There were no predictors of virtual preference for future interview formats.

CONCLUSIONS

Faculty and applicants who experienced virtual recruitment had no preference for future recruitment format. Virtual recruitment had no impact on the racial/gender diversity of matched classes or on applicants matching at their top-ranked institution. Institutions should consider the potential non-inferiority of virtual interviews with financial and other benefits when making decisions about future interview formats.

Enhancing collaborative learning in health management education: an investigation of Padlet-mediated interventions and the influence of flexible thinking

This study had three primary goals. First, it aimed to craft an intervention program centered around collaborative learning enabled by Padlet. Second, it aimed to gauge the perceptions of health management students regarding this intervention and how it affected their collaborative learning experiences. Additionally, the third objective of the study aimed to investigate how students' flexible thinking within the learning process might shape their perceptions of the advantages derived from this instructional activity within the domain of online collaborative learning. Data for the analysis were gathered from 100 Israeli undergraduate students by two measurements: Flexible thinking in learning and Student perceptions of collaborative learning via Padlet. The intervention program included several stages. First, the students discussed the pedagogic objective of using Padlet. In the second stage, the students were presented with ill-structured problems related to the course content. Each group had to choose one problem and analyze it from three perspectives discussed in the course-healthcare provider, patient, and organization. Next, the students presented and explained their solutions employing the shared knowledge base. The final work was presented in different formats using various technologies. The PLS-SEM analysis has corroborated our hypothesis that students' flexible thinking might positively contribute to their perception of Padlet utilization. According to the empirical model, in general, students who perceived themselves as more flexible were found more receptive to utilizing the proposed technological tool (Padlet) and hence tended to appreciate its function as a collaborative learning platform enabler. This study mainly underscores the important role flexible thinking plays in motivating managers and medical professionals to embrace innovative technologies or methods for teamwork, that could enable them to weigh arguments, seek alternative solutions to authentic problems, and adjust their approaches effectively and collaboratively as new challenges emerge.

Video-based interviewing in medicine: a scoping review

BACKGROUND

The Coronavirus 2019 pandemic necessitated a rapid uptake of video-based interviewing within the personnel selection process in healthcare. While video-based interviews have been evaluated previously, we identified a gap in the literature on the implementation of video-based interviews and how they compare to their face-to-face counterparts.

METHODS

A scoping review was conducted to consolidate the available literature on the benefits and limitations of video-based interviews and to understand the perceived barriers associated with transitioning away from face-to-face interviews. A search strategy, developed in concert with an academic health sciences librarian, was run on Ovid MEDLINE, Embase, PsycInfo, and Cochrane Central. The search was performed on March 31, 2020, and updated on February 21, 2021. Studies that implemented and evaluated the impact of video-based interviewing in healthcare were included in our study. Review articles and editorials were excluded.

RESULTS

Forty-three studies were included in our scoping review, of which 17 were conference abstracts and 26 were peer-reviewed manuscripts. The risk of bias was moderate or high in most studies, with only four studies having a low risk of bias. Both financial costs and opportunity costs associated with the selection process were reported to be improved with video-based interviewing, while no studies explored the impact on environmental costs. Technical limitations, which were not prevalent, were easily managed during the interview process. Overall, video-based interviews were well received by both applicants and interviewers, although most participants still reported a preference for face-to-face interviews.

CONCLUSIONS

While video-based interviewing has become necessary during the Coronavirus 2019 era, there are benefits from a financial, opportunistic, and environmental point of view that argue for its continued use even after the pandemic. Despite its successful implementation with minimal technical issues, a preference still remains for face-to-face interviews. Reasons for this preference are not clear from the available literature. Future studies on the role of nonverbal communication during the video-based interview process are important to better understand how video-based interviewing can be optimized.

SYSTEMATIC REVIEW REGISTRATION

This scoping review was registered with Open Science Framework.

Holistic Admissions Review in Nursing Education: A Concept Analysis Using Rodgers' Evolutionary Method

AIM

The aim of this study was to analyze holistic review within the context of collegiate nursing program admissions.

BACKGROUND

Holistic review is the evaluation of individual qualifications for college admission based on a pattern of individualized factors, including standardized scores, personal characteristics, experiences, abilities, and educational/professional "fit." Medical schools regularly use holistic review, and more nursing schools may adopt holistic review in the future.

METHOD

Rodgers' evolutionary method of concept analysis was used to examine usage in current praxis.

RESULTS

Findings support a desire to enhance diversity in collegiate, health-related programs while increasing numbers of culturally competent, qualified providers representative of the US population.

CONCLUSION

Holistic review promotes diversity of nursing students and future nurse providers. This concept analysis provides a unified definition of holistic review to promote educational and health-care-related equity through increased representativeness of nursing students and future nurses.

The virtues of the virtual medical school interview

The COVID-19 pandemic has mandated the use of virtual interactions in medical school. Although this falls mainly in the area of didactic instruction, of necessity, it has extended to the critical Admissions Process and the Medical School Interview itself. The California University of Science and Medicine (CUSM) with their flipped classroom approach had previously entered a virtual space of instruction even before COVID-19. Because CUSM was, in a sense, already committed to 'virtual' education, in the face of the COVID-19 pandemic, CUSM focused not on what it might lose but what it might gain and what their applicants to medical school might gain with the virtual format. The COVID-19 pandemic provided a unique opportunity to initially compare the Virtual Interview with the traditional On-Campus (In-Person) Interview during the hybrid 2020 year when the COVID-19 pandemic began. The Virtual Interview was patterned after the On-Campus Interview with some modifications. The same faculty conducted both interviews. A number of inherent advantages of the Virtual Interview surfaced to these faculty interviewers based on their subjective observations and conclusions. The overall interviewee satisfaction with the Virtual Interview was very positive based on their subjective observations and conclusions. The objective data from the Virtual Interviews compared to the On-Campus Interviews in the hybrid year resulted in a greater percentage of both offers of acceptance (p = .001) and matriculations (p = .001). In order to strengthen our initial observations, we expanded our study to include 2 pre-COVID-19 years (2018, 2019) of exclusively On-Campus interviews (n = 743) and 1 additional COVID-19 year (2021) of exclusively Virtual Interviews (n = 529). In this expanded study, interviewee demographics were not confounding and the Virtual Interview gave rise to overall greater interviewee satisfaction (p = .001), a trend to greater interviewer satisfaction and a greater percentage of both offers of acceptance (p = .047) and matriculations (p = .036).

Conducting multiple mini-interviews in the midst of COVID-19 pandemic

Multiple mini-interview (MMI) is a 'multiple sample-based' approach comprising multiple focused encounters intended to access and assess a range of attributes in order to gain more objectively multiple impressions of an applicant's interpersonal skills, thoughtfulness and general demeanour. It is designed to focus on four domains that are not considered to be comprehensive, but are considered to be vital for a successful career in the health sciences: critical thinking, ethical decision making, communication and knowledge of the healthcare system. Traditionally, the MMI is conducted face-to-face, but with COVID-19 pandemic and the implementation of social distancing measures, no onsite or campus teaching, banning of mass gatherings and cancellation of face-to-face interviews, Pengiran Anak Puteri Rashidah Sa'adatul Bolkiah Institute of Health Sciences at Universiti Brunei Darussalam explored the feasibility of conducting MMI through virtual means. This report provides an account of our experience in conducting internet-MMI for the selection of new applicants into the August 2020 cohort of the Medicine programme. We also aimed to determine whether the scores derived from internet-MMI were reliable and equivalent to the scores derived from traditional MMI.

Virtual Interviews During COVID-19: Perspectives of Female Pelvic Medicine and Reconstructive Surgery Program Directors

Supplemental digital content is available in the text.

Objective

The aim of the study was to evaluate female pelvic medicine and reconstructive surgery (FPMRS) fellowship program directors’ opinions regarding the effectiveness of virtual interviews for selecting fellows and their future interview mode preferences.

Methods

This was a cross-sectional online survey of all FPMRS program directors in the United States conducted from April 29, 2020, to May 30, 2020. At the time of this study, there were 73 program directors and 69 obstetrics and gynecology and urology–accredited FPMRS programs nationwide. The primary outcome was to subjectively assess the effectiveness of virtual interviews as compared with in-person interviews for evaluating applicants.

Results

Fifty seven (82.6%) of the program directors completed the survey. A total of 80.7% (46/57) of the respondents had participated in interviews for the active match cycle. Of the programs that participated in the interview process, almost all conducted interviews using virtual platforms (97.8%, 45/46). Program directors who conducted interviews virtually found them effective in evaluating applicants (88.9%, 40/45) and were satisfied with the virtual interview process (86.7%, 39/45). A total of 31.1% of respondents (14/45) preferred a virtual platform to an in-person setting for future interviews, and 60% (27/45) reported that they will likely perform future interviews virtually.

Conclusions

Although the pandemic resulted in a sudden reformatting of FPMRS fellowship interviews, most program directors nationally were satisfied with the process and found virtual interviews effective for assessing applicants. More than 50% of FPMRS program directors are likely to consider the virtual format for future interviews.

Key Insights From the Development and Implementation of a Novel Virtual Interview Process for Medical School Admissions During the COVID-19 Pandemic

PROBLEM

During the COVID-19 pandemic, medical schools have offered a virtual application process. Minimal literature is available to guide best practices.

APPROACH

The Icahn School of Medicine at Mount Sinai (ISMMS) implemented a completely virtual interview (VI) process in April/May 2020. Large-group sessions for applicants, interviews with applicants, and ISMMS Admissions Committee meetings occurred via Zoom (Zoom Video Communications, Inc., San Jose, California). Large-group sessions and committee meetings occurred via communal conference calls, while one-on-one, semistructured interviews occurred in individual breakout rooms. ISMMS offered live, virtual question-and-answer sessions with students and faculty, plus digital resources describing program features. After the interview day, applicants and interviewers were invited to complete anonymous surveys regarding their experiences with and perspectives of VI.

OUTCOMES

Of 125 applicants and 20 interviewers, 99 (79%) and 18 (90%), respectively, completed at least part of the survey. Of the applicants, 85/95 (89%) agreed VI met or exceeded expectations, with many praising the day's organization and convenience, and 71/95 (75%) agreed they received a sufficient sense of the student body. A minority (n = 39/95 [41%]) felt limited in their ability to learn about the institution (commonly related to their inability to tour campus), and a majority (n =74/91 [81%]) would have preferred an in-person interview. Most interviewers felt comfortable assessing applicants' verbal communication skills (n = 13/16 [81%]), and most (n = 12/17 [71%]) felt VI should be an option for future applicants.

NEXT STEPS

VI, likely to be a temporary-but-universal fixture of upcoming application cycles, may benefit applicants and interviewers alike by saving resources and diversifying those involved. Next steps are developing programming that will permit applicants to virtually explore the institution and connect more with current students. Future research should evaluate potential bias in VI to ensure an equitable application process for all.

Medical Student Selection

Medical school admissions committees are tasked with fulfilling the values of their institutions through careful recruitment. Making accurate predictions regarding the enrollment behavior of admitted students is critical to intentionally formulating class composition and impacts long-term physician representation.

Perceptions and lessons from web-based interviews for the reproductive endocrinology and infertility fellowship application cycle

OBJECTIVE

To evaluate the experience and perceptions of reproductive endocrinology and infertility fellowship applicants and program directors (PDs) regarding the current and future use of web-based interviews (WBIs).

DESIGN

Cross-sectional study.

SETTING

Nationwide cohort.

PATIENT(S)

Reproductive endocrinology and infertility fellowship applicants and PDs participating in the 2020 application cycle.

INTERVENTION(S)

Anonymous survey sent to applicants and PDs.

MAIN OUTCOME MEASURE(S)

Descriptive study evaluating the experience and satisfaction of applicants and PDs with WBIs.

RESULT(S)

Forty-six percent of applicants and eligible PDs responded to our survey. Most applicants and PDs responded that WBIs were adequate for conveying a sense of a program's strengths, faculty, diversity, clinical training, and research opportunities, but less than half responded that WBIs were adequate in providing a sense of the program's clinical site and facilities. After WBIs, both applicants (73%) and PDs (86%) were able to rank with confidence. The cost of WBIs was significantly lower for both applicants (median: $100) and programs (median: $100) than the costs previously reported for in-person interviews. The applicants interviewed at more programs than they would have if the interviews were on-site, and Zoom was the highest rated platform used. Most applicants and PDs responded that WBIs were an adequate substitute, and that they should continue after the coronavirus disease 2019 pandemic. Furthermore, most of the PDs were planning to continue to use WBIs in some capacity.

CONCLUSION(S)

Both applicants and PDs had favorable experiences with and perceptions of WBIs, and most endorse the continued use of this interview modality. The findings of this study can help guide and optimize future WBI practices.

Preparing for the interviewing process during Coronavirus disease-19 pandemic: Virtual interviewing experiences of applicants and interviewers, a systematic review

Purpose

Coronavirus disease-19 (COVID-19) has forced upon all academic institutions to conduct virtual interviewing (VI) instead of face-to-face interviewing (FTFI) this interviewing cycle. The purpose of this systematic review was to understand the process of VI, its effectiveness as an alternative to FTFI, and the experiences of applicants and institutions with VI. We also share best practice strategies for applicants and institutions in VI preparation.

Method

PubMed/MEDLINE, Cochrane Library of Systematic Reviews, Web of Science Core Collection, Scopus and CINAHL databases were searched through May 2020. Articles in English evaluating the effectiveness of VI were included, without applying any date limits. Two reviewers selected articles and extracted data.

Results

Of the 934 articles screened, 22 articles underwent full-text article analysis to include 15 studies. There were 4 studies that reported the use of VI as a screening tool. 11 studies completely replaced FTFI with VI. Most applicants could appropriately convey themselves through VI. Most applicants and interviewing programs expressed reservations about VI’s use as an alternative to FTFI.

Conclusion

There is dearth of evidence supporting the efficacy of VI. There is an opportunity for potential research at multi-institutional level to gain better understanding of the efficacy of VI. The knowledge obtained from this systematic review has the potential of helping applicants and institutions in preparing for VI process. Additionally, authors propose supportive strategies to help prepare applicants and institutions for VI.

Direct Comparison of In-Person Versus Virtual Interviews for Complex General Surgical Oncology Fellowship in the COVID-19 Era

BACKGROUND

In the era of coronavirus disease 2019 (COVID-19), many Complex General Surgical Oncology (CGSO) fellowship programs implemented virtual interviews (VI) during the 2020 interview season. At our institution, we had the unique opportunity to conduct an in-person interview (IPI) prior to the pandemic-related travel restrictions, and a VI after the restrictions were in place.

OBJECTIVE

The goal of this study was to understand how the VI model compares with the traditional IPI approach.

METHODS

Online surveys were distributed to both groups, collecting feedback on their interview experience. Responses were evaluated using a two-sample t test assuming equal variances.

RESULTS

Twenty-three of 26 (88%) applicants completed the survey. Most applicants reported that the interview gave them a satisfactory understanding of the CGSO fellowship (100% IPI, 92% VI) and the majority in both groups felt that the interview experience allowed them to accurately represent themselves (92% and 82%, respectively). All participants in the IPI group felt they were able to get an adequate understanding of the culture of the program, while only 64% in the VI group agreed with that statement (p = 0.02). IPI applicants were more likely to agree that the interview experience was sufficient to allow them to make a ranking decision (92% vs. 54%; p = 0.04).

CONCLUSIONS

While the VI modality offers several advantages over the IPI, it still falls short in conveying some of the more subjective aspects of the programs, including program culture. Strategies to provide applicants with better insight into these areas during the VI will be important moving forward.

The economic costs of selecting medical students: An Australian case study

CONTEXT

The design of selection methods must balance, amongst a range of factors, the desire to select the best possible future doctors with the reality of our resource-constrained environment. Examining the cost of selection processes enables us to identify areas in which efficiencies may be gained.

METHODS

A cost description study was conducted based on selection for 2018 entry into medical school directly from secondary school. The perspectives of applicants, volunteer interviewers and the admitting institution were considered. Costs were modelled based on the Monash University (Australia) selection process, which uses a combination of secondary school matriculation score, aptitude test score (Undergraduate Medicine and Health Sciences Admission Test) and multiple mini-interview score. A variety of data sources were utilised, including bespoke surveys, audit data and existing literature. All costs are expressed in 2018 Australian dollars (AU$). Applicant behaviours in preparing for selection tests were also evaluated.

RESULTS

A total of 381 of 383 applicants returned the survey. Over 70% of applicants had utilised commercial preparation materials. The median total cost to applicants was AU$2586 (interquartile range [IQR] AU$1574-3999), including costs to both prepare for and attend selection tests. Of 217 volunteer interviewers, 108 returned the survey. These were primarily health professional clinicians at a mid-career stage. The median total cost to interviewers was AU$452 (IQR AU$252-715) for participation in a half-day interview session, largely due to the loss of income. The cost to the admitting institution was AU$269 per applicant, accounted for by the costs of equipment and consumables (52%), personnel (34%) and facilities (14%).

CONCLUSIONS

The costs of student selection for medical school are substantial. Understanding costs facilitates achievement of the objective of selecting the desired future medical workforce within the constraints of the resources available. Opportunities for change may arise from changes in applicant preparation behaviours, opportunities for economies of scale, and efficiencies driven by technological solutions.