Virtual interviewing for graduate medical education recruitment and selection: A BEME systematic review: BEME Guide No. 80

Hitting the target and missing the point? A BEME systematic review of evidence regarding the efficacy of statutory and mandatory training in health and care: BEME Guide No. 87

BACKGROUND

Mandatory training is considered fundamental to establishing and maintaining high standards of professional practice. There is little evidence however, of the training either achieving its required learning outcomes, or delivering improvement in outcomes for patients. Whist organisations may be hitting their compliance target for mandatory training, is the purpose missing the point? This systematic review aims to synthesize and evaluate the efficacy of statutory and mandatory training.

METHODS

PubMed, EMBASE, CNAHL, ERIC and Cochrane Central registers were searched on 23rd May 2023. All research designs were included and reported training had to specify an organisational mandate within a healthcare setting. Data was coded using a modified Kirkpatrick (KP) rating system. Critical appraisal was undertaken using the Modified Medical Education Research Study Quality Instrument, Critical Appraisal Skills Programme Qualitative Studies checklist and Mixed Methods Assessment Tool.

RESULTS

Twenty-five studies were included, featuring 9132 participants and 1348 patient cases audited. Studies described evaluation of mandatory training according to Kirkpatrick's outcomes levels 1-4b, with the majority (68%) undertaken in the UK and within acute settings. Training duration varied from 5 min to 3 days. There is a lack of consensus regarding mandatory training rationale, core topics, duration, and optimum refresher training period. Currently, mandatory training does not consistently translate to widescale improvements in safe practice or improved patient outcomes.

CONCLUSIONS

Due to the lack of international consensus regarding the need for mandated training, most papers originated from countries with centrally administered national health care systems. The rationale for mandating training programmes remains undefined. The assumption that mandatory training is delivering safe practice outcomes is not supported by studies included in this review. The findings of this review offer a basis for further research to be undertaken to assist with the design, facilitation, and impact of mandatory training.

Applicant-Fellow Virtual Sessions in Recruitment for Gastroenterology Fellowship

BACKGROUND

Since the COVID-19 pandemic in 2020, virtual interviews have become a norm for gastroenterology (GI) fellowship recruitment. Most interviews hold a session for applicant and current fellow interaction. There is wide variability of the sessions across programs. There are a paucity of data on the influence of these sessions on applicants' ranking of programs.

AIMS

We aim to describe applicants' experiences and perceptions of virtual happy hours (i.e., applicant-fellow sessions) during the GI fellowship application process.

METHODS

We surveyed applicants participating in the 2022 GI fellowship match cycle to understand their experience with virtual fellow-only happy hours. Mixed methods analyses were performed.

RESULTS

The survey was completed by 68 (13.91%) applicants, of which, 75% reported that at least half of the interviews they attended had conducted a virtual, fellow-only happy hour. Most respondents preferred that the virtual happy hours should be conducted prior to the interview day (58%) and that breakout rooms with a smaller ratio of applicants to fellows are helpful (78%). The majority (87%) of respondents reported attending these sessions at least 75% of the time. Nearly half (44%) of respondents reported that these sessions influenced/altered their ranking decisions with respect to programs.

CONCLUSION

Given the advantages associated with virtual interviews and their ongoing support by professional societies, the virtual platform is likely here to stay in future. Virtual fellow-only happy hours help provide a representation of the program's mission and when successfully implemented, can be leveraged to optimize recruitment and attract qualified, diverse candidates.

A scoping review of artificial intelligence in medical education: BEME Guide No. 84

BACKGROUND

Artificial Intelligence (AI) is rapidly transforming healthcare, and there is a critical need for a nuanced understanding of how AI is reshaping teaching, learning, and educational practice in medical education. This review aimed to map the literature regarding AI applications in medical education, core areas of findings, potential candidates for formal systematic review and gaps for future research.

METHODS

This rapid scoping review, conducted over 16 weeks, employed Arksey and O'Malley's framework and adhered to STORIES and BEME guidelines. A systematic and comprehensive search across PubMed/MEDLINE, EMBASE, and MedEdPublish was conducted without date or language restrictions. Publications included in the review spanned undergraduate, graduate, and continuing medical education, encompassing both original studies and perspective pieces. Data were charted by multiple author pairs and synthesized into various thematic maps and charts, ensuring a broad and detailed representation of the current landscape.

RESULTS

The review synthesized 278 publications, with a majority (68%) from North American and European regions. The studies covered diverse AI applications in medical education, such as AI for admissions, teaching, assessment, and clinical reasoning. The review highlighted AI's varied roles, from augmenting traditional educational methods to introducing innovative practices, and underscores the urgent need for ethical guidelines in AI's application in medical education.

CONCLUSION

The current literature has been charted. The findings underscore the need for ongoing research to explore uncharted areas and address potential risks associated with AI use in medical education. This work serves as a foundational resource for educators, policymakers, and researchers in navigating AI's evolving role in medical education. A framework to support future high utility reporting is proposed, the FACETS framework.

A Narrative Review of Key Studies in Medical Education in 2022: Applying the Current Literature to Educational Practice and Scholarship

The expectation of every academic pediatrician is to stay updated on current evidence in their field; this is especially true of pediatric clinician educators who are training the next generation of pediatricians. Since 2016, select members of the Academic Pediatric Association Education Committee have curated educational research articles in order to distill the increasing volume of research related to medical education. The purpose of this narrative review is to summarize 14 articles published in 2022 related to medical education that may impact the work of pediatric clinician educators and educational leadership. These articles are organized into 6 overarching domains: selection and recruitment, promoting learner growth and development, learning environment and wellness, curriculum development, assessment, and educator development.

The mitigated carbon emissions of transitioning to virtual medical school and residency interviews: A survey-based study

PURPOSE

Prior to COVID, thousands of medical school and residency applicants traversed their countries for in-person interviews each year. However, data on the greenhouse gas emissions from in-person interviews is limited. This study estimated greenhouse gas emissions associated with in-person medical school and residency interviews and explored applicant interview structure preferences.

METHODS

From March to June 2022, we developed and distributed a nine-question, website-based survey to collect information on applicant virtual interview schedule, demographics and preference for future interview format. We calculated theoretical emissions for all interviews requiring air travel and performed a content analysis of interview preference explanations.

RESULTS

We received responses from 258 first-year and 253 fourth-year medical students at 26 allopathic US medical schools who interviewed virtually in 2020-2021 and 2021-2022, respectively. Residency applicants participating in the study were interviewed at a mean of 15.3 programs (SD 5.4) and had mean theoretical emissions of 4.31 tons CO2 eq. Medical school applicants participating in the study were interviewed at a mean of 6.9 programs and had mean theoretical emissions of 2.19 tons CO2 eq. Ninety percent of medical school applicants and 91% of residency applicants participating in the study expressed a preference for hybrid or virtual interviews going forward.

CONCLUSION

In-person medical training interviews have significant greenhouse gas emissions. Virtual and hybrid alternatives have a high degree of acceptability among applicants.

The impact of virtual interviews on the Fellowship Council application and main match process

BACKGROUND

The COVID-19 pandemic led the Fellowship Council (FC) to transition rapidly from in-person to virtual interviews. We investigated the impact of this transition on the FC application and main match process.

METHODS

Five years (2018-2022) of deidentified FC applicant, program, and match rank data were used to assess differences between in-person (2018-2019) and virtual interview (2021-2022) cycles. Data are expressed as mean ± SD and one-way and two-way MANOVA tests were applied.

RESULTS

Trainees applied to an average of 30.4 ± 24.3 programs and ranked an average of 10.7 ± 9.7 programs with a 57% match rate and average rank position of 3.6 ± 3.3. Fellowship programs received an average of 64.9 ± 28.6 applications and ranked an average of 15.4 ± 8.8 applicants with a 95% match rate and average applicant rank position of 3.0 ± 3.4. Applicants who interviewed virtually applied to a greater number of programs (32.7 vs. 27.0; p < 0.001) and ranked a greater number of programs (11.5 vs. 10.0; p = 0.004) with no difference in match rates (58% vs. 55%, p = 0.291). Among matched applicants, there was a significant difference in average rank position (3.20 vs. 4.30, p < 0.001), favoring the in-person cohort. Fellowship programs had more applicants per program (69.2 vs. 57.8; p < 0.001) and ranked more applicants (17.4 vs. 13.3; p < 0.001) during the virtual interview cycles. No difference in either match rates (93% vs. 96%, p = 0.178) or applicant rank position (3.09 vs. 2.93, p = 0.561) was seen between in-person and virtual application cycles.

CONCLUSION

Virtual interviews were associated with an increased number of applications for fellowship and applicants ranked by programs but did not impact match rates of either group. Rank match position declined somewhat for applicants but not for fellowship programs. Virtual interviews offer more opportunities for applicants and a greater number of candidates for fellowship programs with only a slight decrement in fellow match rank position.

Reimagining the Rheumatology Fellowship Interview: Using Participatory Design-Thinking Process to Create a More Applicant-Centered Experience

OBJECTIVE

Design thinking is a creative problem-solving process used to better understand users' needs and experiences so that a product or service can be improved. Its emphasis on empathy, iterative prototyping, and participatory collaboration make it an ideal methodology for innovation in medical education. We apply this framework to the virtual rheumatology fellowship interview process so that interviews can become more applicant centered.

METHODS

This educational quality improvement project uses a design-thinking framework to identify opportunities and challenges for rheumatology fellowship applicants. The investigators use the 5-step process (Empathize, Define, Ideate, Prototype, Test) and incorporate rapid qualitative analysis of semistructured interviews to innovate the interview experience. The iterative and collaborative nature of this process has empowered participants to codesign an applicant-centered interview experience.

RESULTS

Interviews with fellowship applicants (n = 9), fellow physicians (n = 4), and faculty members (n = 3) identified three major dynamics of the interview process: (1) Is it a safe environment to ask questions? (2) How do I exchange information effectively? and (3) How do I fit all these data into the bigger picture? Creative brainstorming techniques at a series of three workshops yielded four prototypes emphasizing customization, hybridization, facilitation, and preparation. A finalized applicant-centered interview template was devised in preparation for the 2023-2024 application season.

CONCLUSION

Design thinking has yielded insights into three important dynamics that drive applicant experiences. These insights allow for a redesign of processes so that virtual interviews can be more applicant centered. This framework allows for further iterations and modifications as the needs of applicants and programs evolve over time.

Distribution of emergency medicine programs by location and reproductive rights: A cross-sectional study of the 2023 Match

INTRODUCTION

The 2023 Match saw over 500 unfilled positions in emergency medicine (EM). Geographic location is the third most important factor for all United States (US) EM-bound senior medical students when selecting programs to rank and can be affected by political climate. Given the perceived importance of geography on program selection and recent changes to reproductive rights in the US, we sought to evaluate the impact of geography and reproductive rights on unmatched positions among EM programs.

METHODS

This was a cross-sectional study assessing Match rates in EM by program US state, region, and degree of reproductive rights. We included all EM programs participating in the 2023 Match year. Our primary outcome was to determine the unfilled programs and positions rate per US state. Secondary outcomes included Match rates by region and by degree of reproductive rights.

RESULTS

We found notable differences in unfilled programs by US state, with the highest percentage of unfilled programs and positions in Arkansas (100%, 56.3%), Nevada (100%, 35.5%), Kansas (100%, 40.0%), Ohio (81.3%, 33.3%), and Michigan (80.0%, (36.8%). Among regions, the highest percentage of unfilled programs (62.5%) and residency positions (26.0%) was East North Central (IL, IN, MI, OH, WI). US states with limited reproductive rights had the highest percent (52.9%) of programs with unmatched positions and the highest percent (20.5%) of unfilled positions.

CONCLUSION

We identified notable differences in unmatched positions by US state and region, as well as the highest rate of unmatched positions in US states with more limited reproductive rights.