Audiovisual quality impacts assessments of job candidates in video interviews: Evidence for an AV quality bias

Cognitive perspectives on maintaining physicians' medical expertise: III. Strengths and weaknesses of self-assessment

Is self-assessment enough to keep physicians' cognitive skills-such as diagnosis, treatment, basic biological knowledge, and communicative skills-current? We review the cognitive strengths and weaknesses of self-assessment in the context of maintaining medical expertise. Cognitive science supports the importance of accurately self-assessing one's own skills and abilities, and we review several ways such accuracy can be quantified. However, our review also indicates a broad challenge in self-assessment is that individuals do not have direct access to the strength or quality of their knowledge and instead must infer this from heuristic strategies. These heuristics are reasonably accurate in many circumstances, but they also suffer from systematic biases. For example, information that feels easy to process in the moment can lead individuals to overconfidence in their ability to remember it in the future. Another notable phenomenon is the Dunning-Kruger effect: the poorest performers in a domain are also the least accurate in self-assessment. Further, explicit instruction is not always sufficient to remove these biases. We discuss what these findings imply about when physicians' self-assessment can be useful and when it may be valuable to supplement with outside sources.

Strategies for recruiting underrepresented in medicine and sexual and gender minority students to emergency medicine

Background

Despite having well-described benefits, diversifying the physician workforce has been an ongoing challenge. Within emergency medicine (EM), multiple professional organizations have identified expanding diversity and inclusion as top priorities. The following is a description of an interactive session held at the SAEM annual meeting addressing recruitment strategies for underrepresented in medicine (URiM) and sexual and gender minority (SGM) students into EM.

Methods

During the session, the authors provided an overview of the current state of diversity in EM. In the small-group portion of the session, a facilitator helped characterize the challenges programs face in recruiting URiM and SGM students. These challenges were described during three distinct phases of the recruitment process: (1) preinterview, (2) interview day, and (3) postinterview.

Results

Our facilitated small-group session allowed for discussing the challenges faced by various programs in recruiting a diverse group of trainees. Common challenges in the preinterview and interview day included messaging and visibility as well as funding and support. Postinterview challenges included communication and the ranking process. Through this exercise, we were able to collaboratively share ideas on tangible solutions that programs may use to overcome their specific challenges.

Conclusions

Given the importance of intentionality in diversifying the physician workforce, the authors describe successful strategies implemented within one residency program and those shared by session participants to overcome recruitment challenges.

A Canadian survey of residency applicants' and interviewers' perceptions of the 2021 CaRMS R1 virtual interviews

BACKGROUND

All Canadian Residency Matching Service (CaRMS) R1 interviews were conducted virtually for the first time in 2021. We explored the facilitators, barriers, and implications of the virtual interview process for the CaRMS R1 match and provide recommendations for improvement.

METHODS

We conducted a cross-sectional survey study of CaRMS R1 residency applicants and interviewers across Canada in 2021. Surveys were distributed by email to the interviewers, and by email, social media, or newsletter to the applicants. Inductive thematic analysis was used for open-ended items. Recommendations were provided as frequencies to demonstrate strength. Close-ended items were described and compared across groups using Chi-Square Fisher's Exact tests.

RESULTS

A total of 127 applicants and 400 interviewers, including 127 program directors, responded to the survey. 193/380 (50.8%) interviewers and 90/118 (76.3%) applicants preferred virtual over in-person interview formats. Facilitators of the virtual interview format included cost and time savings, ease of scheduling, reduced environmental impact, greater equity, less stress, greater reach and participation, and safety. Barriers of the virtual interview format included reduced informal conversations, limited ability for applicants to explore programs at different locations, limited ability for programs to assess applicants' interest, technological issues, concern for interview integrity, limited non-verbal communication, and reduced networking. The most helpful media for applicants to learn about residency programs were program websites, the CaRMS/AFMC websites, and recruitment videos. Additionally, panel interviews were preferred by applicants for their ability to showcase themselves and build connections with multiple interviewers. Respondents provided recommendations regarding: (1) dissemination of program information, (2) the use of technology, and (3) the virtual interview format.

CONCLUSIONS

Perceptions of 2021 CaRMS R1 virtual interviews were favourable among applicants and interviewers. Recommendations from this study can help improve future iterations of virtual interviews.

Cooking through perceptual disfluencies: The effects of auditory and visual distortions on predicted and actual memory performance

The current study investigated the joint contribution of visual and auditory disfluencies, or distortions, to actual and predicted memory performance with naturalistic, multi-modal materials through three experiments. In Experiments 1 and 2, participants watched food recipe clips containing visual and auditory information that were either fully intact or else distorted in one or both of the two modalities. They were asked to remember these for a later memory test and made memory predictions after each clip. Participants produced lower memory predictions for distorted auditory and visual information than intact ones. However, these perceptual distortions revealed no actual memory differences across encoding conditions, expanding the metacognitive illusion of perceptual disfluency for static, single-word materials to naturalistic, dynamic, multi-modal materials. Experiment 3 provided naïve participants with a hypothetical scenario about the experimental paradigm used in Experiment 1, revealing lower memory predictions for distorted than intact information in both modalities. Theoretically, these results imply that both in-the-moment experiences and a priori beliefs may contribute to the perceptual disfluency illusion. From an applied perspective, the study suggests that when audio-visual distortions occur, individuals might use this information to predict their memory performance, even when it does not factor into actual memory performance.

Effect of teleconferencing variables on faculty impression of mock residency applicants

Purpose

The objective of this study was to assess how teleconferencing variables influence faculty impressions of mock residency applicants.

Methods

In October 2020, we conducted an online experiment studying five teleconferencing variables: background, lighting, eye contact, internet connectivity, and audio quality. We created interview videos of three mock residency applicants and systematically modified variables in control and intervention conditions. Faculty viewed the videos and rated their immediate impression on a 1-10 scale. The effect of each variable was measured as the mean difference between the intervention and control impression ratings. One-way analysis of variance (ANOVA) was performed to assess whether ratings varied across applicants. Paired-samples Wilcoxon signed-rank tests were conducted to assess the significance of the effect of each variable.

Results

Of 711 faculty members who were emailed a link to the experiment, 97 participated (13.6%). The mean ratings for control videos were 8.1, 7.2, and 7.6 (P < .01). Videos with backlighting, off-center eye contact, choppy internet connectivity, or muffled audio quality had lower ratings when compared with control videos (P < .01). There was no rating difference between home and conference room backgrounds (P = .77). Many faculty participants reported that their immediate impressions were very much or extremely influenced by audio quality (60%), eye contact (57%), and internet connectivity (49%).

Conclusions

Teleconferencing variables may serve as a source of assessment bias during residency interviews. Mock residency applicants received significantly lower ratings when they had off-center eye contact, muffled audio, or choppy internet connectivity, compared to optimal teleconferencing conditions.

Supplementary Information

The online version contains supplementary material available at 10.1007/s44186-022-00053-w.

Avoiding the Virtual Pitfall: Identifying and Mitigating Biases in Graduate Medical Education Videoconference Interviews

Public health concerns related to the COVID-19 pandemic are leading many residency and fellowship programs to transition from in-person to videoconference interviews (VCIs). The magnitude and speed of the shift to VCIs, the lack of existing research around bias and VCIs, and the underlying stress on all involved related to the pandemic put programs at risk of implementing virtual interviews without fully exploring their implications for diversity and equity. VCIs can promote diversity efforts by reducing the need for travel, making interviews more convenient and cost effective for applicants. However, VCIs may also introduce new biases and amplify existing biases in recruitment. VCIs introduce a dependence on technology to conduct the interview process, which may amplify systemic inequities in access to broadband internet and high-quality hardware. Communication delays due to technology challenges may negatively affect interview scores. Additionally, users experience increased cognitive load when participating in videoconferences, which can activate implicit biases. Exposure to cues in the interviewee's personal living situation previously unavailable to interviewers may lead to unconscious assumptions by interviewers, which may also influence scoring. Graduate medical education programs committed to maintaining equitable recruitment processes must be able to recognize potential biases in VCIs and implement strategies to mitigate them. This article identifies some of the biases VCIs can introduce to the recruitment process and offers strategies for programs to mitigate them. These include making interviewers aware of potential technology-based inequities, encouraging interviewers to minimize multitasking, and offering guidance on use of standardized backgrounds. The authors also recognize the limitations of offering behavioral strategies to mitigate systemic inequities and suggest that structural changes are needed to ensure equitable access to technology.

Answering a factual question today increases one's confidence in the same answer tomorrow - independent of fluency

We investigated the cognitive processes that cause confidence to increase. Participants were asked 48 general-knowledge questions either once or three times, without feedback. After 2 min (Experiment 1) or 48 h (Experiment 2) they were asked the same questions again, and rated their confidence. Repeated questioning increased confidence but not accuracy. This increase, which replicated research on episodic memory in the eyewitness literature (e.g., Shaw, Journal of Experimental Psychology: Applied 2: 126-146, 1996), occurred even though accuracy was only around 25%. A mediation analysis identified response repetition, but not fluency, as a mechanism underlying growth in confidence. Thus, the basis for confidence judgments appears to be whether one's current response has been generated previously. In sum, answering a factual question increases confidence, but not accuracy, and this happens because learners use response repetition as a cue for confidence judgments.