The Use of Task-Evoked Pupillary Response as an Objective Measure of Cognitive Load in Novices and Trained Physicians: A New Tool for the Assessment of Expertise

Understanding The Role of Cognitive Load In Paramedical Contexts: A Systematic Review

Objectives: Cognitive load refers to the working memory resources required during a task. When the load is too high or too low this has implications for an individual's task performance. In the context of paramedicine and emergency medical services (EMS) broadly, high cognitive load could potentially put patient and personnel safety at risk. This systematic review aimed to determine the current understanding of the role of cognitive load in paramedical contexts.Methods: To do this, five databases were searched (Elsevier Embase, ProQuest Psychology, CINAHL, Ovid Medline, and Ovid PsychINFO) using synonyms of cognitive load and paramedical contexts. Included articles were full text, peer reviewed empirical research, with a focus on cognitive load and EMS work. Two reviewers screened titles, abstracts, and full text using a traffic light system against the inclusion and exclusion criteria. The quality of evidence was assessed using the GRADE framework. This study was registered on PROSPERO (CRD42022384246). No funding was received for this research.Results: The searches identified 73 unique articles and after title/abstract and full text screening, 25 articles were included in the final review. Synthesis of the research revealed 10 categories of findings in the area. These are clinical performance, cognitive processes, emotional responses, physical expenditure, physiological responses, equipment and ergonomics, expertise and experience, multiple loads, cognitive load measures, and task complexity.Conclusions: From these findings it was determined that there is agreement in terms of what factors influence cognitive load in paramedical contexts, such as cognitive processes, task complexity, physical expenditure, level of experience, multiple types of loads, and the use of equipment. Cognitive load influences clinical task performance and has a bi-directional relationship with emotion. However, the literature is mixed regarding physiological responses to cognitive load, and how they are best measured. These findings highlight potential intervention points where cognitive load can be managed or reduced to improve working conditions for EMS clinicians and safety for their patients.

Physical and biophysical markers of assessment in medical training: A scoping review of the literature

PURPOSE

Assessment in medical education has changed over time to measure the evolving skills required of current medical practice. Physical and biophysical markers of assessment attempt to use technology to gain insight into medical trainees' knowledge, skills, and attitudes. The authors conducted a scoping review to map the literature on the use of physical and biophysical markers of assessment in medical training.

MATERIALS AND METHODS

The authors searched seven databases on 1 August 2022, for publications that utilized physical or biophysical markers in the assessment of medical trainees (medical students, residents, fellows, and synonymous terms used in other countries). Physical or biophysical markers included: heart rate and heart rate variability, visual tracking and attention, pupillometry, hand motion analysis, skin conductivity, salivary cortisol, functional magnetic resonance imaging (fMRI), and functional near-infrared spectroscopy (fNIRS). The authors mapped the relevant literature using Bloom's taxonomy of knowledge, skills, and attitudes and extracted additional data including study design, study environment, and novice vs. expert differentiation from February to June 2023.

RESULTS

Of 6,069 unique articles, 443 met inclusion criteria. The majority of studies assessed trainees using heart rate variability (n = 160, 36%) followed by visual attention (n = 143, 32%), hand motion analysis (n = 67, 15%), salivary cortisol (n = 67, 15%), fMRI (n = 29, 7%), skin conductivity (n = 26, 6%), fNIRs (n = 19, 4%), and pupillometry (n = 16, 4%). The majority of studies (n = 167, 38%) analyzed non-technical skills, followed by studies that analyzed technical skills (n = 155, 35%), knowledge (n = 114, 26%), and attitudinal skills (n = 61, 14%). 169 studies (38%) attempted to use physical or biophysical markers to differentiate between novice and expert.

CONCLUSION

This review provides a comprehensive description of the current use of physical and biophysical markers in medical education training, including the current technology and skills assessed. Additionally, while physical and biophysical markers have the potential to augment current assessment in medical education, there remains significant gaps in research surrounding reliability, validity, cost, practicality, and educational impact of implementing these markers of assessment.

Recognition and evaluation of mental workload in different stages of perceptual and cognitive information processing using a multimodal approach

This study explores the effects of different perceptual and cognitive information processing stages on mental workload by assessing multimodal indicators of mental workload such as the NASA-TLX, task performance, ERPs and eye movements. Repeated measures ANOVA of the data showed that among ERP indicators, P1, N1 and N2 amplitudes were sensitive to perceptual load (P-load), P3 amplitude was sensitive to P-load only in the prefrontal region during high cognitive load (C-load) states, and P3 amplitude in the occipital and parietal regions was sensitive to C-load. Among the eye movement indicators, blink frequency was sensitive to P-load in all C-load states, but to C-load in only low P-load states; pupil diameter and blink duration were sensitive to both P-load and C-load. Based on the above indicators, the k-nearest neighbours (KNN) algorithm was used to propose a classification method for the four different mental workload states with an accuracy of 97.89%.Practitioner summary: Based on the results of this study, it is possible to implement the monitoring of mental workload states and optimise brain task allocation in operations involving high mental workload, such as human-computer interaction.

Quantifying Workload and Stress in Intensive Care Unit Nurses: Preliminary Evaluation Using Continuous Eye-Tracking

OBJECTIVE

(1) To assess mental workloads of intensive care unit (ICU) nurses in 12-hour working shifts (days and nights) using eye movement data; (2) to explore the impact of stress on the ocular metrics of nurses performing patient care in the ICU.

BACKGROUND

Prior studies have employed workload scoring systems or accelerometer data to assess ICU nurses' workload. This is the first naturalistic attempt to explore nurses' mental workload using eye movement data.

METHODS

Tobii Pro Glasses 2 eye-tracking and Empatica E4 devices were used to collect eye movement and physiological data from 15 nurses during 12-hour shifts (252 observation hours). We used mixed-effect models and an ordinal regression model with a random effect to analyze the changes in eye movement metrics during high stress episodes.

RESULTS

While the cadence and characteristics of nurse workload can vary between day shift and night shift, no significant difference in eye movement values was detected. However, eye movement metrics showed that the initial handoff period of nursing shifts has a higher mental workload compared with other times. Analysis of ocular metrics showed that stress is positively associated with an increase in number of eye fixations and gaze entropy, but negatively correlated with the duration of saccades and pupil diameter.

CONCLUSION

Eye-tracking technology can be used to assess the temporal variation of stress and associated changes with mental workload in the ICU environment. A real-time system could be developed for monitoring stress and workload for intervention development.

Webcam-based eye-tracking to measure visual expertise of medical students during online histology training

Objectives

Visual expertise is essential for image-based tasks that rely on visual cues, such as in radiology or histology. Studies suggest that eye movements are related to visual expertise and can be measured by near-infrared eye-tracking. With the popularity of device-embedded webcam eye-tracking technology, cost-effective use in educational contexts has recently become amenable. This study investigated the feasibility of such methodology in a curricular online-only histology course during the 2021 summer term.

Methods

At two timepoints (t1 and t2), third-semester medical students were asked to diagnose a series of histological slides while their eye movements were recorded. Students' eye metrics, performance and behavioral measures were analyzed using variance analyses and multiple regression models.

Results

First, webcam-eye tracking provided eye movement data with satisfactory quality (mean accuracy=115.7 px±31.1). Second, the eye movement metrics reflected the students' proficiency in finding relevant image sections (fixation count on relevant areas=6.96±1.56 vs. irrelevant areas=4.50±1.25). Third, students' eye movement metrics successfully predicted their performance (R2adj=0.39, p<0.001).

Conclusion

This study supports the use of webcam-eye-tracking expanding the range of educational tools available in the (digital) classroom. As the students' interest in using the webcam eye-tracking was high, possible areas of implementation will be discussed.

Evaluating Medic Performance in Combat Casualty Care Simulation and Training: A Scoping Review of Prospective Research

INTRODUCTION

Combat medics are required to perform highly technical medical procedures in austere environments with minimal error. Effective means to quantify medic performance in field and simulated environments are critical to optimize medic training procedures as well as to evaluate the influence of medical equipment and other supportive technologies on medic performance. Human performance evaluation in combat casualty care presents many unique challenges due to the unique environment (battlefields) and population (medics) that must be represented. Recent advances in simulation and measurement technology have presented opportunities to improve simulation fidelity and measurement quality; however, it is currently unclear to what extent these advances have been adopted in this domain.

METHODOLOGY

In this work, a scoping review of recent (2011-2021) prospective research on Army medic (68 W and Special Operations) performance is presented. The Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines served as the framework for the review. The goal of this work was to summarize recent trends and practices and to illuminate opportunities for future work. Two human factors experts performed an exhaustive review of relevant, peer-reviewed literature and systematically identified articles for inclusion in the final analysis. The articles were examined in detail, and data elements of interest were extracted.

RESULTS

Forty-eight articles were identified based on the defined inclusion criteria. Thirty three of the articles focused on technological evaluation, 25 focused on medic training procedures, and 5 focused on evaluating medical techniques. Study contributions were predominantly related to medic training materials/procedures and simulator technology. Supportive medical technologies, including telemedical systems, hemorrhage control devices, and ultrasound devices, also received significant attention. Timing was the most common metric used to quantify medic performance, followed by skill pass/fail ratings. There was a notable lack of neurophysiological data used to examine medic physical/cognitive workload during procedures, a growing practice in many other related domains. The most commonly simulated procedures were hemorrhage control, airway management, and thoracostomy. Notable limitations cited across articles were insufficient simulation fidelity, inadequate sample size or sample representativeness, and poor study design.

CONCLUSIONS

This work provided a summary of recent peer-reviewed research related to medic simulation and training, and performance evaluation. This article should be used to contextualize existing research and inspire new research questions. Expanding and advancing research on medic simulation and training will help to ensure optimal casualty care at the front lines.

Novice Doctors in the Emergency Department: A Scoping Review

In many emergency departments (EDs), young, inexperienced doctors treat patients who are critically ill. At the start of their career, these novice doctors are not sufficiently qualified to take care of these potentially critically ill patients in the highly demanding environment of an ED. This not only poses a threat to the well-being of the doctor, who feels inadequately prepared and experiences a lot of stress, but also to that of the patients, who may not receive optimal care. Lastly, young doctors may influence the efficiency of the organization, with longer throughput times, more orders of ancillary investigations, and more admissions.

Training novice doctors with regard to simple or complex skills using simulation techniques is part of the solution. However, the transfer of newly learned skills to clinical practice remains unexplored, and not everything can be trained before the actual skill is required. Therefore, it is important to train young doctors in their learning abilities, for instance, teach them how to be adaptive and how to use their skills in new situations. Lastly, the way care is organized is essential. Good supervision, leaving room for the learning processes of young doctors, developing a team with more experienced professionals (paramedics, nurses, and doctors), and well-organized processes, aiming to reduce the complexity of the work, are ways to improve the quality of care, independent of the experience level of the novice doctor.

Cognitive load during training for out-of-department emergency responses

Background

Emergency medicine (EM) physicians sometimes respond to critical events outside the emergency department. To prepare for these complex cases-typically called "rapid responses" (RRs)-EM residents receive simulation-based training involving four practice tasks and three exam tasks during a 1-day session. Cognitive load (CL) theory describes how humans function with limited working memories to perform complex tasks. RRs are expected to generate high levels of CL, but the profile of CL across providers and RR cases is not well understood. In this study, we analyzed resident's CL during RR training. We hypothesized variations in CL across individual and case and that exam cases would cause higher CLs than practice cases.

Methods

Residents anonymously self-reported CL levels after each case using the Paas scale, a single-item, 9-point scale from "very, very low CL" to "very, very high CL." To examine case-based differences in CL, data were rescaled by individual residents. "High CL" was defined as a score of 9/9.

Results

Among 18 residents participating, CLs ranged from 4 to 9, with median of 7 and interquartile range of 7-8. While many cases showed bell curve-like distributions of CLs, one case-a bleeding tracheostomy-showed a rightward skew reflecting higher levels of CL. No significant difference was found in CL between practice and exam cases. There were 20 reports (16.5%) of "high" CL with variation across residents (0/7 [0%] to 5/6 [83.3%] cases) and across cases (1/18 [5.6%) to 8/18 [44.4%]).

Conclusions

The CL that EM residents experienced did show considerable interpersonal and intercase variation, but there was no significant difference between practice and exam cases. These results highlight several questions about how to optimally design future training, including how best to balance low and high CL training cases and which cases may require further training.

When medical trainees encountering a performance difficulty: evidence from pupillary responses

BACKGROUND

Medical trainees are required to learn many procedures following instructions to improve their skills. This study aims to investigate the pupillary response of trainees when they encounter moment of performance difficulty (MPD) during skill learning. Detecting the moment of performance difficulty is essential for educators to assist trainees when they need it.

METHODS

Eye motions were recorded while trainees practiced the thoracostomy procedure in the simulation model. To make pupillary data comparable among trainees, we proposed the adjusted pupil size (APS) normalizing pupil dilation for each trainee in their entire procedure. APS variables including APS, maxAPS, minAPS, meanAPS, medianAPS, and max interval indices were compared between easy and difficult subtasks; the APSs were compared among the three different performance situations, the moment of normal performance (MNP), MPD, and moment of seeking help (MSH).

RESULTS

The mixed ANOVA revealed that the adjusted pupil size variables, such as the maxAPS, the minAPS, the meanAPS, and the medianAPS, had significant differences between performance situations. Compared to MPD and MNP, pupil size was reduced during MSH. Trainees displayed a smaller accumulative frequency of APS during difficult subtask when compared to easy subtasks.

CONCLUSIONS

Results from this project suggest that pupil responses can be a good behavioral indicator. This study is a part of our research aiming to create an artificial intelligent system for medical trainees with automatic detection of their performance difficulty and delivering instructional messages using augmented reality technology.

The Relationship of Technical Skills and Cognitive Workload to Errors During Robotic Surgical Exercises

Purpose We attempt to understand the relationship between surgeon technical skills, cognitive workload and errors during a simulated robotic dissection task. Materials and Methods Participant surgeons performed a robotic surgery dissection exercise. Participants were grouped based on surgical experience. Technical skills were evaluated utilizing the validated Global Evaluative Assessment of Robotic Skills (GEARS) assessment tool. The dissection task was evaluated for errors during active dissection or passive retraction maneuvers. We quantified cognitive workload of surgeon participants as an Index of Cognitive Activity (ICA), derived from Task-Evoked-Pupillary-Response metrics; ICA ranged 0-1, with 1 representing maximum ICA. Generalized Estimating Equation (GEE) was used for all modellings to establish relationships between surgeon technical skills, cognitive workload and errors. Results We found a strong association between technical skills as measured by multiple GEARS domains (depth perception, force sensitivity and robotic control) and passive errors, with higher GEARS scores associated with a lower relative risk of errors (all p < 0.01). For novice surgeons, as average GEARS scores increased, the average estimated ICA decreased. In contrast, as average GEARS increased for expert surgeons, the average estimated ICA increased. When exhibiting optimal technical skill (maximal GEARS scores) novices and experts reached a similar range of ICA scores (ICA 0.47 and 0.42, respectively). Conclusions This study found that there is an optimal cognitive workload level for surgeons of all experience levels during our robotic surgical exercise. Select technical skill domains were strong predictors of errors. Future research will explore whether an ideal cognitive workload range truly optimizes surgical training and reduce surgical errors.

Detecting Lies is a Child (Robot)’s Play: Gaze-Based Lie Detection in HRI

Robots destined to tasks like teaching or caregiving have to build a long-lasting social rapport with their human partners. This requires, from the robot side, to be capable of assessing whether the partner is trustworthy. To this aim a robot should be able to assess whether someone is lying or not, while preserving the pleasantness of the social interaction. We present an approach to promptly detect lies based on the pupil dilation, as intrinsic marker of the lie-associated cognitive load that can be applied in an ecological human–robot interaction, autonomously led by a robot. We demonstrated the validity of the approach with an experiment, in which the iCub humanoid robot engages the human partner by playing the role of a magician in a card game and detects in real-time the partner deceptive behavior. On top of that, we show how the robot can leverage on the gained knowledge about the deceptive behavior of each human partner, to better detect subsequent lies of that individual. Also, we explore whether machine learning models could improve lie detection performances for both known individuals (within-participants) over multiple interaction with the same partner, and with novel partners (between-participant). The proposed setup, interaction and models enable iCub to understand when its partners are lying, which is a fundamental skill for evaluating their trustworthiness and hence improving social human–robot interaction.

The medical pause: Importance, processes and training

Research has shown that taking 'timeouts' in medical practice improves performance and patient safety. However, the benefits of taking timeouts, or pausing, are not sufficiently acknowledged in workplaces and training programmes. To promote this acknowledgement, we suggest a systematic conceptualisation of the medical pause, focusing on its importance, processes and implementation in training programmes. By employing insights from educational and cognitive psychology, we first identified pausing as an important skill to interrupt negative momentum and bolster learning. Subsequently, we categorised constituent cognitive processes for pausing skills into two phases: the decision-making phase (determining when and how to take pauses) and the executive phase (applying relaxation or reflection during pauses). We present a model that describes how relaxation and reflection during pauses can optimise cognitive load in performance. Several strategies to implement pause training in medical curricula are proposed: intertwining pause training with training of primary skills, providing second-order scaffolding through shared control and employing auxiliary tools such as computer-based simulations with a pause function.

An exploratory investigation of the measurement of cognitive load on shift: Application of cognitive load theory in emergency medicine

BACKGROUND

Emergency physicians often experience a high cognitive load (CL) due to the inherent nature of working in acute care settings. CL has traditionally been measured in educational studies but has not been well studied in the clinical environment.

METHODS

Emergency medicine attending physicians and residents working in an academic urgent care center completed psychometric questionnaires while on shift to measure overall CL, intrinsic cognitive load (ICL), extraneous cognitive load (ECL), and acute stress. Data regarding the patient load, patient acuity, and the number of patients in the waiting room were also collected. Correlational analysis and simple linear regression were used to evaluate predictors of CL on shift.

RESULTS

Forty-two questionnaires were completed (26 by attending physicians, 16 by residents). Attending physicians carried a significantly higher patient load compared to residents (p < 0.001). No differences in mean overall CL, ICL, ECL, and acute stress were observed between attending physicians and residents. Bivariate analysis demonstrated associations between ICL, ECL, acute stress, and overall CL in attending physicians. In residents, acute stress was the only variable associated with overall CL and the number of high-acuity patients was associated with ICL.

CONCLUSIONS

Factors influencing reported CL during clinical work are different between attending emergency physicians and residents. Further study to appreciate the impact of these differences is required and may help educators elucidate strategies to better manage CL, thereby improving clinical performance and potentially improving patient care.

Evolving robotic surgery training and improving patient safety, with the integration of novel technologies

INTRODUCTION

Robot-assisted surgery is becoming increasingly adopted by multiple surgical specialties. There is evidence of inherent risks of utilising new technologies that are unfamiliar early in the learning curve. The development of standardised and validated training programmes is crucial to deliver safe introduction. In this review, we aim to evaluate the current evidence and opportunities to integrate novel technologies into modern digitalised robotic training curricula.

METHODS

A systematic literature review of the current evidence for novel technologies in surgical training was conducted online and relevant publications and information were identified. Evaluation was made on how these technologies could further enable digitalisation of training.

RESULTS

Overall, the quality of available studies was found to be low with current available evidence consisting largely of expert opinion, consensus statements and small qualitative studies. The review identified that there are several novel technologies already being utilised in robotic surgery training. There is also a trend towards standardised validated robotic training curricula. Currently, the majority of the validated curricula do not incorporate novel technologies and training is delivered with more traditional methods that includes centralisation of training services with wet laboratories that have access to cadavers and dedicated training robots.

CONCLUSIONS

Improvements to training standards and understanding performance data have good potential to significantly lower complications in patients. Digitalisation automates data collection and brings data together for analysis. Machine learning has potential to develop automated performance feedback for trainees. Digitalised training aims to build on the current gold standards and to further improve the 'continuum of training' by integrating PBP training, 3D-printed models, telementoring, telemetry and machine learning.

The future of simulation-based medical education: Adaptive simulation utilizing a deep multitask neural network

BACKGROUND

In resuscitation medicine, effectively managing cognitive load in high-stakes environments has important implications for education and expertise development. There exists the potential to tailor educational experiences to an individual's cognitive processes via real-time physiologic measurement of cognitive load in simulation environments.

OBJECTIVE

The goal of this research was to test a novel simulation platform that utilized artificial intelligence to deliver a medical simulation that was adaptable to a participant's measured cognitive load.

METHODS

The research was conducted in 2019. Two board-certified emergency physicians and two medical students participated in a 10-minute pilot trial of a novel simulation platform. The system utilized artificial intelligence algorithms to measure cognitive load in real time via electrocardiography and galvanic skin response. In turn, modulation of simulation difficulty, determined by a participant's cognitive load, was facilitated through symptom severity changes of an augmented reality (AR) patient. A postsimulation survey assessed the participants' experience.

RESULTS

Participants completed a simulation that successfully measured cognitive load in real time through physiological signals. The simulation difficulty was adapted to the participant's cognitive load, which was reflected in changes in the AR patient's symptoms. Participants found the novel adaptive simulation platform to be valuable in supporting their learning.

CONCLUSION

Our research team created a simulation platform that adapts to a participant's cognitive load in real-time. The ability to customize a medical simulation to a participant's cognitive state has potential implications for the development of expertise in resuscitation medicine.

Post Procedural Cognitive Load Measurement with Immediate Feedback to Guide Curriculum Development

BACKGROUND

Methods to assess competency in cardiothoracic training are essential. Here, we report a system that allows us to better assess competency from the perspective of both the trainee and educator. We hypothesized that post-procedural cognitive burden measurement (by the trainee) with immediate feedback (from educator) could aid in identifying barriers to the acquisition of skills and knowledge so that training curricula can be individualized.

METHODS

The NASA-TLX (National Aeronautics and Space Administration Task Load Index), a validated instrument to measure cognitive load, was administered with an online platform following bronchoscopy, esophagogastroduodenoscopy (EGD), and video-assisted thoracoscopic surgery (VATS) for 11 residents. Immediate post-procedure feedback and standardized debriefing occurred for each procedure.

RESULTS

Mean NASA-TLX scores were highest (indicating greater cognitive load) for EGD and VATS (p<0.001). When comparing sub-scale measures, "mental demand" was significantly higher for VATS (p=0.026) compared to the other procedures, whereas "physical demand" was highest for EGD (p=0.018). Self-reported "frustration" was similar for all case types (p=0.247). Cognitive burden decreased with a greater number of procedures for bronchoscopy (p=0.027). Significant improvement was noted by the trainee at the end of the rotation in self-assessed procedural competency and preparedness for thoracic board topics (all p<0.05). Post-procedure feedback by the attending surgeon correlated with more frequent completion of self-evaluations by the residents.

CONCLUSIONS

Longitudinal assessment of cognitive load in combination with post-procedural feedback identified barriers to skill acquisition for both residents and educators. This information allows for individualized rotation development as a step towards a competency-based curriculum.

The Application of a System of Eye Tracking in Laparoscopic Surgery: A New Didactic Tool to Visual Instructions

Introduction: Laparoscopic surgery is an increasingly used technique, but it requires a high degree of learning, and communication between the operating room crew is considerably difficult. The use of eye tracking has been proposed as a didactic and evaluation tool in several settings, including in laparoscopy in simulators.

Objectives: This study aimed to evaluate the usefulness of the use of eye tracking systems (Tobii glasses 2) in laparoscopic surgery as a didactic and assessment tool to improve communication in the operating room and improve patients' security.

Methodology: An anonymous survey was sent to the students and medical teachers of a faculty of medicine and practicing doctors and residents. The message contained an explanation about the use of the Tobii glasses, a link to watch the video showing its use in a laparoscopic surgery, and the survey to complete after watching the video.

Results: The survey was answered by 113 participants (51.3% medical students, 27.4% medical teachers, 18.6% practicing doctors, and 2.7% medicine residents). Eighty-three percent agreed with the usefulness of the “Tobii glasses” in the operating room for improving communication between the main surgeon and the assistant, for learning complex surgery techniques, for obtaining didactic videos, and for indicating anatomical structures. The item scored worst was the price of the glasses.

Conclusions: It is possible to record and project expert gaze patterns in the operating room in real time using the Tobii glasses. This device allows improving communication among the surgical crew and the learning of residents and also improving the security of surgical patients.

Using Eye Tracking for Measuring Cognitive Workload During Clinical Simulations: Literature Review and Synthesis

High-fidelity clinical simulations can be used by clinicians to acquire technical (physical ability and knowledge) and non-technical (cognitive and social processes) skills. Excessive cognitive workload contributes to medical errors because of the impact on both technical and non-technical skills. Many studies measure cognitive workload with psychometric instruments that limit the assessment of cognitive workload to a single time period and may involve response bias. Using eye tracking to measure task-evoked pupillary responses allows the measurement of changes in pupil diameter related to the cognitive workload associated with a specific activity. Incorporating eye tracking with high-fidelity clinical simulations provides a reliable and continuous assessment of cognitive workload. The purpose of this literature review is to summarize the use of eye-tracking technology to measure cognitive workload of healthcare providers to generate evidence-based guidelines for measuring cognitive workload during high-fidelity clinical simulations. What this manuscript adds to the body of literature is a summary of best practices related to the different methods of measuring cognitive workload, benefits and limitations of using eye tracking, and high-fidelity clinical simulation design considerations for successful integration of eye tracking.

Pupillary Responses for Cognitive Load Measurement to Classify Difficulty Levels in an Educational Video Game: Empirical Study

Background

A learning task recurrently perceived as easy (or hard) may cause poor learning results. Gamer data such as errors, attempts, or time to finish a challenge are widely used to estimate the perceived difficulty level. In other contexts, pupillometry is widely used to measure cognitive load (mental effort); hence, this may describe the perceived task difficulty.

Objective

This study aims to assess the use of task-evoked pupillary responses to measure the cognitive load measure for describing the difficulty levels in a video game. In addition, it proposes an image filter to better estimate baseline pupil size and to reduce the screen luminescence effect.

Methods

We conducted an experiment that compares the baseline estimated from our filter against that estimated from common approaches. Then, a classifier with different pupil features was used to classify the difficulty of a data set containing information from students playing a video game for practicing math fractions.

Results

We observed that the proposed filter better estimates a baseline. Mauchly’s test of sphericity indicated that the assumption of sphericity had been violated (χ²₁₄=0.05; P=.001); therefore, a Greenhouse-Geisser correction was used (ε=0.47). There was a significant difference in mean pupil diameter change (MPDC) estimated from different baseline images with the scramble filter (F_5,78=30.965; P<.001). Moreover, according to the Wilcoxon signed rank test, pupillary response features that better describe the difficulty level were MPDC (z=−2.15; P=.03) and peak dilation (z=−3.58; P<.001). A random forest classifier for easy and hard levels of difficulty showed an accuracy of 75% when the gamer data were used, but the accuracy increased to 87.5% when pupillary measurements were included.

Conclusions

The screen luminescence effect on pupil size is reduced with a scrambled filter on the background video game image. Finally, pupillary response data can improve classifier accuracy for the perceived difficulty of levels in educational video games.

From Theory to Practice: The Application of Cognitive Load Theory to the Practice of Medicine

Cognitive load theory has become a leading model in educational psychology and has started to gain traction in the medical education community over the last decade. The theory is rooted in our current understanding of human cognitive architecture in which an individual's limited working memory and unlimited long-term memory interact during the process of learning. Though initially described as primarily a theory of learning, parallels between cognitive load theory and broader aspects of medical education as well as clinical practice are now becoming clear. These parallels are particularly relevant and evident in complex clinical environments, like resuscitation medicine. The authors have built on these connections to develop a recontextualized version of cognitive load theory that applies to complex professional domains and in which the connections between the theory and clinical practice are made explicit, with resuscitation medicine as a case study. Implications of the new model for medical education are also presented along with suggested applications.

Simulation-based training of critical events during cardiopulmonary bypass: importance of a critical events checklist

Objectives:

Evaluation of critical events training for clinical perfusionists is necessary to improve this educational approach. Critical events checklists are effective in reducing clinical complications, but should be tested in a simulation environment first. Individual behavior and stress response of clinical perfusionists during simulated critical events on cardiopulmonary bypass have not been evaluated yet. This study focuses on the evaluation of critical events training and critical events checklists in simulated cardiopulmonary bypass.

Methods:

A total of 19 clinical perfusionists from a single hospital took part in two simulated critical event scenarios. Clinical perfusionist behavior and physiological responses were recorded using eye tracking, heart rate variability, video, and audio. In addition, workloads were determined and participants were interviewed.

Results:

Relevant areas of interest were identified for each simulation phase. During critical event detection and subsequent decision-making, areas of interest hits and fixation durations varied with the use of a critical events checklist. Times to decision were shorter, decision quality was higher, and temporal workload was increased when the checklist was used. Evaluation of selected heart rate variability measures revealed a good correlation with pupil diameters.

Conclusion:

Evaluation of critical events during simulated cardiopulmonary bypass shows that the scenario is realistic and relevant for clinical practice. Integrating a critical events checklist improves the probability of correct decision-making and shortens the correct decision time. Temporal workload is increased when using a checklist. Eye tracking and heart rate variability are well suited to evaluate participants’ behaviors and stress levels. All participants welcomed simulation training for critical incidents.

Pupil diameter differentiates expertise in dental radiography visual search

Expert behavior is characterized by rapid information processing abilities, dependent on more structured schemata in long-term memory designated for their domain-specific tasks. From this understanding, expertise can effectively reduce cognitive load on a domain-specific task. However, certain tasks could still evoke different gradations of load even for an expert, e.g., when having to detect subtle anomalies in dental radiographs. Our aim was to measure pupil diameter response to anomalies of varying levels of difficulty in expert and student dentists’ visual examination of panoramic radiographs. We found that students’ pupil diameter dilated significantly from baseline compared to experts, but anomaly difficulty had no effect on pupillary response. In contrast, experts’ pupil diameter responded to varying levels of anomaly difficulty, where more difficult anomalies evoked greater pupil dilation from baseline. Experts thus showed proportional pupillary response indicative of increasing cognitive load with increasingly difficult anomalies, whereas students showed pupillary response indicative of higher cognitive load for all anomalies when compared to experts.

Resuscitation Education Science: Educational Strategies to Improve Outcomes From Cardiac Arrest: A Scientific Statement From the American Heart Association

The formula for survival in resuscitation describes educational efficiency and local implementation as key determinants in survival after cardiac arrest. Current educational offerings in the form of standardized online and face-to-face courses are falling short, with providers demonstrating a decay of skills over time. This translates to suboptimal clinical care and poor survival outcomes from cardiac arrest. In many institutions, guidelines taught in courses are not thoughtfully implemented in the clinical environment. A current synthesis of the evidence supporting best educational and knowledge translation strategies in resuscitation is lacking. In this American Heart Association scientific statement, we provide a review of the literature describing key elements of educational efficiency and local implementation, including mastery learning and deliberate practice, spaced practice, contextual learning, feedback and debriefing, assessment, innovative educational strategies, faculty development, and knowledge translation and implementation. For each topic, we provide suggestions for improving provider performance that may ultimately optimize patient outcomes from cardiac arrest.

Using objective robotic automated performance metrics and task-evoked pupillary response to distinguish surgeon expertise

PURPOSE

In this study, we investigate the ability of automated performance metrics (APMs) and task-evoked pupillary response (TEPR), as objective measures of surgeon performance, to distinguish varying levels of surgeon expertise during generic robotic surgical tasks. Additionally, we evaluate the association between APMs and TEPR.

METHODS

Participants completed ten tasks on a da Vinci Xi Surgical System (Intuitive Surgical, Inc.), each representing a surgical skill type: EndoWrist^® manipulation, needle targeting, suturing/knot tying, and excision/dissection. Automated performance metrics (instrument motion tracking, EndoWrist^® articulation, and system events data) and TEPR were recorded by a systems data recorder (Intuitive Surgical, Inc.) and Tobii Pro Glasses 2 (Tobii Technologies, Inc.), respectively. The Kruskal-Wallis test determined significant differences between groups of varying expertise. Spearman's rank correlation coefficient measured associations between APMs and TEPR.

RESULTS

Twenty-six participants were stratified by robotic surgical experience: novice (no prior experience; n = 9), intermediate (< 100 cases; n = 9), and experts (≥ 100 cases; n = 8). Several APMs differentiated surgeon experience including task duration (p < 0.01), time active of instruments (p < 0.03), linear velocity of instruments (p < 0.04), and angular velocity of dominant instrument (p < 0.04). Task-evoked pupillary response distinguished surgeon expertise for three out of four task types (p < 0.04). Correlation trends between APMs and TEPR revealed that expert surgeons move more slowly with high cognitive workload (ρ < - 0.60, p < 0.05), while novices move faster under the same cognitive experiences (ρ > 0.66, p < 0.05).

CONCLUSIONS

Automated performance metrics and TEPR can distinguish surgeon expertise levels during robotic surgical tasks. Furthermore, under high cognitive workload, there can be a divergence in robotic movement profiles between expertise levels.

Can You Ink While You Blink? Assessing Mental Effort in a Sensor-Based Calligraphy Trainer

Sensors can monitor physical attributes and record multimodal data in order to provide feedback. The application calligraphy trainer, exploits these affordances in the context of handwriting learning. It records the expert's handwriting performance to compute an expert model. The application then uses the expert model to provide guidance and feedback to the learners. However, new learners can be overwhelmed by the feedback as handwriting learning is a tedious task. This paper presents the pilot study done with the calligraphy trainer to evaluate the mental effort induced by various types of feedback provided by the application. Ten participants, five in the control group and five in the treatment group, who were Ph.D. students in the technology-enhanced learning domain, took part in the study. The participants used the application to learn three characters from the Devanagari script. The results show higher mental effort in the treatment group when all types of feedback are provided simultaneously. The mental efforts for individual feedback were similar to the control group. In conclusion, the feedback provided by the calligraphy trainer does not impose high mental effort and, therefore, the design considerations of the calligraphy trainer can be insightful for multimodal feedback designers.

The human task-evoked pupillary response function is linear: Implications for baseline response scaling in pupillometry

The human task-evoked pupillary response provides a sensitive physiological index of the intensity and online resource demands of numerous cognitive processes (e.g., memory retrieval, problem solving, or target detection). Cognitive pupillometry is a well-established technique that relies upon precise measurement of these subtle response functions. Baseline variability of pupil diameter is a complex artifact that typically necessitates mathematical correction. A methodological paradox within pupillometry is that linear and nonlinear forms of baseline scaling both remain accepted baseline correction techniques, despite yielding highly disparate results. The task-evoked pupillary response (TEPR) could potentially scale nonlinearly, similar to autonomic functions such as heart rate, in which the amplitude of an evoked response diminishes as the baseline rises. Alternatively, the TEPR could scale similarly to the cortical hemodynamic response, as a linear function that is independent of its baseline. However, the TEPR cannot scale both linearly and nonlinearly. Our aim was to adjudicate between linear and nonlinear scaling of human TEPR. We manipulated baseline pupil size by modulating the illuminance in the testing room as participants heard abrupt pure-tone transitions (Exp. 1) or visually monitored word lists (Exp. 2). Phasic pupillary responses scaled according to a linear function across all lighting (dark, mid, bright) and task (tones, words) conditions, demonstrating that the TEPR is independent of its baseline amplitude. We discuss methodological implications and identify a need to reevaluate past pupillometry studies.

Cortical modulation of pupillary function: systematic review

BACKGROUND

The pupillary light reflex is the main mechanism that regulates the pupillary diameter; it is controlled by the autonomic system and mediated by subcortical pathways. In addition, cognitive and emotional processes influence pupillary function due to input from cortical innervation, but the exact circuits remain poorly understood. We performed a systematic review to evaluate the mechanisms behind pupillary changes associated with cognitive efforts and processing of emotions and to investigate the cerebral areas involved in cortical modulation of the pupillary light reflex.

METHODOLOGY

We searched multiple databases until November 2018 for studies on cortical modulation of pupillary function in humans and non-human primates. Of 8,809 papers screened, 258 studies were included.

RESULTS

Most investigators focused on pupillary dilatation and/or constriction as an index of cognitive and emotional processing, evaluating how changes in pupillary diameter reflect levels of attention and arousal. Only few tried to correlate specific cerebral areas to pupillary changes, using either cortical activation models (employing micro-stimulation of cortical structures in non-human primates) or cortical lesion models (e.g., investigating patients with stroke and damage to salient cortical and/or subcortical areas). Results suggest the involvement of several cortical regions, including the insular cortex (Brodmann areas 13 and 16), the frontal eye field (Brodmann area 8) and the prefrontal cortex (Brodmann areas 11 and 25), and of subcortical structures such as the locus coeruleus and the superior colliculus.

CONCLUSIONS

Pupillary dilatation occurs with many kinds of mental or emotional processes, following sympathetic activation or parasympathetic inhibition. Conversely, pupillary constriction may occur with anticipation of a bright stimulus (even in its absence) and relies on a parasympathetic activation. All these reactions are controlled by subcortical and cortical structures that are directly or indirectly connected to the brainstem pupillary innervation system.

IR-camera-based measurements of 2D/3D cognitive fatigue in 2D/3D display system using task-evoked pupillary response

This study was carried out to evaluate a method used to measure three-dimensional (3D) cognitive fatigue based on the pupillary response. This technique was designed to overcome measurement burdens by using non-contact methods. The pupillary response is related to cognitive function by a neural pathway and may be an indicator of 3D cognitive fatigue. Twenty-six undergraduate students (including 14 women) watched both 2D and 3D versions of a video for 70 min. The participants experienced visual fatigue after viewing the 3D content. Measures such as subjective rating, response time, event-related potential latency, heartbeat-evoked potential (HEP) alpha power, and task-evoked pupillary response (TEPR) latency were significantly different. Multitrait-multimethod matrix analysis indicated that HEP and TEPR latency measures had stronger reliability and higher correlations with 3D cognitive fatigue than other measures. TEPR latency may be useful for quantitatively determining 3D visual fatigue, as it can be easily used to evaluate 3D visual fatigue using a non-contact method without measuring burden.

Evaluation of attention, perception, and stress levels of clinical cardiovascular perfusionists during cardiac operations: a pilot study

BACKGROUND

Performing cardiopulmonary bypass is a complex task which involves evaluating visual input from patient monitors and technical parameters displayed at the heart-lung machine console as well as reacting to other sensory input. Only few studies are available concerning the competency requirements for clinical cardiovascular perfusionists, including attention, perception, and coping with mental stress. This study aims at evaluating attention, perception, and stress levels of clinical cardiovascular perfusionists during cardiopulmonary bypass.

METHODS

Nine clinical cardiovascular perfusionists voluntarily offered to participate in the study. Participants were asked to wear Tobii 2 eye-tracking glasses throughout the procedures. Specific time points were analyzed (cardiopulmonary bypass on, initial cardioplegia delivery, steady state, cross-clamp off, and weaning from cardiopulmonary bypass). Data acquisition was supplemented by participants' self-evaluation regarding their stress levels and by National Aeronautics and Space Administration Task Load Index (NASA TLX) questionnaires.

RESULTS

Seven datasets were sufficient to be evaluated. The clinical cardiovascular perfusionists' professional experience ranged from 0.5 to 24 years. Evaluation of eye-tracking data revealed large variations in areas of interest hits, fixation, and dwell times. Across all phases, the venous reservoir, mean arterial pressure, arterial pump display, cardioplegia control, and data management system received the highest levels of attention. Pupil diameter measurements increased at start of cardiopulmonary bypass, cardioplegia delivery, and weaning off, but returned to base level during steady state. Clinical cardiovascular perfusionists' self-evaluation showed that subjective stress level was highest at the start and the end of the procedure. NASA TLX questionnaires revealed medium-to-high mental and temporal workloads, but low physical workloads. Performance, effort, and frustration indices showed medium workloads.

CONCLUSION

During cardiopulmonary bypass, perfusionists are subjected to stress. Peak stress levels were highest during start and end of cardiopulmonary bypass. Furthermore, visual attention and perception varied between the operative phases. Further studies are indicated to evaluate the design of heart-lung machines and stress-coping strategies during cardiopulmonary bypass.

Cognitive load theory for training health professionals in the workplace: A BEME review of studies among diverse professions: BEME Guide No. 53

AIM

Cognitive load theory (CLT) is of increasing interest to health professions education researchers. CLT has intuitive applicability to workplace settings, yet how CLT should inform teaching, learning, and research in health professions workplaces is unclear.

METHOD

To map the existing literature, we performed a scoping review of studies involving cognitive load, mental effort and/or mental workload in professional workplace settings within and outside of the health professions. We included actual and simulated workplaces and workplace tasks.

RESULT

Searching eight databases, we identified 4571 citations, of which 116 met inclusion criteria. Studies were most often quantitative. Methods to measure cognitive load included psychometric, physiologic, and secondary task approaches. Few covariates of cognitive load or performance were studied. Overall cognitive load and intrinsic load were consistently negatively associated with the level of experience and performance. Studies consistently found distractions and other aspects of workplace environments as contributing to extraneous load. Studies outside the health professions documented similar findings to those within the health professions, supporting relevance of CLT to workplace learning.

CONCLUSION

The authors discuss implications for workplace teaching, curricular design, learning environment, and metacognition. To advance workplace learning, the authors suggest future CLT research should address higher-level questions and integrate other learning frameworks.

A new way to look at simulation-based assessment: the relationship between gaze-tracking and exam performance

Objective

A key task of the team leader in a medical emergency is effective information gathering. Studying information gathering patterns is readily accomplished with the use of gaze-tracking glasses. This technology was used to generate hypotheses about the relationship between performance scores and expert-hypothesized visual areas of interest in residents across scenarios in simulated medical resuscitation examinations.

Methods

Emergency medicine residents wore gaze-tracking glasses during two simulation-based examinations (n=29 and 13 respectively). Blinded experts assessed video-recorded performances using a simulation performance assessment tool that has validity evidence in this context. The relationships between gaze patterns and performance scores were analyzed and potential hypotheses generated. Four scenarios were assessed in this study: diabetic ketoacidosis, bradycardia secondary to beta-blocker overdose, ruptured abdominal aortic aneurysm and metabolic acidosis caused by antifreeze ingestion.

Results

Specific gaze patterns were correlated with objective performance. High performers were more likely to fixate on task-relevant stimuli and appropriately ignore task-irrelevant stimuli compared with lower performers. For example, shorter latency to fixation on the vital signs in a case of diabetic ketoacidosis was positively correlated with performance (r=0.70, p<0.05). Conversely, total time spent fixating on lab values in a case of ruptured abdominal aortic aneurysm was negatively correlated with performance (r= −0.50, p<0.05).

Conclusions

There are differences between the visual patterns of high and low-performing residents. These findings may allow for better characterization of expertise development in resuscitation medicine and provide a framework for future study of visual behaviours in resuscitation cases.

Getting Inside the Expert's Head: An Analysis of Physician Cognitive Processes During Trauma Resuscitations

STUDY OBJECTIVE

Crisis resource management skills are integral to leading the resuscitation of a critically ill patient. Despite their importance, crisis resource management skills (and their associated cognitive processes) have traditionally been difficult to study in the real world. The objective of this study was to derive key cognitive processes underpinning expert performance in resuscitation medicine, using a new eye-tracking-based video capture method during clinical cases.

METHODS

During an 18-month period, a sample of 10 trauma resuscitations led by 4 expert trauma team leaders was analyzed. The physician team leaders were outfitted with mobile eye-tracking glasses for each case. After each resuscitation, participants were debriefed with a modified cognitive task analysis, based on a cued-recall protocol, augmented by viewing their own first-person perspective eye-tracking video from the clinical encounter.

RESULTS

Eye-tracking technology was successfully applied as a tool to aid in the qualitative analysis of expert performance in a clinical setting. All participants stated that using these methods helped uncover previously unconscious aspects of their cognition. Overall, 5 major themes were derived from the interviews: logistic awareness, managing uncertainty, visual fixation behaviors, selective attendance to information, and anticipatory behaviors.

CONCLUSION

The novel approach of cognitive task analysis augmented by eye tracking allowed the derivation of 5 unique cognitive processes underpinning expert performance in leading a resuscitation. An understanding of these cognitive processes has the potential to enhance educational methods and to create new assessment modalities of these previously tacit aspects of expertise in this field.

Critical Appraisal of Emergency Medicine Educational Research: The Best Publications of 2015

OBJECTIVE

The objectives were to critically appraise the medical education research literature of 2015 and review the highest-quality quantitative and qualitative examples.

METHODS

A total of 434 emergency medicine (EM)-related articles were discovered upon a search of ERIC, PsychINFO, PubMED, and SCOPUS. These were both quantitative and qualitative in nature. All were screened by two of the authors using previously published exclusion criteria, and the remaining were appraised by all authors using a previously published scoring system. The highest scoring articles were then reviewed.

RESULTS

Sixty-one manuscripts were scored, and 10 quantitative and two qualitative papers were the highest scoring and are reviewed and summarized in this article.

CONCLUSIONS

This installment in this critical appraisal series reviews 12 of the highest-quality EM-related medical education research manuscripts published in 2015.

Critical Appraisal of Emergency Medicine Educational Research: The Best Publications of 2015

OBJECTIVE

The objectives were to critically appraise the medical education research literature of 2015 and review the highest-quality quantitative and qualitative examples.

METHODS

A total of 434 emergency medicine (EM)-related articles were discovered upon a search of ERIC, PsychINFO, PubMED, and SCOPUS. These were both quantitative and qualitative in nature. All were screened by two of the authors using previously published exclusion criteria, and the remaining were appraised by all authors using a previously published scoring system. The highest scoring articles were then reviewed.

RESULTS

Sixty-one manuscripts were scored, and 10 quantitative and two qualitative papers were the highest scoring and are reviewed and summarized in this article.

CONCLUSIONS

This installment in this critical appraisal series reviews 12 of the highest-quality EM-related medical education research manuscripts published in 2015.

Measuring physician cognitive load: validity evidence for a physiologic and a psychometric tool

In general, researchers attempt to quantify cognitive load using physiologic and psychometric measures. Although the construct measured by both of these metrics is thought to represent overall cognitive load, there is a paucity of studies that compares these techniques to one another. The authors compared data obtained from one physiologic tool (pupillometry) to one psychometric tool (Paas scale) to explore whether they actually measured the construct of cognitive load as purported. Thirty-two participants with a range of resuscitation medicine experience and expertise completed resuscitation-medicine based multiple-choice-questions as well as arithmetic questions. Cognitive load, as measured by both tools, was found to be higher for the more difficult questions as well as for questions that were answered incorrectly (p < 0.001). The group with the least medical experience had higher cognitive load than both the intermediate and experienced groups when answering domain-specific questions (p = 0.023 and p = 0.003 respectively for the physiologic tool; p = 0.006 and p < 0.001 respectively for the psychometric tool). There was a strong positive correlation (Spearman's ρ = 0.827, p < 0.001 for arithmetic questions; Spearman's ρ = 0.606, p < 0.001 for medical questions) between the two cognitive load measurement tools. These findings support the validity argument that both physiologic and psychometric metrics measure the construct of cognitive load.

The World (of Warcraft) through the eyes of an expert

Negative correlations between pupil size and the tendency to look at salient locations were found in recent studies (e.g., Mathôt et al., 2015). It is hypothesized that this negative correlation might be explained by the mental effort put by participants in the task that leads in return to pupil dilation. Here we present an exploratory study on the effect of expertise on eye-movement behavior. Because there is no available standard tool to evaluate WoW players' expertise, we built an off-game questionnaire testing players' knowledge about WoW and acquired skills through completed raids, highest rated battlegrounds, Skill Points, etc. Experts (N = 4) and novices (N = 4) in the massively multiplayer online role-playing game World of Warcraft (WoW) viewed 24 designed video segments from the game that differ in regards with their content (i.e, informative locations) and visual complexity (i.e, salient locations). Consistent with previous studies, we found a negative correlation between pupil size and the tendency to look at salient locations (experts, r =  - .17, p < .0001, and novices, r =  - .09, p < .0001). This correlation has been interpreted in terms of mental effort: People are inherently biased to look at salient locations (sharp corners, bright lights, etc.), but are able (i.e., experts) to overcome this bias if they invest sufficient mental effort. Crucially, we observed that this correlation was stronger for expert WoW players than novice players (Z =  - 3.3, p = .0011). This suggests that experts learned to improve control over eye-movement behavior by guiding their eyes towards informative, but potentially low-salient areas of the screen. These findings may contribute to our understanding of what makes an expert an expert.

Well-Being in Graduate Medical Education: A Call for Action

Job burnout is highly prevalent in graduate medical trainees. Numerous demands and stressors drive the development of burnout in this population, leading to significant and potentially tragic consequences, not only for trainees but also for the patients and communities they serve. The literature on interventions to reduce resident burnout is limited but suggests that both individual- and system-level approaches are effective. Work hours limitations and mindfulness training are each likely to have modest benefit. Despite concerns that physician trainee wellness programs might be costly, attention to physician wellness may lead to important benefits such as greater patient satisfaction, long-term physician satisfaction, and increased physician productivity. A collaborative of medical educators, academic leaders, and researchers recently formed with the goal of improving trainee well-being and mitigating burnout. Its first task is outlining this framework of initial recommendations in a call to action. These recommendations are made at the national, hospital, program, and nonwork levels and are meant to inform stakeholders who have taken up the charge to address trainee well-being. Regulatory bodies and health care systems need to be accountable for the well-being of trainees under their supervision and drive an enforceable mandate to programs under their charge. Programs and individuals should develop and engage in a "menu" of wellness options to reach a variety of learners and standardize the effort to ameliorate burnout. The impact of these multilevel changes will promote a culture where trainees can learn in settings that will sustain them over the course of their careers.

The challenges of studying visual expertise in medical image diagnosis

CONTEXT

Visual expertise is the superior visual skill shown when executing domain-specific visual tasks. Understanding visual expertise is important in order to understand how the interpretation of medical images may be best learned and taught. In the context of this article, we focus on the visual skill of medical image diagnosis and, more specifically, on the methodological set-ups routinely used in visual expertise research.

METHODS

We offer a critique of commonly used methods and propose three challenges for future research to open up new avenues for studying characteristics of visual expertise in medical image diagnosis. The first challenge addresses theory development. Novel prospects in modelling visual expertise can emerge when we reflect on cognitive and socio-cultural epistemologies in visual expertise research, when we engage in statistical validations of existing theoretical assumptions and when we include social and socio-cultural processes in expertise development. The second challenge addresses the recording and analysis of longitudinal data. If we assume that the development of expertise is a long-term phenomenon, then it follows that future research can engage in advanced statistical modelling of longitudinal expertise data that extends the routine use of cross-sectional material through, for example, animations and dynamic visualisations of developmental data. The third challenge addresses the combination of methods. Alternatives to current practices can integrate qualitative and quantitative approaches in mixed-method designs, embrace relevant yet underused data sources and understand the need for multidisciplinary research teams.

CONCLUSION

Embracing alternative epistemological and methodological approaches for studying visual expertise can lead to a more balanced and robust future for understanding superior visual skills in medical image diagnosis as well as other medical fields.

Before your very eyes: the value and limitations of eye tracking in medical education

CONTEXT

Medicine is a highly visual discipline. Physicians from many specialties constantly use visual information in diagnosis and treatment. However, they are often unable to explain how they use this information. Consequently, it is unclear how to train medical students in this visual processing. Eye tracking is a research technique that may offer answers to these open questions, as it enables researchers to investigate such visual processes directly by measuring eye movements. This may help researchers understand the processes that support or hinder a particular learning outcome.

AIM

In this article, we clarify the value and limitations of eye tracking for medical education researchers. For example, eye tracking can clarify how experience with medical images mediates diagnostic performance and how students engage with learning materials. Furthermore, eye tracking can also be used directly for training purposes by displaying eye movements of experts in medical images.

CONCLUSIONS

Eye movements reflect cognitive processes, but cognitive processes cannot be directly inferred from eye-tracking data. In order to interpret eye-tracking data properly, theoretical models must always be the basis for designing experiments as well as for analysing and interpreting eye-tracking data. The interpretation of eye-tracking data is further supported by sound experimental design and methodological triangulation.

Pupil diameter changes reflect difficulty and diagnostic accuracy during medical image interpretation

Background

No automated methods exist to objectively monitor and evaluate the diagnostic process while physicians review computerized medical images. The present study tested whether using eye tracking to monitor tonic and phasic pupil dynamics may prove valuable in tracking interpretive difficulty and predicting diagnostic accuracy.

Methods

Pathologists interpreted digitized breast biopsies varying in diagnosis and rated difficulty, while pupil diameter was monitored. Tonic diameter was recorded during the entire duration of interpretation, and phasic diameter was examined when the eyes fixated on a pre-determined diagnostic region during inspection.

Results

Tonic pupil diameter was higher with increasing rated difficulty levels of cases. Phasic diameter was interactively influenced by case difficulty and the eventual agreement with consensus diagnosis. More difficult cases produced increases in pupil diameter, but only when the pathologists’ diagnoses were ultimately correct. All results were robust after adjusting for the potential impact of screen brightness on pupil diameter.

Conclusions

Results contribute new understandings of the diagnostic process, theoretical positions regarding locus coeruleus-norepinephrine system function, and suggest novel approaches to monitoring, evaluating, and guiding medical image interpretation.

Measuring cognitive load: performance, mental effort and simulation task complexity

CONTEXT

Interest in applying cognitive load theory in health care simulation is growing. This line of inquiry requires measures that are sensitive to changes in cognitive load arising from different instructional designs. Recently, mental effort ratings and secondary task performance have shown promise as measures of cognitive load in health care simulation.

OBJECTIVES

We investigate the sensitivity of these measures to predicted differences in intrinsic load arising from variations in task complexity and learner expertise during simulation-based surgical skills training.

METHODS

We randomly assigned 28 novice medical students to simulation training on a simple or complex surgical knot-tying task. Participants completed 13 practice trials, interspersed with computer-based video instruction. On trials 1, 5, 9 and 13, knot-tying performance was assessed using time and movement efficiency measures, and cognitive load was assessed using subjective rating of mental effort (SRME) and simple reaction time (SRT) on a vibrotactile stimulus-monitoring secondary task.

RESULTS

Significant improvements in knot-tying performance (F(1.04,24.95)  = 41.1, p < 0.001 for movements; F(1.04,25.90)  = 49.9, p < 0.001 for time) and reduced cognitive load (F(2.3,58.5)  = 57.7, p < 0.001 for SRME; F(1.8,47.3)  = 10.5, p < 0.001 for SRT) were observed in both groups during training. The simple-task group demonstrated superior knot tying (F(1,24)  = 5.2, p = 0.031 for movements; F(1,24)  = 6.5, p = 0.017 for time) and a faster decline in SRME over the first five trials (F(1,26)  = 6.45, p = 0.017) compared with their peers. Although SRT followed a similar pattern, group differences were not statistically significant.

CONCLUSIONS

Both secondary task performance and mental effort ratings are sensitive to changes in intrinsic load among novices engaged in simulation-based learning. These measures can be used to track cognitive load during skills training. Mental effort ratings are also sensitive to small differences in intrinsic load arising from variations in the physical complexity of a simulation task. The complementary nature of these subjective and objective measures suggests their combined use is advantageous in simulation instructional design research.