Eye Tracking: A Novel Approach for Evaluating and Improving the Safety of Healthcare Processes in the Simulated Setting

Assessment of Visual Attention in Teams with or without Dedicated Team Leaders: A Neonatal Simulation-Based Pilot Randomised Cross-Over Trial Utilising Low-Cost Eye-Tracking Technology

BACKGROUND

Eye-tracking technology could be used to study human factors during teamwork.

OBJECTIVES

This work aimed to compare the visual attention (VA) of a team member acting as both a team leader and managing the airway, compared to a team member performing the focused task of managing the airway in the presence of a dedicated team leader. This work also aimed to report differences in team performance, behavioural skills, and workload between the two groups using validated tools.

METHODS

We conducted a simulation-based, pilot randomised controlled study. The participants included were volunteer paediatric trainees, nurse practitioners, and neonatal nurses. Three teams consisting of four team members were formed. Each team participated in two identical neonatal resuscitation simulation scenarios in a random order, once with and once without a team leader. Using a commercially available eye-tracking device, we analysed VA regarding attention to (1) a manikin, (2) a colleague, and (3) a monitor. Only the trainee who was the airway operator would wear eye-tracking glasses in both simulations.

RESULTS

In total, 6 simulation scenarios and 24 individual role allocations were analysed. Participants in a no-team-leader capacity had a greater number of total fixations on manikin and monitors, though this was not significant. There were no significant differences in team performance, behavioural skills, and individual workload. Physical demand was reported as significantly higher by participants in the group without a team leader. During debriefing, all the teams expressed their preference for having a dedicated team leader.

CONCLUSION

In our pilot study using low-cost technology, we could not demonstrate the difference in VA with the presence of a team leader.

Application of eye-tracking in nursing research: A scoping review

AIMS

To map the themes and methods of nursing researches involving eye-tracking as a measurement, and offer suggestion for future nursing research using eye-tracking.

DESIGN

We conducted a scoping review following the methodology outlined in the JBI Manual for Evidence Synthesis on scoping reviews.

METHODS

Eligibility criteria were established based on Population (involving nursing or nursing students), Concept (utilizing eye-tracking as a research method), and Context (in any setting). Articles were retrieved from the PubMed, Web of Science, Embase, CINAHL, APA PsycInfo, and Scopus databases, spanning from database inception to November 17, 2023. The included studies were analysed using descriptive statistics and content analysis.

RESULTS

After duplicates were removed, 815 citations were identified from searches of electronic databases and other resources, and 66 met the inclusion criteria finally. Thirty-eight studies were conducted in a simulated environment. Five application domains were identified, and most of the studies (N = 50) were observational. The domains found in our review did not cover all topics of nursing research in the same depth. Additionally, 39 studies did not solely explicate eye-tracking data but instead integrated behavioural measures, scales/questionnaires, or other physiological data.

CONCLUSIONS

Eye-tracking emerges as a significant research tool in uncovering visual behaviour, particularly in nursing research focused on nursing education. This study not only summarized the application and interpretation of eye-tracking data but also recognized its potential in advancing clinical nursing research and practice. To effectively harness the capabilities of eye-tracking in elucidating cognitive processes, future research should aim for a clearer grasp of the theoretical underpinnings of the addressed research problems and methodological choices. It is crucial to emphasize the standardization of eye-tracking method reporting and ensuring data quality. No Patient or Public Contribution.

Measuring teamwork for training in healthcare using eye tracking and pose estimation

Teamwork is critical for safe patient care. Healthcare teams typically train teamwork in simulated clinical situations, which require the ability to measure teamwork via behavior observation. However, the required observations are prone to human biases and include significant cognitive load even for trained instructors. In this observational study we explored how eye tracking and pose estimation as two minimal invasive video-based technologies may measure teamwork during simulation-based teamwork training in healthcare. Mobile eye tracking, measuring where participants look, and multi-person pose estimation, measuring 3D human body and joint position, were used to record 64 third-year medical students who completed a simulated handover case in teams of four. On one hand, we processed the recorded data into the eye contact metric, based on eye tracking and relevant for situational awareness and communication patterns. On the other hand, the distance to patient metric was processed, based on multi-person pose estimation and relevant for team positioning and coordination. After successful data recording, we successfully processed the raw videos to specific teamwork metrics. The average eye contact time was 6.46 s [min 0 s - max 28.01 s], while the average distance to the patient resulted in 1.01 m [min 0.32 m - max 1.6 m]. Both metrics varied significantly between teams and simulated roles of participants (p < 0.001). With the objective, continuous, and reliable metrics we created visualizations illustrating the teams' interactions. Future research is necessary to generalize our findings and how they may complement existing methods, support instructors, and contribute to the quality of teamwork training in healthcare.

Provider Visual Attention Correlates With the Quality of Pediatric Resuscitation: An Observational Eye-Tracking Study

Background

Eye-tracking devices are an innovative tool to understand providers' attention during stressful medical tasks. The knowledge about what gaze behaviors improve (or harm) the quality of clinical care can substantially improve medical training. The aim of this study is to identify gaze behaviors that are related to the quality of pediatric resuscitation.

Methods

Forty students and healthcare providers performed a simulated pediatric life support scenario, consisting of a chest compression task and a ventilation task, while wearing eye-tracking glasses. Skill Reporter software measured chest compression (CC) quality and Neo Training software measured ventilation quality. Main eye-tracking parameters were ratio [the number of participants who attended a certain area of interest (AOI)], dwell time (total amount of time a participant attended an AOI), the number of revisits (how often a participant returned his gaze to an AOI), and the number of transitions between AOIs.

Results

The most salient AOIs were infant chest and ventilation mask (ratio = 100%). During CC task, 41% of participants also focused on ventilation bag and 59% on study nurse. During ventilation task, the ratio was 61% for ventilation bag and 36% for study nurse. Percentage of correct CC rate was positively correlated with dwell time on infant chest (p = 0.044), while the overall CC quality was negatively correlated with dwelling outside of pre-defined task-relevant AOIs (p = 0.018). Furthermore, more dwell time on infant chest predicted lower leakage (p = 0.042). The number of transitions between AOIs was unrelated to CC parameters, but correlated negatively with mask leak during ventilations (p = 0.014). Participants with high leakage shifted their gaze more often between ventilation bag, ventilation mask, and task-irrelevant environment.

Conclusion

Infant chest and ventilation mask are the most salient AOIs in pediatric basic life support. Especially the infant chest AOI gives beneficial information for the resuscitation provider. In contrast, attention to task-irrelevant environment and frequent gaze shifts seem to harm the quality of care.

Detecting Task Difficulty of Learners in Colonoscopy: Evidence from Eye-Tracking

Eye-tracking can help decode the intricate control mechanism in human performance. In healthcare, physicians-in-training require extensive practice to improve their healthcare skills. When a trainee encounters any difficulty in the practice, they will need feedback from experts to improve their performance. Personal feedback is time-consuming and subjected to bias. In this study, we tracked the eye movements of trainees during their colonoscopic performance in simulation. We examined changes in eye movement behavior during the moments of navigation loss (MNL), a signature sign for task difficulty during colonoscopy, and tested whether deep learning algorithms can detect the MNL by feeding data from eye-tracking. Human eye gaze and pupil characteristics were learned and verified by the deep convolutional generative adversarial networks (DCGANs); the generated data were fed to the Long Short-Term Memory (LSTM) networks with three different data feeding strategies to classify MNLs from the entire colonoscopic procedure. Outputs from deep learning were compared to the expert's judgment on the MNLs based on colonoscopic videos. The best classification outcome was achieved when we fed human eye data with 1000 synthesized eye data, where accuracy (91.80%), sensitivity (90.91%), and specificity (94.12%) were optimized. This study built an important foundation for our work of developing an education system for training healthcare skills using simulation.

Using eye-tracking augmented cognitive task analysis to explore healthcare professionals' cognition during neonatal resuscitation

AIM

We aimed to describe the cognitive processes of healthcare providers participating as airway leads in delivery room neonatal resuscitations using eye-tracking assisted debriefing to facilitate recall and provide situational context.

METHODS

Delivery room neonatal resuscitations were recorded using eye-tracking glasses worn by participants who acted as airway leads. These glasses analyze eye-movements to produce an audio-visual recording approximating what was "seen" by the participant and marking their visual attention. Participants then reviewed and debriefed their recordings. Debriefing involved a retrospective think-aloud prompted by eye-tracked recordings and an integrated semi-structured interview. Debriefing sessions were transcribed and subjected to thematic analysis.

RESULTS

Eight healthcare providers participated in 10 interviews; two providers participated twice in two separate resuscitations. Most visual attention was directed at the infant (62%), with 16% directed to monitors/gauges, 3% to team members. Five major themes emerged including situation awareness, performance, working in teams, addressing threats to performance, and perception of eye-tracking. Information processing was complex and involved top-down and bottom-up processing of environmental stimuli, integration of knowledge/experience, and anticipation of patient response. Despite the focus on individual cognition, interpersonal interactions and teamwork emerged as key aspects of resuscitation performance. Potential threats to performance include equipment issues, mental stress, distractions, and parental presence. Eye-tracking recordings were well-received by the participants.

CONCLUSION

Retrospective think-aloud prompted by point-of-view eye-tracked recordings is a useful means of examining cognition of healthcare providers during neonatal resuscitation. Themes identified in this project aligned with existing models of clinical reasoning.

Using Eye Tracking for Measuring Cognitive Workload During Clinical Simulations: Literature Review and Synthesis

High-fidelity clinical simulations can be used by clinicians to acquire technical (physical ability and knowledge) and non-technical (cognitive and social processes) skills. Excessive cognitive workload contributes to medical errors because of the impact on both technical and non-technical skills. Many studies measure cognitive workload with psychometric instruments that limit the assessment of cognitive workload to a single time period and may involve response bias. Using eye tracking to measure task-evoked pupillary responses allows the measurement of changes in pupil diameter related to the cognitive workload associated with a specific activity. Incorporating eye tracking with high-fidelity clinical simulations provides a reliable and continuous assessment of cognitive workload. The purpose of this literature review is to summarize the use of eye-tracking technology to measure cognitive workload of healthcare providers to generate evidence-based guidelines for measuring cognitive workload during high-fidelity clinical simulations. What this manuscript adds to the body of literature is a summary of best practices related to the different methods of measuring cognitive workload, benefits and limitations of using eye tracking, and high-fidelity clinical simulation design considerations for successful integration of eye tracking.

Eye-Tracking Metrics Predict Perceived Workload in Robotic Surgical Skills Training

OBJECTIVE

The aim of this study is to assess the relationship between eye-tracking measures and perceived workload in robotic surgical tasks.

BACKGROUND

Robotic techniques provide improved dexterity, stereoscopic vision, and ergonomic control system over laparoscopic surgery, but the complexity of the interfaces and operations may pose new challenges to surgeons and compromise patient safety. Limited studies have objectively quantified workload and its impact on performance in robotic surgery. Although not yet implemented in robotic surgery, minimally intrusive and continuous eye-tracking metrics have been shown to be sensitive to changes in workload in other domains.

METHODS

Eight surgical trainees participated in 15 robotic skills simulation sessions. In each session, participants performed up to 12 simulated exercises. Correlation and mixed-effects analyses were conducted to explore the relationships between eye-tracking metrics and perceived workload. Machine learning classifiers were used to determine the sensitivity of differentiating between low and high workload with eye-tracking features.

RESULTS

Gaze entropy increased as perceived workload increased, with a correlation of .51. Pupil diameter and gaze entropy distinguished differences in workload between task difficulty levels, and both metrics increased as task level difficulty increased. The classification model using eye-tracking features achieved an accuracy of 84.7% in predicting workload levels.

CONCLUSION

Eye-tracking measures can detect perceived workload during robotic tasks. They can potentially be used to identify task contributors to high workload and provide measures for robotic surgery training.

APPLICATION

Workload assessment can be used for real-time monitoring of workload in robotic surgical training and provide assessments for performance and learning.

Analysis of visual attention and team communications during neonatal endotracheal intubations using eye-tracking: An observational study

PURPOSE

Eye-tracking can be used to analyse visual attention (VA) of health care providers during clinical tasks. No study has examined eye-tracking during neonatal endotracheal intubation. We aimed to examine VA and team communications during endotracheal intubation using eye-tracking in the Neonatal Intensive Care Unit.

METHODS

Twenty-seven video-recordings were obtained using eye-tracking glasses worn by intubators during endotracheal intubation of stable neonates. Videos were analysed to obtain i) intubation duration and success, ii) areas of interest (AOIs), iii) duration spent on each AOI, iv) types and frequency of gaze-shifts between AOIs, and v) monitor looking behaviour, and vi) team communications of vital signs and verbal medication orders.

RESULTS

Twenty-four videos were of acceptable quality and analysed. Median attempt duration was 44.7 s. Success rate was 79%. Overall, 50% of VA was directed at the infant, with 23% of VA directed at equipment. There were 415 gaze-shifts types and 0.55 gaze-shifts/sec. Intubators glanced at the monitor spontaneously and rarely reported vital signs afterwards. Language used to communicate vital signs and medication orders varied.

CONCLUSION

During neonatal intubations, 50% of VA was directed away from the infant. Team communications were non-standard. Eye-tracking can be used to analyse human performance during neonatal resuscitation.

Effect of monitor positioning on visual attention and situation awareness during neonatal resuscitation: a randomised simulation study

OBJECTIVES

To compare situation awareness (SA), visual attention (VA) and protocol adherence in simulated neonatal resuscitations using two different monitor positions.

DESIGN

Randomised controlled simulation study.

SETTINGS

Simulation lab at the Royal Alexandra Hospital, Edmonton, Canada.

PARTICIPANTS

Healthcare providers (HCPs) with Neonatal Resuscitation Program (NRP) certification within the last 2 years and trained in neonatal endotracheal intubations.

INTERVENTION

HCPs were randomised to either central (eye-level on the radiant warmer) or peripheral (above eye-level, wall-mounted) monitor positions. Each led a complex resuscitation with a high-fidelity mannequin and a standardised assistant. To measure SA, situation awareness global assessment tool (SAGAT) was used, where simulations were paused at three predetermined points, with five questions asked each pause. Videos were analysed for SAGAT and adherence to a NRP checklist. Eye-tracking glasses recorded participants' VA.

MAIN OUTCOME MEASURE

The main outcome was SA as measured by composite SAGAT score. Secondary outcomes included VA and adherence to NRP checklist.

RESULTS

Thirty simulations were performed; 29 were completed per protocol and analysed. Twenty-two eye-tracking recordings were of sufficient quality and analysed. Median composite SAGAT was 11.5/15 central versus 11/15 peripheral, p=0.56. Checklist scores 46/50 central versus 46/50 peripheral, p=0.75. Most VA was directed at the mannequin (30.6% central vs 34.1% peripheral, p=0.76), and the monitor (28.7% central vs 20.5% peripheral, p=0.06).

CONCLUSIONS

Simulation, SAGAT and eye-tracking can be used to evaluate human factors of neonatal resuscitation. During simulated neonatal resuscitation, monitor position did not affect SA, VA or protocol adherence.

Eye-tracking during simulation-based neonatal airway management

BACKGROUND

Eye-tracking devices help to understand provider behavior during medical tasks. The aim of this study was to assess participants' gaze behavior and usability of eye-tracking glasses during airway management in a simulated neonatal resuscitation.

METHODS

This study was an observational simulation-based study. The team member assigned to airway management wore head-mounted eye-tracking glasses. Main outcome measures were airway providers' gaze, dwell time (total amount of time a participant fixates certain areas of interest), and usability of eye-tracking glasses.

RESULTS

Data from 13 participants were included. There were significant differences in dwell time during the scenario (p < 0.001), with participants spending twice as much time on the newborn and instruments as on the monitor and other staff. Participants spent about 25% more time focusing on another provider while the provider was inserting the umbilical vein catheter than in all other times of interest (intervals of time with meaningful events) (p = 0.04). The use of the glasses was perceived easy and not disturbing.

CONCLUSIONS

Eye-tracking glasses enhance our understanding of providers' gaze and perspective during simulated neonatal airway management. Future studies will better characterize the ideal use in real situations.

ABCDE approach to victims by lifeguards: How do they manage a critical patient? A cross sectional simulation study

Introduction

Decision-making in emergencies is a multifactorial process based on the rescuer, patient, setting and resources. The eye-tracking system is a proven method for assessing decision-making processes that have been used in different fields of science. Our aim was to evaluate the lifeguards’ capacity to perform the ABCDE (Airway-Breathing-Circulation-Disability-Exposure) approach when facing a simulated critically ill-drowned victim.

Methods

A cross-sectional simulation study was designed to assess the skills and sequence of the ABCDE approach by 20 professional lifeguards. They had to assess a victim and act according to his/her clinical status by following the ABCDE primary assessment approach. The two kinds of variables were recorder: those related to the quality of each step of the ABCDE approach and the visual behaviour using a portable eye-movement system. The eye-tracking system was the Mobile Eye system (Bedford, USA).

Results

None of the study participants were able to complete correctly the ABCDE approach. Lifeguards spent more time in the Circulation step: Airway (15.5±11.1 s), Breathing (25.1±21.1 s), Circulation (44.6±29.5 s), Disability (38.5±0.7 s). Participants spent more time in viewpoints considered as important (65.5±17.4 s) compared with secondary ones (34.6±17.4 s, p = 0.008). This was also represented in the percentage of visual fixations (fixations in important viewpoints: 63.36±15.06; fixation in secondary viewpoints: 36.64±15.06; p = 0.008).

Conclusion

Professional lifeguards failed to fully perform the ABCDE sequence. Evaluation by experts with the help of eye-tracking technology detected the lifeguards’ limitations in the assessment and treatment of an eventual critically ill victim. Such deficits should be considered in the design and implementation of lifeguards’ training programmes.

A new way to look at simulation-based assessment: the relationship between gaze-tracking and exam performance

Objective

A key task of the team leader in a medical emergency is effective information gathering. Studying information gathering patterns is readily accomplished with the use of gaze-tracking glasses. This technology was used to generate hypotheses about the relationship between performance scores and expert-hypothesized visual areas of interest in residents across scenarios in simulated medical resuscitation examinations.

Methods

Emergency medicine residents wore gaze-tracking glasses during two simulation-based examinations (n=29 and 13 respectively). Blinded experts assessed video-recorded performances using a simulation performance assessment tool that has validity evidence in this context. The relationships between gaze patterns and performance scores were analyzed and potential hypotheses generated. Four scenarios were assessed in this study: diabetic ketoacidosis, bradycardia secondary to beta-blocker overdose, ruptured abdominal aortic aneurysm and metabolic acidosis caused by antifreeze ingestion.

Results

Specific gaze patterns were correlated with objective performance. High performers were more likely to fixate on task-relevant stimuli and appropriately ignore task-irrelevant stimuli compared with lower performers. For example, shorter latency to fixation on the vital signs in a case of diabetic ketoacidosis was positively correlated with performance (r=0.70, p<0.05). Conversely, total time spent fixating on lab values in a case of ruptured abdominal aortic aneurysm was negatively correlated with performance (r= −0.50, p<0.05).

Conclusions

There are differences between the visual patterns of high and low-performing residents. These findings may allow for better characterization of expertise development in resuscitation medicine and provide a framework for future study of visual behaviours in resuscitation cases.

Analysis of neonatal resuscitation using eye tracking: a pilot study

BACKGROUND

Visual attention (VA) is important for situation awareness and decision-making. Eye tracking can be used to analyse the VA of healthcare providers. No study has examined eye tracking during neonatal resuscitation.

OBJECTIVE

To test the use of eye tracking to examine VA during neonatal resuscitation.

METHODS

Six video recordings were obtained using eye tracking glasses worn by resuscitators during the first 5 min of neonatal resuscitation. Videos were analysed to obtain (i) areas of interest (AOIs), (ii) time spent on each AOI and (iii) frequency of saccades between AOIs.

RESULTS

Five videos were of acceptable quality and analysed. Only 35% of VA was directed at the infant, with 33% at patient monitors and gauges. There were frequent saccades (0.45/s) and most involved patient monitors.

CONCLUSION

During neonatal resuscitation, VA is often directed away from the infant towards patient monitors. Eye tracking can be used to analyse human performance during neonatal resuscitation.

Quantifying the cognitive cost of laparo-endoscopic single-site surgeries: Gaze-based indices

BACKGROUND

Despite the growing interest concerning the laparo-endoscopic single-site surgery (LESS) procedure, LESS presents multiple difficulties and challenges that are likely to increase the surgeon's cognitive cost, in terms of both cognitive load and performance. Nevertheless, there is currently no objective index capable of assessing the surgeon cognitive cost while performing LESS. We assessed if gaze-based indices might offer unique and unbiased measures to quantify LESS complexity and its cognitive cost. We expect that the assessment of surgeon's cognitive cost to improve patient safety by measuring fitness-for-duty and reducing surgeons overload.

METHODS

Using a wearable eye tracker device, we measured gaze entropy and velocity of surgical trainees and attending surgeons during two surgical procedures (LESS vs. multiport laparoscopy surgery [MPS]). None of the participants had previous experience with LESS. They performed two exercises with different complexity levels (Low: Pattern Cut vs. High: Peg Transfer). We also collected performance and subjective data.

RESULTS

LESS caused higher cognitive demand than MPS, as indicated by increased gaze entropy in both surgical trainees and attending surgeons (exploration pattern became more random). Furthermore, gaze velocity was higher (exploration pattern became more rapid) for the LESS procedure independently of the surgeon's expertise. Perceived task complexity and laparoscopic accuracy confirmed gaze-based results.

CONCLUSION

Gaze-based indices have great potential as objective and non-intrusive measures to assess surgeons' cognitive cost and fitness-for-duty. Furthermore, gaze-based indices might play a relevant role in defining future guidelines on surgeons' examinations to mark their achievements during the entire training (e.g. analyzing surgical learning curves).

Recognizing the Ordinary as Extraordinary: Insight Into the "Way We Work" to Improve Patient Safety Outcomes

The Institute of Medicine (now National Academy of Medicine) reports "To Err is Human" and "Crossing the Chasm" made explicit 3 previously unappreciated realities: (1) Medical errors are common and result in serious, preventable adverse events; (2) The majority of medical errors are the result of system versus human failures; and (3) It would be impossible for any system to prevent all errors. With these realities, the role of the nurse in the "near miss" process and as the final safety net for the patient is of paramount importance. The nurse's role in patient safety is described from both a systems perspective and a human factors perspective. Critical care nurses use specific strategies to identify, interrupt, and correct medical errors. Strategies to identify errors include knowing the patient, knowing the plan of care, double-checking, and surveillance. Nursing strategies to interrupt errors include offering assistance, clarifying, and verbally interrupting. Nurses correct errors by persevering, being physically present, reviewing/confirming the plan of care, or involving another nurse or physician. Each of these strategies has implications for education, practice, and research. Surveillance is a key nursing strategy for identifying medical errors and reducing adverse events. Eye-tracking technology is a novel approach for evaluating the surveillance process during common, high-risk processes such as blood transfusion and medication administration. Eye tracking has also been used to examine the impact of interruptions to care caused by bedside alarms as well as by other health care personnel. Findings from this safety-related eye-tracking research provide new insight into effective bedside surveillance and interruption management strategies.

Toward Designing Information Display to Support Critical Care. A Qualitative Contextual Evaluation and Visioning Effort

OBJECTIVES

Electronic health information overload makes it difficult for providers to quickly find and interpret information to support care decisions. The purpose of this study was to better understand how clinicians use information in critical care to support the design of improved presentation of electronic health information.

METHODS

We conducted a contextual analysis and visioning project. We used an eye-tracker to record 20 clinicians' information use activities in critical care settings. We played video recordings back to clinicians in retrospective cued interviews and queried: 1) context and goals of information use, 2) impacts of current display design on use, and 3) processes related to information use. We analyzed interview transcripts using grounded theory-based content analysis techniques and identified emerging themes. From these, we conducted a visioning activity with a team of subject matter experts and identified key areas for focus of design and research for future display designs.

RESULTS

Analyses revealed four unique critical care information use activities including new patient assessment, known patient status review, specific directed information seeking, and review and prioritization of multiple patients. Emerging themes were primarily related to a need for better representation of dynamic data such as vital signs and laboratory results, usability issues associated with reducing cognitive load and supporting efficient interaction, and processes for managing information. Visions for the future included designs that: 1) provide rapid access to new information, 2) organize by systems or problems as well as by current versus historical patient data, and 3) apply intelligence toward detecting and representing change and urgency.

CONCLUSIONS

The results from this study can be used to guide the design of future acute care electronic health information display. Additional research and collaboration is needed to refine and implement intelligent graphical user interfaces to improve clinical information organization and prioritization to support care decisions.