Action-related eye measures to assess surgical expertise

Eye tracking in digital pathology: A comprehensive literature review

Eye tracking has been used for decades in attempt to understand the cognitive processes of individuals. From memory access to problem-solving to decision-making, such insight has the potential to improve workflows and the education of students to become experts in relevant fields. Until recently, the traditional use of microscopes in pathology made eye tracking exceptionally difficult. However, the digital revolution of pathology from conventional microscopes to digital whole slide images allows for new research to be conducted and information to be learned with regards to pathologist visual search patterns and learning experiences. This has the promise to make pathology education more efficient and engaging, ultimately creating stronger and more proficient generations of pathologists to come. The goal of this review on eye tracking in pathology is to characterize and compare the visual search patterns of pathologists. The PubMed and Web of Science databases were searched using 'pathology' AND 'eye tracking' synonyms. A total of 22 relevant full-text articles published up to and including 2023 were identified and included in this review. Thematic analysis was conducted to organize each study into one or more of the 10 themes identified to characterize the visual search patterns of pathologists: (1) effect of experience, (2) fixations, (3) zooming, (4) panning, (5) saccades, (6) pupil diameter, (7) interpretation time, (8) strategies, (9) machine learning, and (10) education. Expert pathologists were found to have higher diagnostic accuracy, fewer fixations, and shorter interpretation times than pathologists with less experience. Further, literature on eye tracking in pathology indicates that there are several visual strategies for diagnostic interpretation of digital pathology images, but no evidence of a superior strategy exists. The educational implications of eye tracking in pathology have also been explored but the effect of teaching novices how to search as an expert remains unclear. In this article, the main challenges and prospects of eye tracking in pathology are briefly discussed along with their implications to the field.

Eye-Tracking in Physical Human-Robot Interaction: Mental Workload and Performance Prediction

BACKGROUND

In Physical Human-Robot Interaction (pHRI), the need to learn the robot's motor-control dynamics is associated with increased cognitive load. Eye-tracking metrics can help understand the dynamics of fluctuating mental workload over the course of learning.

OBJECTIVE

The aim of this study was to test eye-tracking measures' sensitivity and reliability to variations in task difficulty, as well as their performance-prediction capability, in physical human-robot collaboration tasks involving an industrial robot for object comanipulation.

METHODS

Participants (9M, 9F) learned to coperform a virtual pick-and-place task with a bimanual robot over multiple trials. Joint stiffness of the robot was manipulated to increase motor-coordination demands. The psychometric properties of eye-tracking measures and their ability to predict performance was investigated.

RESULTS

Stationary Gaze Entropy and pupil diameter were the most reliable and sensitive measures of workload associated with changes in task difficulty and learning. Increased task difficulty was more likely to result in a robot-monitoring strategy. Eye-tracking measures were able to predict the occurrence of success or failure in each trial with 70% sensitivity and 71% accuracy.

CONCLUSION

The sensitivity and reliability of eye-tracking measures was acceptable, although values were lower than those observed in cognitive domains. Measures of gaze behaviors indicative of visual monitoring strategies were most sensitive to task difficulty manipulations, and should be explored further for the pHRI domain where motor-control and internal-model formation will likely be strong contributors to workload.

APPLICATION

Future collaborative robots can adapt to human cognitive state and skill-level measured using eye-tracking measures of workload and visual attention.

Comparison of virtual reality and computed tomography in the preoperative planning of complex tibial plateau fractures

INTRODUCTION

Preoperative planning is a critical step in the success of any complex surgery. The pur-pose of this study is to evaluate the advantage of VR glasses in surgical planning of complex tibial plateau fractures compared to CT planning.

MATERIALS AND METHODS

Five orthopedic surgeons performed preoperative planning for 30 fractures using either conventional CT slices or VR visualization with a VR headset. Planning was performed in a randomized order with a 3-month interval between planning sessions. A standardized questionnaire assessed planned operative time, planning time, fracture classification and understanding, and surgeons' subjective confidence in surgical planning.

RESULTS

The mean planned operative time of 156 (SD 47) minutes was significantly lower (p < 0.001) in the VR group than in the CT group (172 min; SD 44). The mean planning time in the VR group was 3.48 min (SD 2.4), 17% longer than in the CT group (2.98 min, SD 1.9; p = 0.027). Relevant parameters influencing planning time were surgeon experience (-0.61 min) and estimated complexity of fracture treatment (+ 0.65 min).

CONCLUSION

The use of virtual reality for surgical planning of complex tibial plateau fractures resulted in significantly shorter planned operative time, while planning time was longer compared to CT planning. After VR planning, more surgeons felt (very) well prepared for surgery.

Early Eye Disengagement Is Regulated by Task Complexity and Task Repetition in Visual Tracking Task

Understanding human actions often requires in-depth detection and interpretation of bio-signals. Early eye disengagement from the target (EEDT) represents a significant eye behavior that involves the proactive disengagement of the gazes from the target to gather information on the anticipated pathway, thereby enabling rapid reactions to the environment. It remains unknown how task difficulty and task repetition affect EEDT. We aim to provide direct evidence of how these factors influence EEDT. We developed a visual tracking task in which participants viewed arrow movement videos while their eye movements were tracked. The task complexity was increased by increasing movement steps. Every movement pattern was performed twice to assess the effect of repetition on eye movement. Participants were required to recall the movement patterns for recall accuracy evaluation and complete cognitive load assessment. EEDT was quantified by the fixation duration and frequency within the areas of eye before arrow. When task difficulty increased, we found the recall accuracy score decreased, the cognitive load increased, and EEDT decreased significantly. The EEDT was higher in the second trial, but significance only existed in tasks with lower complexity. EEDT was positively correlated with recall accuracy and negatively correlated with cognitive load. Performing EEDT was reduced by task complexity and increased by task repetition. EEDT may be a promising sensory measure for assessing task performance and cognitive load and can be used for the future development of eye-tracking-based sensors.

Exploring eye-tracking data as an indicator of situational awareness in nursing students during a cardiorespiratory arrest simulation

AIM

To examine the components of visual attention that maintain situational awareness during simulation training in undergraduate nursing students with different instruction levels.

BACKGROUND

Eye-tracking can provide deep insight into the nurses' attention during simulated practice. Knowing which gaze patterns promote situational awareness can significantly improve nurse instruction.

DESIGN

A comparative observational study investigated the role of visual attention on the performance quality, psychophysiological parameters (vital signs, anxiety and stress) and socioemotional competencies (cognitive workload, motivation and self-efficacy) of nursing students with various experience levels.

METHODS

Thirty nursing students divided into two groups according to their academic level: first cycle (n=14) and second-cycle (n=16) faced a clinical simulation scenario to resolve a cardiorespiratory arrest event. Eye tracking-based analysis required the selection of six areas of interest. The monitorization of vital signs included measuring blood pressure, heart rate, respiratory rate and oxygen saturation before and after the simulation practice. Participants completed the socioemotional questionnaire (NASA-TLX). They answered the state subscale of the State-Trait Anxiety Inventory (STAI), the Visual Analogue Scale (VAS) of stress, the Situational Motivation Scale (SIMS) and the Baessler and Schwarzer General Self-Efficacy Scale.

RESULTS

The first-cycle group displayed higher vital sign scores than the second cycle, apart from the post-simulation respiratory rate. All physiological parameters increased in mean value after the clinical simulation, except oxygen saturation. Anxiety was the only parameter in the socioemotional domain to present a statistically significant difference between the groups. First-year nursing students showed greater anxiety, stress, mental workload, identified regulation and intrinsic motivation, while second-year students showed higher levels of amotivation, external regulation and perceived self-efficacy. Eye-tracking data (revisits, gaze and duration of fixations) exhibited statistically significant differences depending on the area of interest in both groups (p =. 05). The performance outcomes showed a negative and moderate association with gaze the total number of gazes in the second-cycle group (rho = -0.640, p = 0.010).

CONCLUSION

Eye-tracking-based analysis can help to predict performance quality while maintaining situational awareness during nursing instruction.

Developing surgical skill level classification model using visual metrics and a gradient boosting algorithm

Objective

Assessment of surgical skills is crucial for improving training standards and ensuring the quality of primary care. This study aimed to develop a gradient boosting classification model (GBM) to classify surgical expertise into inexperienced, competent, and experienced levels in robot-assisted surgery (RAS) using visual metrics.

Methods

Eye gaze data were recorded from 11 participants performing four subtasks; blunt dissection, retraction, cold dissection, and hot dissection using live pigs and the da Vinci robot. Eye gaze data were used to extract the visual metrics. One expert RAS surgeon evaluated each participant's performance and expertise level using the modified Global Evaluative Assessment of Robotic Skills (GEARS) assessment tool. The extracted visual metrics were used to classify surgical skill levels and to evaluate individual GEARS metrics. Analysis of Variance (ANOVA) was used to test the differences for each feature across skill levels.

Results

Classification accuracies for blunt dissection, retraction, cold dissection, and burn dissection were 95%, 96%, 96%, and 96%, respectively. The time to complete only the retraction was significantly different among the 3 skill levels (p-value = 0.04). Performance was significantly different for 3 categories of surgical skill level for all subtasks (p-values<0.01). The extracted visual metrics were strongly associated with GEARS metrics (R2>0.7 for GEARS metrics evaluation models).

Conclusions

Machine learning (ML) algorithms trained by visual metrics of RAS surgeons can classify surgical skill levels and evaluate GEARS measures. The time to complete a surgical subtask may not be considered a stand-alone factor for skill level assessment.

Eye gaze metrics for skill assessment and feedback in kidney stone surgery

PURPOSE

Surgical skill assessment is essential for safe operations. In endoscopic kidney stone surgery, surgeons must perform a highly skill-dependent mental mapping from the pre-operative scan to the intraoperative endoscope image. Poor mental mapping can lead to incomplete exploration of the kidney and high reoperation rates. Yet there are few objective ways to evaluate competency. We propose to use unobtrusive eye-gaze measurements in the task space to evaluate skill and provide feedback.

METHODS

We capture the surgeons' eye gaze on the surgical monitor with the Microsoft Hololens 2. To enable stable and accurate gaze detection, we develop a calibration algorithm to refine the eye tracking of the Hololens. In addition, we use a QR code to locate the eye gaze on the surgical monitor. We then run a user study with three expert and three novice surgeons. Each surgeon is tasked to locate three needles representing kidney stones in three different kidney phantoms.

RESULTS

We find that experts have more focused gaze patterns. They complete the task faster, have smaller total gaze area, and the gaze fewer times outside the area of interest. While fixation to non-fixation ratio did not show significant difference in our findings, tracking the ratio over time shows different patterns between novices and experts.

CONCLUSION

We show that a non-negligible difference holds between novice and expert surgeons' gaze metrics in kidney stone identification in phantoms. Expert surgeons demonstrate more targeted gaze throughout a trial, indicating their higher level of proficiency. To improve the skill acquisition process for novice surgeons, we suggest providing sub-task specific feedback. This approach presents an objective and non-invasive method to assess surgical competence.

Telestration with augmented reality improves surgical performance through gaze guidance

BACKGROUND

In minimally invasive surgery (MIS), trainees need to learn how to interpret the operative field displayed on the laparoscopic screen. Experts currently guide trainees mainly verbally during laparoscopic procedures. A newly developed telestration system with augmented reality (iSurgeon) allows the instructor to display hand gestures in real-time on the laparoscopic screen in augmented reality to provide visual expert guidance (telestration). This study analysed the effect of telestration guided instructions on gaze behaviour during MIS training.

METHODS

In a randomized-controlled crossover study, 40 MIS naive medical students performed 8 laparoscopic tasks with telestration or with verbal instructions only. Pupil Core eye-tracking glasses were used to capture the instructor's and trainees' gazes. Gaze behaviour measures for tasks 1-7 were gaze latency, gaze convergence and collaborative gaze convergence. Performance measures included the number of errors in tasks 1-7 and trainee's ratings in structured and standardized performance scores in task 8 (ex vivo porcine laparoscopic cholecystectomy).

RESULTS

There was a significant improvement 1-7 on gaze latency [F(1,39) = 762.5, p < 0.01, ηp2 = 0.95], gaze convergence [F(1,39) = 482.8, p < 0.01, ηp2 = 0.93] and collaborative gaze convergence [F(1,39) = 408.4, p < 0.01, ηp2 = 0.91] upon instruction with iSurgeon. The number of errors was significantly lower in tasks 1-7 (0.18 ± 0.56 vs. 1.94 ± 1.80, p < 0.01) and the score ratings for laparoscopic cholecystectomy were significantly higher with telestration (global OSATS: 29 ± 2.5 vs. 25 ± 5.5, p < 0.01; task-specific OSATS: 60 ± 3 vs. 50 ± 6, p < 0.01).

CONCLUSIONS

Telestration with augmented reality successfully improved surgical performance. The trainee's gaze behaviour was improved by reducing the time from instruction to fixation on targets and leading to a higher convergence of the instructor's and the trainee's gazes. Also, the convergence of trainee's gaze and target areas increased with telestration. This confirms augmented reality-based telestration works by means of gaze guidance in MIS and could be used to improve training outcomes.

Gaze Behaviors of Neurosurgeon in Performing Gross Movements under Microscope

BACKGROUND

Frequent intermittent bleeding control and suction are often necessary during microneurosurgical procedures. We compared the visual searching strategy that guides these types of gross hand movements between expert surgeons and neurosurgical residents.

METHODS

A total of 70 trials of a microsurgical task consisting of moving a cotton ball with bipolar forceps and a suction instrument under the microscope were recorded. Based on surgical videos, we extracted total task time, boarding time, gaze-tool time gap, number of fixations during the boarding time, and target-locked fixation duration. A comparison of these measures between experts and resident surgeons were performed.

RESULTS

No significant difference was found for total task time, boarding time, and number of fixations during the boarding time between the 2 groups of surgeons. However, we found significant differences in the measure of gaze-tool time gap (P < 0.001, partial η² = 0.54) and the target-locked fixation (P < 0.001, partial η² = 0.24). Specifically, expert microsurgeons' eyes move to the target board earlier before their tools in hands move (1.01 ± 0.30 seconds) compared with resident microsurgeons (0.44 ± 0.17 seconds). The target-locked fixation duration was also longer among experts comparing to residents (experts: 1.16 ± 0.82 seconds, residents: 0.40 ± 0.30 seconds).

CONCLUSIONS

Task time analysis for this basic microsurgical task is not sufficient to evaluate participants' level of expertise. Gaze behavior analysis helps to reveal hidden differences between experts and residents. This research provides more evidence that supports the use of gaze analysis for assessing surgeons' skills in microsurgery.