What Am I Looking at? Interpreting Dynamic and Static Gaze Displays

Teaching through their eyes: effects on optometry teachers' adaptivity and students' learning when teachers see students' gaze

Adaptive teacher support fosters effective learning in one-to-one teaching sessions, which are a common way of learning complex visual tasks in the health sciences. Adaptive support is tailored to student needs, and this is difficult in complex visual tasks as visual problem-solving processes are covert and thus cannot be directly observed by the teacher. Eye-tracking apparatus can measure covert processes and make them visible in gaze displays: visualizations of where a student looks while executing a task. We investigate whether live dynamic gaze displays help teachers in being more adaptive to students' needs when teaching optical coherence tomography interpretation in one-to-one teaching sessions and whether this fosters learning. Forty-nine students and 10 teachers participated in a one-to-one teaching session in clinical optometry. In the control condition, teachers saw the learning task of the student and could discuss it with them, whereas in the gaze-display condition, teachers could additionally see where the student looked. After the 15-minute teaching session, a test was administered to examine achievement. Furthermore, students filled in the 'questionnaire on teacher support adaptivity', and teachers rated how adaptive their support was. Bayesian analyses provide some initial evidence that students did not experience support to be more adaptive in the gaze-display condition versus the control condition, nor were their post-test scores higher. Teachers rated their provided support as being more adaptive in the gaze-display versus the control condition. Further research could investigate if live dynamic gaze displays impact adaptive teaching when used over longer periods or with more teacher training.

Visualizing a Task Performer’s Gaze to Foster Observers’ Performance and Learning—a Systematic Literature Review on Eye Movement Modeling Examples

Eye movement modeling examples (EMMEs) are instructional videos (e.g., tutorials) that visualize another person’s gaze location while they demonstrate how to perform a task. This systematic literature review provides a detailed overview of studies on the effects of EMME to foster observers’ performance and learning and highlights their differences in EMME designs. Through a broad, systematic search on four relevant databases, we identified 72 EMME studies (78 experiments). First, we created an overview of the different study backgrounds. Studies most often taught tasks from the domains of sports/physical education, medicine, aviation, and STEM areas and had different rationales for displaying EMME. Next, we outlined how studies differed in terms of participant characteristics, task types, and the design of the EMME materials, which makes it hard to infer how these differences affect performance and learning. Third, we concluded that the vast majority of the experiments showed at least some positive effects of EMME during learning, on tests directly after learning, and tests after a delay. Finally, our results provide a first indication of which EMME characteristics may positively influence learning. Future research should start to more systematically examine the effects of specific EMME design choices for specific participant populations and task types.

Do we know our visual preferences?

Humans differ in the amount of time they direct their gaze toward different types of stimuli. Individuals' preferences are known to be reliable and can predict various cognitive and affective processes. However, it remains unclear whether humans are aware of their visual gaze preferences and are able to report it. In this study, across three different tasks and without prior warning, participants were asked to estimate the amount of time they had looked at a certain visual content (e.g., faces or texts) at the end of each experiment. The findings show that people can report accurately their visual gaze preferences. The implications are discussed in the context of visual perception, metacognition, and the development of applied diagnostic tools based on eye tracking.

Did You Get That? Predicting Learners' Comprehension of a Video Lecture from Visualizations of Their Gaze Data

In online lectures, unlike in face-to-face lectures, teachers lack access to (nonverbal) cues to check if their students are still "with them" and comprehend the lecture. The increasing availability of low-cost eye-trackers provides a promising solution. These devices measure unobtrusively where students look and can visualize these data to teachers. These visualizations might inform teachers about students' level of "with-me-ness" (i.e., do students look at the information that the teacher is currently talking about) and comprehension of the lecture, provided that (1) gaze measures of "with-me-ness" are related to comprehension, (2) people not trained in eye-tracking can predict students' comprehension from gaze visualizations, (3) we understand how different visualization techniques impact this prediction. We addressed these issues in two studies. In Study 1, 36 students watched a video lecture while being eye-tracked. The extent to which students looked at relevant information and the extent to which they looked at the same location as the teacher both correlated with students' comprehension (score on an open question) of the lecture. In Study 2, 50 participants watched visualizations of students' gaze (from Study 1), using six visualization techniques (dynamic and static versions of scanpaths, heatmaps, and focus maps) and were asked to predict students' posttest performance and to rate their ease of prediction. We found that people can use gaze visualizations to predict learners' comprehension above chance level, with minor differences between visualization techniques. Further research should investigate if teachers can act on the information provided by gaze visualizations and thereby improve students' learning.

Adaptive Gaze Behavior and Decision Making of Penalty Corner Strikers in Field Hockey

In recent years, studies have increasingly dealt with the interaction of gaze behavior and decision making of team sports athletes. However, there is still a variety of important game situations, for example, in the case of penalty corners in field hockey, in which this interaction has not been investigated in detail yet. Penalty corners present a meaningful goal scoring opportunity by providing a relatively free shot. This paper considers two studies. The first study investigated a possible connection between the gaze behavior and the quality of decisions of experienced field hockey players and evaluated the level of success of different gaze strategies. A preliminary study (Study 1) was designed as a survey questionnaire with the aim of preparing for the main study by obtaining subjective assessments of the individual gaze behavior and decision making of professional athletes. In the second and the main study (Study 2), the gaze behavior of experienced field hockey players was recorded using mobile eye-tracking systems to analyze different strategical approaches in associated gaze behavior and decision making. Study 1 showed that players consider reacting to the defenders' behavior during a penalty corner a promising avenue for improving success at penalty corner attempts. It also indicated that such defense-dependent strategies are currently only rarely employed. Study 2 demonstrated how gaze behavior differs between different strategical approaches of the offense. It was shown that the gaze direction on the ball, the stopper, and the goal area is important to allow for a more optimal adaptation to the tactical behavior of defense. It can be concluded that adaptive decision making (i.e., choosing which variation will be carried out just after the "injection" of the ball) seems promising but requires further training to improve the success rate of penalty corner.

Characteristics of expert search behavior in volumetric medical image interpretation

Purpose: Experienced radiologists have enhanced global processing ability relative to novices, allowing experts to rapidly detect medical abnormalities without performing an exhaustive search. However, evidence for global processing models is primarily limited to two-dimensional image interpretation, and it is unclear whether these findings generalize to volumetric images, which are widely used in clinical practice. We examined whether radiologists searching volumetric images use methods consistent with global processing models of expertise. In addition, we investigated whether search strategy (scanning/drilling) differs with experience level. Approach: Fifty radiologists with a wide range of experience evaluated chest computed-tomography scans for lung nodules while their eye movements and scrolling behaviors were tracked. Multiple linear regressions were used to determine: (1) how search behaviors differed with years of experience and the number of chest CTs evaluated per week and (2) which search behaviors predicted better performance. Results: Contrary to global processing models based on 2D images, experience was unrelated to measures of global processing (saccadic amplitude, coverage, time to first fixation, search time, and depth passes) in this task. Drilling behavior was associated with better accuracy than scanning behavior when controlling for observer experience. Greater image coverage was a strong predictor of task accuracy. Conclusions: Global processing ability may play a relatively small role in volumetric image interpretation, where global scene statistics are not available to radiologists in a single glance. Rather, in volumetric images, it may be more important to engage in search strategies that support a more thorough search of the image.

How Experts Adapt Their Gaze Behavior When Modeling a Task to Novices

Domain experts regularly teach novice students how to perform a task. This often requires them to adjust their behavior to the less knowledgeable audience and, hence, to behave in a more didactic manner. Eye movement modeling examples (EMMEs) are a contemporary educational tool for displaying experts’ (natural or didactic) problem‐solving behavior as well as their eye movements to learners. While research on expert‐novice communication mainly focused on experts’ changes in explicit, verbal communication behavior, it is as yet unclear whether and how exactly experts adjust their nonverbal behavior. This study first investigated whether and how experts change their eye movements and mouse clicks (that are displayed in EMMEs) when they perform a task naturally versus teach a task didactically. Programming experts and novices initially debugged short computer codes in a natural manner. We first characterized experts’ natural problem‐solving behavior by contrasting it with that of novices. Then, we explored the changes in experts’ behavior when being subsequently instructed to model their task solution didactically. Experts became more similar to novices on measures associated with experts’ automatized processes (i.e., shorter fixation durations, fewer transitions between code and output per click on the run button when behaving didactically). This adaptation might make it easier for novices to follow or imitate the expert behavior. In contrast, experts became less similar to novices for measures associated with more strategic behavior (i.e., code reading linearity, clicks on run button) when behaving didactically.

What do we know about volumetric medical image interpretation?: a review of the basic science and medical image perception literatures

Interpretation of volumetric medical images represents a rapidly growing proportion of the workload in radiology. However, relatively little is known about the strategies that best guide search behavior when looking for abnormalities in volumetric images. Although there is extensive literature on two-dimensional medical image perception, it is an open question whether the conclusions drawn from these images can be generalized to volumetric images. Importantly, volumetric images have distinct characteristics (e.g., scrolling through depth, smooth-pursuit eye-movements, motion onset cues, etc.) that should be considered in future research. In this manuscript, we will review the literature on medical image perception and discuss relevant findings from basic science that can be used to generate predictions about expertise in volumetric image interpretation. By better understanding search through volumetric images, we may be able to identify common sources of error, characterize the optimal strategies for searching through depth, or develop new training and assessment techniques for radiology residents.

Searching with and against each other: Spatiotemporal coordination of visual search behavior in collaborative and competitive settings

Although in real life people frequently perform visual search together, in lab experiments this social dimension is typically left out. Here, we investigate individual, collaborative and competitive visual search with visualization of search partners' gaze. Participants were instructed to search a grid of Gabor patches while being eye tracked. For collaboration and competition, searchers were shown in real time at which element the paired searcher was looking. To promote collaboration or competition, points were rewarded or deducted for correct or incorrect answers. Early in collaboration trials, searchers rarely fixated the same elements. Reaction times of couples were roughly halved compared with individual search, although error rates did not increase. This indicates searchers formed an efficient collaboration strategy. Overlap, the proportion of dwells that landed on hexagons that the other searcher had already looked at, was lower than expected from simulated overlap of two searchers who are blind to the behavior of their partner. The proportion of overlapping dwells correlated positively with ratings of the quality of collaboration. During competition, overlap increased earlier in time, indicating that competitors divided space less efficiently. Analysis of the entropy of the dwell locations and scan paths revealed that in the competition condition, a less fixed looking pattern was exhibited than in the collaborate and individual search conditions. We conclude that participants can efficiently search together when provided only with information about their partner's gaze position by dividing up the search space. Competing search exhibited more random gaze patterns, potentially reflecting increased interaction between searchers in this condition.

Even if I showed you where you looked, remembering where you just looked is hard

People know surprisingly little about their own visual behavior, which can be problematic when learning or executing complex visual tasks such as search of medical images. We investigated whether providing observers with online information about their eye position during search would help them recall their own fixations immediately afterwards. Seventeen observers searched for various objects in “Where's Waldo” images for 3 s. On two-thirds of trials, observers made target present/absent responses. On the other third (critical trials), they were asked to click twelve locations in the scene where they thought they had just fixated. On half of the trials, a gaze-contingent window showed observers their current eye position as a 7.5° diameter “spotlight.” The spotlight “illuminated” everything fixated, while the rest of the display was still visible but dimmer. Performance was quantified as the overlap of circles centered on the actual fixations and centered on the reported fixations. Replicating prior work, this overlap was quite low (26%), far from ceiling (66%) and quite close to chance performance (21%). Performance was only slightly better in the spotlight condition (28%, p = 0.03). Giving observers information about their fixation locations by dimming the periphery improved memory for those fixations modestly, at best.