A hidden Markov model for analyzing eye-tracking of moving objects : Case study in a sustained attention paradigm

Modeling Eye Movements During Decision Making: A Review

This article reviews recent advances in the psychometric and econometric modeling of eye-movements during decision making. Eye movements offer a unique window on unobserved perceptual, cognitive, and evaluative processes of people who are engaged in decision making tasks. They provide new insights into these processes, which are not easily available otherwise, allow for explanations of fundamental search and choice phenomena, and enable predictions of future decisions. We propose a theoretical framework of the search and choice tasks that people commonly engage in and of the underlying cognitive processes involved in those tasks. We discuss how these processes drive specific eye-movement patterns. Our framework emphasizes the central role of task and strategy switching for complex goal attainment. We place the extant literature within that framework, highlight recent advances in modeling eye-movement behaviors during search and choice, discuss limitations, challenges, and open problems. An agenda for further psychometric modeling of eye movements during decision making concludes the review.

Sound Source Selection Based on Head Movements in Natural Group Conversation

To optimally improve signal-to-noise ratio in noisy environments, a hearing assistance device must correctly identify what is signal and what is noise. Many of the biosignal-based approaches to solving this question are themselves subject to noise, but head angle is an overt behavior that may be possible to capture in practical devices in the real world. Previous orientation studies have demonstrated that head angle is systematically related to listening target; our study aimed to examine whether this relationship is sufficiently reliable to be used in group conversations where participants may be seated in different layouts and the listener is free to turn their body as well as their head. In addition to this simple method, we developed a source-selection algorithm based on a hidden Markov model (HMM) trained on listeners’ head movement. The performance of this model and the simple head-steering method was evaluated using publicly available behavioral data. Head angle during group conversation was predictive of active talker, exhibiting an undershoot with a slope consistent with that found in simple orientation studies, but the intercept of the linear relationship was different for different talker layouts, suggesting it would be problematic to rely exclusively on this information to predict the location of auditory attention. Provided the location of all target talkers is known, the HMM source selection model implemented here, however, showed significantly lower error in identifying listeners’ auditory attention than the linear head-steering method.

Comparison of Visual Fixation Trajectories in Toddlers with Autism Spectrum Disorder and Typical Development: A Markov Chain Model

Autism spectrum disorder (ASD) is a neurodevelopmental condition in which visual attention and visual search strategies are altered. Eye-tracking paradigms have been used to detect these changes. In our study, 18 toddlers with ASD and 18 toddlers with typical development (TD; age range 12-36 months) underwent an eye-tracking paradigm where a face was shown together with a series of objects. Eye gaze was coded according to three areas of interest (AOIs) indicating where the toddlers' gaze was directed: 'Face', 'Object', and 'No-stimulus fixation'. The fixation sequence for the ASD and TD groups was modelled with a Markov chain model, obtaining transition probabilities between AOIs. Our results indicate that the transition between AOIs could differentiate between toddlers with ASD or TD, highlighting different visual exploration patterns between the groups. The sequence of exploration is strictly conditioned based on previous fixations, among which 'No-stimulus fixation' has a critical role in differentiating the two groups. Furthermore, our analyses underline difficulties of individuals with ASD to engage in stimulus exploration. These results could improve clinical and interventional practice by considering this dimension among the evaluation process.

A Data-Driven Framework for Intention Prediction via Eye Movement With Applications to Assistive Systems

Fast and accurate human intention prediction can significantly advance the performance of assistive devices for patients with limited motor or communication abilities. Among available modalities, eye movement can be valuable for inferring the user's intention, as it can be tracked non-invasively. However, existing limited studies in this domain do not provide the level of accuracy required for the reliable operation of assistive systems. By taking a data-driven approach, this paper presents a new framework that utilizes the spatial and temporal patterns of eye movement along with deep learning to predict the user's intention. In the proposed framework, the spatial patterns of gaze are identified by clustering the gaze points based on their density over displayed images in order to find the regions of interest (ROIs). The temporal patterns of gaze are identified via hidden Markov models (HMMs) to find the transition sequence between ROIs. Transfer learning is utilized to identify the objects of interest in the displayed images. Finally, models are developed to predict the user's intention after completing the task as well as at early stages of the task. The proposed framework is evaluated in an experiment involving predicting intended daily-life activities. Results indicate that an average classification accuracy of 97.42% is achieved, which is considerably higher than existing gaze-based intention prediction studies.

Applying Eye Movement Modeling Examples to Guide Novices' Attention in the Comprehension of Process Models

Process models are crucial artifacts in many domains, and hence, their proper comprehension is of importance. Process models mediate a plethora of aspects that are needed to be comprehended correctly. Novices especially face difficulties in the comprehension of process models, since the correct comprehension of such models requires process modeling expertise and visual observation capabilities to interpret these models correctly. Research from other domains demonstrated that the visual observation capabilities of experts can be conveyed to novices. In order to evaluate the latter in the context of process model comprehension, this paper presents the results from ongoing research, in which gaze data from experts are used as Eye Movement Modeling Examples (EMMEs) to convey visual observation capabilities to novices. Compared to prior results, the application of EMMEs improves process model comprehension significantly for novices. Novices achieved in some cases similar performances in process model comprehension to experts. The study’s insights highlight the positive effect of EMMEs on fostering the comprehension of process models.