Saliency-based gaze prediction based on head direction

Differential correlates of fear and anxiety in salience perception: A behavioral and ERP study with adolescents

Anxiety disorders are the most common psychopathologies among adolescents. Their diagnostic criteria include both fear and anxiety symptomatology, although according to the literature, we can find evidence for some distinction between these two emotions. The present study contribute to this distinction, exploring the effects of trait fear and trait anxiety on behavioral and neural correlates. Thirty-two participants (aged 11-16 years) performed two experimental tasks of salient target detection, including visual stimuli that were manipulated to become salient, while reaction times and EEG were recorded. Results of both tasks revealed differential effects of trait fear and trait anxiety assessed through the Fear Survey Schedule for Children-Revised and the Youth Anxiety Measure for DSM-5 on reaction times and ERP components amplitudes. Specifically, higher symptoms from Separation Anxiety Disorder increased early neural visual processing and decreased reaction times for more salient stimuli. Also, trait fear reduced later neural visual processing of salient stimuli. These findings may provide a significant contribution to guiding psychological interventions, especially with adolescents presenting higher levels of anxiety-related symptomatology.

Gaze Point Tracking Based on a Robotic Body-Head-Eye Coordination Method

When the magnitude of a gaze is too large, human beings change the orientation of their head or body to assist their eyes in tracking targets because saccade alone is insufficient to keep a target at the center region of the retina. To make a robot gaze at targets rapidly and stably (as a human does), it is necessary to design a body-head-eye coordinated motion control strategy. A robot system equipped with eyes and a head is designed in this paper. Gaze point tracking problems are divided into two sub-problems: in situ gaze point tracking and approaching gaze point tracking. In the in situ gaze tracking state, the desired positions of the eye, head and body are calculated on the basis of minimizing resource consumption and maximizing stability. In the approaching gaze point tracking state, the robot is expected to approach the object at a zero angle. In the process of tracking, the three-dimensional (3D) coordinates of the object are obtained by the bionic eye and then converted to the head coordinate system and the mobile robot coordinate system. The desired positions of the head, eyes and body are obtained according to the object's 3D coordinates. Then, using sophisticated motor control methods, the head, eyes and body are controlled to the desired position. This method avoids the complex process of adjusting control parameters and does not require the design of complex control algorithms. Based on this strategy, in situ gaze point tracking and approaching gaze point tracking experiments are performed by the robot. The experimental results show that body-head-eye coordination gaze point tracking based on the 3D coordinates of an object is feasible. This paper provides a new method that differs from the traditional two-dimensional image-based method for robotic body-head-eye gaze point tracking.

Testing the effect of display organization in the collinear search impairment

Previous studies established that a salient collinear structure impairs local visual search. A display organization hypothesis states that the vertical grouping of elemental bars in the search display may selectively increase the salience of the local target in the background than that in the collinear distractor, leading to the collinear search impairment. Three displays were designed to test this hypothesis. A classical search display was adopted as a baseline. A diagonal search display was created with tilted bars, making perceptual organization diagonal and should reduce collinear search impairment. An illusory search display was designed by using abutting line illusion to emphasize the vertical grouping direction, which should increase collinear search impairment. A manipulation check was conducted with an online survey to understand the perceptual organization of the three displays. Results showed that the probability to perceive the stimuli grouping in the vertical direction was strongest in the illusory display and the least in the diagonal display. Nevertheless, the collinear search impairment did not vary with these manipulations, argue against the display organization hypothesis. We speculate that the search impairment might associate with the perceptual organization of the collinear distractor per se, rather than the perceptual organization of the background.

Vergence eye movements during figure-ground perception

Figure-ground, that is the segmentation of visual information into objects and their surrounding backgrounds, provides structure for visual attention. Recent evidence shows a novel role of vergence eye movements in visual attention. In the present work, vergence responses during figure-ground segregation tasks are psychophysically investigated. We show that during a figure-ground detection task, subjects convergence their eyes. Vergence eye movements are larger in figure trials than in ground trials. In detected figures trials, vergence are stronger than in trials where the figure went unnoticed. Moreover in figure trials, vergence responses are stronger to low-contrast figures than to high-contrast figures. We argue that these discriminative vergence responses have a role in figure-ground.

DGaze: CNN-Based Gaze Prediction in Dynamic Scenes

We conduct novel analyses of users' gaze behaviors in dynamic virtual scenes and, based on our analyses, we present a novel CNN-based model called DGaze for gaze prediction in HMD-based applications. We first collect 43 users' eye tracking data in 5 dynamic scenes under free-viewing conditions. Next, we perform statistical analysis of our data and observe that dynamic object positions, head rotation velocities, and salient regions are correlated with users' gaze positions. Based on our analysis, we present a CNN-based model (DGaze) that combines object position sequence, head velocity sequence, and saliency features to predict users' gaze positions. Our model can be applied to predict not only realtime gaze positions but also gaze positions in the near future and can achieve better performance than prior method. In terms of realtime prediction, DGaze achieves a 22.0% improvement over prior method in dynamic scenes and obtains an improvement of 9.5% in static scenes, based on using the angular distance as the evaluation metric. We also propose a variant of our model called DGaze_ET that can be used to predict future gaze positions with higher precision by combining accurate past gaze data gathered using an eye tracker. We further analyze our CNN architecture and verify the effectiveness of each component in our model. We apply DGaze to gaze-contingent rendering and a game, and also present the evaluation results from a user study.

Perceiving gaze from head and eye rotations: An integrative challenge for children and adults

Gaze is an emergent visual feature. A person's gaze direction is perceived not just based on the rotation of their eyes, but also their head. At least among adults, this integrative process appears to be flexible such that one feature can be weighted more heavily than the other depending on the circumstances. Yet it is unclear how this weighting might vary across individuals or across development. When children engage emergent gaze, do they prioritize cues from the head and eyes similarly to adults? Is the perception of gaze among individuals with autism spectrum disorder (ASD) emergent, or is it reliant on a single feature? Sixty adults (M = 29.86 years-of-age), thirty-seven typically developing children and adolescents (M = 9.3 years-of-age; range = 7-15), and eighteen children with ASD (M = 9.72 years-of-age; range = 7-15) viewed faces with leftward, rightward, or direct head rotations in conjunction with leftward or rightward pupil rotations, and then indicated whether the face was looking leftward or rightward. All individuals, across development and ASD status, used head rotation to infer gaze direction, albeit with some individual differences. However, the use of pupil rotation was heavily dependent on age. Finally, children with ASD used pupil rotation significantly less than typically developing (TD) children when inferring gaze direction, even after accounting for age. Our approach provides a novel framework for understanding individual and group differences in gaze as it is actually perceived-as an emergent feature. Furthermore, this study begins to address an important gap in ASD literature, taking the first look at emergent gaze perception in this population.

SGaze: A Data-Driven Eye-Head Coordination Model for Realtime Gaze Prediction

We present a novel, data-driven eye-head coordination model that can be used for realtime gaze prediction for immersive HMD-based applications without any external hardware or eye tracker. Our model (SGaze) is computed by generating a large dataset that corresponds to different users navigating in virtual worlds with different lighting conditions. We perform statistical analysis on the recorded data and observe a linear correlation between gaze positions and head rotation angular velocities. We also find that there exists a latency between eye movements and head movements. SGaze can work as a software-based realtime gaze predictor and we formulate a time related function between head movement and eye movement and use that for realtime gaze position prediction. We demonstrate the benefits of SGaze for gaze-contingent rendering and evaluate the results with a user study.

Saliency model based on a neural population for integrating figure direction and organizing Border Ownership

Attentional selection is a function of the brain that allocates computational resources momentarily to the most important part of a visual scene. Saliency map models have been used to predict the location of attentional selection and gaze. Border Ownership (BO) indicates the direction of the figure with respect to the border. I here propose a biologically plausible saliency model based on neural population for integrating the activities of intermediate-level visual areas with neurons selective to BO. A variety of BO organizations produces a population of model neurons that represent the grouping structure. In the model I propose, the interactions and the population responses of these model neurons underlie the determination of saliency and the accurate prediction of gaze location. I tested 100 patterns for BO organizations and found that the proposed saliency model not only reproduced the characteristics of perceptual organization but also captured object locations in natural images. Furthermore, the saliency model based on the population responses of the BO organization significantly improved the gaze prediction accuracy compared with previous saliency-based models. These results suggest a crucial role for a wide variety of BO organizations and neural population coding to determine saliency mediating attentional selection and to predict gaze location.

Saliency in VR: How Do People Explore Virtual Environments?

Understanding how people explore immersive virtual environments is crucial for many applications, such as designing virtual reality (VR) content, developing new compression algorithms, or learning computational models of saliency or visual attention. Whereas a body of recent work has focused on modeling saliency in desktop viewing conditions, VR is very different from these conditions in that viewing behavior is governed by stereoscopic vision and by the complex interaction of head orientation, gaze, and other kinematic constraints. To further our understanding of viewing behavior and saliency in VR, we capture and analyze gaze and head orientation data of 169 users exploring stereoscopic, static omni-directional panoramas, for a total of 1980 head and gaze trajectories for three different viewing conditions. We provide a thorough analysis of our data, which leads to several important insights, such as the existence of a particular fixation bias, which we then use to adapt existing saliency predictors to immersive VR conditions. In addition, we explore other applications of our data and analysis, including automatic alignment of VR video cuts, panorama thumbnails, panorama video synopsis, and saliency-basedcompression.

Real-time estimation of horizontal gaze angle by saccade integration using in-ear electrooculography

The manuscript proposes and evaluates a real-time algorithm for estimating eye gaze angle based solely on single-channel electrooculography (EOG), which can be obtained directly from the ear canal using conductive ear moulds. In contrast to conventional high-pass filtering, we used an algorithm that calculates absolute eye gaze angle via statistical analysis of detected saccades. The estimated eye positions of the new algorithm were still noisy. However, the performance in terms of Pearson product-moment correlation coefficients was significantly better than the conventional approach in some instances. The results suggest that in-ear EOG signals captured with conductive ear moulds could serve as a basis for light-weight and portable horizontal eye gaze angle estimation suitable for a broad range of applications. For instance, for hearing aids to steer the directivity of microphones in the direction of the user's eye gaze.

The whereabouts of visual attention: Involuntary attentional bias toward the default gaze direction

This study proposed and verified a new hypothesis on the relationship between gaze direction and visual attention: attentional bias by default gaze direction based on eye-head coordination. We conducted a target identification task in which visual stimuli appeared briefly to the left and right of a fixation cross. In Experiment 1, the direction of the participant's head (aligned with the body) was manipulated to the left, front, or right relative to a central fixation point. In Experiment 2, head direction was manipulated to the left, front, or right relative to the body direction. This manipulation was based on results showing that bias of eye position distribution was highly correlated with head direction. In both experiments, accuracy was greater when the target appeared at a position where the eyes would potentially be directed. Consequently, eye-head coordination influences visual attention. That is, attention can be automatically biased toward the location where the eyes tend to be directed.