Orienting of attention via observed eye gaze is head-centred

Age Differences in Gaze Following: Older Adults Follow Gaze More than Younger Adults When free-viewing Scenes

Previous research investigated age differences in gaze following with an attentional cueing paradigm where participants view a face with averted gaze, and then respond to a target appearing in a location congruent or incongruent with the gaze cue. However, this paradigm is far removed from the way we use gaze cues in everyday settings. Here we recorded the eye movements of younger and older adults while they freely viewed naturalistic scenes where a person looked at an object or location. Older adults were more likely to fixate and made more fixations to the gazed-at location, compared to younger adults. Our findings suggest that, contrary to what was observed in the traditional gaze-cueing paradigm, in a non-constrained task that uses contextualized stimuli older adults follow gaze as much as or even more than younger adults.

The influence of perceptual load on gaze-induced attentional orienting: The modulation of expectation

Humans tend to focus on others' gaze. Previous studies have shown that the gaze direction of others can induce corresponding attentional orienting. However, gaze cues have typically been presented alone in these studies. It is unclear how gaze cues induce observers' attention in complicated contexts with additional perceptual information. Therefore, the present study investigated gaze-induced attentional orienting at different levels of perceptual load. Results indicated that the attentional effect of the dynamic gaze cue (i.e., GCE: gaze cue effect) emerged under low perceptual load and disappeared under high perceptual load. The absence of GCE could not attribute to perceptual capacity exhaustion. Moreover, the influence of perceptual load on gaze-induced attentional orienting was modulated by individuals' expectation. Specifically, the GCE occurred under high perceptual load when the gaze cue was predictive (with individuals' expectation). These findings provide new evidence on the mode of gaze-induced attentional orienting under different perceptual load conditions.

Head and body cues guide eye movements and facilitate target search in real-world videos

Static gaze cues presented in central vision result in observer shifts of covert attention and eye movements, and benefits in perceptual performance in the detection of simple targets. Less is known about how dynamic gazer behaviors with head and body motion influence search eye movements and performance in perceptual tasks in real-world scenes. Participants searched for a target person (yes/no task, 50% presence), whereas watching videos of one to three gazers looking at a designated person (50% valid gaze cue, looking at the target). To assess the contributions of different body parts, we digitally erase parts of the gazers in the videos to create three different body parts/whole conditions for gazers: floating heads (only head movements), headless bodies (only lower body movements), and the baseline condition with intact head and body. We show that valid dynamic gaze cues guided participants' eye movements (up to 3 fixations) closer to the target, speeded the time to foveate the target, reduced fixations to the gazers, and improved target detection. The effect of gaze cues in guiding eye movements to the search target was the smallest when the gazer's head was removed from the videos. To assess the inherent information about gaze goal location for each body parts/whole condition, we collected perceptual judgments estimating gaze goals by a separate group of observers with unlimited time. Observers' perceptual judgments showed larger estimate errors when the gazer's head was removed. This suggests that the reduced eye movement guidance from lower body cueing is related to observers' difficulty extracting gaze information without the presence of the head. Together, the study extends previous work by evaluating the impact of dynamic gazer behaviors on search with videos of real-world cluttered scenes.

Evaluating attention deficit hyperactivity disorder symptoms in children and adolescents through tracked head movements in a virtual reality classroom: The effect of social cues with different sensory modalities

Background

Attention deficit hyperactivity disorder (ADHD) is clinically diagnosed; however, quantitative analysis to statistically analyze the symptom severity of children with ADHD via the measurement of head movement is still in progress. Studies focusing on the cues that may influence the attention of children with ADHD in classroom settings, where children spend a considerable amount of time, are relatively scarce. Virtual reality allows real-life simulation of classroom environments and thus provides an opportunity to test a range of theories in a naturalistic and controlled manner. The objective of this study was to investigate the correlation between participants’ head movements and their reports of inattention and hyperactivity, and to investigate how their head movements are affected by different social cues of different sensory modalities.

Methods

Thirty-seven children and adolescents with (n = 20) and without (n = 17) ADHD were recruited for this study. All participants were assessed for diagnoses, clinical symptoms, and self-reported symptoms. A virtual reality-continuous performance test (VR-CPT) was conducted under four conditions: (1) control, (2) no-cue, (3) visual cue, and (4) visual/audio cue. A quantitativecomparison of the participants’ head movements was conducted in three dimensions (pitch [head nods], yaw [head turns], and roll [lateral head inclinations]) using a head-mounted display (HMD) in a VR classroom environment. Task-irrelevant head movements were analyzed separately, considering the dimension of movement needed to perform the VR-CPT.

Results

The magnitude of head movement, especially task-irrelevant head movement, significantly correlated with the current standard of clinical assessment in the ADHD group. Regarding the four conditions, head movement showed changes according to the complexity of social cues in both the ADHD and healthy control (HC) groups.

Conclusion

Children and adolescents with ADHD showed decreasing task-irrelevant movements in the presence of social stimuli toward the intended orientation. As a proof-of-concept study, this study preliminarily identifies the potential of VR as a tool to understand and investigate the classroom behavior of children with ADHD in a controlled, systematic manner.

Gaze-cued shifts of attention and microsaccades are sustained for whole bodies but are transient for body parts

Gaze direction is an evolutionarily important mechanism in daily social interactions. It reflects a person’s internal cognitive state, spatial locus of interest, and predicts future actions. Studies have used static head images presented foveally and simple synthetic tasks to find that gaze orients attention and facilitates target detection at the cued location in a sustained manner. Little is known about how people’s natural gaze behavior, including eyes, head, and body movements, jointly orient covert attention, microsaccades, and facilitate performance in more ecological dynamic scenes. Participants completed a target person detection task with videos of real scenes. The videos showed people looking toward (valid cue) or away from a target (invalid cue) location. We digitally manipulated the individuals in the videos directing gaze to create three conditions: whole-intact (head and body movements), floating heads (only head movements), and headless bodies (only body movements). We assessed their impact on participants’ behavioral performance and microsaccades during the task. We show that, in isolation, an individual’s head or body orienting toward the target-person direction led to facilitation in detection that is transient in time (200 ms). In contrast, only the whole-intact condition led to sustained facilitation (500 ms). Furthermore, observers executed microsaccades more frequently towards the cued direction for valid trials, but this bias was sustained in time only with the joint presence of head and body parts. Together, the results differ from previous findings with foveally presented static heads. In more real-world scenarios and tasks, sustained attention requires the presence of the whole-intact body of the individuals dynamically directing their gaze.

Rapid detection of social interactions is the result of domain general attentional processes

Using visual search displays of interacting and non-interacting pairs, it has been demonstrated that detection of social interactions is facilitated. For example, two people facing each other are found faster than two people with their backs turned: an effect that may reflect social binding. However, recent work has shown the same effects with non-social arrow stimuli, where towards facing arrows are detected faster than away facing arrows. This latter work suggests a primary mechanism is an attention orienting process driven by basic low-level direction cues. However, evidence for lower level attentional processes does not preclude a potential additional role of higher-level social processes. Therefore, in this series of experiments we test this idea further by directly comparing basic visual features that orient attention with representations of socially interacting individuals. Results confirm the potency of orienting of attention via low-level visual features in the detection of interacting objects. In contrast, there is little evidence for the representation of social interactions influencing initial search performance.

Context Modulates Congruency Effects in Selective Attention to Social Cues

Head and gaze directions are used during social interactions as essential cues to infer where someone attends. When head and gaze are oriented toward opposite directions, we need to extract socially meaningful information despite stimulus conflict. Recently, a cognitive and neural mechanism for filtering-out conflicting stimuli has been identified while performing non-social attention tasks. This mechanism is engaged proactively when conflict is anticipated in a high proportion of trials and reactively when conflict occurs infrequently. Here, we investigated whether a similar mechanism is at play for limiting distraction from conflicting social cues during gaze or head direction discrimination tasks in contexts with different probabilities of conflict. Results showed that, for the gaze direction task only (Experiment 1), inverse efficiency (IE) scores for distractor-absent trials (i.e., faces with averted gaze and centrally oriented head) were larger (indicating worse performance) when these trials were intermixed with congruent/incongruent distractor-present trials (i.e., faces with averted gaze and tilted head in the same/opposite direction) relative to when the same distractor-absent trials were shown in isolation. Moreover, on distractor-present trials, IE scores for congruent (vs. incongruent) head-gaze pairs in blocks with rare conflict were larger than in blocks with frequent conflict, suggesting that adaptation to conflict was more efficient than adaptation to infrequent events. However, when the task required discrimination of head orientation while ignoring gaze direction, performance was not impacted by both block-level and current trial congruency (Experiment 2), unless the cognitive load of the task was increased by adding a concurrent task (Experiment 3). Overall, our study demonstrates that during attention to social cues proactive cognitive control mechanisms are modulated by the expectation of conflicting stimulus information at both the block- and trial-sequence level, and by the type of task and cognitive load. This helps to clarify the inherent differences in the distracting potential of head and gaze cues during speeded social attention tasks.

Investigating gaze processing in euthymic bipolar disorder: Impaired ability to infer mental state and intention, but preservation of social attentional orienting

Euthymic bipolar disorder (BD) has been associated with subtle impairment in face processing. However, it is not known whether their difficulties extend to the processing of gaze. In the present study, two tasks, both of which rely on the ability to make use of the eye region of a pictured face, were used: the Reading the Mind in the Eyes test and the Eye-gaze cueing task. Compared to healthy controls, BD patients were impaired at judging mental state from images of the face but showed normal susceptibility to the direction of gaze as an attentional cue. These findings suggest that BD patients present selective gaze processing impairment, limited to the sensitivity to intention and emotion. This impairment could account at least partially for the higher levels of interpersonal problems generally observed in BD.

Gaze-Cueing With Crossed Eyes: Asymmetry Between Nasal and Temporal Shifts

A person's direction of gaze (and visual attention) can be inferred from the direction of the parallel shift of the eyes. However, the direction of gaze is ambiguous when there is a misalignment between the eyes. The use of schematic drawings of faces in a previous study demonstrated that gaze-cueing was equally effective, even when one eye looked straight and the other eye was averted. In the current study, we used more realistic computer-generated face models to re-examine if the diverging direction of the eyes affected gaze-cueing. The condition where one eye was averted nasally while the other looked straight produced a significantly smaller gaze-cueing effect in comparison with when both eyes were averted in parallel or one eye was averted temporally. The difference in the gaze-cueing effect disappeared when the position of one eye was occluded with a rectangular surface or an eye-patch. These results highlight the possibility that the gaze-cueing effect might be weakened when a direct gaze exists between the cueing eye (i.e., nasally oriented eye) and the target and the effect magnitude might depend on which type of face stimulus are used as a cue.

Assessing early processing of eye gaze in schizophrenia: measuring the cone of direct gaze and reflexive orienting of attention

INTRODUCTION

The accurate discrimination of another person's eye-gaze direction is vital as it provides a cue to the gazer's focus of attention, which in turn supports joint attention. Patients with schizophrenia have shown a "direct gaze bias" when judging gaze direction. However, current tasks do not dissociate an early perceptual bias from high-level top-down effects. We investigated early stages of gaze processing in schizophrenia by measuring perceptual sensitivity to fine deviations in gaze direction (i.e., the cone of direct gaze: CoDG) and ability to reflexively orient to locations cued by the same deviations.

METHODS

Twenty-four patients and 26 controls completed a CoDG discrimination task that used realistic direct-face images with six fine degrees of deviation (i.e., 3, 6 or 9 pixels to the left and right) and direct gaze, and a gaze cueing task that assessed reflexive orienting to the same fine-grained deviations.

RESULTS

Our data showed patients exhibited no impairment in gaze discrimination, nor did we observe a reduced orienting response.

CONCLUSIONS

These results suggest that while patients may suffer deficits associated with interpreting another person's gaze, the earliest processes concerned with detecting averted gaze and reflexively orienting to the gazed-at location are intact.

Examining the durability of incidentally learned trust from gaze cues

In everyday interactions we find our attention follows the eye gaze of faces around us. As this cueing is so powerful and difficult to inhibit, gaze can therefore be used to facilitate or disrupt visual processing of the environment, and when we experience this we infer information about the trustworthiness of the cueing face. However, to date no studies have investigated how long these impressions last. To explore this we used a gaze-cueing paradigm where faces consistently demonstrated either valid or invalid cueing behaviours. Previous experiments show that valid faces are subsequently rated as more trustworthy than invalid faces. We replicate this effect (Experiment 1) and then include a brief interference task in Experiment 2 between gaze cueing and trustworthiness rating, which weakens but does not completely eliminate the effect. In Experiment 3, we explore whether greater familiarity with the faces improves the durability of trust learning and find that the effect is more resilient with familiar faces. Finally, in Experiment 4, we push this further and show that evidence of trust learning can be seen up to an hour after cueing has ended. Taken together, our results suggest that incidentally learned trust can be durable, especially for faces that deceive.

Automatic attentional orienting to other people's gaze in schizophrenia

Explicit tests of social cognition have revealed pervasive deficits in schizophrenia. Less is known of automatic social cognition in schizophrenia. We used a spatial orienting task to investigate automatic shifts of attention cued by another person's eye gaze in 29 patients and 28 controls. Central photographic images of a face with eyes shifted left or right, or looking straight ahead, preceded targets that appeared left or right of the cue. To examine automatic effects, cue direction was non-predictive of target location. Cue-target intervals were 100, 300, and 800 ms. In non-social control trials, arrows replaced eye-gaze cues. Both groups showed automatic attentional orienting indexed by faster reaction times (RTs) when arrows were congruent with target location across all cue-target intervals. Similar congruency effects were seen for eye-shift cues at 300 and 800 ms intervals, but patients showed significantly larger congruency effects at 800 ms, which were driven by delayed responses to incongruent target locations. At short 100-ms cue-target intervals, neither group showed faster RTs for congruent than for incongruent eye-shift cues, but patients were significantly slower to detect targets after direct-gaze cues. These findings conflict with previous studies using schematic line drawings of eye-shifts that have found automatic attentional orienting to be reduced in schizophrenia. Instead, our data indicate that patients display abnormalities in responding to gaze direction at various stages of gaze processing-reflected by a stronger preferential capture of attention by another person's direct eye contact at initial stages of gaze processing and difficulties disengaging from a gazed-at location once shared attention is established.

Space-based and object-centered gaze cuing of attention in right hemisphere-damaged patients

Gaze cuing of attention is a well established phenomenon consisting of the tendency to shift attention to the location signaled by the averted gaze of other individuals. Evidence suggests that such phenomenon might follow intrinsic object-centered features of the head containing the gaze cue. In the present exploratory study, we aimed to investigate whether such object-centered component is present in neuropsychological patients with a lesion involving the right hemisphere, which is known to play a critical role both in orienting of attention and in face processing. To this purpose, we used a modified gaze-cuing paradigm in which a centrally placed head with averted gaze was presented either in the standard upright position or rotated 90° clockwise or anti-clockwise. Afterward, a to-be-detected target was presented either in the right or in the left hemifield. The results showed that gaze cuing of attention was present only when the target appeared in the left visual hemifield and was not modulated by head orientation. This suggests that gaze cuing of attention in right hemisphere-damaged patients can operate within different frames of reference.

The shape of the face template: geometric distortions of faces and their detection in natural scenes

Human face detection might be driven by skin-coloured face-shaped templates. To explore this idea, this study compared the detection of faces for which the natural height-to-width ratios were preserved with distorted faces that were stretched vertically or horizontally. The impact of stretching on detection performance was not obvious when faces were equated to their unstretched counterparts in terms of their height or width dimension (Experiment 1). However, stretching impaired detection when the original and distorted faces were matched for their surface area (Experiment 2), and this was found with both vertically and horizontally stretched faces (Experiment 3). This effect was evident in accuracy, response times, and also observers' eye movements to faces. These findings demonstrate that height-to-width ratios are an important component of the cognitive template for face detection. The results also highlight important differences between face detection and face recognition.

The effects of short-term and long-term learning on the responses of lateral intraparietal neurons to visually presented objects

The lateral intraparietal area (LIP) is thought to play an important role in the guidance of where to look and pay attention. LIP can also respond selectively to differently shaped objects. We sought to understand to what extent short-term and long-term experience with visual orienting determines the responses of LIP to objects of different shapes. We taught monkeys to arbitrarily associate centrally presented objects of various shapes with orienting either toward or away from a preferred spatial location of a neuron. The training could last for less than a single day or for several months. We found that neural responses to objects are affected by such experience, but that the length of the learning period determines how this neural plasticity manifests. Short-term learning affects neural responses to objects, but these effects are only seen relatively late after visual onset; at this time, the responses to newly learned objects resemble those of familiar objects that share their meaning or arbitrary association. Long-term learning affects the earliest bottom-up responses to visual objects. These responses tend to be greater for objects that have been associated with looking toward, rather than away from, LIP neurons' preferred spatial locations. Responses to objects can nonetheless be distinct, although they have been similarly acted on in the past and will lead to the same orienting behavior in the future. Our results therefore indicate that a complete experience-driven override of LIP object responses may be difficult or impossible. We relate these results to behavioral work on visual attention.

Atypical integration of social cues for orienting to gaze direction in adults with autism

BACKGROUND

Gaze direction provides important information about social attention, and people tend to reflexively orient in the direction others are gazing. Perceiving the gaze of others relies on the integration of multiple social cues, which include perceptual information related to the eyes, gaze direction, head position, and body orientation of others. Autism spectrum conditions (ASC) are characterised by social and emotional deficits, including atypical gaze behaviour. The social-emotional deficits may emerge from a reliance on perceptual information involving details and features, at the expense of more holistic processing, which includes the integration of features. While people with ASC are often able to physically compute gaze direction and show intact reflexive orienting to others' gaze, they show deficits in reading mental states from the eyes.

METHODS

The present study recruited 23 adult males with a diagnosis of ASC and 23 adult males without ASC as a control group. They were tested using a spatial cuing paradigm involving head and body cues in a photograph of a person followed by a laterally presented target. The task manipulated the orientation of head with respect to body orientation to test subsequent shifts of attention in observers.

RESULTS

The results replicated previous findings showing facilitated shifts of attention by the healthy control participants toward laterally presented targets cued by a congruently rotated head combined with a front view of a body. In contrast, the ASC group showed facilitated orienting to targets when both the head and body were rotated towards the target.

CONCLUSIONS

The findings reveal atypical integration of social cues in ASC for orienting of attention. This is suggested to reflect abnormalities in cognitive and neural mechanisms specialized for processing of social cues for attention orienting in ASC.

From observation to action simulation: the role of attention, eye-gaze, emotion, and body state

This paper reviews recent aspects of my research. It focuses, first, on the idea that during the perception of objects and people, action-based representations are automatically activated and, second, that such action representations can feed back and influence the perception of people and objects. For example, when one is merely viewing an object such as a coffee cup, the action it affords, such as a reach to grasp, is activated even though there is no intention to act on the object. Similarly, when one is observing a person's behaviour, their actions are automatically simulated, and such action simulation can influence our perception of the person and the object with which they interacted. The experiments to be described investigate the role of attention in such vision-to-action processes, the effects of such processes on emotion, and the role of a perceiver's body state in their interpretation of visual stimuli.

Anticipating intentional actions: the effect of eye gaze direction on the judgment of head rotation

Using a representational momentum paradigm, this study investigated the hypothesis that judgments of how far another agent's head has rotated are influenced by the perceived gaze direction of the head. Participants observed a video-clip of a face rotating 60 degrees towards them starting from the left or right profile view. The gaze direction of the face was either congruent with, ahead of, or lagging behind the angle of rotation. Following this, two static faces, at varying angles of rotation with respect to the end-point angle of the face in the video-clip, were presented simultaneously. The task of the participants was to decide which of the two heads was at an angle best resembling the angle of the end-point of the moving face. The critical test condition consisted of one test face oriented at 10 degrees before, and the other at 10 degrees after the end-point. The 'lagging behind' gaze condition elicited a significant underestimation of the rotation compared to the 'congruent' and 'ahead' gaze conditions. Participants did not exhibit similar biases when judging the rotation of several non-face control stimuli with visual features that mimicked different aspects of gaze direction. The findings suggest that when the gaze direction of a perceived agent is incongruent with the direction of the agent's head motion observers automatically utilise this discrepancy to adjust their inferences about the agent's intended heading direction.

Effects of head orientation on gaze perception: how positive congruency effects can be reversed

Several past studies have considered how perceived head orientation may be combined with perceived gaze direction in judging where someone else is attending. In three experiments we tested the impact of different sources of information by examining the role of head orientation in gaze-direction judgements when presenting: (a) the whole face; (b) the face with the nose masked; (c) just the eye region, removing all other head-orientation cues apart from some visible part of the nose; or (d) just the eyes, with all parts of the nose masked and no head orientation cues present other than those within the eyes themselves. We also varied time pressure on gaze direction judgements. The results showed that gaze judgements were not solely driven by the eye region. Gaze perception can also be affected by parts of the head and face, but in a manner that depends on the time constraints for gaze direction judgements. While "positive" congruency effects were found with time pressure (i.e., faster left/right judgements of seen gaze when the seen head deviated towards the same side as that gaze), the opposite applied without time pressure.

Gaze cueing of attention: visual attention, social cognition, and individual differences

During social interactions, people's eyes convey a wealth of information about their direction of attention and their emotional and mental states. This review aims to provide a comprehensive overview of past and current research into the perception of gaze behavior and its effect on the observer. This encompasses the perception of gaze direction and its influence on perception of the other person, as well as gaze-following behavior such as joint attention, in infant, adult, and clinical populations. Particular focus is given to the gaze-cueing paradigm that has been used to investigate the mechanisms of joint attention. The contribution of this paradigm has been significant and will likely continue to advance knowledge across diverse fields within psychology and neuroscience.

Gaze cuing and affective judgments of objects: I like what you look at

When we see another person look somewhere, we automatically attend to the same location in space. This joint attention emerges early in life and has a great impact on social interactions in development and in everyday adult life. The direction of another's gaze indicates what object is of current interest, which may be the target for a subsequent action. In this study, we found that objects that are looked at by other people are liked more than objects that do not receive the attention of other people (Experiment 1). This suggests that observing averted gaze can have an impact on the affective appraisals of objects in the environment. This liking effect was absent when an arrow was used to cue attention (Experiment 2). This underlines the importance of other people's interactions with objects for generating our own impressions of such stimuli in the world.

Gaze cues evoke both spatial and object-centered shifts of attention

When someone observes another individual suddenly shifting gaze, the observer's attention automatically and rapidly orients to the same location. Such gaze cuing of attention has properties similar to those of exogenous cuing. We investigated whether gaze cuing is also like exogenous cuing in that it is observed for both spatial and object-/head-centered frames of reference. That is, when the face that produces the gaze cue is presented on its side, tilted 90 degrees from upright, will attention be simultaneously directed to where the eyes would have been looking if the face had been presented upright and toward the actual spatial direction of gaze? It is demonstrated that gaze cues do indeed orient attention in both spatial and object-centered frames, that these effects are of similar magnitude, and that such orienting is relatively rapidly computed.

Objects Capture Perceived Gaze Direction

The interpretation of another person's eye gaze is a key element of social cognition. Previous research has established that this ability develops early in life and is influenced by the person's head orientation, as well as local features of the person's eyes. Here we show that the presence of objects in the attended space also has an impact on gaze interpretation. Eleven normal adults identified the fixation points of photographed faces with a mouse cursor. Their responses were systematically biased toward the locations of nearby objects. This capture of perceived gaze direction probably reflects the attribution of intentionality and has methodological implications for research on gaze perception.

Effect of image orientation on the eye direction aftereffect

After observing a face with the eyes looking to the left or right (adaptation stimulus), the perception of the eye direction of the subsequent face (test stimulus) is biased in the opposite direction of the adapted eye direction; this is called the eye direction aftereffect (EDAE). In the present study, the adaptation stimuli were rotated 90 degrees (clockwise or counterclockwise) or 180 degrees relative to the viewer. The EDAE was measured using upright test stimuli. For the 90 degrees rotation, prior observation of the 'leftward' and 'rightward' eye directions biased the perceived eye directions of the upright test stimuli to the right and left, respectively. These results suggest that the adaptation was induced utilizing an object-based (or face-based) reference frame. For the 180 degrees rotation, however, the results suggest that the adaptation was induced in a viewer-centered reference frame. The involvement of an object-based reference frame suggests that the EDAE reflected the adaptation of a relatively higher-level mechanism at least when the rotation angle from the upright position did not exceed 90 degrees .