Eye Direction, Not Movement Direction, Predicts Attention Shifts in Those with Autism Spectrum Disorders

Head and body cues guide eye movements and facilitate target search in real-world videos

Static gaze cues presented in central vision result in observer shifts of covert attention and eye movements, and benefits in perceptual performance in the detection of simple targets. Less is known about how dynamic gazer behaviors with head and body motion influence search eye movements and performance in perceptual tasks in real-world scenes. Participants searched for a target person (yes/no task, 50% presence), whereas watching videos of one to three gazers looking at a designated person (50% valid gaze cue, looking at the target). To assess the contributions of different body parts, we digitally erase parts of the gazers in the videos to create three different body parts/whole conditions for gazers: floating heads (only head movements), headless bodies (only lower body movements), and the baseline condition with intact head and body. We show that valid dynamic gaze cues guided participants' eye movements (up to 3 fixations) closer to the target, speeded the time to foveate the target, reduced fixations to the gazers, and improved target detection. The effect of gaze cues in guiding eye movements to the search target was the smallest when the gazer's head was removed from the videos. To assess the inherent information about gaze goal location for each body parts/whole condition, we collected perceptual judgments estimating gaze goals by a separate group of observers with unlimited time. Observers' perceptual judgments showed larger estimate errors when the gazer's head was removed. This suggests that the reduced eye movement guidance from lower body cueing is related to observers' difficulty extracting gaze information without the presence of the head. Together, the study extends previous work by evaluating the impact of dynamic gazer behaviors on search with videos of real-world cluttered scenes.

Attention to faces in images is associated with personality and psychopathology

Humans show a robust tendency to look at faces in images, but also differ consistently in the strength of this attentional preference. Previous research remained inconclusive as to how a stronger face preference may be indicative of an individual's personality or clinical characteristics. Here we investigated face preferences in 120 participants (primarily drawn from a student population) who freely viewed photos in an internet browser showing a person in the context of a visually rich environment while attention was assessed using a cursor-based technique. Participants differed consistently in the strength of their face preference across images. A stronger preference for faces was correlated positively with openness to experience, extraversion, agreeableness and empathizing and was correlated negatively with social anxiety, depression levels and alexithymia. Trait measures were linked through a strong common factor which was additionally correlated with face preference. We conclude that face preferences may be linked to personality traits and to psychopathology but that an attribution to a specific facet of psychopathology may not be warranted. Future research should investigate links between face preferences and personality features in more diverse samples and across differing social situations.

Does agreement mean accuracy? Evaluating glance annotation in naturalistic driving data

Naturalistic driving studies often make use of cameras to monitor driver behavior. To analyze the resulting video images, human annotation is often adopted. These annotations then serve as the ‘gold standard’ to train and evaluate automated computer vision algorithms, even though it is uncertain how accurate human annotation is. In this study, we provide a first evaluation of glance direction annotation by comparing instructed, actual glance direction of truck drivers with annotated direction. Findings indicate that while for some locations high annotation accuracy is achieved, for most locations accuracy is well below 50%. Higher accuracy can be obtained by clustering these locations, but this also leads to reduced detail of the annotation, suggesting that decisions to use clustering should take into account the purpose of the annotation. The data also show that high agreement between annotators does not guarantee high accuracy. We argue that the accuracy of annotation needs to be verified experimentally more often.

Heritability of reflexive social attention triggered by eye gaze and walking direction: common and unique genetic underpinnings

BACKGROUND

Social attention ability is crucial for human adaptive social behaviors and interpersonal communications, and the malfunction of which has been implicated in autism spectrum disorder (ASD), a highly genetic neurodevelopmental disorder marked by striking social deficits.

METHODS

Using a classical twin design, the current study investigated the genetic contribution to individual variation in social and non-social attention abilities, and further probed their potential genetic linkage. Moreover, individual autistic traits were further measured in an independent group of non-twin participants to examine the hypothetical link between the core social attention ability and ASD.

RESULTS

We found reliable genetic influences on the social attentional effects induced by two distinct cues (eye gaze and walking direction), with 91% of their covariance accounted for by common genetic effects. However, no evidence of heritability or shared genetic effects was observed for the attentional effect directed by a non-social cue (i.e. arrow direction) and its correlation with the social attention ability. Remarkably, one's autistic traits could well predict his/her heritable core social attention ability extracted from the conventional social attentional effect.

CONCLUSIONS

These findings together suggest that human social attention ability is supported by unique genetic mechanisms that can be shared across different social, but not non-social, processing. Moreover, they also encourage the identification of 'social attention genes' and highlight the critical role of the core human social attention ability in seeking the endophenotypes of social cognitive disorders including ASD.

Social anxiety modulates visual exploration in real life - but not in the laboratory

In clinical reports, individuals high on social anxiety are often described to avoid gaze at other people, whereas several experimental studies employing images of persons yielded conflicting results. Here, we show that gaze avoidance crucially depends on the possibility of social interactions. We examined gaze behaviour in individuals with varying degrees of social anxiety in real‐life and in a second group of participants using a closely matched laboratory condition. In the real‐life situation, individuals with a higher degree of social anxiety had a reduced bias to look at near persons compared to individuals with a lower degree of social anxiety, while gaze behaviour in the laboratory group was not modulated by social anxiety. This effect was specific to social attention since there was no corresponding effect regarding fixations on objects. The presence of anxiety effects in real‐life but not in the laboratory condition, where participants do not expect to be evaluated by gazed‐at conspecifics, points to critical deficits of current laboratory research paradigms in eliciting authentic social attentional mechanisms, possibly leading to spurious results.

Brief Report: Attentional Cueing to Images of Social Interactions is Automatic for Neurotypical Individuals But Not Those with ASC

Human actions induce attentional orienting toward the target of the action. We examined the influence of action cueing in social (man throwing toward a human) and non-social (man throwing toward a tree) contexts in observers with and without autism spectrum condition (ASC). Results suggested that a social interaction enhanced the cueing effect for neurotypical participants. Participants with ASC did not benefit from non-predictive cues and were slower in social contexts, although they benefitted from reliably predictive cues. Social orienting appears to be automatic in the context of an implied social interaction for neurotypical observers, but not those with ASC. Neurotypical participants’ behavior may be driven by automatic processing, while participants with ASC use an alternative, effortful strategy.

Neural Mechanisms Underlying Conscious and Unconscious Gaze-Triggered Attentional Orienting in Autism Spectrum Disorder

Impaired joint attention represents the core clinical feature of autism spectrum disorder (ASD). Behavioral studies have suggested that gaze-triggered attentional orienting is intact in response to supraliminally presented eyes but impaired in response to subliminally presented eyes in individuals with ASD. However, the neural mechanisms underlying conscious and unconscious gaze-triggered attentional orienting remain unclear. We investigated this issue in ASD and typically developing (TD) individuals using event-related functional magnetic resonance imaging. The participants viewed cue stimuli of averted or straight eye gaze direction presented either supraliminally or subliminally and then localized a target. Reaction times were shorter when eye-gaze cues were directionally valid compared with when they were neutral under the supraliminal condition in both groups; the same pattern was found in the TD group but not the ASD group under the subliminal condition. The temporo–parieto–frontal regions showed stronger activation in response to averted eyes than to straight eyes in both groups under the supraliminal condition. The left amygdala was more activated while viewing averted vs. straight eyes in the TD group than in the ASD group under the subliminal condition. These findings provide an explanation for the neural mechanisms underlying the impairment in unconscious but not conscious gaze-triggered attentional orienting in individuals with ASD and suggest possible neurological and behavioral interventions to facilitate their joint attention behaviors.

Atypical Gaze Cueing Pattern in a Complex Environment in Individuals with ASD

Clinically, social interaction, including gaze-triggered attention, has been reported to be impaired in autism spectrum disorder (ASD), but psychological studies have generally shown intact gaze-triggered attention in ASD. These studies typically examined gaze-triggered attention under simple environmental conditions. In real life, however, the environment is complex. Previous studies have shown that an enhanced cueing effect was found when using eye gaze compared with arrow cues in unpredictably complex conditions in typically developing (TD) individuals. However, in the current study, compared with TD individuals, the cueing effect failed to enhance when using eye gaze compared with arrow cues under complex conditions in individuals with ASD. This may reflect the atypical style of gaze-triggered attention when individuals with ASD adapt to environmental complexity.

Feature and motion-based gaze cuing is linked with reduced social competence

Gaze following is a fundamental ability that plays an important role in human social function. However, the link between these two processes remains elusive. On the one hand, typically developing persons show robust gaze following in laboratory cuing tasks. On the other hand, investigations with individuals with autism suggest that reduced social competence in this population may partly reflect an atypical access to social information through attending to perceptual changes that normally accompany gaze shifts, like luminance or motion transients. Here we investigated if gaze cuing in typically developing individuals was modulated by similar task-irrelevant perceptual changes. In Experiment 1, a social gaze cue was presented with or without a luminance change. In Experiment 2, a social gaze cue was presented together with a motion cue. Both experiments indicated reduced magnitudes of gaze cuing in persons with low social competence on trials containing an irrelevant perceptual change. This suggests that similarly to individuals with autism, typically developing persons with low social competence also utilize idiosyncratic perceptual changes in the environment to access social content, revealing strong links between basic gaze following abilities and a range of social competence within typical individuals.

Self make-up: the influence of self-referential processing on attention orienting

For humans, both eye gaze and arrows serve as powerful signals for orienting attention. Recent studies have shown important differences between gaze and arrows in attention orienting; however, the mechanisms underlying these differences are not known. One such mechanism may be self-referential processing. To investigate this possibility, we trained participants to associate two cues (a red and green arrow in Experiment 1A and two different faces in Experiment 1B) with distinct words (“self” and “other”). Then, we manipulated two types of sound (voice and tone) as targets to investigate whether the cueing effect to self- and other-referential cues differs in a manner similar to that reported for gaze and arrows. We found that self-, but not other-, referential cues induced an enhanced cueing effect to the voice target relative to the tone target regardless of the cue characteristic (i.e., biological or non-biological). Our results suggest that the difference between gaze and arrows in orienting attention can be explained, at least in part, by the self-referentiality of gaze. Furthermore, in Experiment 2, we found a reverse cueing pattern between gaze and arrow cues by manipulating subjects’ experiences, suggesting that differences in the self-referentiality of gaze and arrow cues are not inherent.

Attention orienting by eye gaze and arrows reveals flexibility to environmental changes

This study aimed to evaluate the difference in non-predictive cues between gaze and arrows in attention orienting. Attention orienting was investigated with gaze or arrows as separate cues in a simple condition (i.e., block design) in Experiment 1 and in an unpredictable condition (i.e., randomized design) in Experiment 2. Two kinds of sound (voice and tone) stimuli were used as targets. Results showed that gaze and arrow cues induced enhanced attention orienting to a voice versus tone target in the block condition. However, in the randomized condition, enhanced attention orienting to a voice versus tone target was found in gaze but not arrow cues. The congruency of the meaning between a social cue (i.e., gaze) and a social target (i.e., voice) was clear in the randomized but not blocked design, because social gaze and non-social arrow cues were implemented in the same block. Thus, attention orienting might be mediated by the associated relationship of cue-target in a randomized condition, as an enhanced orienting effect was found when the associated relationship of cue-target was strong (i.e., social cue and target). The present study suggests that the difference in attention orienting between gaze and arrows is apparent in a randomized design (the unpredictable condition), and people employ a flexibly strategy of orienting to better respond to environmental changes.

A meta-analysis of visual orienting in autism

BACKGROUND

Visual orienting is inconsistently reported to be impaired in autism.

METHODS

We conducted a meta-analysis on visual orienting in autism. We focused on studies that used a Posner-type task. A total of 18 research papers published between 1993 and 2011 were included in our meta-analysis. We examined the effects of differences in experimental design as well as differences in participant samples. We examined both orienting reaction times of participants with autism, and the effect size relative to comparison group in each experiment.

RESULTS

We found that participants with autism oriented across conditions (mean orienting effect = 40.73 ms), which was of an overall smaller magnitude than that of comparison groups (Cohen's d = 0.44). Participants with autism were most impaired on arrow cue tasks, and least impaired on eye-gaze cue tasks, more impaired with rapid trials, and the impairment increased with age.

CONCLUSIONS

Variations in experimental design and participant age group contribute to whether participants with autism appear impaired at visual orienting. Critical gaps exist in the literature; developmental studies are needed across and comparing broader age ranges, and more attention should be focused on basic endogenous orienting processes.

Atypical gaze following in autism: a comparison of three potential mechanisms

In order to evaluate the following potential mechanisms underlying atypical gaze following in autism, impaired reflexive gaze following, difficulty integrating gaze and affect, or reduced understanding of the referential significance of gaze, we administered three paradigms to young children with autism (N = 21) and chronological (N = 21) and nonverbal mental age (N = 21) matched controls. Children with autism exhibited impaired reflexive gaze following. The absence of evidence of integration of gaze and affect, regardless of diagnosis, indicates ineffective measurement of this construct. Reduced gaze following was apparent among children with autism during eye-tracking and in-person assessments. Word learning from gaze cues was better explained by developmental level than autism. Thus, gaze following may traverse an atypical, rather than just delayed, trajectory in autism.

Social orienting of children with autism to facial expressions and speech: a study with a wearable eye-tracker in naturalistic settings

This study investigates attention orienting to social stimuli in children with Autism Spectrum Conditions (ASC) during dyadic social interactions taking place in real-life settings. We study the effect of social cues that differ in complexity and distinguish between social cues produced by facial expressions of emotion and those produced during speech. We record the children's gazes using a head-mounted eye-tracking device and report on a detailed and quantitative analysis of the motion of the gaze in response to the social cues. The study encompasses a group of children with ASC from 2 to 11-years old (n = 14) and a group of typically developing (TD) children (n = 17) between 3 and 6-years old. While the two groups orient overtly to facial expressions, children with ASC do so to a lesser extent. Children with ASC differ importantly from TD children in the way they respond to speech cues, displaying little overt shifting of attention to speaking faces. When children with ASC orient to facial expressions, they show reaction times and first fixation lengths similar to those presented by TD children. However, children with ASC orient to speaking faces slower than TD children. These results support the hypothesis that individuals affected by ASC have difficulties processing complex social sounds and detecting intermodal correspondence between facial and vocal information. It also corroborates evidence that people with ASC show reduced overt attention toward social stimuli.

Orienting in response to gaze and the social use of gaze among children with autism spectrum disorder

Potential relations between gaze cueing, social use of gaze, and ability to follow line of sight were examined in children with autism and typically developing peers. Children with autism (mean age = 10 years) demonstrated intact gaze cueing. However, they preferred to follow arrows instead of eyes to infer mental state, and showed decreased accuracy in following line of sight when several visual distracters were present. Performance across tasks was not correlated for either group. Findings suggest that children with autism are less inclined to prioritize and select eyes, particularly in visually-rich environments. Gaze-following deficits may lie at the level of selective attention, rather than cueing-a possibility that can be explored with more complex and ecologically valid tasks.

Social and non-social cueing of visuospatial attention in autism and typical development

Three experiments explored attention to eye gaze, which is incompletely understood in typical development and is hypothesized to be disrupted in autism. Experiment 1 (n = 26 typical adults) involved covert orienting to box, arrow, and gaze cues at two probabilities and cue-target times to test whether reorienting for gaze is endogenous, exogenous, or unique; experiment 2 (total n = 80: male and female children and adults) studied age and sex effects on gaze cueing. Gaze cueing appears endogenous and may strengthen in typical development. Experiment 3 tested exogenous, endogenous, and gaze-based orienting in 25 typical and 27 Autistic Spectrum Disorder (ASD) children. ASD children made more saccades, slowing their reaction times; however, exogenous and endogenous orienting, including gaze cueing, appear intact in ASD.

Deficits in adults with autism spectrum disorders when processing multiple objects in dynamic scenes

People with autism spectrum disorders (ASD) process visual information in a manner that is distinct from typically developing individuals. They may be less sensitive to people's goals and, more generally, focus on visual details instead of the entire scene. To examine these differences, people with and without ASD were asked to detect changes in dynamic scenes with multiple elements. Participants viewed a brief video of a person or an inanimate object (the "figure") moving from one object to another; after a delay, they reported whether a second video was the same or different. Possible changes included the figure, the object the figure was moving from, or the object the figure was moving toward (the "goal"). We hypothesized that individuals with ASD would be less sensitive to changes in scenes with people, particularly elements that might be the person's goal. Alternately, people with ASD might attend to fewer elements regardless of whether the scene included a person. Our results indicate that, like controls, people with ASD noticed a change in the "goal" object at the end of a person's movement more often than the object at the start. However, the group with ASD did not undergo the developmental improvement that was evident typically when detecting changes in both the start and end objects. This atypical development led to deficits in adults with ASD that were not specific to scenes with people or to "goals." Improvements in visual processing that underlie mature representation of scenes may not occur in ASD, suggesting that late developing brain processes are affected.

Dynamic fearful gaze does not enhance attention orienting in individuals with Asperger's disorder

Although impaired joint attention is one of the core clinical features of pervasive developmental disorder including autistic disorder and Asperger's disorder, experimental studies failed to report its impairment. This discrepancy might be the result of differences between real-life and experimental situations. The present study examined joint attention in 11 individuals with Asperger's disorder and 11 age-matched controls under naturalistic conditions using a target detection paradigm with dynamic emotional gaze cues. Although both groups showed gaze-triggered attention orienting as assessed by the differences in reaction time for invalid minus valid cues, enhancement of joint attention by fearful (vs. neutral) gaze was observed in the control, but not in the Asperger group. This suggests that the integration of emotion and gaze direction that elicits strong joint attention is impaired in individuals with Asperger's disorder.

Vision in autism spectrum disorders

Autism spectrum disorders (ASDs) are developmental disorders which are thought primarily to affect social functioning. However, there is now a growing body of evidence that unusual sensory processing is at least a concomitant and possibly the cause of many of the behavioural signs and symptoms of ASD. A comprehensive and critical review of the phenomenological, empirical, neuroscientific and theoretical literature pertaining to visual processing in ASD is presented, along with a brief justification of a new theory which may help to explain some of the data, and link it with other current hypotheses about the genetic and neural aetiologies of this enigmatic condition.

Human social attention: A new look at past, present, and future investigations

The present review examines the neural-behavioral correlates of human social attention, with special regard to the neural mechanisms involved in processing gaze information and the functional impact of gaze direction on the spatial orienting of attention. Our review suggests that there is strong evidence that specific brain systems are preferentially biased toward processing gaze information, yet this specificity is not mirrored by the behavioral data as measured in highly controlled model attention tasks such as the Posner cueing paradigm. In less controlled tasks, however, such as when observers are left free to select what they want to attend, they focus on people and their eyes, consistent with one's intuition and with the neural evidence that eyes are special. We discuss a range of implications of these data, including that much is to be gained by examining brain and behavioral processes to social stimuli as they occur in complex real-world settings.