Objects Capture Perceived Gaze Direction

Integration of Facial Expression and Gaze Direction in Individuals with a High Level of Autistic Traits

Background. We investigated whether individuals with high levels of autistic traits integrate relevant communicative signals, such as facial expression, when decoding eye-gaze direction. Methods. Students with high vs. low scores on the Autism Spectrum Quotient (AQ) performed a task in which they responded to the eye directions of faces, presented on the left or the right side of a screen, portraying different emotional expressions. Results. In both groups, the identification of gaze direction was faster when the eyes were directed towards the center of the scene. However, in the low AQ group, this effect was larger for happy faces than for neutral faces or faces showing other emotional expressions, whereas participants from high AQ group were not affected by emotional expressions. Conclusions. These results suggest that individuals with more autistic traits may not integrate multiple communicative signals based on their emotional value.

Hidden intentions: Visual awareness prioritizes perceived attention even without eyes or faces

Eye contact is a powerful social signal, and it readily captures attention. Recent work has suggested that direct gaze is prioritized even unconsciously: faces rendered invisible via interocular suppression enter awareness faster when they look directly at (vs. away from) you. Such effects may be driven in a relatively low level way by the special visual properties of eyes, per se, but here we asked whether they might instead arise from the perception of a deeper property: being the focus of another agent's attention and/or intentions. We report five experiments which collectively explore whether visual awareness also prioritizes distinctly non-eye-like stimuli that nevertheless convey directedness. We first showed that directed (vs. averted) 'mouth' shapes also break through into awareness faster, after being rendered invisible by continuous flash suppression - a direct 'gaze' effect without any eyes. But such effects could still be specific to faces (if not eyes), so we next asked whether the prioritization of directed intentions would still occur even for stimuli that have no faces at all. In fact, even simple geometric shapes can be seen as intentional, as when numerous randomly scattered cones are all consistently pointing at you. And indeed, even such directed (vs. averted) cones entered awareness faster - a direct 'gaze' effect without any facial cues. Additional control experiments ruled out effects of both symmetry and response biases. We conclude that the perception of directed intentions is sufficient to boost objects into awareness, and that putative eye-contact effects might instead reflect more general phenomena of 'mind contact'.

Distinguishing Social From Private Intentions Through the Passive Observation of Gaze Cues

Observing others’ gaze is most informative during social encounters between humans: We can learn about potentially salient objects in the shared environment, infer others’ mental states and detect their communicative intentions. We almost automatically follow the gaze of others in order to check the relevance of the target of the other’s attention. This phenomenon called gaze cueing can be conceptualized as a triadic interaction involving a gaze initiator, a gaze follower and a gaze target, i.e., an object or person of interest in the environment. Gaze cueing can occur as “gaze pointing” with a communicative or “social” intention by the initiator, telling the observer that she/he is meant to follow, or as an incidental event, in which the observer follows spontaneously without any intention of the observed person. Here, we investigate which gaze cues let an observer ascribe a social intention to the observed person’s gaze and whether and to which degree previous eye contact in combination with an object fixation contributes to this ascription. We varied the orientation of the starting position of gaze toward the observer and the orientation of the end position of a lateral gaze shift. In two experiments participants had to infer from the gaze behavior either mere approach (“the person looked at me”) vs. a social (“the person wanted to show me something”) or a social vs. a private motivation (“the person was interested in something”). Participants differentially attributed either approach behavior, a social, or a private intention to the agent solely based on the passive observation of the two specific gaze cues of start and end position. While for the attribution of privately motivated behavior, participants relied solely on the end position of the gaze shift, the social interpretation of the observed behavior depended additionally upon initial eye contact. Implications of these results for future social gaze and social cognition research in general are discussed.

A review of eye tracking for understanding and improving diagnostic interpretation

Inspecting digital imaging for primary diagnosis introduces perceptual and cognitive demands for physicians tasked with interpreting visual medical information and arriving at appropriate diagnoses and treatment decisions. The process of medical interpretation and diagnosis involves a complex interplay between visual perception and multiple cognitive processes, including memory retrieval, problem-solving, and decision-making. Eye-tracking technologies are becoming increasingly available in the consumer and research markets and provide novel opportunities to learn more about the interpretive process, including differences between novices and experts, how heuristics and biases shape visual perception and decision-making, and the mechanisms underlying misinterpretation and misdiagnosis. The present review provides an overview of eye-tracking technology, the perceptual and cognitive processes involved in medical interpretation, how eye tracking has been employed to understand medical interpretation and promote medical education and training, and some of the promises and challenges for future applications of this technology.

Eyes that bind us: Gaze leading induces an implicit sense of agency

Humans feel a sense of agency over the effects their motor system causes. This is the case for manual actions such as pushing buttons, kicking footballs, and all acts that affect the physical environment. We ask whether initiating joint attention - causing another person to follow our eye movement - can elicit an implicit sense of agency over this congruent gaze response. Eye movements themselves cannot directly affect the physical environment, but joint attention is an example of how eye movements can indirectly cause social outcomes. Here we show that leading the gaze of an on-screen face induces an underestimation of the temporal gap between action and consequence (Experiments 1 and 2). This underestimation effect, named 'temporal binding,' is thought to be a measure of an implicit sense of agency. Experiment 3 asked whether merely making an eye movement in a non-agentic, non-social context might also affect temporal estimation, and no reliable effects were detected, implying that inconsequential oculomotor acts do not reliably affect temporal estimations under these conditions. Together, these findings suggest that an implicit sense of agency is generated when initiating joint attention interactions. This is important for understanding how humans can efficiently detect and understand the social consequences of their actions.

Constructing Perceptual Common Ground Between Human and Robot Through Joint Attention

Joint attention is a communicative activity that allows social partners to share perceptual experiences by jointly attending to an environmental object. Unlike the common approach towards joint attention, which is based on the developmental view in robotics, here it is conceptualized with a psychophysical paradigm known as cueing. The triadic interaction of joint attention is formalized as the conditional probability of an attentional response for a given target candidate derived from object features and a cue derived from a human partner's indication. A robotic system to which the joint attention model is applied conducted a series of tasks to demonstrate the properties of the computational model. The robotic system successfully performed the tasks, which could not be specified by the information derived from a target object alone; furthermore, the system demonstrated how perceptual and selection ambiguity is resolved through joint attentive interaction and made to converge into a common perceptual state. The results imply that a perceptual common ground is constructed on the triadic relationship between user, robot, and objects through joint attentive interaction.

Gaze and Arrows: The Effect of Element Orientation on Apparent Motion is Modulated by Attention

In two experiments we investigated whether stimuli that elicit automatic orienting of attention (i.e., arrow or averted gaze) could drive apparent motion perception in one of two possible directions, modulating the effect of a low-level property (the orientation of elements along the motion direction). To this end, the competing motion paradigm was used, in which at time 1, a stimulus appears in the center of the display, and at time 2, two other stimuli appear in different spatial locations. Three kinds of stimuli with eight possible orientations were used in separate blocks; (1) a line; (2) an arrow; and, (3) an averted gaze. First, since the three stimuli present in the display at time 2 should be perceived to be located at the same distance (i.e., equidistant), the threshold for perceived equidistance was calculated for each participant and then used as the customized inter-stimulus distance. Participants were asked to press the button corresponding to the direction of the perceived motion. Results show a preference for collinear motion (motions between elements oriented along the motion direction), with a higher percentage of responses for gaze and arrow stimuli. In Experiment 1, a difference between gaze- and arrow-stimuli was observed. Apparent motion was seen towards the collinear position more often for gaze than for arrow when the stimulus was pointing to the vertical directions, while the opposite was true when the stimulus was pointing to the horizontal directions. In Experiment 2, where the lightness contrast between the gaze and the background was reduced, no difference between gaze- and arrow-stimuli emerged. We interpret our results as due to the social and biological value of gaze, which solved a possible ambiguity between gaze direction and the directions conveyed by the figural properties of the contrasted background in Experiment 1. These findings are consistent with the idea that stimuli known to automatically orient visual attention modulate motion perception.

Detecting and identifying offset gaze

A number of experiments have demonstrated that observers can accurately identify stimuli that they fail to detect (Rollman and Nachmias, 1972; Harris and Fahle, 1995; Allik et al. 1982, 2014). Using a 2x2AFC double judgements procedure, we demonstrated an analogous pattern of performance in making judgements about the direction of eye gaze. Participants were shown two faces in succession: one with direct gaze and one with gaze offset to the left or right. We found that they could identify the direction of gaze offset (left/right) better than they could detect which face contained the offset gaze. A simple Thurstonian model, under which the detection judgement is shown to be more computationally complex, was found to explain the empirical data. A further experiment incorporated metacognitive ratings into the double judgements procedure to measure observers' metacognitive awareness (Meta-d') across the two judgements and to assess whether observers were aware of the evidence for offset gaze when detection performance was at and below threshold. Results suggest that metacognitive awareness is tied to performance, with approximately equal Meta-d' across the two judgements, when sensitivity is taken into account. These results show that both performance and metacognitive awareness rely not only on the strength of sensory evidence but also on the computational complexity of the decision, which determines the relative distance of that evidence from the decision axes.

A bias-minimising measure of the influence of head orientation on perceived gaze direction

The orientation of the head is an important cue for gaze direction, and its role has been explained in a dual route model. The model incorporates both an attractive and a repulsive effect of head orientation, which act to support accurate gaze perception across large changes in natural stimuli. However, in all previous studies of which we are aware, measurements of the influence of head orientation on perceived gaze direction were obtained using a single-interval methodology, which may have been affected by response bias. Here we compare the single-interval methodology with a two-interval (bias-minimising) design. We find that although measures obtained using the two-interval design showed a stronger attractive effect of head orientation than previous studies, the influence of head orientation on perceived gaze direction still represents a genuine perceptual effect. Measurements obtained using the two-interval design were also shown to be more stable across sessions one week apart. These findings suggest the two-interval design should be used in future experiments, especially if comparing groups who may systematically differ in their biases, such as patients with schizophrenia or autism.

Social orienting in gaze leading: a mechanism for shared attention

Here, we report a novel social orienting response that occurs after viewing averted gaze. We show, in three experiments, that when a person looks from one location to an object, attention then shifts towards the face of an individual who has subsequently followed the person's gaze to that same object. That is, contrary to 'gaze following', attention instead orients in the opposite direction to observed gaze and towards the gazing face. The magnitude of attentional orienting towards a face that 'follows' the participant's gaze is also associated with self-reported autism-like traits. We propose that this gaze leading phenomenon implies the existence of a mechanism in the human social cognitive system for detecting when one's gaze has been followed, in order to establish 'shared attention' and maintain the ongoing interaction.

Prior expectations about where other people are likely to direct their attention systematically influence gaze perception

Different locations in the visual environment vary greatly in terms of how likely they are to draw a person's attention. When inferring the most likely target of another person's gaze, it would therefore be a reasonable strategy to incorporate expectations about the relative visual salience of these various locations, weighing this prior knowledge against incoming social signals (e.g., eye cues). This Bayesian approach to modeling gaze perception has informed computer vision techniques, but whether this model accounts well for human performance remains an untested hypothesis. We present subjects with a "gazer" fixating his eyes on various locations on a two-dimensional surface, and project arbitrary photographic images onto that surface. Subjects judge where the gazer is looking in each image. A full Bayesian model, which takes image salience information into account, fits subjects' gaze judgments better than a reduced model that only considers the perceived direction of the gazer's eyes. Varying the amount of time the subject is allowed to view the gazer reveals that center bias tends to dominate gaze judgments early, whereas salient features specific to the projected image influence judgments at longer viewing durations.

The importance of context information for the spatial specificity of gaze cueing

In three experiments, we investigated the spatial allocation of attention in response to central gaze cues. In particular, we examined whether the allocation of attentional resources is influenced by context information-that is, the presence or absence of reference objects (i.e., placeholders) in the periphery. On each trial, gaze cues were followed by a target stimulus to which participants had to respond by keypress or by performing a target-directed saccade. Targets were presented either in an empty visual field (Exps. 1 and 2) or in previewed location placeholders (Exp. 3) and appeared at one of either 18 (Exp. 1) or six (Exps. 2 and 3) possible positions. The spatial distribution of attention was determined by comparing response times as a function of the distance between the cued and target positions. Gaze cueing was not specific to the exact cued position, but instead generalized equally to all positions in the cued hemifield, when no context information was provided. However, gaze direction induced a facilitation effect specific to the exact gazed-at position when reference objects were presented. We concluded that the presence of possible objects in the periphery to which gaze cues could refer is a prerequisite for attention shifts being specific to the gazed-at position.

All eyes on me?! Social anxiety and self-directed perception of eye gaze

To date, only little is known about the self-directed perception and processing of subtle gaze cues in social anxiety that might however contribute to excessive feelings of being looked at by others. Using a web-based approach, participants (n=174) were asked whether or not briefly (300 ms) presented facial expressions modulated in gaze direction (0°, 2°, 4°, 6°, 8°) and valence (angry, fearful, happy, neutral) were directed at them. The results demonstrate a positive, linear relationship between self-reported social anxiety and stronger self-directed perception of others' gaze directions, particularly for negative (angry, fearful) and neutral expressions. Furthermore, faster responding was found for gaze more clearly directed at socially anxious individuals (0°, 2°, and 4°) suggesting a tendency to avoid direct gaze. In sum, the results illustrate an altered self-directed perception of subtle gaze cues. The possibly amplifying effects of social stress on biased self-directed perception of eye gaze are discussed.

Stress increases the feeling of being looked at

Eye gaze direction and facial expression are important social cues. Recent studies have shown that emotional expression affects interpretation of gaze direction in such a way that positive emotions are more favorably interpreted as making eye contact than negative or neutral expressions. Here we examine whether stress affects this positivity bias in gaze perception. Stress was induced in 25 healthy young adults by means of the cold pressure stress test (CPS), 24 participants serving as controls. Stimuli were created from three-dimensional face models of 8 actors expressing happy, fearful, angry and neutral emotions. From each of these 3D models we extracted 9 different views (0°, 2°, 4°, 6° and 8° to the left and to the right). This resulted in 288 stimuli, which were randomly presented for 700 ms. Using a forced choice paradigm participants judged whether or not each face was looking at them. The results show that the CPS group falsely interpreted faces with averted gaze direction as making eye contact more often than did controls, independent of the expressed emotion. These results suggest that a stress-induced raise in cortisol level increases the sense of being looked at.

The world smiles at me: self-referential positivity bias when interpreting direction of attention

Recent research suggests that eye-gaze direction modulates perceived emotional expression. Here we explore the extent to which emotion affects interpretation of attention direction. We captured three-dimensional face models of 8 actors expressing happy, fearful, angry and neutral emotions. From these 3D models 9 views were extracted (0°, 2°, 4°, 6°, 8° to the left and right). These stimuli were randomly presented for 150 ms. Using a forced-choice paradigm 28 participants judged for each face whether or not it was attending to them. Two conditions were tested: either the whole face was visible, or the eyes were covered. In both conditions happy faces elicited most "attending-to-me" answers. Thus, emotional expression has a more general effect than an influence on gaze direction: emotion affects interpretation of attention direction. We interpret these results as a self-referential positivity bias, suggesting a general preference to associate a happy face with the self.

Potential social interactions are important to social attention

Social attention, or how spatial attention is allocated to biologically relevant stimuli, has typically been studied using simplistic paradigms that do not provide any opportunity for social interaction. To study social attention in a complex setting that affords social interaction, we measured participants' looking behavior as they were sitting in a waiting room, either in the presence of a confederate posing as another research participant, or in the presence of a videotape of the same confederate. Thus, the potential for social interaction existed only when the confederate was physically present. Although participants frequently looked at the videotaped confederate, they seldom turned toward or looked at the live confederate. Ratings of participants' social skills correlated with head turns to the live, but not videotaped, confederate. Our results demonstrate the importance of studying social attention within a social context, and suggest that the mere opportunity for social interaction can alter social attention.

"You talkin' to me?" Self-relevant auditory signals influence perception of gaze direction

In humans, direct gaze typically signals a deliberate attempt to communicate with an observer. An auditory signal with similar signal value is calling someone's name. We investigated whether the presence of this personally relevant signal in the auditory modality would influence perception of another individual's gaze. Participants viewed neutral faces displaying different gaze deviations while hearing someone call their own name or the name of another person. Results were consistent with our predictions, as participants judged faces with a wider range of gaze deviations as looking directly at them when they simultaneously heard their own name. The influence of this personally relevant signal was present only at ambiguous gaze deviations; thus, an overall response bias to categorize gaze as direct when hearing one's own name cannot account for the results. This study provides the first evidence that communicative intent signaled via the auditory modality influences the perception of another individual's gaze.

Reflexive orienting to gaze is not luminance dependent

In the investigation of reflexive orienting to cues, two major theories have emerged: One proposes that transients created by the cue trigger attentional shifts, whereas the other argues that object changes are responsible for instigating orienting. In the present study, we examined whether luminance transients produced by the cue can generate reflexive orienting to gaze. Using a temporal order judgment paradigm under luminant or subjectively equiluminant conditions, participants judged which of two peripheral targets onset first. An uninformative gaze cue served to reflexively shift attention toward one object location, thereby temporally prioritizing the target presented there. The results revealed that attention was successfully shifted toward the cued object, as was evidenced by the participants' selecting the cued object as appearing first significantly more often than the uncued object, even when the two onset simultaneously. Critically, the results were comparable across luminance conditions. Our findings reveal that luminance transients are not necessary for triggering orienting to gaze cues. We suggest that the orienting observed here can be better explained via an object-based hypothesis whereby object changes, not transients, trigger reflexive orienting.

Anticipating intentional actions: the effect of eye gaze direction on the judgment of head rotation

Using a representational momentum paradigm, this study investigated the hypothesis that judgments of how far another agent's head has rotated are influenced by the perceived gaze direction of the head. Participants observed a video-clip of a face rotating 60 degrees towards them starting from the left or right profile view. The gaze direction of the face was either congruent with, ahead of, or lagging behind the angle of rotation. Following this, two static faces, at varying angles of rotation with respect to the end-point angle of the face in the video-clip, were presented simultaneously. The task of the participants was to decide which of the two heads was at an angle best resembling the angle of the end-point of the moving face. The critical test condition consisted of one test face oriented at 10 degrees before, and the other at 10 degrees after the end-point. The 'lagging behind' gaze condition elicited a significant underestimation of the rotation compared to the 'congruent' and 'ahead' gaze conditions. Participants did not exhibit similar biases when judging the rotation of several non-face control stimuli with visual features that mimicked different aspects of gaze direction. The findings suggest that when the gaze direction of a perceived agent is incongruent with the direction of the agent's head motion observers automatically utilise this discrepancy to adjust their inferences about the agent's intended heading direction.

Gaze cueing of attention: visual attention, social cognition, and individual differences

During social interactions, people's eyes convey a wealth of information about their direction of attention and their emotional and mental states. This review aims to provide a comprehensive overview of past and current research into the perception of gaze behavior and its effect on the observer. This encompasses the perception of gaze direction and its influence on perception of the other person, as well as gaze-following behavior such as joint attention, in infant, adult, and clinical populations. Particular focus is given to the gaze-cueing paradigm that has been used to investigate the mechanisms of joint attention. The contribution of this paradigm has been significant and will likely continue to advance knowledge across diverse fields within psychology and neuroscience.