Joint attention: Inferring what others perceive (and don’t perceive)

Mentalistic attention orienting triggered by android eyes

The eyes play a special role in human communications. Previous psychological studies have reported reflexive attention orienting in response to another individual's eyes during live interactions. Although robots are expected to collaborate with humans in various social situations, it remains unclear whether robot eyes have the potential to trigger attention orienting similarly to human eyes, specifically based on mental attribution. We investigated this issue in a series of experiments using a live gaze-cueing paradigm with an android. In Experiment 1, the non-predictive cue was the eyes and head of an android placed in front of human participants. Light-emitting diodes in the periphery served as target signals. The reaction times (RTs) required to localize the valid cued targets were faster than those for invalid cued targets for both types of cues. In Experiment 2, the gaze direction of the android eyes changed before the peripheral target lights appeared with or without barriers that made the targets non-visible, such that the android did not attend to them. The RTs were faster for validly cued targets only when there were no barriers. In Experiment 3, the targets were changed from lights to sounds, which the android could attend to even in the presence of barriers. The RTs to the target sounds were faster with valid cues, irrespective of the presence of barriers. These results suggest that android eyes may automatically induce attention orienting in humans based on mental state attribution.

Co-perceiving: Bringing the social into perception

Humans and other animals possess the remarkable ability to effectively navigate a shared perceptual environment by discerning which objects and spaces are perceived by others and which remain private to themselves. Traditionally, this capacity has been encapsulated under the umbrella of joint attention or joint action. In this comprehensive review, we advocate for a broader and more mechanistic understanding of this phenomenon, termed co-perception. Co-perception encompasses the sensitivity to the perceptual engagement of others and the capability to differentiate between objects perceived privately and those perceived commonly with others. It represents a distinct concept from mere simultaneous individual perception. Moreover, discerning between private and common objects doesn't necessitate intricate mind-reading abilities or mutual coordination. The act of perceiving objects as either private or common provides a comprehensive account for social scenarios where individuals simply share the same context or may even engage in competition. This conceptual framework encourages a re-examination of classical paradigms that demonstrate social influences on perception. Furthermore, it suggests that the impacts of shared experiences extend beyond affective responses, also influencing perceptual processes. This article is categorized under: Psychology > Attention Philosophy > Foundations of Cognitive Science Philosophy > Psychological Capacities.

Can the robot "see" what I see? Robot gaze drives attention depending on mental state attribution

Mentalizing, where humans infer the mental states of others, facilitates understanding and interaction in social situations. Humans also tend to adopt mentalizing strategies when interacting with robotic agents. There is an ongoing debate about how inferred mental states affect gaze following, a key component of joint attention. Although the gaze from a robot induces gaze following, the impact of mental state attribution on robotic gaze following remains unclear. To address this question, we asked forty-nine young adults to perform a gaze cueing task during which mental state attribution was manipulated as follows. Participants sat facing a robot that turned its head to the screen at its left or right. Their task was to respond to targets that appeared either at the screen the robot gazed at or at the other screen. At the baseline, the robot was positioned so that participants would perceive it as being able to see the screens. We expected faster response times to targets at the screen the robot gazed at than targets at the non-gazed screen (i.e., gaze cueing effect). In the experimental condition, the robot's line of sight was occluded by a physical barrier such that participants would perceive it as unable to see the screens. Our results revealed gaze cueing effects in both conditions although the effect was reduced in the occluded condition compared to the baseline. These results add to the expanding fields of social cognition and human-robot interaction by suggesting that mentalizing has an impact on robotic gaze following.

A Trained Humanoid Robot can Perform Human-Like Crossmodal Social Attention and Conflict Resolution

To enhance human-robot social interaction, it is essential for robots to process multiple social cues in a complex real-world environment. However, incongruency of input information across modalities is inevitable and could be challenging for robots to process. To tackle this challenge, our study adopted the neurorobotic paradigm of crossmodal conflict resolution to make a robot express human-like social attention. A behavioural experiment was conducted on 37 participants for the human study. We designed a round-table meeting scenario with three animated avatars to improve ecological validity. Each avatar wore a medical mask to obscure the facial cues of the nose, mouth, and jaw. The central avatar shifted its eye gaze while the peripheral avatars generated sound. Gaze direction and sound locations were either spatially congruent or incongruent. We observed that the central avatar's dynamic gaze could trigger crossmodal social attention responses. In particular, human performance was better under the congruent audio-visual condition than the incongruent condition. Our saliency prediction model was trained to detect social cues, predict audio-visual saliency, and attend selectively for the robot study. After mounting the trained model on the iCub, the robot was exposed to laboratory conditions similar to the human experiment. While the human performance was overall superior, our trained model demonstrated that it could replicate attention responses similar to humans.

Gaze cueing, mental States, and the effect of autistic traits

The ability to interpret and follow the gaze of our social partners is an integral skill in human communication. Recent research has demonstrated that gaze following behaviour is influenced by theory of mind (ToM) processes. However, it has yet to be determined whether the modulation of gaze cueing by ToM is affected by individual differences, such as autistic traits. The aim of this experiment was to establish whether autistic traits in neurotypical populations affect the mediation of gaze cueing by ToM processes. This study used a gaze cueing paradigm within a change detection task. Participants' perception of a gaze cue was manipulated such that they only believed the cue to be able to 'see' in one condition. The results revealed that participants in the Low Autistic Traits group were significantly influenced by the mental state of the gaze cue and were more accurate on valid trials when they believed the cue could 'see'. By contrast, participants in the High Autistic Traits group were also more accurate on valid trials, but this was not influenced by the mental state of the gaze cue. This study therefore provides evidence that autistic traits influence the extent to which mental state attributions modulate social attention in neurotypical adults.

Hidden intentions: Visual awareness prioritizes perceived attention even without eyes or faces

Eye contact is a powerful social signal, and it readily captures attention. Recent work has suggested that direct gaze is prioritized even unconsciously: faces rendered invisible via interocular suppression enter awareness faster when they look directly at (vs. away from) you. Such effects may be driven in a relatively low level way by the special visual properties of eyes, per se, but here we asked whether they might instead arise from the perception of a deeper property: being the focus of another agent's attention and/or intentions. We report five experiments which collectively explore whether visual awareness also prioritizes distinctly non-eye-like stimuli that nevertheless convey directedness. We first showed that directed (vs. averted) 'mouth' shapes also break through into awareness faster, after being rendered invisible by continuous flash suppression - a direct 'gaze' effect without any eyes. But such effects could still be specific to faces (if not eyes), so we next asked whether the prioritization of directed intentions would still occur even for stimuli that have no faces at all. In fact, even simple geometric shapes can be seen as intentional, as when numerous randomly scattered cones are all consistently pointing at you. And indeed, even such directed (vs. averted) cones entered awareness faster - a direct 'gaze' effect without any facial cues. Additional control experiments ruled out effects of both symmetry and response biases. We conclude that the perception of directed intentions is sufficient to boost objects into awareness, and that putative eye-contact effects might instead reflect more general phenomena of 'mind contact'.

Theory of mind affects the interpretation of another person's focus of attention

People are drawn to social, animate things more than inanimate objects. Previous research has also shown gaze following in humans, a process that has been linked to theory of mind (ToM). In three experiments, we investigated whether animacy and ToM are involved when making judgements about the location of a cursor in a scene. In Experiment 1, participants were told that this cursor represented the gaze of an observer and were asked to decide whether the observer was looking at a target object. This task is similar to that carried out by researchers manually coding eye-tracking data. The results showed that participants were biased to perceive the gaze cursor as directed towards animate objects (faces) compared to inanimate objects. In Experiments 2 and 3 we tested the role of ToM, by presenting the same scenes to new participants but now with the statement that the cursor was generated by a ‘random’ computer system or by a computer system designed to seek targets. The bias to report that the cursor was directed toward faces was abolished in Experiment 2, and minimised in Experiment 3. Together, the results indicate that people attach minds to the mere representation of an individual's gaze, and this attribution of mind influences what people believe an individual is looking at.

"Be careful what you do": How social threat influences social attention driven by reach-to-grasp movements

Social attention is the ability to share with social partners the attentional focus on an object and to orient attentional resources after observing others' behaviors. In the present work, we analyzed whether social threat may influence social attention during the observation of social agents' reach-to-grasp movements. In two experimental studies, we adopted a Posner-like paradigm: participants were required to classify neutral stimuli appearing in a congruent or incongruent position with the actor's movement. Membership (ingroup vs. outgroup) and contextual cues (neutral vs. threatening objects) were manipulated. The results showed a robust action-cueing effect: responses to stimuli in a congruent position with the actor's action were faster than responses to incongruent stimuli. Interestingly, we found a stronger effect for the threatening outgroup (i.e., Iraqi) than for the ingroup (i.e., Italian), especially in presence of menacing objects. No effects were found when observing the non-threatening outgroup (i.e., Japanese).

Visual perspective-taking and image-like representations: We don't see it

The ability to represent another agent's visual perspective has recently been attributed to a process called "perceptual simulation", whereby we generate an image-like or "quasi-perceptual" representation of another agent's vision. In an extensive series of experiments we tested this notion. Adult observers were presented with pictures of an agent looking at two horizontal lines, one of which was closer to the agent and hence appeared longer from his/her visual perspective. In each case approximately as many participants judged the closer line to appear shorter as longer (to the agent), i.e., failures to take the agent's perspective. This occurred when clear depth cues were added to emphasise the agent's location relative to the stimuli, when the agent was moved closer to the lines, when the lines were oriented vertically, when judgments could be made while viewing the image, and when participants imagined themselves in the agent's place. It also persisted when we asked participants to imagine what a photo taken from the same location as the agent would show, ruling out a misinterpretation of the instructions. Overall, our data suggest that adults attempt to solve visual perspective-taking problems by drawing upon naïve and often erroneous ideas about how vision works.

Attention in naïve psychology

In everyday life, mentalizing is nested in a rich context of cognitive faculties and background information that potentially contribute to its success. Yet, we know little about these modulating effects. Here we propose that humans develop a naïve psychological model of attention (featured as a goal-dependent, intentional relation to the environment) and use this to fine-tune their mentalizing attempts, presuming that the way people represent their environment is influenced by the cognitive priorities (attention) their current intentions create. The attention model provides an opportunity to tailor mental state inferences to the temporary features of the agent whose mind is in the focus of mentalizing. The ability to trace attention is an exceptionally powerful aid for mindreading. Knowledge about the partner's attention provides background information, however being grounded in his current intentions, attention has direct relevance to the ongoing interaction. Furthermore, due to its causal connection to intentions, the output of the attention model remains valid for a prolonged but predictable amount of time: till the evoking intention is in place. The naïve attention model theory is offered as a novel theory on social attention that both incorporates existing evidence and identifies new directions in research.

Cross-modal social attention triggered by biological motion cues

Previous research has demonstrated that biological motion (BM) cues can induce reflexive attentional orienting. This BM-triggered social attention has hitherto only been investigated within visual modality. It remains unknown whether and to what extent social attention induced by BM cues can occur across different sensory modalities. By introducing auditory stimuli to a modified central cueing paradigm, we showed that observers responded significantly faster to auditory targets presented in the walking direction of BM than in the opposite direction, reflecting the notion that BM cues can trigger cross-modal social attention. This effect was not due to the viewpoint effect of the global configuration and could be extended to local BM cues without any global configuration. Critically, such cross-modal social attention was sensitive to the orientation of BM cues and completely disappeared when critical biological characteristics were removed. Our findings, taken together, support the existence of a special multimodal attention mechanism tuned to life motion signals and shed new light on the unique and cross-modal nature of social attention.

Altercentric Cognition: How Others Influence Our Cognitive Processing

Humans are ultrasocial, yet, theories of cognition have often been occupied with the solitary mind. Over the past decade, an increasing volume of work has revealed how individual cognition is influenced by the presence of others. Not only do we rapidly identify others in our environment, but we also align our attention with their attention, which influences what we perceive, represent, and remember, even when our immediate goals do not involve coordination. Here, we refer to the human sensitivity to others and to the targets and content of their attention as 'altercentrism'; and aim to bring seemingly disparate findings together, suggesting that they are all reflections of the altercentric nature of human cognition.

Perspective-Taking: In Search of a Theory

Perspective-taking has been one of the central concerns of work on social attention and developmental psychology for the past 60 years. Despite its prominence, there is no formal description of what it means to represent another’s viewpoint. The present article argues that such a description is now required in the form of theory—a theory that should address a number of issues that are central to the notion of assuming another’s viewpoint. After suggesting that the mental imagery debate provides a good framework for understanding some of the issues and problems surrounding perspective-taking, we set out nine points that we believe any theory of perspective-taking should consider.

Visual Attentional Bias Induced by Face Direction

The effect of spatial cueing on eye gaze has been confirmed by a large number of studies, but the effect of spatial cueing on face direction and the impact of eye gaze on this effect are less known. In four experiments, we investigated the attentional bias induced by face direction. A modified paradigm of spatial cueing was adopted with stimuli that were static faces rotated by 90 or 45° to the left or right from the frontal view. To control the effect of eyes, face stimuli with eyes open and those with eyes closed were both used in each experiment. In Experiment 1, the facial cue (face rotated by 90°) and target were presented simultaneously, and the stimulus onset asynchrony (SOA) between the facial cue and target was set to be 300, 600, and 900 ms in Experiments 2 (face rotated by 90°), 3 (inverted face rotated by 90°), and 4 (face rotated by 45°), respectively. The response time of detecting the target position was recorded. The spatial cueing effects were nonsignificant in Experiment 1, in which the cue and target were presented simultaneously. However, significant spatial cueing effects of face direction were found in Experiments 2 and 3, in which the upright and inverted faces rotated by 90° were adopted, respectively, in both the eyes open and eyes closed conditions. In addition, we did not find an effect of spatial cueing with the face rotated by 45° (Experiment 4). Our results indicate that face direction can bias visual attention. This effect might not be based on the holistic processing of faces.

Mechanisms for the Cognitive Processing of Attractiveness in Adult and Infant Faces: From the Evolutionary Perspective

Research on the cognitive processing of facial attractiveness has mainly focused on adult faces. Recent studies have revealed that the cognitive processing of facial attractiveness in infant faces is not the same as that in adult faces. Therefore, it is necessary to summarize the evidence on the processing of facial attractiveness in each kind of face and compare their underlying mechanisms. In this paper, we first reviewed studies on the cognitive processing of facial attractiveness in adult faces, including attentional and mnemonic processing, and then discussed the underlying mechanisms. Afterward, studies on facial attractiveness in infant faces were reviewed, and the underlying mechanisms were also discussed. Direct comparisons between the two kinds of cognitive processing were subsequently made. The results showed that the mechanisms for the processing of attractiveness in adult faces and infant faces are mainly motivated by the perspectives of mate selection and raising offspring, respectively, in evolutionary psychology. Finally, directions for future research are proposed.

The closing of the theory of mind: A critique of perspective-taking

Theory of mind (ToM) is defined as the ability to attribute mental states to oneself and others and is often said to be one of the cornerstones of efficient social interaction. In recent years, a number of authors have suggested that one particular ToM process occurs spontaneously in that it is rapid and outside of conscious control. This work has argued that humans efficiently compute the visual perspective of other individuals. In this article, we present a critique of this notion both on empirical and theoretical grounds. We argue that the experiments and paradigms that purportedly demonstrate spontaneous perspective-taking have not as yet convincingly demonstrated the existence of such a phenomenon. We also suggest that it is not possible to represent the percept of another person, spontaneous or otherwise. Indeed, the perspective-taking field has suggested that humans can represent the visual experience of others. That is, going beyond assuming that we can represent another's viewpoint in anything other than symbolic form. In this sense, the field suffers from the same problem that afflicted the "pictorial" theory in the mental imagery debate. In the last section we present a number of experiments designed to provide a more thorough assessment of whether humans can indeed represent the visual experience of others.

Heritability of reflexive social attention triggered by eye gaze and walking direction: common and unique genetic underpinnings

BACKGROUND

Social attention ability is crucial for human adaptive social behaviors and interpersonal communications, and the malfunction of which has been implicated in autism spectrum disorder (ASD), a highly genetic neurodevelopmental disorder marked by striking social deficits.

METHODS

Using a classical twin design, the current study investigated the genetic contribution to individual variation in social and non-social attention abilities, and further probed their potential genetic linkage. Moreover, individual autistic traits were further measured in an independent group of non-twin participants to examine the hypothetical link between the core social attention ability and ASD.

RESULTS

We found reliable genetic influences on the social attentional effects induced by two distinct cues (eye gaze and walking direction), with 91% of their covariance accounted for by common genetic effects. However, no evidence of heritability or shared genetic effects was observed for the attentional effect directed by a non-social cue (i.e. arrow direction) and its correlation with the social attention ability. Remarkably, one's autistic traits could well predict his/her heritable core social attention ability extracted from the conventional social attentional effect.

CONCLUSIONS

These findings together suggest that human social attention ability is supported by unique genetic mechanisms that can be shared across different social, but not non-social, processing. Moreover, they also encourage the identification of 'social attention genes' and highlight the critical role of the core human social attention ability in seeking the endophenotypes of social cognitive disorders including ASD.

Watching a video together creates social closeness between children and adults

Human social relationships are often formed through shared social activities in which individuals share mental states about external stimuli. Previous work on joint attention has shown that even minimal shared experiences such as watching something together facilitates social closeness between individuals. Here, we examined whether young children already connect with others through joint attention. In the current studies, children sat next to a novel adult who either watched a film with them or was not able to see the film and read a book instead. After the video, we measured children's willingness (i.e., latency) to approach the experimenter holding out a toy. In both studies, the 2.5-year-olds who watched the film together approached more quickly than the other children. These results show that both minimally interactive shared experiences and noninteractive shared experiences lead children to feel more comfortable with a novel adult. This suggests that joint attention interactions, and shared experiences in general, play an important role not only in children's cognitive development but also in their social development and the formation of their social relationships.

The Role of Perspective Taking on Attention: A Review of the Special Issue on the Reflexive Attentional Shift Phenomenon

Attention is a process that alters how cognitive resources are allocated, and it allows individuals to efficiently process information at the attended location. The presence of visual or auditory cues in the environment can direct the focus of attention toward certain stimuli even if the cued stimuli are not the individual’s primary target. Samson et al. demonstrated that seeing another person in the scene (i.e., a person-like cue) caused a delay in responding to target stimuli not visible to that person: “alter-centric intrusion.” This phenomenon, they argue, is dependent upon the fact that the cue used resembled a person as opposed to a more generic directional indicator. The characteristics of the cue are the core of the debate of this special issue. Some maintain that the perceptual-directional characteristics of the cue are sufficient to generate the bias while others argue that the cuing is stronger when the cue has social characteristics (relates to what another individual can perceive). The research contained in this issue confirms that human attention is biased by the presence of a directional cue. We discuss and compare the different studies. The pattern that emerges seems to suggest that the social relevance of the cue is necessary in some contexts but not in others, depending on the cognitive demand of the experimental task. One possibility is that the social mechanisms are involved in perspective taking when the task is cognitively demanding, while they may not play a role in automatic attention allocation.

Attribution of vision and knowledge in 'spontaneous perspective taking'

Theory of mind is a ubiquitous notion that permeates many aspects of social cognition. A recent application has been in the context of 'spontaneous perspective taking' in which responses to target stimuli are facilitated if a human agent, present in a display, sees the same stimuli as an experimental participant. In the present work, we replicated results from a paradigm purporting to show such perspective taking in which participants tend to judge an ambiguous number from the position of an agent. We find, however (in Experiment 1) that this effect still occurs even when the agent cannot see the number due to an occluding object. This, therefore, does not support the perspective-taking hypothesis. An alternative explanation to the theory of mind account is posited in which the agent acts as a reference point that cues the observer to view the critical stimuli from that position and in the direction to which the agent faces. We test this hypothesis in Experiments 2 and 3, and show that non-human reference points can generate perspective-taking-like data. Overall, these results do not support the theory of mind account of previous studies.

Mental attribution is not sufficient or necessary to trigger attentional orienting to gaze

Attention can be shifted in the direction that another person is looking, but the role played by an observer's mental attribution to the looker is controversial. And whether mental attribution to the looker is sufficient to trigger an attention shift is unknown. The current study introduces a novel paradigm to investigate this latter issue. An actor is presented on video turning his head to the left or right before a target appears, randomly, at the gazed-at or non-gazed at location. Time to detect the target is measured. The standard finding is that target detection is more efficient at the gazed-at than the nongazed-at location, indicating that attention is shifted to the gazed-at location. Critically, in the current study, an actor is wearing two identical masks - one covering his face and the other the back of his head. Thus, after the head turn, participants are presented with the profile of two faces, one looking left and one looking right. For a gaze cuing effect to emerge, participants must attribute a mental state to the actor - as looking through one mask and not the other. Over the course of four experiments we report that when mental attribution is necessary, a shift in social attention does not occur (i.e., mental attribution is not sufficient to produce a social attention effect); and when mental attribution is not necessary, a shift in social attention does occur. Thus, mental attribution is neither sufficient nor necessary for the occurrence of an involuntary shift in social attention. The present findings constrain future models of social attention that wish to link gaze cuing to mental attribution.

The Role of Eye Gaze During Natural Social Interactions in Typical and Autistic People

Social interactions involve complex exchanges of a variety of social signals, such as gaze, facial expressions, speech and gestures. Focusing on the dual function of eye gaze, this review explores how the presence of an audience, communicative purpose and temporal dynamics of gaze allow interacting partners to achieve successful communication. First, we focus on how being watched modulates social cognition and behavior. We then show that the study of interpersonal gaze processing, particularly gaze temporal dynamics, can provide valuable understanding of social behavior in real interactions. We propose that the Interpersonal Gaze Processing model, which combines both sensing and signaling functions of eye gaze, provides a framework to make sense of gaze patterns in live interactions. Finally, we discuss how autistic individuals process the belief in being watched and interpersonal dynamics of gaze, and suggest that systematic manipulation of factors modulating gaze signaling can reveal which aspects of social eye gaze are challenging in autism.

Do Beliefs About Whether Others Can See Modulate Social Seeking in Autism?

Autistic people process gaze differently than typical people, but it is not yet clear if these differences lie in the processing of eye-shapes or the belief in whether others can see (perceptual mentalizing). We aimed to investigate whether these two models of gaze processing modulate social seeking in typical and autistic adults. We measured preferences of participants to view videos of an actress with visible or hidden eyes, who can or cannot see out. While typical participants preferred videos where the actress can see through and has visible eyes, autistic people showed no preference for these videos. These findings are discussed in the context of perceptual mentalizing and the social motivation theory of autism.

Reasoning About Visibility in Mirrors: A Comparison Between a Human Observer and a Camera

Human observers make errors when predicting what is visible in a mirror. This is true for perception with real mirrors as well as for reasoning about mirrors shown in diagrams. We created an illustration of a room, a top-down view, with a mirror on a wall and objects (nails) on the opposite wall. The task was to select which nails were visible in the mirror from a given position (viewpoint). To study the importance of the social nature of the viewpoint, we divided the sample ( N = 108) in two groups. One group ( n = 54) were tested with a scene in which there was the image of a person. The other group ( n = 54) were tested with the same scene but with a camera replacing the person. Participants were instructed to think about what would be captured by a camera on a tripod. This manipulation tests the effect of social perspective-taking in reasoning about mirrors. As predicted, performance on the task shows an overestimation of what can be seen in a mirror and a bias to underestimate the role of the different viewpoints, that is, a tendency to treat the mirror as if it captures information independently of viewpoint. In terms of the comparison between person and camera, there were more errors for the camera, suggesting an advantage for evaluating a human viewpoint as opposed to an artificial viewpoint. We suggest that social mechanisms may be involved in perspective-taking in reasoning rather than in automatic attention allocation.

Mental State Attributions Mediate the Gaze Cueing Effect

Understanding the mental states of our social partners allows us to successfully interact with the world around us. Mental state attributions are argued to underpin social attention, and have been shown to modulate attentional orienting to social cues. However, recent research has disputed this claim, arguing that this effect may arise as an unintentional side effect of study design, rather than through the involvement of mentalising processes. This study therefore aimed to establish whether the mediation of gaze cueing by mental state attributions generalises beyond the specific experimental paradigm used in previous research. The current study used a gaze cueing paradigm within a change detection task, and the gaze cue was manipulated such that participants were aware that the cue-agent was only able to 'see' in one condition. The results revealed that participants were influenced by the mental state of the cue-agent, and were significantly better at identifying if a change had occurred on valid trials when they believed the cue-agent could 'see'. The computation of the cue-agent's mental state therefore mediated the gaze cueing effect, demonstrating that the modulation of gaze cueing by mental state attributions generalises to other experimental paradigms.

The Effect of Eye Contact Is Contingent on Visual Awareness

The present study explored how eye contact at different levels of visual awareness influences gaze-induced joint attention. We adopted a spatial-cueing paradigm, in which an averted gaze was used as an uninformative central cue for a joint-attention task. Prior to the onset of the averted-gaze cue, either supraliminal (Experiment 1) or subliminal (Experiment 2) eye contact was presented. The results revealed a larger subsequent gaze-cueing effect following supraliminal eye contact compared to a no-contact condition. In contrast, the gaze-cueing effect was smaller in the subliminal eye-contact condition than in the no-contact condition. These findings suggest that the facilitation effect of eye contact on coordinating social attention depends on visual awareness. Furthermore, subliminal eye contact might have an impact on subsequent social attention processes that differ from supraliminal eye contact. This study highlights the need to further investigate the role of eye contact in implicit social cognition.

Investigating gaze processing in euthymic bipolar disorder: Impaired ability to infer mental state and intention, but preservation of social attentional orienting

Euthymic bipolar disorder (BD) has been associated with subtle impairment in face processing. However, it is not known whether their difficulties extend to the processing of gaze. In the present study, two tasks, both of which rely on the ability to make use of the eye region of a pictured face, were used: the Reading the Mind in the Eyes test and the Eye-gaze cueing task. Compared to healthy controls, BD patients were impaired at judging mental state from images of the face but showed normal susceptibility to the direction of gaze as an attentional cue. These findings suggest that BD patients present selective gaze processing impairment, limited to the sensitivity to intention and emotion. This impairment could account at least partially for the higher levels of interpersonal problems generally observed in BD.

Perceiving crowd attention: Gaze following in human crowds with conflicting cues

People automatically redirect their visual attention by following others' gaze orientation, a phenomenon called "gaze following." This is an evolutionarily generated socio-cognitive process that provides people with information about their environments. Often, however, people in crowds can have rather different gaze orientations. This study investigated how gaze following occurs in situations with many conflicting gazes. In two experiments, we modified the gaze cueing paradigm to use a crowd rather than a single individual. Specifically, participants were presented with a group of human avatars with differing gaze orientations, and the target appeared randomly on the left or right side of a display. We found that (a) when a marked difference existed in the number of avatars with divergent gaze orientations, participants automatically followed the majority's gaze orientation, and (b) the strongest gaze cue effect occurred when all gazes shared the same orientation, with the response superiority of the majority's oriented location monotonically diminishing with the number of gazes with divergent orientations. These findings suggested that the majority rule plays a role in gaze following behavior when individuals are confronted with conflicting multigaze scenes, and that an increasing subgroup size appears to enlarge the strength of the gaze cueing effect.

Social Beliefs and Visual Attention: How the Social Relevance of a Cue Influences Spatial Orienting

We are highly tuned to each other's visual attention. Perceiving the eye or hand movements of another person can influence the timing of a saccade or the reach of our own. However, the explanation for such spatial orienting in interpersonal contexts remains disputed. Is it due to the social appearance of the cue—a hand or an eye—or due to its social relevance—a cue that is connected to another person with attentional and intentional states? We developed an interpersonal version of the Posner spatial cueing paradigm. Participants saw a cue and detected a target at the same or a different location, while interacting with an unseen partner. Participants were led to believe that the cue was either connected to the gaze location of their partner or was generated randomly by a computer (Experiment 1), and that their partner had higher or lower social rank while engaged in the same or a different task (Experiment 2). We found that spatial cue‐target compatibility effects were greater when the cue related to a partner's gaze. This effect was amplified by the partner's social rank, but only when participants believed their partner was engaged in the same task. Taken together, this is strong evidence in support of the idea that spatial orienting is interpersonally attuned to the social relevance of the cue—whether the cue is connected to another person, who this person is, and what this person is doing—and does not exclusively rely on the social appearance of the cue. Visual attention is not only guided by the physical salience of one's environment but also by the mental representation of its social relevance.

Constructing Perceptual Common Ground Between Human and Robot Through Joint Attention

Joint attention is a communicative activity that allows social partners to share perceptual experiences by jointly attending to an environmental object. Unlike the common approach towards joint attention, which is based on the developmental view in robotics, here it is conceptualized with a psychophysical paradigm known as cueing. The triadic interaction of joint attention is formalized as the conditional probability of an attentional response for a given target candidate derived from object features and a cue derived from a human partner's indication. A robotic system to which the joint attention model is applied conducted a series of tasks to demonstrate the properties of the computational model. The robotic system successfully performed the tasks, which could not be specified by the information derived from a target object alone; furthermore, the system demonstrated how perceptual and selection ambiguity is resolved through joint attentive interaction and made to converge into a common perceptual state. The results imply that a perceptual common ground is constructed on the triadic relationship between user, robot, and objects through joint attentive interaction.

Attentional interference is modulated by salience not sentience

Spatial cueing of attention occurs when attention is oriented by the onset of a stimulus or by other information that creates a bias towards a particular location. The presence of a cue that orients attention can also interfere with participants' reporting of what they see. It has been suggested that this type of interference is stronger in the presence of socially-relevant cues, such as human faces or avatars, and is therefore indicative of a specialised role for perspective calculation within the social domain. However, there is also evidence that the effect is a domain-general form of processing that is elicited equally with non-social directional cues. The current paper comprises four experiments that systematically manipulated the social factors believed necessary to elicit the effect. The results show that interference persists when all social components are removed, and that visual processes are sufficient to explain this type of interference, thus supporting a domain-general perceptual interpretation of interference.

Spontaneous Perspective Taking in Humans?

A number of social cognition studies posit that humans spontaneously compute the viewpoint of other individuals. This is based on experiments showing that responses are shorter when a human agent, located in a visual display, can see the stimuli relevant to the observer's task. Similarly, responses are slower when the agent cannot see the task-relevant stimuli. We tested the spontaneous perspective taking theory by incorporating it within two classic visual cognition paradigms (i.e., the flanker effect and the Simon effect), as well as reassessing its role in the gaze cueing effect. Results showed that these phenomena (e.g., the Simon effect) are not modulated according to whether a gazing agent can see the critical stimuli or not. We also examined the claim that previous results attributed to spontaneous perspective taking are due to the gazing agent's ability to shift attention laterally. Results found no evidence of this. Overall, these data challenge both the spontaneous perspective taking theory, as well as the attentional shift hypothesis.

Backward-walking biological motion orients attention to moving away instead of moving toward

Walking direction is an important attribute of biological motion because it carries key information, such as the specific intention of the walker. Although it is known that spatial attention is guided by walking direction, it remains unclear whether this attentional shift is reflexive (i.e., constantly shifts to the walking direction) or not. A richer interpretation of this effect is that attention is guided to seek the information that is necessary to understand the motion. To investigate this issue, we examined how backward-walking biological motion orients attention because the intention of walking backward is usually to avoid something that walking forward would encounter. The results showed that attention was oriented to the walking-away direction of biological motion instead of the walking-toward direction (Experiment 1), and this effect was not due to the gaze direction of biological motion (Experiment 2). Our findings suggest that the attentional shift triggered by walking direction is not reflexive, thus providing support for the rich interpretation of these attentional effects.

Social Influence on Metacognitive Evaluations: The Power of Nonverbal Cues

Metacognitive evaluations refer to the processes by which people assess their own cognitive operations with respect to their current goal. Little is known about whether this process is susceptible to social influence. Here we investigate whether nonverbal social signals spontaneously influence metacognitive evaluations. Participants performed a two-alternative forced-choice task, which was followed by a face randomly gazing towards or away from the response chosen by the participant. Participants then provided a metacognitive evaluation of their response by rating their confidence in their answer. In Experiment 1, the participants were told that the gaze direction was irrelevant to the task purpose and were advised to ignore it. The results revealed an effect of implicit social information on confidence ratings even though the gaze direction was random and therefore unreliable for task purposes. In addition, nonsocial cues (car) did not elicit this effect. In Experiment 2, the participants were led to believe that cue direction (face or car) reflected a previous participant's response to the same question—that is, the social information provided by the cue was made explicit, yet still objectively unreliable for the task. The results showed a similar social influence on confidence ratings, observed with both cues (car and face) but with an increased magnitude relative to Experiment 1. We additionally showed in Experiment 2 that social information impaired metacognitive accuracy. Together our results strongly suggest an involuntary susceptibility of metacognitive evaluations to nonverbal social information, even when it is implicit (Experiment 1) and unreliable (Experiments 1 and 2).

Real Person Interaction in Visual Attention Research

Abstract. An important development in cognitive psychology in the past decade has been the examination of visual attention during real social interaction. This contrasts traditional laboratory studies of attention, including “social attention,” in which observers perform tasks alone. In this review, we show that although the lone-observer method has been central to attention research, real person interaction paradigms have not only uncovered the processes that occur during “joint attention,” but have also revealed attentional processes previously thought not to occur. Furthermore, the examination of some visual attention processes almost invariably requires the use of real person paradigms. While we do not argue for an increase in “ecological validity” for its own sake, we do suggest that research using real person interaction has greatly benefited the development of visual attention theories.

Eyes only? Perceiving eye contact is neither sufficient nor necessary for attentional capture by face direction

Direct eye contact and motion onset both constitute powerful cues that capture attention. Recent research suggests that (social) gaze and (non-social) motion onset influence information processing in parallel, even when combined as sudden onset direct gaze cues (i.e., faces suddenly establishing eye contact). The present study investigated the role of eye visibility for attention capture by these sudden onset face cues. To this end, face direction was manipulated (away or towards onlooker) while faces had closed eyes (eliminating visibility of eyes, Experiment 1), wore sunglasses (eliminating visible eyes, but allowing for the expectation of eyes to be open, Experiment 2), and were inverted with visible eyes (disrupting the integration of eyes and faces, Experiment 3). Participants classified targets appearing on one of four faces. Initially, two faces were oriented towards participants and two faces were oriented away from participants. Simultaneous to target presentation, one averted face became directed and one directed face became averted. Attention capture by face direction (i.e., facilitation for faces directed towards participants) was absent when eyes were closed, but present when faces wore sunglasses. Sudden onset direct faces can, hence, induce attentional capture, even when lacking eye cues. Inverted faces, by contrast, did not elicit attentional capture. Thus, when eyes cannot be integrated into a holistic face representation they are not sufficient to capture attention. Overall, the results suggest that visibility of eyes is neither necessary nor sufficient for the sudden direct face effect.

[Observing grasping action from egocentric perspective modulates observer's visual attention]

Previous studies demonstrated that observing another person's grasp action modulated the observer's attention to the object in a manner congruent with another person's action goal. These studies suggest that this grasp-cueing effect results from representation of the observer's understanding of action intention in the mirror neuron system. This system serves as the neural mechanism underlying action-oriented goals, regardless of whether they are those of the observer or others. The present study examined whether the grasp-cueing effect emerged even when a grasp action was observed from an egocentric (observer's own) perspective. Reaction times were faster when the target was presented over the object that was congruent with the action goal implied by the size of the hand aperture. This indicated that the grasp cueing effect occurred when the stimulus was observed from an egocentric perspective, consistent with the view that this effect resulted from the observer's understanding of action intention as represented in the mirror neuron system.

The effects of social misdirection on magic tricks: How deceived and undeceived groups differ

Characteristics of perception and cognition in our daily lives can be elucidated through studying misdirection, a technique used by magicians to manipulate attention. Recent findings on the effects of social misdirection induced by joint attention have been disputed, and differences between deceived (failed to detect the magic trick) and undeceived (detected the magic trick) groups remain unclear. To examine how social misdirection affects deceived and undeceived groups, we showed participants movie clips of the “cups & balls,” a classic magic trick, and measured participants' eye positions (i.e. where participants looked while viewing the clips) using an eye tracker. We found that the undeceived group looked less at the magician's face than the deceived group. These results indicate that deceived individuals have difficulty trying not to allocate attention to the face. We conclude that social misdirection captures attention, influencing the emergence of deception.

Effects of Peripheral Eccentricity and Head Orientation on Gaze Discrimination

Visual search tasks support a special role for direct gaze in human cognition, while classic gaze judgment tasks suggest the congruency between head orientation and gaze direction plays a central role in gaze perception. Moreover, whether gaze direction can be accurately discriminated in the periphery using covert attention is unknown. In the present study, individual faces in frontal and in deviated head orientations with a direct or an averted gaze were flashed for 150 ms across the visual field; participants focused on a centred fixation while judging the gaze direction. Gaze discrimination speed and accuracy varied with head orientation and eccentricity. The limit of accurate gaze discrimination was less than ±6° eccentricity. Response times suggested a processing facilitation for direct gaze in fovea, irrespective of head orientation, however, by ±3° eccentricity, head orientation started biasing gaze judgments, and this bias increased with eccentricity. Results also suggested a special processing of frontal heads with direct gaze in central vision, rather than a general congruency effect between eye and head cues. Thus, while both head and eye cues contribute to gaze discrimination, their role differs with eccentricity.

From gaze cueing to dual eye-tracking: novel approaches to investigate the neural correlates of gaze in social interaction

Tracking eye-movements provides easy access to cognitive processes involved in visual and sensorimotor processing. More recently, the underlying neural mechanisms have been examined by combining eye-tracking and functional neuroimaging methods. Apart from extracting visual information, gaze also serves important functions in social interactions. As a deictic cue, gaze can be used to direct the attention of another person to an object. Conversely, by following other persons' gaze we gain access to their attentional focus, which is essential for understanding their mental states. Social gaze has therefore been studied extensively to understand the social brain. In this endeavor, gaze has mostly been investigated from an observational perspective using static displays of faces and eyes. However, there is growing consent that observational paradigms are insufficient for an understanding of the neural mechanisms of social gaze behavior, which typically involve active engagement in social interactions. Recent methodological advances have allowed increasing ecological validity by studying gaze in face-to-face encounters in real-time. Such improvements include interactions using virtual agents in gaze-contingent eye-tracking paradigms, live interactions via video feeds, and dual eye-tracking in two-person setups. These novel approaches can be used to analyze brain activity related to social gaze behavior. This review introduces these methodologies and discusses recent findings on the behavioral functions and neural mechanisms of gaze processing in social interaction.

What is social about social perception research?

A growing consensus in social cognitive neuroscience holds that large portions of the primate visual brain are dedicated to the processing of social information, i.e., to those aspects of stimuli that are usually encountered in social interactions such as others' facial expressions, actions, and symbols. Yet, studies of social perception have mostly employed simple pictorial representations of conspecifics. These stimuli are social only in the restricted sense that they physically resemble objects with which the observer would typically interact. In an equally important sense, however, these stimuli might be regarded as "non-social": the observer knows that they are viewing pictures and might therefore not attribute current mental states to the stimuli or might do so in a qualitatively different way than in a real social interaction. Recent studies have demonstrated the importance of such higher-order conceptualization of the stimulus for social perceptual processing. Here, we assess the similarity between the various types of stimuli used in the laboratory and object classes encountered in real social interactions. We distinguish two different levels at which experimental stimuli can match social stimuli as encountered in everyday social settings: (1) the extent to which a stimulus' physical properties resemble those typically encountered in social interactions and (2) the higher-level conceptualization of the stimulus as indicating another person's mental states. We illustrate the significance of this distinction for social perception research and report new empirical evidence further highlighting the importance of mental state attribution for perceptual processing. Finally, we discuss the potential of this approach to inform studies of clinical conditions such as autism.