Seeing other minds: attributed mental states influence perception

Not to follow because of distrust: perceived trust modulates the gaze cueing effect

In social life, people spontaneously form stable trustworthiness impressions from faces. However, the precise role of extracting trustworthiness information remains unclear. This study aims to elucidate whether discerning facial trustworthiness influences social interactions. Specifically, it explores the gaze cueing effect (GCE), wherein individuals exhibit quicker responses to targets appearing in the direction of gaze compared to other locations. Given conflicting perspectives in existing literature regarding the potential modulation of trustworthiness on the GCE, two plausible hypotheses are proposed to explain divergent result patterns. The reflexive hypothesis posits that the GCE operates automatically. In contrast, the flexible hypothesis underscores the potential modulatory role of trustworthiness in the GCE. To provide a comprehensive understanding of whether trustworthiness modulates the GCE, we employed face stimuli incorporating trustworthiness information within Posner' s cue-target task. The findings of Experiment 1 revealed that the perception of trustworthiness indeed influenced the GCE. Specifically, when facial stimuli were perceived as trustworthy, they elicited a more pronounced GCE compared to untrustworthy stimuli. This modulation effect was replicated using a different stimulus set in Experiment 2. In Experiment 3, we employed the same stimuli as in Experiment 2, setting the trustworthiness information to baseline as a control experiment. The results demonstrated that the trustworthiness modulation effect disappeared, indicating its specificity to the trustworthiness attribute of the stimuli rather than other characteristics. Collectively, these findings lend support to the flexible hypothesis, highlighting that the extraction of trustworthiness information plays a pivotal role in modulating the GCE, consequently influencing social interactions.

Beyond human-likeness: Socialness is more influential when attributing mental states to robots

We sought to replicate and expand previous work showing that the more human-like a robot appears, the more willing people are to attribute mind-like capabilities and socially engage with it. Forty-two participants played games against a human, a humanoid robot, a mechanoid robot, and a computer algorithm while undergoing functional neuroimaging. We confirmed that the more human-like the agent, the more participants attributed a mind to them. However, exploratory analyses revealed that the perceived socialness of an agent appeared to be as, if not more, important for mind attribution. Our findings suggest top-down knowledge cues may be equally or possibly more influential than bottom-up stimulus cues when exploring mind attribution in non-human agents. While further work is now required to test this hypothesis directly, these preliminary findings hold important implications for robotic design and to understand and test the flexibility of human social cognition when people engage with artificial agents.

The role of self-representation in emotional contagion

Although prior research has implied that emotional contagion occurs automatically and unconsciously, convincing evidence suggests that it is significantly influenced by individuals' perceptions of their relationships with others or with collectives within specific social contexts. This implies a role for self-representation in the process. The present study aimed to offer a novel explanation of the interplay between social contexts and emotional contagion, focusing on the contextualized nature of self-representation and exploring the social factors that shape emotional contagion. It further posits a causal loop among social contexts, self-representation, and emotional contagion. Drawing from the lens of self-representation, this study concludes with a discussion on potential research directions in this field, commencing with an exploration of the antecedents and consequences of emotional contagion and self-representation.

An assessment of dynamic facial emotion recognition and theory of mind in children with ADHD: An eye-tracking study

Deficits in social cognition in attention deficit hyperactivity disorder (ADHD) have been associated with difficulties in functioning. Since recognizing emotional facial expressions is essential for developing the perceptual components of the theory of mind (ToM), it is important to assess this relationship in children with ADHD. This study therefore compared the recognition of emotional stimuli and gaze patterns between children with ADHD and healthy children using eye-tracking with dynamic facial images. It also examined the relationship between facial emotion recognition accuracy, gaze patterns, ToM scores, and ADHD symptoms. Children with ADHD aged 8-13 (n = 47) and a control group (n = 38) completed a facial emotion recognition test, ToM tests, and the Conners' Parent Rating Scale. Participants' gaze patterns in response to dynamic facial emotion expressions were recorded using eye-tracking technology. Children with ADHD exhibited significantly lower accuracy in the recognition of the facial expressions of disgust and anger. The percentage fixation in the eye region was also significantly lower for happy, angry, sad, disgusted, and neutral emotions in the children with ADHD compared to the control group. No relationship was determined between the percentage of fixations on facial areas of interests and ADHD symptoms or ToM tests. This study provides evidence that children with ADHD experience deficits in visual attention to emotional cues. In addition, it suggests that facial emotion recognition deficits in children with ADHD represent a separate domain of social cognition that develops independently of ToM skills and core symptoms. Understanding and treating the social difficulties of individuals with ADHD may help improve their social functioning.

Top-Down Modulation of Motor Priming by Belief About Animacy

Research has shown that people automatically imitate others and that this tendency is stronger when the other person is a human compared with a nonhuman agent. However, a controversial question is whether automatic imitation is also modulated by whether people believe the other person is a human. Although early research supported this hypothesis, not all studies reached the same conclusion and a recent meta-analysis found that there is currently neither evidence in favor nor against an influence of animacy beliefs on automatic imitation. One of the most prominent studies supporting such an influence is the study by Liepelt and Brass (2010), who found that automatic imitation was stronger when participants believed an ambiguous, gloved hand to be human, as opposed to wooden. In this registered report, we provide a high-powered replication of this study (N = 199). In contrast to Liepelt and Brass (2010), we did not find an effect of animacy beliefs on automatic imitation. However, we did find a correlation between automatic imitation and perceived self-other similarity. Together, these results suggest that the gloved hand procedure does not reliably influence automatic imitation, but interindividual differences in perceived similarity do.

The influence of perceptual load on gaze-induced attentional orienting: The modulation of expectation

Humans tend to focus on others' gaze. Previous studies have shown that the gaze direction of others can induce corresponding attentional orienting. However, gaze cues have typically been presented alone in these studies. It is unclear how gaze cues induce observers' attention in complicated contexts with additional perceptual information. Therefore, the present study investigated gaze-induced attentional orienting at different levels of perceptual load. Results indicated that the attentional effect of the dynamic gaze cue (i.e., GCE: gaze cue effect) emerged under low perceptual load and disappeared under high perceptual load. The absence of GCE could not attribute to perceptual capacity exhaustion. Moreover, the influence of perceptual load on gaze-induced attentional orienting was modulated by individuals' expectation. Specifically, the GCE occurred under high perceptual load when the gaze cue was predictive (with individuals' expectation). These findings provide new evidence on the mode of gaze-induced attentional orienting under different perceptual load conditions.

Expectations of efficient actions bias social perception: a pre-registered online replication

Humans take a teleological stance when observing others' actions, interpreting them as intentional and goal directed. In predictive processing accounts of social perception, this teleological stance would be mediated by a perceptual prediction of an ideal energy-efficient reference trajectory with which a rational actor would achieve their goals within the current environmental constraints. Hudson and colleagues (2018 Proc. R. Soc. B 285, 20180638. (doi:10.1098/rspb.2018.0638)) tested this hypothesis in a series of experiments in which participants reported the perceived disappearance points of hands reaching for objects. They found that these judgements were biased towards the expected efficient reference trajectories. Observed straight reaches were reported higher when an obstacle needed to be overcome than if the path was clear. By contrast, unnecessarily high reaches over empty space were perceptually flattened. Moreover, these perceptual biases increased the more the environmental constraints and expected action trajectories were explicitly processed. These findings provide an important advance to our understanding of the mechanisms underlying social perception. The current replication tests the robustness of these findings and whether they uphold in an online setting.

Body Form Modulates the Prediction of Human and Artificial Behaviour from Gaze Observation

The future of human–robot collaboration relies on people’s ability to understand and predict robots' actions. The machine-like appearance of robots, as well as contextual information, may influence people’s ability to anticipate the behaviour of robots. We conducted six separate experiments to investigate how spatial cues and task instructions modulate people’s ability to understand what a robot is doing. Participants observed goal-directed and non-goal directed gaze shifts made by human and robot agents, as well as directional cues displayed by a triangle. We report that biasing an observer's attention, by showing just one object an agent can interact with, can improve people’s ability to understand what humanoid robots will do. Crucially, this cue had no impact on people’s ability to predict the upcoming behaviour of the triangle. Moreover, task instructions that focus on the visual and motor consequences of the observed gaze were found to influence mentalising abilities. We suggest that the human-like shape of an agent and its physical capabilities facilitate the prediction of an upcoming action. The reported findings expand current models of gaze perception and may have important implications for human–human and human–robot collaboration.

Computational psychiatry: from synapses to sentience

This review considers computational psychiatry from a particular viewpoint: namely, a commitment to explaining psychopathology in terms of pathophysiology. It rests on the notion of a generative model as underwriting (i) sentient processing in the brain, and (ii) the scientific process in psychiatry. The story starts with a view of the brain-from cognitive and computational neuroscience-as an organ of inference and prediction. This offers a formal description of neuronal message passing, distributed processing and belief propagation in neuronal networks; and how certain kinds of dysconnection lead to aberrant belief updating and false inference. The dysconnections in question can be read as a pernicious synaptopathy that fits comfortably with formal notions of how we-or our brains-encode uncertainty or its complement, precision. It then considers how the ensuing process theories are tested empirically, with an emphasis on the computational modelling of neuronal circuits and synaptic gain control that mediates attentional set, active inference, learning and planning. The opportunities afforded by this sort of modelling are considered in light of in silico experiments; namely, computational neuropsychology, computational phenotyping and the promises of a computational nosology for psychiatry. The resulting survey of computational approaches is not scholarly or exhaustive. Rather, its aim is to review a theoretical narrative that is emerging across subdisciplines within psychiatry and empirical scales of investigation. These range from epilepsy research to neurodegenerative disorders; from post-traumatic stress disorder to the management of chronic pain, from schizophrenia to functional medical symptoms.

Social Perception Deficit as a Factor of Vulnerability to Psychosis: A Brief Proposal for a Definition

Disturbances in social cognition are a core feature of schizophrenia. While most research in the field has focused on emotion perception, social knowledge, theory of mind, and attribution styles, the domain of social perception has received little specific attention. In this paper, we suggest that this issue can be explained by the lack of a precise and unitary definition of the concept, this leads to the existence of different competing uses of the concept and their conflation with other domains of social cognition. Relying on resources coming from the ecological approach to psychology and the phenomenological tradition in psychiatry, we propose that the concept of Social Perception should be used to refer to low-level pre-reflective processes underlying the awareness of interpersonal interactions with and between others. Clinical data suggests that people with schizophrenia have problems perceiving social situations as opportunities for social engagement, so, in order to fulfil this explanatory need, we propose that the term should be used to capture this important-yet neglected-domain of social cognition. We conclude with the discussion of some future directions for research derived from our proposal.

Social signalling as a framework for second-person neuroscience

Despite the recent increase in second-person neuroscience research, it is still hard to understand which neurocognitive mechanisms underlie real-time social behaviours. Here, we propose that social signalling can help us understand social interactions both at the single- and two-brain level in terms of social signal exchanges between senders and receivers. First, we show how subtle manipulations of being watched provide an important tool to dissect meaningful social signals. We then focus on how social signalling can help us build testable hypotheses for second-person neuroscience with the example of imitation and gaze behaviour. Finally, we suggest that linking neural activity to specific social signals will be key to fully understand the neurocognitive systems engaged during face-to-face interactions.

Shared Attention Amplifies the Neural Processing of Emotional Faces

Sharing an experience, without communicating, affects people's subjective perception of the experience, often by intensifying it. We investigated the neural mechanisms underlying shared attention by implementing an EEG study where participants attended to and rated the intensity of emotional faces, simultaneously or independently. Participants performed the task in three experimental conditions: (a) alone; (b) simultaneously next to each other in pairs, without receiving feedback of the other's responses (shared without feedback); and (c) simultaneously while receiving the feedback (shared with feedback). We focused on two face-sensitive ERP components: The amplitude of the N170 was greater in the "shared with feedback" condition compared to the alone condition, reflecting a top-down effect of shared attention on the structural encoding of faces, whereas the EPN was greater in both shared context conditions compared to the alone condition, reflecting an enhanced attention allocation in the processing of emotional content of faces, modulated by the social context. Taken together, these results suggest that shared attention amplifies the neural processing of faces, regardless of the valence of facial expressions.

Are adaptation aftereffects for facial emotional expressions affected by prior knowledge about the emotion?

Accurate perception of the emotional signals conveyed by others is crucial for successful social interaction. Such perception is influenced not only by sensory input, but also by knowledge we have about the others' emotions. This study addresses the issue of whether knowing that the other's emotional state is congruent or incongruent with their displayed emotional expression ("genuine" and "fake", respectively) affects the neural mechanisms underpinning the perception of their facial emotional expressions. We used a visual adaptation paradigm to investigate this question in three experiments employing increasing adaptation durations. The adapting stimuli consisted of photographs of emotional facial expressions of joy and anger, purported to reflect (in-)congruency between felt and expressed emotion, displayed by professional actors. A Validity checking procedure ensured participants had the correct knowledge about the (in-)congruency. Significantly smaller adaptation aftereffects were obtained when participants knew that the displayed expression was incongruent with the felt emotion, following all tested adaptation periods. This study shows that knowledge relating to the congruency between felt and expressed emotion modulates face expression aftereffects. We argue that this reflects that the neural substrate responsible for the perception of facial expressions of emotion incorporates the presumed felt emotion underpinning the expression.

Irrelevant Robot Signals in a Categorization Task Induce Cognitive Conflict in Performance, Eye Trajectories, the N2 ERP-EEG Component, and Frontal Theta Oscillations

Understanding others' nonverbal behavior is essential for social interaction, as it allows, among others, to infer mental states. Although gaze communication, a well-established nonverbal social behavior, has shown its importance in inferring others' mental states, not much is known about the effects of irrelevant gaze signals on cognitive conflict markers during collaborative settings. Here, participants completed a categorization task where they categorized objects based on their color while observing images of a robot. On each trial, participants observed the robot iCub grasping an object from a table and offering it to them to simulate a handover. Once the robot "moved" the object forward, participants were asked to categorize the object according to its color. Before participants were allowed to respond, the robot made a lateral head/gaze shift. The gaze shifts were either congruent or incongruent with the object's color. We expected that incongruent head cues would induce more errors (Study 1), would be associated with more curvature in eye-tracking trajectories (Study 2), and induce larger amplitude in electrophysiological markers of cognitive conflict (Study 3). Results of the three studies show more oculomotor interference as measured in error rates (Study 1), larger curvatures eye-tracking trajectories (Study 2), and higher amplitudes of the N2 ERP of the EEG signals as well as higher event-related spectral perturbation amplitudes (Study 3) for incongruent trials compared with congruent trials. Our findings reveal that behavioral, ocular, and electrophysiological markers can index the influence of irrelevant signals during goal-oriented tasks.

Adaptive Prediction for Social Contexts: The Cerebellar Contribution to Typical and Atypical Social Behaviors

Social interactions involve processes ranging from face recognition to understanding others' intentions. To guide appropriate behavior in a given context, social interactions rely on accurately predicting the outcomes of one's actions and the thoughts of others. Because social interactions are inherently dynamic, these predictions must be continuously adapted. The neural correlates of social processing have largely focused on emotion, mentalizing, and reward networks, without integration of systems involved in prediction. The cerebellum forms predictive models to calibrate movements and adapt them to changing situations, and cerebellar predictive modeling is thought to extend to nonmotor behaviors. Primary cerebellar dysfunction can produce social deficits, and atypical cerebellar structure and function are reported in autism, which is characterized by social communication challenges and atypical predictive processing. We examine the evidence that cerebellar-mediated predictions and adaptation play important roles in social processes and argue that disruptions in these processes contribute to autism.

Seeing things differently: Gaze shapes neural signal during mentalizing according to emotional awareness

Studies on social cognition often use complex visual stimuli to asses neural processes attributed to abilities like "mentalizing" or "Theory of Mind" (ToM). During the processing of these stimuli, eye gaze, however, shapes neural signal patterns. Individual differences in neural operations on social cognition may therefore be obscured if individuals' gaze behavior differs systematically. These obstacles can be overcome by the combined analysis of neural signal and natural viewing behavior. Here, we combined functional magnetic resonance imaging (fMRI) with eye-tracking to examine effects of unconstrained gaze on neural ToM processes in healthy individuals with differing levels of emotional awareness, i.e. alexithymia. First, as previously described for emotional tasks, people with higher alexithymia levels look less at eyes in both ToM and task-free viewing contexts. Further, we find that neural ToM processes are not affected by individual differences in alexithymia per se. Instead, depending on alexithymia levels, gaze on critical stimulus aspects reversely shapes the signal in medial prefrontal cortex (MPFC) and anterior temporoparietal junction (TPJ) as distinct nodes of the ToM system. These results emphasize that natural selective attention affects fMRI patterns well beyond the visual system. Our study implies that, whenever using a task with multiple degrees of freedom in scan paths, ignoring the latter might obscure important conclusions.

Predictive action perception from explicit intention information in autism

Social difficulties in autism spectrum disorder (ASD) may originate from a reduced top-down modulation of sensory information that prevents the spontaneous attribution of intentions to observed behaviour. However, although people with autism are able to explicitly reason about others' mental states, the effect of abstract intention information on perceptual processes has remained untested. ASD participants (n = 23) and a neurotypical (NT) control group (n = 23) observed a hand either reaching for an object or withdrawing from it. Prior to action onset, the participant either instructed the actor to "Take it" or "Leave it", or heard the actor state "I'll take it" or "I'll leave it", which provided an explicit intention that was equally likely to be congruent or incongruent with the subsequent action. The hand disappeared before completion of the action, and participants reported the last seen position of the tip of the index finger by touching the screen. NT participants exhibited a predictive bias in response to action direction (reaches perceived nearer the object, withdrawals perceived farther away), and in response to prior knowledge of the actor's intentions (nearer the object after "Take it", farther away after "Leave it"). However, ASD participants exhibited a predictive perceptual bias only in response to the explicit intentions, but not in response to the motion of the action itself. Perception in ASD is not immune from top-down modulation. However, the information must be explicitly presented independently from the stimulus itself, and not inferred from cues inherent in the stimulus.

Revisiting Human-Agent Communication: The Importance of Joint Co-construction and Understanding Mental States

The study of human-human communication and the development of computational models for human-agent communication have diverged significantly throughout the last decade. Yet, despite frequently made claims of “super-human performance” in, e.g., speech recognition or image processing, so far, no system is able to lead a half-decent coherent conversation with a human. In this paper, we argue that we must start to re-consider the hallmarks of cooperative communication and the core capabilities that we have developed for it, and which conversational agents need to be equipped with: incremental joint co-construction and mentalizing. We base our argument on a vast body of work on human-human communication and its psychological processes that we reason to be relevant and necessary to take into account when modeling human-agent communication. We contrast those with current conceptualizations of human-agent interaction and formulate suggestions for the development of future systems.

Enhanced detection of gaze toward an object: Sociocognitive influences on visual search

Another person's gaze direction is a rich source of social information, especially eyes gazing toward prominent or relevant objects. To guide attention to these important stimuli, visual search mechanisms may incorporate sophisticated coding of eye-gaze and its spatial relationship to other objects. Alternatively, any guidance might reflect the action of simple perceptual 'templates' tuned to visual features of socially relevant objects, or intrinsic salience of direct-gazing eyes for human vision. Previous findings that direct gaze (toward oneself) is prioritised over averted gaze do not distinguish between these accounts. To resolve this issue, we compared search for eyes gazing toward a prominent object versus gazing away, finding more efficient search for eyes 'gazing toward' the object. This effect was most clearly seen in target-present trials when gaze was task-relevant. Visual search mechanisms appear to specify gazer-object relations, a computational building-block of theory of mind.

Exploring Influences on Theory of Mind Impairment in Opioid Dependent Patients

Theory of mind (ToM) is an aspect of social cognition impaired in different addictive disorders, including opioid addiction. This study aimed at replicating ToM deficits in opioid dependent patients undergoing opioid maintenance treatment (OMT) and exploring the influence of substance use related variables, executive functions and childhood maltreatment on ToM in opioid dependent patients. 66 opioid dependent patients were tested using the Movie for Assessment of Social Cognition (MASC) and compared with the data of healthy controls. Furthermore, the opioid dependent patients underwent testing for executive functions and filled in the Childhood Trauma Questionnaire (CTQ). Performance on the MASC was significantly poorer in the opioid dependence group than in the control group, even when recent additional drug use and psychiatric comorbidities were controlled for. No correlations were found between ToM and substance use related factors. Aspects of ToM performance in opioid dependent patients correlated significantly with different EF domains. ToM correlated significantly with the CTQ scales for physical maltreatment. The results confirm impaired ToM in opioid dependent patients and highlight executive functions and childhood maltreatment as influential factors. The lack of associations between ToM and substance use related variables and the association with childhood maltreatment suggest that ToM impairments might be a risk factor predating substance abuse.

Attention neglects a stare-in-the-crowd: Unanticipated consequences of prediction-error coding

Direct gaze - someone looking at you - is an important and subjectively-salient stimulus. Its processing is thought to be enhanced by the brain's internalised predictions - priors - that effectively specify it as the most likely gaze direction. Current consensus holds that, befitting its presumed importance, direct gaze attracts attention more powerfully than other gazes. Conversely, some Predictive Coding (PC) models, in which exogenous attention is drawn to stimuli that violate predictions, may be construed as making the opposite claim - i.e., exogenous attention should be biased away from direct gaze (which conforms to internal predictions), toward averted gaze (which does not). Here, searching displays with salient, 'odd-one-out' gazes, we observed attentional bias (in rapid, initial saccades) toward averted gaze, as would be expected by PC models. However, this pattern obtained only when conditions highlighted gaze-uniqueness. We speculate that, in our experiments, task requirements determined how prediction influenced perception.

Investigating Visual Perception in Teaching and Learning with Advanced Eye-Tracking Methodologies: Rewards and Challenges of an Innovative Research Paradigm

In this discussion paper, teaching and learning are characterized as being situated, complex, and reciprocally interactive activities. Accordingly, a teacher’s pedagogical actions are always action and reaction at the same time. Irrespective of the reciprocally interactive nature of teaching and learning, educational research has sought to identify characteristics of teacher expertise that enable teachers to influence students’ learning in a systematic and positive way. In this respect, the contributions to this special issue offer an innovative research paradigm, because they bring together different and originally separated strands of research: (1) research on professional vision in teacher education, (2) research on the structure and development of expertise in cognitive science, (3) research on cognitive processes by means of eye-tracking technologies in psychology, and (4) research on educational effectiveness and instructional quality in educational science. Following this introduction, a detailed discussion of the rewards and challenges of each of the seven contributions is provided. This discussion leads to the following conclusions: (a) The emphasis on the “reacting teacher” in professional vision research undervalues the role of lesson planning for the flexible handling of sudden events during class instruction. (b) The assumption of professional vision as mediator between a teacher’s knowledge and pedagogical actions overlooks that teachers can sometimes do more than they can tell. (c) Not all of the gaze behavior that is currently studied needs to be instructed. (d) Research should move beyond explorative expert-novice comparisons to hypotheses-driven designs that investigate how teachers can learn to successfully apply evidence-based pedagogical principles.

Psychophysiological responses to eye contact in a live interaction and in video call

Another person's gaze directed to oneself elicits autonomic arousal and facial reactions indicating positive affect in its observer. These effects have only been found to occur with mutual, live eye contact and not in response to direct gaze pictures or when the observer believes that the live person cannot see them. The question remains whether the physical presence of the other person is necessary for these effects. We measured psychophysiological responses to another person's direct versus averted gaze in three conditions: live interaction, bidirectional video call, and watching a mere video. Autonomic arousal was measured with skin conductance responses and facial reactions with facial electromyography. In the live and video call conditions, but not in the mere video condition, direct gaze increased autonomic arousal in comparison to averted gaze. In all three conditions, however, direct gaze elicited positive affective facial reactions. Therefore, an experience of being seen is essential for the autonomic reactions but not for the facial responses that are elicited by another person's direct gaze. Most importantly, the results suggest that the physical presence or proximity of the other person is not necessary for these psychophysiological responses to eye contact.

Effects of being watched on self-referential processing, self-awareness and prosocial behaviour

Reputation management theory suggests that our behaviour changes in the presence of others to signal good reputation (audience effect). However, the specific cognitive mechanisms by which being watched triggers these changes are poorly understood. Here we test the hypothesis that these changes happen because the belief in being watched increases self-referential processing. We used a novel deceptive video-conference paradigm, where participants believe a video-clip is (or is not) a live feed of a confederate watching them. Participants completed four tasks measuring self-referential processing, prosocial behaviour and self-awareness under these two belief settings. Although the belief manipulation and self-referential effect task were effective, there were no changes on self-referential processing between the two settings, nor on prosocial behaviour and self-awareness. Based on previous evidence and these findings, we propose that further research on the role of the self, social context and personality traits will help elucidating the mechanisms underlying audience effects.

Perceptual sensitivity is modulated by what others can see

Previous work has established that social cues such as the direction of others' gaze or their perspective on a scene may influence one's own perceptual judgments. However, up until now it has remained unclear whether such influences are exerted at a perceptual or decisional locus, as most previous studies have used response times as their primary dependent measure. Here, we asked whether perceptual sensitivity is also dependent on social cognition. To test this hypothesis, we asked participants to evaluate whether low-contrast Gabor patterns embedded in noise were visible from either their own or an avatar's perspective. Across three experiments, we found that observers' detection performance was increased if an avatar also shared perception of the stimulus location. By leveraging signal detection modelling, we show that this effect is driven by a change in perceptual sensitivity (d'), independent of decisional or response interference. Furthermore, by "blindfolding" the avatar, we show that the boosting effect of shared perception on detection sensitivity is only obtained when the participant believes the avatar can also see the stimulus, ruling out an influence of low-level directional cues. We interpret these results within a framework in which the avatar's perspective boosts top-down spatial attention by prioritising particular spatial locations at which perception is shared. In summary, we reveal that perceptual sensitivity is modulated by the perspective of others.

Brain activation and adaptation of deception processing during dyadic face-to-face interaction

Though deception is consistently characterized by the slippery-slope effect, i.e., the escalation of small lies over time, differing interactive situations and interacting processes may influence the trajectories of deception. To explore this influence, we investigated naturalistic face-to-face (FF) and computer-mediated face-blocked (FB) interactions using functional near-infrared spectroscopy (fNIRS). Pairs of participants acted as deceivers and receivers in an adapted ultimatum game while brain activity in the right dorsolateral prefrontal cortex (rDLPFC) and temporoparietal junction (rTPJ) was recorded. Comparison of deception in the two types of interactions showed that the FF interactions resulted in more successful deception, as well as acceptance of deception, and prompted more neural activation in the rDLPFC than the FB interactions. We found that the deception magnitude escalated in both FF and FB interactions, but rDLPFC activity during deception diminished over time only in the FF interactions but not in FB interactions, suggesting that the deceivers behaviourally adapted to deception over time in both types of interactions, but the neural adaptation occurred only in the FF interactions. Furthermore, neural adaptation in FF interactions was associated with behavioural switching after deception, indicating that the rDLPFC contributes to deception adaptation and the control of switching between deception and honesty. The FF interactions were also characterized by activity in the rTPJ, which showed an adaptation to deception. These findings highlight the importance of interactive situations in dyadic naturalistic settings for deception and the role of the rDLPFC and rTPJ in the slippery-slope effect in deception.

Socially Communicative Eye Contact and Gender Affect Memory

Because of their value as a socially communicative cue, researchers have strived to understand how the gaze of other people influences a variety of cognitive processes. Recent work in social attention suggests that the use of images of people in laboratory studies, as a substitute for real people, may not effectively test socially communicative aspects of eye gaze. As attention affects many other cognitive processes, it is likely that social attention between real individuals could also affect other cognitive processes, such as memory. However, from previous work alone, it is unclear whether, and if so how, socially communicative eye gaze affects memory. The present studies test the assumption that socially communicative aspects of eye gaze may impact memory by manipulating the eye gaze of a live speaker in the context of a traditional recognition paradigm used frequently in the laboratory. A female (Experiment 1) or male (Experiment 2) investigator read words aloud and varied whether eye contact was, or was not, made with a participant. With both female and male investigators, eye contact improved word recognition only for female participants and hindered word recognition in male participants. When a female investigator prolonged their eye contact (Experiment 3) to provide a longer opportunity to both observe and process the investigator's eye gaze, the results replicated the findings from Experiments 1 and 2. The findings from Experiments 1-3 suggest that females interpret and use the investigator's eye gaze differently than males. When key aspects from the previous experiments were replicated in a noncommunicative situation (i.e., when a video of a speaker is used instead of a live speaker; Experiment 4), the memory effects observed previously in response to eye gaze were eliminated. Together, these studies suggest that it is the socially communicative aspects of eye gaze from a real person that influence memory. The findings reveal the importance of using social cues that are communicative in nature (e.g., real people) when studying the relationship between social attention and memory.

The Role of Eye Gaze During Natural Social Interactions in Typical and Autistic People

Social interactions involve complex exchanges of a variety of social signals, such as gaze, facial expressions, speech and gestures. Focusing on the dual function of eye gaze, this review explores how the presence of an audience, communicative purpose and temporal dynamics of gaze allow interacting partners to achieve successful communication. First, we focus on how being watched modulates social cognition and behavior. We then show that the study of interpersonal gaze processing, particularly gaze temporal dynamics, can provide valuable understanding of social behavior in real interactions. We propose that the Interpersonal Gaze Processing model, which combines both sensing and signaling functions of eye gaze, provides a framework to make sense of gaze patterns in live interactions. Finally, we discuss how autistic individuals process the belief in being watched and interpersonal dynamics of gaze, and suggest that systematic manipulation of factors modulating gaze signaling can reveal which aspects of social eye gaze are challenging in autism.

Do Beliefs About Whether Others Can See Modulate Social Seeking in Autism?

Autistic people process gaze differently than typical people, but it is not yet clear if these differences lie in the processing of eye-shapes or the belief in whether others can see (perceptual mentalizing). We aimed to investigate whether these two models of gaze processing modulate social seeking in typical and autistic adults. We measured preferences of participants to view videos of an actress with visible or hidden eyes, who can or cannot see out. While typical participants preferred videos where the actress can see through and has visible eyes, autistic people showed no preference for these videos. These findings are discussed in the context of perceptual mentalizing and the social motivation theory of autism.

Perceiving Animacy in Own-and Other-Species Faces

Though artificial faces of various kinds are rapidly becoming more and more life-like due to advances in graphics technology (Suwajanakorn et al., 2015; Booth et al., 2017), observers can typically distinguish real faces from artificial faces. In general, face recognition is tuned to experience such that expert-level processing is most evident for faces that we encounter frequently in our visual world, but the extent to which face animacy perception is also tuned to in-group vs. out-group categories remains an open question. In the current study, we chose to examine how the perception of animacy in human faces and dog faces was affected by face inversion and the duration of face images presented to adult observers. We hypothesized that the impact of these manipulations may differ as a function of species category, indicating that face animacy perception is tuned for in-group faces. Briefly, we found evidence of such a differential impact, suggesting either that distinct mechanisms are used to evaluate the "life" in a face for in-group and out-group faces, or that the efficiency of a common mechanism varies substantially as a function of visual expertise.

Brain networks for engaging oneself in positive-social emotion regulation

Positive emotions facilitate cognitive performance, and their absence is associated with burdening psychiatric disorders. However, the brain networks regulating positive emotions are not well understood, especially with regard to engaging oneself in positive-social situations. Here we report convergent evidence from a multimodal approach that includes functional magnetic resonance imaging (fMRI) brain activations, meta-analytic functional characterization, Bayesian model-driven analysis of effective brain connectivity, and personality questionnaires to identify the brain networks mediating the cognitive up-regulation of positive-social emotions. Our comprehensive approach revealed that engaging in positive-social emotion regulation with a self-referential first-person perspective is characterized by dynamic interactions between functionally specialized prefrontal cortex (PFC) areas, the temporoparietal junction (TPJ) and the amygdala. Increased top-down connectivity from the superior frontal gyrus (SFG) controls affective valuation in the ventromedial and dorsomedial PFC, self-referential processes in the TPJ, and modulate emotional responses in the amygdala via the ventromedial PFC. Understanding the brain networks engaged in the regulation of positive-social emotions that involve a first-person perspective is important as they are known to constitute an effective strategy in therapeutic settings.

The eyes know it: Toddlers' visual scanning of sad faces is predicted by their theory of mind skills

The current research explored toddlers' gaze fixation during a scene showing a person expressing sadness after a ball is stolen from her. The relation between the duration of gaze fixation on different parts of the person's sad face (e.g., eyes, mouth) and theory of mind skills was examined. Eye tracking data indicated that before the actor experienced the negative event, toddlers divided their fixation equally between the actor's happy face and other distracting objects, but looked longer at the face after the ball was stolen and she expressed sadness. The strongest predictor of increased focus on the sad face versus other elements of the scene was toddlers' ability to predict others' emotional reactions when outcomes fulfilled (happiness) or failed to fulfill (sadness) desires, whereas toddlers' visual perspective-taking skills predicted their more specific focusing on the actor's eyes and, for boys only, mouth. Furthermore, gender differences emerged in toddlers' fixation on parts of the scene. Taken together, these findings suggest that top-down processes are involved in the scanning of emotional facial expressions in toddlers.

Active Inference and Auditory Hallucinations

Auditory verbal hallucinations (AVH) are often distressing symptoms of several neuropsychiatric conditions, including schizophrenia. Using a Markov decision process formulation of active inference, we develop a novel model of AVH as false (positive) inference. Active inference treats perception as a process of hypothesis testing, in which sensory data are used to disambiguate between alternative hypotheses about the world. Crucially, this depends upon a delicate balance between prior beliefs about unobserved (hidden) variables and the sensations they cause. A false inference that a voice is present, even in the absence of auditory sensations, suggests that prior beliefs dominate perceptual inference. Here we consider the computational mechanisms that could cause this imbalance in perception. Through simulation, we show that the content of (and confidence in) prior beliefs depends on beliefs about policies (here sequences of listening and talking) and on beliefs about the reliability of sensory data. We demonstrate several ways in which hallucinatory percepts could occur when an agent expects to hear a voice in the presence of imprecise sensory data. This model expresses, in formal terms, alternative computational mechanisms that underwrite AVH and, speculatively, can be mapped onto neurobiological changes associated with schizophrenia. The interaction of action and perception is important in modeling AVH, given that speech is a fundamentally enactive and interactive process-and that hallucinators often actively engage with their voices.

Can Monetary Reward Modulate Social Attention?

Selective visual attention is a primary cognitive function, which allows the selection of the most relevant stimuli in the environment by prioritizing their processing. Several studies showed that this process can be influenced by both social signals, such as gaze direction (i.e., the Gaze Cueing Effect, GCE) and by the motivational valence of gratifying stimuli, such as monetary rewards. The aim of this study was to explore whether GCE could be modulated by a monetary reward. To this end, we created an experiment in which participants performed a gaze cuing task before and after an implicit learning task aiming to induce an association between gaze direction and monetary reward (experimental condition), or after a perceptual task (control condition). Statistical analyses were conducted following both a frequentist and a Bayesian approach. Results supported previous findings showing the presence of the GCE, i.e., faster responses in congruent trials when the target appeared in the gazed-at location. Interestingly, our results did not reveal significant differences among the conditions. Therefore, contrary to what was reported by previous attentional orienting studies with non-social stimuli, monetary reward does not seem to be able to modulate (or interfere with) the orienting of attention mediated by gaze direction as measured by the GCE. Taken together our results suggest that social signals such as gaze direction have a greater impact than monetary reward in orienting selective attention.

Adaptation to the Direction of Others' Gaze: A Review

The direction of another person’s gaze provides us with a strong cue to their intentions and future actions, and, correspondingly, the human visual system has evolved to extract information about others’ gaze from the sensory stream. The perception of gaze is a remarkably plastic process: adaptation to a particular direction of gaze over a matter of seconds or minutes can cause marked aftereffects in our sense of where other people are looking. In this review, we first discuss the measurement, specificity, and neural correlates of gaze aftereffects. We then examine how studies that have explored the perceptual and neural determinants of gaze aftereffects have provided key insights into the nature of how other people’s gaze direction is represented within the visual hierarchy. This includes the level of perceptual representation of gaze direction (e.g., relating to integrated vs. local facial features) and the interaction of this system with higher-level social-cognitive functions, such as theory of mind. Moreover, computational modeling of data from behavioral studies of gaze adaptation allows us to make inferences about the functional principles that govern the neural encoding of gaze direction. This in turn provides a foundation for testing computational theories of neuropsychiatric conditions in which gaze processing is compromised, such as autism.

The Role of Pattern Extrapolation in the Perception of Dynamic Facial Expressions in Autism Spectrum Disorder

Changes in the intensity and type of facial expressions reflect alterations in the emotional state of the agent. Such "direct" access to the other's affective state might, top-down, influence the perception of the facial expressions that gave rise to the affective state inference. Previously, we described a perceptual bias occurring when the last, neutral, expression of offsets of facial expressions (joy-to-neutral and anger-to-neutral), was evaluated. Individuals with high-functioning autism (HFA) and matched typically developed (TD) individuals rated the neutral expression at the end of the joy-offset videos as slightly angry and the identical neutral expression at the end of the anger-offset videos as slightly happy ("overshoot" bias). That study suggested that the perceptual overshoot response bias in the TD group could be best explained by top-down "emotional anticipation," i.e., the involuntary/automatic anticipation of the agent's next emotional state of mind, generated by the immediately preceding perceptual history (low-level mind reading). The experimental manipulations further indicated that in the HFA group the "overshoot" was better explained by contrast effects between the first and last facial expressions, both presented for a relatively long period of 400 ms. However, in principle, there is another, more parsimonious, explanation, which is pattern extrapolation or representational momentum (RM): the extrapolation of a pattern present in the dynamic sequence. This hypothesis is tested in the current study, in which 18 individuals with HFA and a matched control group took part. In a base-line condition, joy-offset and anger-offset video-clips were presented. In the new experimental condition, the clips were modified so as to create an offset-onset-offset pattern within each sequence (joy-to-anger-to-neutral and anger-to-joy-to-neutral). The final neutral expressions had to be evaluated. The overshoot bias was confirmed in the base-line condition for both TD and HFA groups, while the experimental manipulation removed the bias in both groups. This outcome ruled out pattern extrapolation or RM as explanation for the perceptual "overshoot" bias in the HFA group and suggested a role for facial contrast effects in HFA. This is compatible with the view that ASD individuals tend to lack the spontaneous "tracking" of changes in the others' affective state and hence show no or reduced emotional anticipation.

From automata to animate beings: the scope and limits of attributing socialness to artificial agents

Understanding the mechanisms and consequences of attributing socialness to artificial agents has important implications for how we can use technology to lead more productive and fulfilling lives. Here, we integrate recent findings on the factors that shape behavioral and brain mechanisms that support social interactions between humans and artificial agents. We review how visual features of an agent, as well as knowledge factors within the human observer, shape attributions across dimensions of socialness. We explore how anthropomorphism and dehumanization further influence how we perceive and interact with artificial agents. Based on these findings, we argue that the cognitive reconstruction within the human observer is likely to be far more crucial in shaping our interactions with artificial agents than previously thought, while the artificial agent's visual features are possibly of lesser importance. We combine these findings to provide an integrative theoretical account based on the "like me" hypothesis, and discuss the key role played by the Theory-of-Mind network, especially the temporal parietal junction, in the shift from mechanistic to social attributions. We conclude by highlighting outstanding questions on the impact of long-term interactions with artificial agents on the behavioral and brain mechanisms of attributing socialness to these agents.

The influence of group membership on cross-contextual imitation

Research on mimicry has demonstrated that individuals imitate in-group members more strongly than out-group members. In the present study, we tested whether such top-down modulation also applies for more extreme forms of direct mapping, such as for cross-contextual imitation settings, in which individuals imitate others' movements without sharing a common goal or context. Models on self-other control suggest that top-down modulations are based merely on a direct link between social sensory processing and imitation. That is, perceived similarities between oneself and another person is sufficient to amplify a shared representation between own and others' actions, which then trigger imitation. However, motivational accounts explain such findings with the assumption that individuals are motivated to affiliate with others. Because imitation is linked to positive social consequences, individuals should imitate in-group members more strongly than out-group members. We tested these two theoretical accounts against each other by applying a cross-contextual imitation paradigm. The results demonstrate that in-group members are more strongly cross-contextually imitated than out-group members the higher individuals' motivation to affiliate with the in-group is. This supports motivational models but not self-other control accounts. Further theoretical implications are discussed.

A Social-Interactive Neuroscience Approach to Understanding the Developing Brain

From birth onward, social interaction is central to our everyday lives. Our ability to seek out social partners, flexibly navigate and learn from social interactions, and develop social relationships is critically important for our social and cognitive development and for our mental and physical health. Despite the importance of our social interactions, the neurodevelopmental bases of such interactions are underexplored, as most research examines social processing in noninteractive contexts. We begin this chapter with evidence from behavioral work and adult neuroimaging studies demonstrating how social-interactive context fundamentally alters cognitive and neural processing. We then highlight four brain networks that play key roles in social interaction and, drawing on existing developmental neuroscience literature, posit the functional roles these networks may play in social-interactive development. We conclude by discussing how a social-interactive neuroscience approach holds great promise for advancing our understanding of both typical and atypical social development.

Social Beliefs and Visual Attention: How the Social Relevance of a Cue Influences Spatial Orienting

We are highly tuned to each other's visual attention. Perceiving the eye or hand movements of another person can influence the timing of a saccade or the reach of our own. However, the explanation for such spatial orienting in interpersonal contexts remains disputed. Is it due to the social appearance of the cue—a hand or an eye—or due to its social relevance—a cue that is connected to another person with attentional and intentional states? We developed an interpersonal version of the Posner spatial cueing paradigm. Participants saw a cue and detected a target at the same or a different location, while interacting with an unseen partner. Participants were led to believe that the cue was either connected to the gaze location of their partner or was generated randomly by a computer (Experiment 1), and that their partner had higher or lower social rank while engaged in the same or a different task (Experiment 2). We found that spatial cue‐target compatibility effects were greater when the cue related to a partner's gaze. This effect was amplified by the partner's social rank, but only when participants believed their partner was engaged in the same task. Taken together, this is strong evidence in support of the idea that spatial orienting is interpersonally attuned to the social relevance of the cue—whether the cue is connected to another person, who this person is, and what this person is doing—and does not exclusively rely on the social appearance of the cue. Visual attention is not only guided by the physical salience of one's environment but also by the mental representation of its social relevance.

Longitudinal reciprocity between theory of mind and aggression in middle childhood

Theory of mind is one of the most important cognitive factors in social information-processing, and deficits in theory of mind have been linked to aggressive behavior in childhood. The present longitudinal study investigated reciprocal links between theory of mind and two forms of aggression – physical and relational – in middle childhood with three data waves over 3 years. Theory of mind was assessed by participants’ responses to cartoons, and physical and relational aggression were assessed through teacher reports in a community sample of 1657 children (mean age at Time 1: 8 years). Structural equation modeling analyses showed that theory of mind was a negative predictor of subsequent physical and relational aggression, both from Time 1 to Time 2 as well as from Time 2 to Time 3. Moreover, relational aggression was a negative predictor of theory of mind from Time 1 to Time 2. There were no significant gender or age differences in the tested pathways. The results suggest that reciprocal and negative longitudinal relations exist between children’s theory of mind and aggressive behavior. Our study extends current knowledge about the development of such relations across middle childhood.

Spontaneous Perspective Taking in Humans?

A number of social cognition studies posit that humans spontaneously compute the viewpoint of other individuals. This is based on experiments showing that responses are shorter when a human agent, located in a visual display, can see the stimuli relevant to the observer's task. Similarly, responses are slower when the agent cannot see the task-relevant stimuli. We tested the spontaneous perspective taking theory by incorporating it within two classic visual cognition paradigms (i.e., the flanker effect and the Simon effect), as well as reassessing its role in the gaze cueing effect. Results showed that these phenomena (e.g., the Simon effect) are not modulated according to whether a gazing agent can see the critical stimuli or not. We also examined the claim that previous results attributed to spontaneous perspective taking are due to the gazing agent's ability to shift attention laterally. Results found no evidence of this. Overall, these data challenge both the spontaneous perspective taking theory, as well as the attentional shift hypothesis.

Backward-walking biological motion orients attention to moving away instead of moving toward

Walking direction is an important attribute of biological motion because it carries key information, such as the specific intention of the walker. Although it is known that spatial attention is guided by walking direction, it remains unclear whether this attentional shift is reflexive (i.e., constantly shifts to the walking direction) or not. A richer interpretation of this effect is that attention is guided to seek the information that is necessary to understand the motion. To investigate this issue, we examined how backward-walking biological motion orients attention because the intention of walking backward is usually to avoid something that walking forward would encounter. The results showed that attention was oriented to the walking-away direction of biological motion instead of the walking-toward direction (Experiment 1), and this effect was not due to the gaze direction of biological motion (Experiment 2). Our findings suggest that the attentional shift triggered by walking direction is not reflexive, thus providing support for the rich interpretation of these attentional effects.