Attention AND mentalizing? Reframing a debate on social orienting of attention

Unmasking social attention: The key distinction between social and non-social attention emerges in disengagement, not engagement

The debate surrounding whether social and non-social attention share the same mechanism has been contentious. While prior studies predominantly focused on engagement, we examined the potential disparity between social and non-social attention from both perspectives of engagement and disengagement, respectively. We developed a two-stage attention-shifting paradigm to capture both attention engagement and disengagement. Combining results from five eye-tracking experiments, we supported that the disengagement of social attention markedly outpaces that of non-social attention, while no significant discrepancy emerges in engagement. We uncovered that the faster disengagement of social attention came from its social nature by eliminating alternative explanations including broader fixation distribution width, reduced directional salience in the peripheral visual field, decreased cue-object categorical consistency, reduced perceived validity, and faster processing time. Our study supported that the distinction between social and non-social attention is rooted in attention disengagement, not engagement.

Tools and methods to study and replicate experiments addressing human social cognition in interactive scenarios

In the last decade, scientists investigating human social cognition have started bringing traditional laboratory paradigms more "into the wild" to examine how socio-cognitive mechanisms of the human brain work in real-life settings. As this implies transferring 2D observational paradigms to 3D interactive environments, there is a risk of compromising experimental control. In this context, we propose a methodological approach which uses humanoid robots as proxies of social interaction partners and embeds them in experimental protocols that adapt classical paradigms of cognitive psychology to interactive scenarios. This allows for a relatively high degree of "naturalness" of interaction and excellent experimental control at the same time. Here, we present two case studies where our methods and tools were applied and replicated across two different laboratories, namely the Italian Institute of Technology in Genova (Italy) and the Agency for Science, Technology and Research in Singapore. In the first case study, we present a replication of an interactive version of a gaze-cueing paradigm reported in Kompatsiari et al. (J Exp Psychol Gen 151(1):121-136, 2022). The second case study presents a replication of a "shared experience" paradigm reported in Marchesi et al. (Technol Mind Behav 3(3):11, 2022). As both studies replicate results across labs and different cultures, we argue that our methods allow for reliable and replicable setups, even though the protocols are complex and involve social interaction. We conclude that our approach can be of benefit to the research field of social cognition and grant higher replicability, for example, in cross-cultural comparisons of social cognition mechanisms.

Co-perceiving: Bringing the social into perception

Humans and other animals possess the remarkable ability to effectively navigate a shared perceptual environment by discerning which objects and spaces are perceived by others and which remain private to themselves. Traditionally, this capacity has been encapsulated under the umbrella of joint attention or joint action. In this comprehensive review, we advocate for a broader and more mechanistic understanding of this phenomenon, termed co-perception. Co-perception encompasses the sensitivity to the perceptual engagement of others and the capability to differentiate between objects perceived privately and those perceived commonly with others. It represents a distinct concept from mere simultaneous individual perception. Moreover, discerning between private and common objects doesn't necessitate intricate mind-reading abilities or mutual coordination. The act of perceiving objects as either private or common provides a comprehensive account for social scenarios where individuals simply share the same context or may even engage in competition. This conceptual framework encourages a re-examination of classical paradigms that demonstrate social influences on perception. Furthermore, it suggests that the impacts of shared experiences extend beyond affective responses, also influencing perceptual processes. This article is categorized under: Psychology > Attention Philosophy > Foundations of Cognitive Science Philosophy > Psychological Capacities.

I prefer what you can see: The role of visual perspective-taking on the gaze-liking effect

Individuals' gaze on an object usually leads others to prefer that object, which is called the gaze-liking effect. However, it is still unclear whether this effect is driven by social factors (i.e., visual perspective-taking) or the domain-general processing (i.e., attention cueing). This research explored the mechanism of the gaze-liking effect by manipulating the objects' visibility to an avatar in six online one-shot experiments. The results showed that participants' affective evaluation for the object was modulated by the avatar's visual perspective. Specifically, the visible object to the avatar received a higher rating of liking degree. However, when the avatar was replaced with a non-social stimulus, the experimental effect was absent. Furthermore, the gaze-liking effect was robust while controlling for confounding factors such as the distance between the object and the avatar or type of stimuli. These findings provided convincing evidence that the gaze-liking effect involves a process of the other's visual experience and is not merely a by-product of the gaze-cueing effect.

Spontaneous perspective-taking in real-time language comprehension: evidence from eye-movements and grain of coordination

Linguistic communication requires interlocutors to consider differences in each other's knowledge (perspective-taking). However, perspective-taking might either be spontaneous or strategic. We monitored listeners' eye movements in a referential communication task. A virtual speaker gave temporally ambiguous instructions with scalar adjectives ("big" in "big cubic block"). Scalar adjectives assume a contrasting object (a small cubic block). We manipulated whether the contrasting object (a small triangle) for a competitor object (a big triangle) was in common ground (visible to both speaker and listener) or was occluded so it was in the listener's privileged ground, in which case perspective-taking would allow earlier reference-resolution. We used a complex visual context with multiple objects, making strategic perspective-taking unlikely when all objects are in the listener's referential domain. A turn-taking, puzzle-solving task manipulated whether participants could anticipate a more restricted referential domain. Pieces were either confined to a small area (requiring fine-grained coordination) or distributed across spatially distinct regions (requiring only coarse-grained coordination). Results strongly supported spontaneous perspective-taking: Although comprehension was less time-locked in the coarse-grained condition, participants in both conditions used perspective information to identify the target referent earlier when the competitor contrast was in privileged ground, even when participants believed instructions were computer-generated.

Gaze-Triggered Communicative Intention Compresses Perceived Temporal Duration

Eye gaze communicates a person's attentional state and intentions toward objects. Here we demonstrate that this important social signal has the potential to distort time perception of gazed-at objects (N = 70 adults). By using a novel gaze-associated learning paradigm combined with the time-discrimination task, we showed that objects previously associated with others' eye gaze were perceived as significantly shorter in duration than the nonassociated counterparts. The time-compression effect cannot be attributed to general attention allocation because it disappeared when objects were associated with nonsocial attention cues (i.e., arrows). Critically, this effect correlated with observers' autistic traits and vanished when the gazing agent's line of sight was blocked by barriers, reflecting the key role of intention processing triggered by gaze in modulating time perception. Our findings support the existence of a special mechanism tuned to social cues, which can shape our perception of the outer world in time domains.

Cultural differences in joint attention and engagement in mutual gaze with a robot face

Joint attention is a pivotal mechanism underlying human ability to interact with one another. The fundamental nature of joint attention in the context of social cognition has led researchers to develop tasks that address this mechanism and operationalize it in a laboratory setting, in the form of a gaze cueing paradigm. In the present study, we addressed the question of whether engaging in joint attention with a robot face is culture-specific. We adapted a classical gaze-cueing paradigm such that a robot avatar cued participants' gaze subsequent to either engaging participants in eye contact or not. Our critical question of interest was whether the gaze cueing effect (GCE) is stable across different cultures, especially if cognitive resources to exert top-down control are reduced. To achieve the latter, we introduced a mathematical stress task orthogonally to the gaze cueing protocol. Results showed larger GCE in the Singapore sample, relative to the Italian sample, independent of gaze type (eye contact vs. no eye contact) or amount of experienced stress, which translates to available cognitive resources. Moreover, since after each block, participants rated how engaged they felt with the robot avatar during the task, we observed that Italian participants rated as more engaging the avatar during the eye contact blocks, relative to no eye contact while Singaporean participants did not show any difference in engagement relative to the gaze. We discuss the results in terms of cultural differences in robot-induced joint attention, and engagement in eye contact, as well as the dissociation between implicit and explicit measures related to processing of gaze.

Research on social attention in autism and the challenges of the research domain criteria (RDoC) framework

The fuzzy nature of categories of psychopathology, such as autism, leads to significant research challenges. Alternatively, focusing research on the study of a common set of important and well-defined psychological constructs across psychiatric conditions may make the fundamental etiological processes of psychopathology easier to discern and treat (Cuthbert, 2022). The development of the research domain criteria (RDoC) framework is designed to guide this new research approach (Insel et al., 2010). However, progress in research may be expected to continually refine and reorganize the understanding of the specifics of these mental processes (Cuthbert & Insel, 2013). Moreover, knowledge gleaned from the study of both normative and atypical development can be mutually informative in the evolution of our understanding of these fundamental processes. A case in point is the study of social attention. This Autism 101 commentary provides an educational summary of research over the last few decades indicates that social attention is major construct in the study of human social-cognitive development, autism and other forms of psychopathology. The commentary also describes how this research can inform the Social Process dimension of the RDoC framework.

Spontaneous perspective-taking and its relation to schizotypy

INTRODUCTION

Patients with schizophrenia differ from healthy controls in the extent that they spontaneously take another's perspective. For such effects, it is difficult to separate the influence of schizophrenia from multiple potential confounders. Here, for the first time, associations between spontaneous perspective-taking and schizotypy were investigated in a nonclinical population.

METHODS

Adult participants completed both a Schizotypal Personality Questionnaire (SPQ-BRU) and a novel online adaptation of a visual perspective-taking task that required participants to make judgements both from their own perspective and that of a human avatar.

RESULTS

Response times were elevated when the avatar's perspective was inconsistent with that of the participant, providing evidence of spontaneous perspective-taking. This demonstrates that the visual perspective-taking task can be successfully implemented in an online format. However, schizotypy did not predict these spontaneous perspective-taking effects.

CONCLUSIONS

Unlike explicit mentalising, this form of implicit mentalising is not affected by nonclinical manifestations of schizotypy traits. This implies that impairment of general neurocognitive function contributes to altered spontaneous perspective-taking in schizophrenia. A novel account based on the cognitive control processes involved in perspective selection and the role of attention in perspective calculation reconciles apparently contradictory findings of earlier studies comparing patients with schizophrenia and healthy controls.

Both the domain-general and the mentalising processes affect visual perspective taking

People's attention cannot help being affected by what others are looking at. The dot-perspective task has been often employed to investigate this visual attentional shift. In this task, participants are presented with virtual scenes with a cue facing some targets and must judge how many targets are visible from their own or the cue perspective. Typically, this task shows an interference pattern: Participants record slower reaction times (RTs) and more errors when the cue is facing away from the targets. Interestingly, this occurs also when participants take their own perspective. Two accounts contend the explanation of this interference. The mentalising account focuses on the social relevance of the cue, while the domain-general account focuses on the directional features of the cue. To investigate the relative contribution of the two accounts, we developed a Social_Only cue, a cue having only social features and compared its effects with a Social+Directional cue, which had both social and directional features. Results show that while the Social+Directional cue generates the typical interference pattern, the Social_Only cue does not generate interference in the RTs, only in the error rate. We advance an integration between the mentalising and the domain-general accounts. We suggest that the dot-perspective task requires two processes: an orienting process, elicited by the directional features of the cue and measured by the RTs, and a decisional process elicited by the social features of the cue and measured also by the error rate.

Emotion recognition in autism spectrum disorder across age groups: A cross-sectional investigation of various visual and auditory communicative domains

Previous research on emotion processing in autism spectrum disorder (ASD) has predominantly focused on human faces and speech prosody, with little attention paid to other domains such as nonhuman faces and music. In addition, emotion processing in different domains was often examined in separate studies, making it challenging to evaluate whether emotion recognition difficulties in ASD generalize across domains and age cohorts. The present study investigated: (i) the recognition of basic emotions (angry, scared, happy, and sad) across four domains (human faces, face-like objects, speech prosody, and song) in 38 autistic and 38 neurotypical (NT) children, adolescents, and adults in a forced-choice labeling task, and (ii) the impact of pitch and visual processing profiles on this ability. Results showed similar recognition accuracy between the ASD and NT groups across age groups for all domains and emotion types, although processing speed was slower in the ASD compared to the NT group. Age-related differences were seen in both groups, which varied by emotion, domain, and performance index. Visual processing style was associated with facial emotion recognition speed and pitch perception ability with auditory emotion recognition in the NT group but not in the ASD group. These findings suggest that autistic individuals may employ different emotion processing strategies compared to NT individuals, and that emotion recognition difficulties as manifested by slower response times may result from a generalized, rather than a domain-specific underlying mechanism that governs emotion recognition processes across domains in ASD.

Explicit vs. implicit spatial processing in arrow vs. eye-gaze spatial congruency effects

Arrows and gaze stimuli lead to opposite spatial congruency effects. While standard congruency effects are observed for arrows (faster responses for congruent conditions), responses are faster when eye-gaze stimuli are presented on the opposite side of the gazed-at location (incongruent trials), leading to a reversed congruency effect (RCE). Here, we explored the effects of implicit vs. explicit processing of arrows and eye-gaze direction. Participants were required to identify the direction (explicit task) or the colour (implicit task) of left or right looking/pointing gaze or arrows, presented to either the left or right of the fixation point. When participants responded to the direction of stimuli, standard congruency effects for arrows and RCE for eye-gaze stimuli were observed. However, when participants responded to the colour of stimuli, no congruency effects were observed. These results suggest that it is necessary to explicitly pay attention to the direction of eye-gaze and arrows for the congruency effect to occur. The same pattern of data was observed when participants responded either manually or verbally, demonstrating that manual motor components are not responsible for the results observed. These findings are not consistent with some hypotheses previously proposed to explain the RCE observed with eye-gaze stimuli and, therefore, call for an alternative plausible hypothesis.

Are there quantitative differences between eye-gaze and arrow cues? A meta-analytic answer to the debate and a call for qualitative differences

Gaze acts from an early age as a cue to orient attention and, thereafter, to infer our social partners' intentions, thoughts, and emotions. Variants of the attentional orienting paradigm have been used to study the orienting capabilities associated to eye gaze. However, to date, it is still unclear whether this methodology truly assesses "social-specific" processes exclusively involved in attention to eye-gaze or the operation of domain-general attentional processes. The present study provides a comprehensive meta-analysis indicating that eye-gaze and non-social directional stimuli, such as arrows, produce equivalent attentional effects. This result casts doubt on the potential utility of the classic cueing task in revealing social-specific processes. On the other hand, we review behavioral evidence suggesting that eye-gaze stimuli may induce higher-order social processes when more specific experimental procedures that analyze qualitative rather than quantitative differences are used. These findings point to an integrated view in which domain-general and social specific processes both contribute to the attentional mechanisms induced by eye-gaze direction. Finally, some proposals about the social components specifically triggered by eye-gaze stimuli are discussed.

The (un)learning of social functions and its significance for mental health

Social interactions are dynamic, context-dependent, and reciprocal events that influence prospective strategies and require constant practice and adaptation. This complexity of social interactions creates several research challenges. We propose a new framework encouraging future research to investigate not only individual differences in capacities relevant for social functioning and their underlying mechanisms, but also the flexibility to adapt or update one's social abilities. We suggest three key capacities relevant for social functioning: (1) social perception, (2) sharing emotions or empathizing, and (3) mentalizing. We elaborate on how adaptations in these capacities may be investigated on behavioral and neural levels. Research on these flexible adaptations of one's social behavior is needed to specify how humans actually "learn to be social". Learning to adapt implies plasticity of the relevant brain networks involved in the underlying social processes, indicating that social abilities are malleable for different contexts. To quantify such measures, researchers need to find ways to investigate learning through dynamic changes in adaptable social paradigms and examine several factors influencing social functioning within the three aformentioned social key capacities. This framework furthers insight concerning individual differences, provides a holistic approach to social functioning, and may improve interventions for ameliorating social abilities in patients.

Direct Gaze Holds Attention, but Not in Individuals with Obsessive-Compulsive Disorder

The attentional response to eye-gaze stimuli is still largely unexplored in individuals with obsessive-compulsive disorder (OCD). Here, we focused on an attentional phenomenon according to which a direct-gaze face can hold attention in a perceiver. Individuals with OCD and a group of matched healthy controls were asked to discriminate, through a speeded manual response, a peripheral target. Meanwhile, a task-irrelevant face displaying either direct gaze (in the eye-contact condition) or averted gaze (in the no-eye-contact condition) was also presented at the centre of the screen. Overall, the latencies were slower for faces with direct gaze than for faces with averted gaze; however, this difference was reliable in the healthy control group but not in the OCD group. This suggests the presence of an unusual attentional response to direct gaze in this clinical population.

Rapid detection of social interactions is the result of domain general attentional processes

Using visual search displays of interacting and non-interacting pairs, it has been demonstrated that detection of social interactions is facilitated. For example, two people facing each other are found faster than two people with their backs turned: an effect that may reflect social binding. However, recent work has shown the same effects with non-social arrow stimuli, where towards facing arrows are detected faster than away facing arrows. This latter work suggests a primary mechanism is an attention orienting process driven by basic low-level direction cues. However, evidence for lower level attentional processes does not preclude a potential additional role of higher-level social processes. Therefore, in this series of experiments we test this idea further by directly comparing basic visual features that orient attention with representations of socially interacting individuals. Results confirm the potency of orienting of attention via low-level visual features in the detection of interacting objects. In contrast, there is little evidence for the representation of social interactions influencing initial search performance.

Irrelevant Robot Signals in a Categorization Task Induce Cognitive Conflict in Performance, Eye Trajectories, the N2 ERP-EEG Component, and Frontal Theta Oscillations

Understanding others' nonverbal behavior is essential for social interaction, as it allows, among others, to infer mental states. Although gaze communication, a well-established nonverbal social behavior, has shown its importance in inferring others' mental states, not much is known about the effects of irrelevant gaze signals on cognitive conflict markers during collaborative settings. Here, participants completed a categorization task where they categorized objects based on their color while observing images of a robot. On each trial, participants observed the robot iCub grasping an object from a table and offering it to them to simulate a handover. Once the robot "moved" the object forward, participants were asked to categorize the object according to its color. Before participants were allowed to respond, the robot made a lateral head/gaze shift. The gaze shifts were either congruent or incongruent with the object's color. We expected that incongruent head cues would induce more errors (Study 1), would be associated with more curvature in eye-tracking trajectories (Study 2), and induce larger amplitude in electrophysiological markers of cognitive conflict (Study 3). Results of the three studies show more oculomotor interference as measured in error rates (Study 1), larger curvatures eye-tracking trajectories (Study 2), and higher amplitudes of the N2 ERP of the EEG signals as well as higher event-related spectral perturbation amplitudes (Study 3) for incongruent trials compared with congruent trials. Our findings reveal that behavioral, ocular, and electrophysiological markers can index the influence of irrelevant signals during goal-oriented tasks.

Neuropsychological and socio-cognitive deficits in patients with obstructive sleep apnea

Introduction: Patients with obstructive sleep apnea (OSA) suffer from several neurocognitive deficits. We investigated the cognitive and socio-cognitive profiles of patients with severe OSA, controlling for potentially relevant mediating variables (i.e. age, body-mass index, cognitive reserve and depression). Moreover, we studied the neuropsychological profile of a high-risk OSA phenotype characterized by severe OSA and severe nocturnal hypoxemia.Method: We assessed 29 previously untreated severe OSA patients with a mean age of 55.6 (± 9.9 years) and a mean apnea-hypopnea index (AHI) of 53.1 (± 17.4). A control group of 34 healthy participants was also enrolled. Participants completed an extensive neuropsychological battery that included social cognition, a relatively new investigation area among OSA patients.Data analysis: Data were analyzed with a Bayesian approach. Specifically, Bayesian ANCOVA was used to investigate whether the grouping variable could predict test performance. Age, body-mass index, cognitive reserve and state of depression were added as covariates to the null model to weight the effects of these potential confounding factors. Three groups were analyzed: healthy controls (H), OSA with severe apnea and severe nocturnal oxygen desaturation (D⁺), and OSA with severe apnea non-desaturators (D^-). Performances on the various neuropsychological tests were treated as the dependent variables.Results: The results indicate that non-verbal reasoning, the theory of mind skills, and mental shifting ability were impaired in OSA patients. Patients with severe nocturnal hypoxemia underperformed compared to patients with the same severity of apnea but non-desaturators. Additionally, we observed a trend toward a worse performance among OSA desaturator patients in the following abilities: constructional ability, short term verbal memory, phonological fluency, and the ability to inhibit automatic and dominant responses.Conclusion: The data suggest a key role of hypoxemia in affecting cognitive functioning in OSA patients. Executive functions and the concomitant involvement of social cognition are particularly affected.

The Gaze Cueing Effect and Its Enhancement by Facial Expressions Are Impacted by Task Demands: Direct Comparison of Target Localization and Discrimination Tasks

The gaze cueing effect is characterized by faster attentional orienting to a gazed-at than a non-gazed-at target. This effect is often enhanced when the gazing face bears an emotional expression, though this finding is modulated by a number of factors. Here, we tested whether the type of task performed might be one such modulating factor. Target localization and target discrimination tasks are the two most commonly used gaze cueing tasks, and they arguably differ in cognitive resources, which could impact how emotional expression and gaze cues are integrated to orient attention. In a within-subjects design, participants performed both target localization and discrimination gaze cueing tasks with neutral, happy, and fearful faces. The gaze cueing effect for neutral faces was greatly reduced in the discrimination task relative to the localization task, and the emotional enhancement of the gaze cueing effect was only present in the localization task and only when this task was performed first. These results suggest that cognitive resources are needed for gaze cueing and for the integration of emotional expressions and gaze cues. We propose that a shift toward local processing may be the mechanism by which the discrimination task interferes with the emotional modulation of gaze cueing. The results support the idea that gaze cueing can be greatly modulated by top-down influences and cognitive resources and thus taps into endogenous attention. Results are discussed within the context of the recently proposed EyeTune model of social attention.

Prior attentional bias is modulated by social gaze

Focusing attention is a key cognitive skill, but how the gaze of others affects engaged attention remains relatively unknown. We investigated if participants' attentional bias toward a location is modulated by the number of people gazing toward or away from it. We presented participants with a nonpredictive directional cue that biased attention towards a specific location. Then, any number of four stimulus faces turned their gaze toward or away from the attended location. When all the faces looked at the attended location participants increased their commitment to it, and response time to targets at that location were speeded. When most or all of the faces looked away from the attended location, attention was withdrawn, and response times were slowed. This study reveals that the gaze of others can penetrate one's ability to focus attention, which in turn can be both beneficial and costly to one's responses to events in the environment.

Pre-exposure to Ambiguous Faces Modulates Top-Down Control of Attentional Orienting to Counterpredictive Gaze Cues

Understanding and reacting to others' nonverbal social signals, such as changes in gaze direction (i.e., gaze cue), are essential for social interactions, as it is important for processes such as joint attention and mentalizing. Although attentional orienting in response to gaze cues has a strong reflexive component, accumulating evidence shows that it can be top-down controlled by context information regarding the signals' social relevance. For example, when a gazer is believed to be an entity "with a mind" (i.e., mind perception), people exert more top-down control on attention orienting. Although increasing an agent's physical human-likeness can enhance mind perception, it could have negative consequences on top-down control of social attention when a gazer's physical appearance is categorically ambiguous (i.e., difficult to categorize as human or nonhuman), as resolving this ambiguity would require using cognitive resources that otherwise could be used to top-down control attention orienting. To examine this question, we used mouse-tracking to explore if categorically ambiguous agents are associated with increased processing costs (Experiment 1), whether categorically ambiguous stimuli negatively impact top-down control of social attention (Experiment 2), and if resolving the conflict related to the agent's categorical ambiguity (using exposure) would restore top-down control to orient attention (Experiment 3). The findings suggest that categorically ambiguous stimuli are associated with cognitive conflict, which negatively impact the ability to exert top-down control on attentional orienting in a counterpredicitive gaze-cueing paradigm; this negative impact, however, is attenuated when being pre-exposed to the stimuli prior to the gaze-cueing task. Taken together, these findings suggest that manipulating physical human-likeness is a powerful way to affect mind perception in human-robot interaction (HRI) but has a diminishing returns effect on social attention when it is categorically ambiguous due to drainage of cognitive resources and impairment of top-down control.

Gaze deflection reveals how gaze cueing is tuned to extract the mind behind the eyes

Suppose you are surreptitiously looking at someone, and then when they catch you staring at them, you immediately turn away. This is a social phenomenon that almost everyone experiences occasionally. In such experiences-which we will call gaze deflection-the "deflected" gaze is not directed at anything in particular but simply away from the other person. As such, this is a rare instance where we may turn to look in a direction without intending to look there specifically. Here we show that gaze cues are markedly less effective at orienting an observer's attention when they are seen as deflected in this way-even controlling for low-level visual properties. We conclude that gaze cueing is a sophisticated mental phenomenon: It is not merely driven by perceived eye or head motions but is rather well tuned to extract the "mind" behind the eyes.