Visual cognition during real social interaction

Age Differences in Gaze Following: Older Adults Follow Gaze More than Younger Adults When free-viewing Scenes

Previous research investigated age differences in gaze following with an attentional cueing paradigm where participants view a face with averted gaze, and then respond to a target appearing in a location congruent or incongruent with the gaze cue. However, this paradigm is far removed from the way we use gaze cues in everyday settings. Here we recorded the eye movements of younger and older adults while they freely viewed naturalistic scenes where a person looked at an object or location. Older adults were more likely to fixate and made more fixations to the gazed-at location, compared to younger adults. Our findings suggest that, contrary to what was observed in the traditional gaze-cueing paradigm, in a non-constrained task that uses contextualized stimuli older adults follow gaze as much as or even more than younger adults.

Avatars with faces of real people: A construction method for scientific experiments in virtual reality

Experimental psychology research typically employs methods that greatly simplify the real-world conditions within which cognition occurs. This approach has been successful for isolating cognitive processes, but cannot adequately capture how perception operates in complex environments. In turn, real-world environments rarely afford the access and control required for rigorous scientific experimentation. In recent years, technology has advanced to provide a solution to these problems, through the development of affordable high-capability virtual reality (VR) equipment. The application of VR is now increasing rapidly in psychology, but the realism of its avatars, and the extent to which they visually represent real people, is captured poorly in current VR experiments. Here, we demonstrate a user-friendly method for creating photo-realistic avatars of real people and provide a series of studies to demonstrate their psychological characteristics. We show that avatar faces of familiar people are recognised with high accuracy (Study 1), replicate the familiarity advantage typically observed in real-world face matching (Study 2), and show that these avatars produce a similarity-space that corresponds closely with real photographs of the same faces (Study 3). These studies open the way to conducting psychological experiments on visual perception and social cognition with increased realism in VR.

The interplay between gaze and consistency in scene viewing: Evidence from visual search by young and older adults

Searching for an object in a complex scene is influenced by high-level factors such as how much the item would be expected in that setting (semantic consistency). There is also evidence that a person gazing at an object directs our attention towards it. However, there has been little previous research that has helped to understand how we integrate top-down cues such as semantic consistency and gaze to direct attention when searching for an object. Also, there are separate lines of evidence to suggest that older adults may be more influenced by semantic factors and less by gaze cues compared to younger counterparts, but this has not been investigated before in an integrated task. In the current study we analysed eye-movements of 34 younger and 30 older adults as they searched for a target object in complex visual scenes. Younger adults were influenced by semantic consistency in their attention to objects, but were more influenced by gaze cues. In contrast, older adults were more guided by semantic consistency in directing their attention, and showed less influence from gaze cues. These age differences in use of high-level cues were apparent early in processing (time to first fixation and probability of immediate fixation) but not in later processing (total time looking at objects and time to make a response). Overall, this pattern of findings indicates that people are influenced by both social cues and prior expectations when processing a complex scene, and the relative importance of these factors depends on age.

Validation of revised reading the mind in the eyes test in the Indian (Bengali) population: A preliminary study

BACKGROUND

Social cognition deficits are common in clinical populations but there is a dearth of standardized social cognition assessment tools in India. Theory of mind (ToM) is an important aspect of social cognition which is often assessed with the revised reading the mind in eyes test (RMET-R). However, we do not have a statistically validated version of the test for the Indian population.

AIM

This study aims to assess the acceptability, reliability, and validity of the Bengali version of the RMET-R.

MATERIALS AND METHODS

We administered the RMET-R to 23 patients with chronic schizophrenia (SCZ), 22 patients with bipolar disorder, and 104 healthy controls (HCs) to evaluate the reliability and validity of the instrument in the Indian (Bengali) population.

RESULTS

We obtained moderate internal consistency (Cronbach's alpha = 0.6) and test-retest reliability (intraclass correlation coefficient = 0.64, P < 0.001). Positive correlations were found between RMET-R and Wechsler picture arrangement (r = 0.60, P < 0.001), picture completion (r = 0.54, P < 0.001), and comprehension subtests (r = 0.48, P < 0.001). Patients with SCZ (M = 49.7, standard deviation [SD] = 16.5) scored significantly lower than HCs (M = 68.9, SD = 13.8) (P = 0.008; Cohen's d = 1.3) on the RMET-R. Thus this tool could discriminate patients who are reported to have Theory of Mind deficits from healthy controls.

CONCLUSION

The Bengali version of the RMET-R is a reliable and valid tool for assessing first-order ToM insofar as the original RMET-R measures this construct.

Reduced egocentric bias when perspective-taking compared with working from rules

Previous research has suggested that adults are sometimes egocentric, erroneously attributing their current beliefs, perspectives, and opinions to others. Interestingly, this egocentricity is sometimes stronger when perspective-taking than when working from functionally identical but non-perspectival rules. Much of our knowledge of egocentric bias comes from Level 1 perspective-taking (e.g., judging whether something is seen) and judgements made about narrated characters or avatars rather than truly social stimuli such as another person in the same room. We tested whether adults would be egocentric on a Level 2 perspective-taking task (judging how something appears), in which they were instructed to indicate on a continuous colour scale the colour of an object as seen through a filter. In our first experiment, we manipulated the participants’ knowledge of the object’s true colour. We also asked participants to judge either what the filtered colour looked like to themselves or to another person present in the room. We found participants’ judgements did not vary across conditions. In a second experiment, we instead manipulated how much participants knew about the object’s colour when it was filtered. We found that participants were biased towards the true colour of the object when making judgements about targets they could not see relative to targets they could, but that this bias disappeared when the instruction was to imagine what the object looked like to another person. We interpret these findings as indicative of reduced egocentricity when considering other people’s experiences of events relative to considering functionally identical but abstract rules.

The role of transients in action observation

A large number of studies have now described the various ways in which the observation of another person's dynamic movement can influence the speed with which the observer is able to prepare a motor action themselves. The typical results are most often explained with reference to theories that link perception and action. Such theories argue that the cognitive structures associated with each share common representations. Consequently, action preparation and action observation are often said to be functionally equivalent. However, the dominance of these theories in explaining action observation effects has masked the potential contribution from processes associated with the detection of low-level "transients" resulting from observing a body movement, such as motion and sound. In the present review, we describe work undertaken in one particular action observation phenomenon ("social inhibition of return") and show that the transient account provides the best explanation of the effect. We argue that future work should consider attention capture and orienting as a potential contributing factor to action observation effects more broadly.

Searching with and against each other: Spatiotemporal coordination of visual search behavior in collaborative and competitive settings

Although in real life people frequently perform visual search together, in lab experiments this social dimension is typically left out. Here, we investigate individual, collaborative and competitive visual search with visualization of search partners' gaze. Participants were instructed to search a grid of Gabor patches while being eye tracked. For collaboration and competition, searchers were shown in real time at which element the paired searcher was looking. To promote collaboration or competition, points were rewarded or deducted for correct or incorrect answers. Early in collaboration trials, searchers rarely fixated the same elements. Reaction times of couples were roughly halved compared with individual search, although error rates did not increase. This indicates searchers formed an efficient collaboration strategy. Overlap, the proportion of dwells that landed on hexagons that the other searcher had already looked at, was lower than expected from simulated overlap of two searchers who are blind to the behavior of their partner. The proportion of overlapping dwells correlated positively with ratings of the quality of collaboration. During competition, overlap increased earlier in time, indicating that competitors divided space less efficiently. Analysis of the entropy of the dwell locations and scan paths revealed that in the competition condition, a less fixed looking pattern was exhibited than in the collaborate and individual search conditions. We conclude that participants can efficiently search together when provided only with information about their partner's gaze position by dividing up the search space. Competing search exhibited more random gaze patterns, potentially reflecting increased interaction between searchers in this condition.

Visual attention and action: How cueing, direct mapping, and social interactions drive orienting

Despite considerable interest in both action perception and social attention over the last 2 decades, there has been surprisingly little investigation concerning how the manual actions of other humans orient visual attention. The present review draws together studies that have measured the orienting of attention, following observation of another's goal-directed action. Our review proposes that, in line with the literature on eye gaze, action is a particularly strong orienting cue for the visual system. However, we additionally suggest that action may orient visual attention using mechanisms, which gaze direction does not (i.e., neural direct mapping and corepresentation). Finally, we review the implications of these gaze-independent mechanisms for the study of attention to action. We suggest that our understanding of attention to action may benefit from being studied in the context of joint action paradigms, where the role of higher level action goals and social factors can be investigated.

Exposure to attachment narratives dynamically modulates cortical arousal during the resting state in the listener

BACKGROUND

Affective stimulation entails changes in brain network patterns at rest, but it is unknown whether exogenous emotional stimulation has a prolonged effect on the temporal dynamics of endogenous cortical arousal. We therefore investigated differences in cortical arousal in the listener following stimulation with different attachment-related narratives.

METHODS

Resting-state EEG was recorded from sixteen healthy subjects for ten minutes each with eyes closed: first at baseline and then after passively listening to three affective narratives from strangers about their early childhood experiences (prototypical for insecure-dismissing, insecure-preoccupied, and secure attachment). Using the VIGALL 2.1 algorithm, low or high vigilance stages in consecutive EEG segments were classified, and their dynamic profile was analyzed. Questionnaires assessed the listeners' emotional response to the content of the narrative.

RESULTS

As a general effect of preceding affective stimulation, vigilance following the stimulation was significantly elevated compared to baseline rest, and carryover effects in dynamic vigilance profiles were observed. A difference between narrative conditions was revealed for the insecure-dismissing condition, in which the decrease in duration of high vigilance stages was fastest compared to the other two conditions. The behavioral data supported the observation that especially the insecure narratives induced a tendency in the listener to affectively disengage from the narrative content.

DISCUSSION

This study revealed carryover effects in endogenous cortical arousal evoked by preceding affective stimulation and provides evidence for attachment-specific dynamic alterations of brain states and individual differences in emotional reactivity.

The "Social Gaze Space": A Taxonomy for Gaze-Based Communication in Triadic Interactions

Humans substantially rely on non-verbal cues in their communication and interaction with others. The eyes represent a “simultaneous input-output device”: While we observe others and obtain information about their mental states (including feelings, thoughts, and intentions-to-act), our gaze simultaneously provides information about our own attention and inner experiences. This substantiates its pivotal role for the coordination of communication. The communicative and coordinative capacities – and their phylogenetic and ontogenetic impacts – become fully apparent in triadic interactions constituted in its simplest form by two persons and an object. Technological advances have sparked renewed interest in social gaze and provide new methodological approaches. Here we introduce the ‘Social Gaze Space’ as a new conceptual framework for the systematic study of gaze behavior during social information processing. It covers all possible categorical states, namely ‘partner-oriented,’ ‘object-oriented,’ ‘introspective,’ ‘initiating joint attention,’ and ‘responding joint attention.’ Different combinations of these states explain several interpersonal phenomena. We argue that this taxonomy distinguishes the most relevant interactional states along their distinctive features, and will showcase the implications for prominent social gaze phenomena. The taxonomy allows to identify research desiderates that have been neglected so far. We argue for a systematic investigation of these phenomena and discuss some related methodological issues.

Own-age biases in adults’ and children’s joint attention: Biased face prioritization, but not gaze following!

Previous studies have reported own-age biases in younger and older adults in gaze following. We investigated own-age biases in social attentional processes between adults and children by focusing on two aspects of the joint attention process; the extent to which people attend towards an individual’s face, and the extent to which they fixate objects that are looked at by this person (i.e., gaze following). Participants viewed images that always contained a child and an adult who either looked towards each other or each looked at objects located to their side. Observers consistently, and rapidly fixated the actor’s faces, though the children were faster to fixate the child’s face than the adult’s faces, whilst the adults were faster to fixate on the adult’s face than the child’s face. The children also spent significantly more time fixating the child’s face than the adult’s face, and the opposite pattern of results was found for the adults. Whilst both adults and children prioritized objects when they were looked at by the actor, both groups showed equivalent levels of gaze following, and there was no own-age bias for gaze following. Our results show an own-age bias for prioritizing faces, but not gaze following.

Social Beliefs and Visual Attention: How the Social Relevance of a Cue Influences Spatial Orienting

We are highly tuned to each other's visual attention. Perceiving the eye or hand movements of another person can influence the timing of a saccade or the reach of our own. However, the explanation for such spatial orienting in interpersonal contexts remains disputed. Is it due to the social appearance of the cue—a hand or an eye—or due to its social relevance—a cue that is connected to another person with attentional and intentional states? We developed an interpersonal version of the Posner spatial cueing paradigm. Participants saw a cue and detected a target at the same or a different location, while interacting with an unseen partner. Participants were led to believe that the cue was either connected to the gaze location of their partner or was generated randomly by a computer (Experiment 1), and that their partner had higher or lower social rank while engaged in the same or a different task (Experiment 2). We found that spatial cue‐target compatibility effects were greater when the cue related to a partner's gaze. This effect was amplified by the partner's social rank, but only when participants believed their partner was engaged in the same task. Taken together, this is strong evidence in support of the idea that spatial orienting is interpersonally attuned to the social relevance of the cue—whether the cue is connected to another person, who this person is, and what this person is doing—and does not exclusively rely on the social appearance of the cue. Visual attention is not only guided by the physical salience of one's environment but also by the mental representation of its social relevance.

Examining ecological validity in social interaction: problems of visual fidelity, gaze, and social potential

Social interaction is an essential part of the human experience, and much work has been done to study it. However, several common approaches to examining social interactions in psychological research may inadvertently either unnaturally constrain the observed behaviour by causing it to deviate from naturalistic performance, or introduce unwanted sources of variance. In particular, these sources are the differences between naturalistic and experimental behaviour that occur from changes in visual fidelity (quality of the observed stimuli), gaze (whether it is controlled for in the stimuli), and social potential (potential for the stimuli to provide actual interaction). We expand on these possible sources of extraneous variance and why they may be important. We review the ways in which experimenters have developed novel designs to remove these sources of extraneous variance. New experimental designs using a 'two-person' approach are argued to be one of the most effective ways to develop more ecologically valid measures of social interaction, and we suggest that future work on social interaction should use these designs wherever possible.

Real Person Interaction in Visual Attention Research

Abstract. An important development in cognitive psychology in the past decade has been the examination of visual attention during real social interaction. This contrasts traditional laboratory studies of attention, including “social attention,” in which observers perform tasks alone. In this review, we show that although the lone-observer method has been central to attention research, real person interaction paradigms have not only uncovered the processes that occur during “joint attention,” but have also revealed attentional processes previously thought not to occur. Furthermore, the examination of some visual attention processes almost invariably requires the use of real person paradigms. While we do not argue for an increase in “ecological validity” for its own sake, we do suggest that research using real person interaction has greatly benefited the development of visual attention theories.

Altered social attention in anorexia nervosa during real social interaction

The capacity to devote attentional resources in response to body-related signals provided by others is still largely unexplored in individuals with Anorexia Nervosa (AN). Here, we tested this capacity through a novel paradigm that mimics a social interaction with a real partner. Healthy individuals (Experiment 1) and individuals with AN (Experiment 2) completed a task with another person which consisted in performing, alternatively, rapid aiming movements to lateralised targets. Generally, this task leads to a form of Inhibition of Return (IOR), which consists of longer reaction times when an individual has to respond to a location previously searched by either himself (individual IOR) or by the partner (social IOR) as compared to previously unexplored locations. IOR is considered as an important attentional mechanism that promotes an effective exploration of the environment during social interaction. Here, healthy individuals displayed both individual and social IOR that were both reliable and of the same magnitude. Individuals with AN displayed a non-significant individual IOR but a reliable social IOR that was also significantly stronger than individual IOR. These results suggest the presence of a reduced sensitivity in processing body-related stimuli conveyed by oneself in individuals with AN which is reflected in action-based attentional processes.

Two Polarities of Attention in Social Contexts: From Attending-to-Others to Attending-to-Self

Social attention is one special form of attention that involves the allocation of limited processing resources in a social context. Previous studies on social attention often regard how attention is directed toward socially relevant stimuli such as faces and gaze directions of other individuals. In contrast to attending-to-others, a different line of researches has shown that self-related information such as own face and name automatically captures attention and is preferentially processed comparing to other-related information. These contrasting behavioral effects between attending-to-others and attending-to-self prompt me to consider a synthetic viewpoint for understanding social attention. I propose that social attention operates at two polarizing states: In one extreme, individual tends to attend to the self and prioritize self-related information over others', and, in the other extreme, attention is allocated to other individuals to infer their intentions and desires. Attending-to-self and attending-to-others mark the two ends of an otherwise continuum spectrum of social attention. For a given behavioral context, the mechanisms underlying these two polarities will interact and compete with each other in order to determine a saliency map of social attention that guides our behaviors. An imbalanced competition between these two behavioral and cognitive processes will cause cognitive disorders and neurological symptoms such as autism spectrum disorders and Williams syndrome. I have reviewed both behavioral and neural evidence that support the notion of polarized social attention, and have suggested several testable predictions to corroborate this integrative theory for understanding social attention.

When gaze opens the channel for communication: Integrative role of IFG and MPFC

Recent advances in the field of cognitive neuroscience have revealed that direct gaze modulates activity in cortical and subcortical key regions of the 'social brain network', including the inferior frontal gyrus (IFG) and the anterior rostral medial prefrontal cortex (arMPFC). However, very little is known about how direct gaze is processed during live interaction with a real partner. Here, for the first time we used an experimental setup allowing the participant inside an MRI scanner to interact face-to-face with a partner located in the scanner room. Depending on condition, the participant and the partner were instructed either to look at each other in the eyes or to direct their gaze away from the other. As control conditions, participants gazed at their own eyes, reflected in a mirror, or gazed at a picture of the partner's eyes. Results revealed that direct gaze by the partner was associated with activity in areas involved in production and comprehension of language and action, including the IFG, the premotor cortex (PM), and the supplementary motor area (SMA). Activations in these areas were observed regardless of the participant's gaze behavior. In contrast, increased activity in arMPFC, an area involved in inference of other mental states during social interaction and communication, was only observed when the participant reciprocated the partner's direct gaze so as to establish mutual gaze. Psychophysiological interaction (PPI) analysis revealed effective connectivity between the IFG and the arMPFC during mutual gaze. This suggests that, within a larger network concerned with the processing of social gaze, mutual gaze with a real partner is established by an increased coupling between areas involved in the detection of communicative intentions, language, and social interaction.

When your decisions are not (quite) your own: action observation influences free choices

A growing number of studies have begun to assess how the actions of one individual are represented in an observer. Using a variant of an action observation paradigm, four experiments examined whether one person’s behaviour can influence the subjective decisions and judgements of another. In Experiment 1, two observers sat adjacent to each other and took turns to freely select and reach to one of two locations. Results showed that participants were less likely to make a response to the same location as their partner. In three further experiments observers were asked to decide which of two familiar products they preferred or which of two faces were most attractive. Results showed that participants were less likely to choose the product or face occupying the location of their partner’s previous reaching response. These findings suggest that action observation can influence a range of free choice preferences and decisions. Possible mechanisms through which this influence occurs are discussed.

Reduced gaze following and attention to heads when viewing a "live" social scene

Social stimuli are known to both attract and direct our attention, but most research on social attention has been conducted in highly controlled laboratory settings lacking in social context. This study examined the role of social context on viewing behaviour of participants whilst they watched a dynamic social scene, under three different conditions. In two social groups, participants believed they were watching a live webcam of other participants. The socially-engaged group believed they would later complete a group task with the people in the video, whilst the non-engaged group believed they would not meet the people in the scene. In a third condition, participants simply free-viewed the same video with the knowledge that it was pre-recorded, with no suggestion of a later interaction. Results demonstrated that the social context in which the stimulus was viewed significantly influenced viewing behaviour. Specifically, participants in the social conditions allocated less visual attention towards the heads of the actors in the scene and followed their gaze less than those in the free-viewing group. These findings suggest that by underestimating the impact of social context in social attention, researchers risk coming to inaccurate conclusions about how we attend to others in the real world.

The role of attention in a joint-action effect

The most common explanation for joint-action effects has been the action co-representation account in which observation of another's action is represented within one's own action system. However, recent evidence has shown that the most prominent of these joint-action effects (i.e., the Social Simon effect), can occur when no co-actor is present. In the current work we examined whether another joint-action phenomenon (a movement congruency effect) can be induced when a participant performs their part of the task with a different effector to that of their co-actor and when a co-actor's action is replaced by an attention-capturing luminance signal. Contrary to what is predicted by the action co-representation account, results show that the basic movement congruency effect occurred in both situations. These findings challenge the action co-representation account of this particular effect and suggest instead that it is driven by bottom-up mechanisms.