Visual attention and action: How cueing, direct mapping, and social interactions drive orienting

Intact gesture cueing of attention but attenuated sensitivity to peripheral social targets in autistic children: An eye-tracking and pupillometric study

BACKGROUND

Altered automatic attention cueing has been reported in autistic children. Yet less is known about how autistic children would respond when the social cue that directs attention occurs in an implied social interaction.

METHODS

By using eye-tracking, the current study examined orienting responses to a socially-relevant target or a nonsocial target cued by a goal-directed social gesture in autistic children. Saccadic reaction time and pupillary responses were employed to measure gaze behavior and physiological arousal of autistic children.

RESULTS

Both groups of children showed reflexive orienting to the target regardless of its sociality, whereas typically developing (TD) children exhibited faster gaze shift than autistic children when the target was a social stimulus. An increased pupil dilation was observed in autistic children in response to stimuli relative to TD children. Further, autistic children showed larger baseline pupil response.

CONCLUSIONS

Autistic children show attenuated sensitivity to social targets and atypical pupil responses, which may be due to the dysfunction of locus coeruleus (LC) - norepinephrine (NE) system.

I prefer what you can see: The role of visual perspective-taking on the gaze-liking effect

Individuals' gaze on an object usually leads others to prefer that object, which is called the gaze-liking effect. However, it is still unclear whether this effect is driven by social factors (i.e., visual perspective-taking) or the domain-general processing (i.e., attention cueing). This research explored the mechanism of the gaze-liking effect by manipulating the objects' visibility to an avatar in six online one-shot experiments. The results showed that participants' affective evaluation for the object was modulated by the avatar's visual perspective. Specifically, the visible object to the avatar received a higher rating of liking degree. However, when the avatar was replaced with a non-social stimulus, the experimental effect was absent. Furthermore, the gaze-liking effect was robust while controlling for confounding factors such as the distance between the object and the avatar or type of stimuli. These findings provided convincing evidence that the gaze-liking effect involves a process of the other's visual experience and is not merely a by-product of the gaze-cueing effect.

Effects of interpreting a dynamic geometric cue as gaze on attention allocation

Gaze is a powerful cue for directing attention. We investigate the interpretation of an abstract figure as gaze modulates its efficacy as an attentional cue. In each trial, two vertical lines on a central disk moved to one side (left or right). Independent of this "feature-cued" side, a target (black disk) subsequently appeared on one side. After 300 trials (phase 1), participants watched a video of a human avatar walking away. For one group, the avatar wore a helmet that visually matched the central disk and looked at black disks to either side. The other group's video was unrelated to the cueing task. After another 300 trials (phase 2), videos were swapped between groups; 300 further trials (phase 3) followed. In all phases, participants responded more quickly for targets appearing on the feature-cued side. There was a significant interaction between group and phase for reaction times: In phase 3, the group who had just watched the avatar with the helmet had a reduced advantage to the feature-cued side. Hence, interpreting the disk as a turning head seen from behind counteracts the cueing by the motion of the disk. This suggests that the mere perceptual interpretation of an abstract stimulus as gaze yields social cueing effects.

When Attentional and Politeness Demands Clash: The Case of Mutual Gaze Avoidance and Chin Pointing in Quiahije Chatino

Pointing with the chin is a practice attested worldwide: it is an effective and highly recognizable device for re-orienting the attention of the addressee. For the chin point to be observed, the addressee must attend carefully to the movements of the sender’s head. This demand comes into conflict with the politeness norms of many cultures, since these often require conversationalists to avoid meeting the gaze of their interlocutor, and can require them to look away from their interlocutor’s face and head. In this paper we explore how the chin point is successfully used in just such a culture, among the Chatino indigenous group of Oaxaca, Mexico. We analyze interactions between multiple dyads of Chatino speakers, examining how senders invite visual attention to the pointing gesture, and how addressees signal that attention, while both participants avoid stretches of mutual gaze. We find that in the Chatino context, the senior (or higher-status) party to the conversation is highly consistent in training their gaze away from their interlocutor. This allows their interlocutor to give visual attention to their face without the risk of meeting the gaze of a higher-status sender, and facilitates close attention to head movements including the chin point.

Abstracts in Spanish and Quiahije Chatino are published as appendices.

Se incluyen como apéndices resúmenes en español y en el chatino de San Juan Quiahije.

SonG ktyiC reC inH, ngyaqC skaE ktyiC noE ndaH sonB naF ngaJ noI ngyaqC loE ktyiC reC, ngyaqC ranF chaqE xlyaK qoE chaqF jnyaJ noA ndywiqA renqA KchinA KyqyaC.

Selective visual attention during public speaking in an immersive context

It has recently become feasible to study selective visual attention to social cues in increasingly ecologically valid ways. In this secondary analysis, we examined gaze behavior in response to the actions of others in a social context. Participants (N = 84) were asked to give a 5-minute speech to a five-member audience that had been filmed in 360° video, displayed in a virtual reality headset containing a built-in eye tracker. Audience members were coached to make movements that would indicate interest or lack of interest (e.g., nodding vs. looking away). The goal of this paper was to analyze whether these actions influenced the speaker's gaze. We found that participants showed reliable evidence of gaze towards audience member actions in general, and towards audience member actions involving their phone specifically (compared with other actions like looking away or leaning back). However, there were no differences in gaze towards actions reflecting interest (like nodding) compared with actions reflecting lack of interest (like looking away). Participants were more likely to look away from audience member actions as well, but there were no specific actions that elicited looking away more or less. Taken together, these findings suggest that the actions of audience members are broadly influential in motivating gaze behaviors in a realistic, contextually embedded (public speaking) setting. Further research is needed to examine the ways in which these findings can be elucidated in more controlled laboratory environments as well as in the real world.

Differences in Rhythmic Neural Activity Supporting the Temporal and Spatial Cueing of Attention

The neural processes serving the orienting of attention toward goal-relevant stimuli are generally examined with informative cues that direct visual attention to a spatial location. However, cues predicting the temporal emergence of an object are also known to be effective in attentional orienting but are implemented less often. Differences in the neural oscillatory dynamics supporting these divergent types of attentional orienting have only rarely been examined. In this study, we utilized magnetoencephalography and an adapted Posner cueing task to investigate the spectral specificity of neural oscillations underlying these different types of attentional orienting (i.e., spatial vs. temporal). We found a spectral dissociation of attentional cueing, such that alpha (10-16 Hz) oscillations were central to spatial orienting and theta (3-6 Hz) oscillations were critical to temporal orienting. Specifically, we observed robust decreases in alpha power during spatial orienting in key attention areas (i.e., lateral occipital, posterior cingulate, and hippocampus), along with strong theta increases during temporal orienting in the primary visual cortex. These results suggest that the oscillatory dynamics supporting attentional orienting are spectrally and anatomically specific, such that spatial orienting is served by stronger alpha oscillations in attention regions, whereas temporal orienting is associated with stronger theta responses in visual sensory regions.

Urgency Promotes Affective Disengagement: Effects From Bivalent Cues on Preference Formation for Abstract Images

Although previous research has characterized the important role for spatial and affective pre-cues in the control of visual attention, less is known about the impact of pre-cues on preference formation. In preference formation, the gaze cascade phenomenon suggests that the gaze serves both to enhance and express “liking” during value-based decision-making. This phenomenon has been interpreted as a type of Pavlovian approach toward preferred objects. Decision-making here reflects a process of gradual commitment in which the gaze functions as a precursor to choice; by this account, overt attention produces a necessarily positive, additive effect on the value of the attended object. The implication is that drawing attention to an object should initiate, and therefore promote, preference formation for that object. Alternatively, information-integration models of attention propose that attention produces a multiplicative effect on the value of the attended object, implying that negative information can impede preference formation. To pitch the gradual-commitment hypothesis against the information-integration hypothesis, we conducted four experiments that combined the spatial-cueing paradigm with a value-based choice paradigm. In each trial in all experiments, subjects were presented with an irrelevant, peripheral pre-cue for a duration of 500 ms, followed by a 500 ms blank, and then a pair of abstract images (one at the pre-cued position; one in the opposite hemifield). The subjects were asked to choose their preferred abstract image by pressing the corresponding button. We manipulated the type of pre-cues (images of faces versus foods; with varying affective associations) and the time constraints (a deadline of 1,500 ms versus self-paced). Overall, the choice data showed a clear pattern of influence from the pre-cues, such that, given a deadline, abstract images were chosen less often if they had been preceded by a pre-cue with a negative affective association (both for face and food images). Analyses of the gaze data showed the emergence of significant gaze biases in line with the subjects’ choices. Taken together, the data pattern provided support for the information-integration hypothesis, particularly under urgency. When tasked with a speeded preference choice, subjects showed affective disengagement following pre-cues that carried a negative association.

The role of transients in action observation

A large number of studies have now described the various ways in which the observation of another person's dynamic movement can influence the speed with which the observer is able to prepare a motor action themselves. The typical results are most often explained with reference to theories that link perception and action. Such theories argue that the cognitive structures associated with each share common representations. Consequently, action preparation and action observation are often said to be functionally equivalent. However, the dominance of these theories in explaining action observation effects has masked the potential contribution from processes associated with the detection of low-level "transients" resulting from observing a body movement, such as motion and sound. In the present review, we describe work undertaken in one particular action observation phenomenon ("social inhibition of return") and show that the transient account provides the best explanation of the effect. We argue that future work should consider attention capture and orienting as a potential contributing factor to action observation effects more broadly.

Neuromodulated attention and goal-driven perception in uncertain domains

In uncertain domains, the goals are often unknown and need to be predicted by the organism or system. In this paper, contrastive Excitation Backprop (c-EB) was used in two goal-driven perception tasks - one with pairs of noisy MNIST digits and the other with a robot in an action-based attention scenario. The first task included attending to even, odd, low, and high digits, whereas the second task included action goals, such as "eat", "work-on-computer", "read", and "say-hi" that led to attention to objects associated with those actions. The system needed to increase attention to target items and decrease attention to distractor items and background noise. Because the valid goal was unknown, an online learning model based on the cholinergic and noradrenergic neuromodulatory systems was used to predict a noisy goal (expected uncertainty) and re-adapt when the goal changed (unexpected uncertainty). This neurobiologically plausible model demonstrates how neuromodulatory systems can predict goals in uncertain domains and how attentional mechanisms can enhance the perception for that goal.

Peer Presence Effects on Eye Movements and Attentional Performance

"Social facilitation" refers to the enhancement or impairment of performance engendered by the mere presence of others. It has been demonstrated for a diversity of behaviors. This study assessed whether it also concerns attention and eye movements and if yes, which decision-making mechanisms it affects. Human volunteers were tested in three different tasks (saccades, visual search, and continuous performance) either alone or in the presence of a familiar peer. The results failed to reveal any significant peer influence on the visual search and continuous performance tasks. For saccades, by contrast, they showed a negative or positive peer influence depending on the complexity of the testing protocol. Pro-and anti-saccades were both inhibited when pseudorandomly mixed, and both facilitated when performed separately. Peer presence impaired or improved reaction times, i.e., the speed to initiate the saccade, as well as peak velocity, i.e., the driving force moving the eye toward the target. Effect sizes were large, with Cohen's d-values ranging for reaction times (RTs) from 0.50 to 0.95. Analyzing RT distributions using the LATER (Linear Approach to Threshold with Ergodic Rate) model revealed that social inhibition of pro- and anti-saccades in the complex protocol was associated with a significant increase in the rate of rise. The present demonstration that the simple presence of a familiar peer can inhibit or facilitate saccades depending on task difficulty strengthens a growing body of evidence showing social modulations of eye movements and attention processes. The present lack of effect on visual search and continuous performance tasks contrasts with peer presence effects reported earlier using similar tasks, and future studies are needed to determine whether it is due to an intermediate level of difficulty maximizing individual variability. Together with an earlier study of the social inhibition of anti-saccades also using the LATER model, which showed an increase of the threshold, the present increase of the rate of rise suggests that peer presence can influence both the top-down and bottom-up attention-related processes guiding the decision to move the eyes.

Gaze and body cues interplay during interactive requests

Although observing other's gaze and body movements provides a crucial source of information to successfully interact with other people, it remains unclear whether observers weigh differently these cues and whether the convergence of gaze and body's directions determines facilitation effects. Here we aim to shed more light on this issue by testing the reliance upon these cues from both a behavioral and a neurophysiological perspective in a social interactive context. In Experiment 1, we manipulated the convergence between the direction of an actor's upper limb movement and gaze direction while he attempts to socially interact with the participants observing the scene. We determined the direction of gaze as well as the duration of participants' ocular fixations during the observation of the scene. In Experiment 2, we measured and correlated the effect of the body/gaze manipulation on corticospinal excitability and on the readiness to interact-a disposition to engage in social situations. Eye-tracking data revealed that participants fixated chiefly the actor's head when his hand and gaze directions were divergent. Possibly a strategy to disambiguate the scene. Whereas participants mainly fixated the actor's hand when he performed an interactive request toward the participants. From a neurophysiological point of view, the more participants felt involved in the interaction, the lower was motor preparation in the muscle potentially needed to fulfill the actor's request. We contend that social contexts are more likely to elicit motor preparation compared to non-social ones, and that muscular inhibition is a necessary mechanism in order to prevent unwanted overt reactions during action observation tasks.

Biological motion and animacy belief induce similar effects on involuntary shifts of attention

Biological motion is salient to the human visual and motor systems and may be intrinsic to the perception of animacy. Evidence for the salience of visual stimuli moving with trajectories consistent with biological motion comes from studies showing that such stimuli can trigger shifts of attention in the direction of that motion. The present study was conducted to determine whether or not top-down beliefs about animacy can modify the salience of a nonbiologically moving stimulus to the visuomotor system. A nonpredictive cuing task was used in which a white dot moved from a central location toward a left- or right-sided target placeholder. The target randomly appeared at either location 200, 600, or 1,300 ms after the motion onset. Five groups of participants experienced different stimulus conditions: (1) biological motion, (2) inverted biological motion, (3) nonbiological motion, (4) animacy belief (paired with nonbiological motion), and (5) computer-generated belief (paired with nonbiological motion). Analysis of response times revealed that the motion in the biological motion and animacy belief groups, but not in the inverted and nonbiological motion groups, affected processing of the target information. These findings indicate that biological motion is salient to the visual system and that top-down beliefs regarding the animacy of the stimulus can tune the visual and motor systems to increase the salience of nonbiological motion.

Look at Me: Early Gaze Engagement Enhances Corticospinal Excitability During Action Observation

Direct gaze is a powerful social cue able to capture the onlooker’s attention. Beside gaze, head and limb movements as well can provide relevant sources of information for social interaction. This study investigated the joint role of direct gaze and hand gestures on onlookers corticospinal excitability (CE). In two experiments we manipulated the temporal and spatial aspects of observed gaze and hand behavior to assess their role in affecting motor preparation. To do this, transcranial magnetic stimulation (TMS) on the primary motor cortex (M1) coupled with electromyography (EMG) recording was used in two experiments. In the crucial manipulation, we showed to participants four video clips of an actor who initially displayed eye contact while starting a social request gesture, and then completed the action while directing his gaze toward a salient object for the interaction. This way, the observed gaze potentially expressed the intention to interact. Eye tracking data confirmed that gaze manipulation was effective in drawing observers’ attention to the actor’s hand gesture. In the attempt to reveal possible time-locked modulations, we tracked CE at the onset and offset of the request gesture. Neurophysiological results showed an early CE modulation when the actor was about to start the request gesture looking straight to the participants, compared to when his gaze was averted from the gesture. This effect was time-locked to the kinematics of the actor’s arm movement. Overall, data from the two experiments seem to indicate that the joint contribution of direct gaze and precocious kinematic information, gained while a request gesture is on the verge of beginning, increases the subjective experience of involvement and allows observers to prepare for an appropriate social interaction. On the contrary, the separation of gaze cues and body kinematics can have adverse effects on social motor preparation. CE is highly susceptible to biological cues, such as averted gaze, which is able to automatically capture and divert observer’s attention. This point to the existence of heuristics based on early action and gaze cues that would allow observers to interact appropriately.