Configural processing of biological motion in human superior temporal sulcus

Pupil dilation reflects the social and motion content of faces

Human facial features (eyes, nose, and mouth) allow us to communicate with others. Observing faces triggers physiological responses, including pupil dilation. Still, the relative influence of social and motion content of a visual stimulus on pupillary reactivity has never been elucidated. A total of 30 adults aged 18-33 years old were recorded with an eye tracker. We analysed the event-related pupil dilation in response to stimuli distributed along a gradient of social salience (non-social to social, going from objects to avatars to real faces) and dynamism (static to micro- to macro-motion). Pupil dilation was larger in response to social (faces and avatars) compared to non-social stimuli (objects), with surprisingly a larger response for avatars. Pupil dilation was also larger in response to macro-motion compared to static. After quantifying each stimulus' real quantity of motion, we found that the higher the quantity of motion, the larger the pupil dilated. However, the slope of this relationship was not higher for social stimuli. Overall, pupil dilation was more sensitive to the real quantity of motion than to the social component of motion, highlighting the relevance of ecological stimulations. Physiological response to faces results from specific contributions of both motion and social processing.

Distinct Yet Proximal Face- and Body-Selective Brain Regions Enable Clutter-Tolerant Representations of the Face, Body, and Whole Person

Faces and bodies are processed in separate but adjacent regions in the primate visual cortex. Yet, the functional significance of dividing the whole person into areas dedicated to its face and body components and their neighboring locations remains unknown. Here we hypothesized that this separation and proximity together with a normalization mechanism generate clutter-tolerant representations of the face, body, and whole person when presented in complex multi-category scenes. To test this hypothesis, we conducted a fMRI study, presenting images of a person within a multi-category scene to human male and female participants and assessed the contribution of each component to the response to the scene. Our results revealed a clutter-tolerant representation of the whole person in areas selective for both faces and bodies, typically located at the border between the two category-selective regions. Regions exclusively selective for faces or bodies demonstrated clutter-tolerant representations of their preferred category, corroborating earlier findings. Thus, the adjacent locations of face- and body-selective areas enable a hardwired machinery for decluttering of the whole person, without the need for a dedicated population of person-selective neurons. This distinct yet proximal functional organization of category-selective brain regions enhances the representation of the socially significant whole person, along with its face and body components, within multi-category scenes.

From Motion to Emotion: Visual Pathways and Potential Interconnections

The two visual pathway description of [Ungerleider, L. G., & Mishkin, M. Two cortical visual systems. In D. J. Dingle, M. A. Goodale, & R. J. W. Mansfield (Eds.), Analysis of visual behavior (pp. 549-586). Cambridge, MA: MIT, 1982] changed the course of late 20th century systems and cognitive neuroscience. Here, I try to reexamine our laboratory's work through the lens of the [Pitcher, D., & Ungerleider, L. G. Evidence for a third visual pathway specialized for social perception. Trends in Cognitive Sciences, 25, 100-110, 2021] new third visual pathway. I also briefly review the literature related to brain responses to static and dynamic visual displays, visual stimulation involving multiple individuals, and compare existing models of social information processing for the face and body. In this context, I examine how the posterior STS might generate unique social information relative to other brain regions that also respond to social stimuli. I discuss some of the existing challenges we face with assessing how information flow progresses between structures in the proposed functional pathways and how some stimulus types and experimental designs may have complicated our data interpretation and model generation. I also note a series of outstanding questions for the field. Finally, I examine the idea of a potential expansion of the third visual pathway, to include aspects of previously proposed "lateral" visual pathways. Doing this would yield a more general entity for processing motion/action (i.e., "[inter]action") that deals with interactions between people, as well as people and objects. In this framework, a brief discussion of potential hemispheric biases for function, and different forms of neuropsychological impairments created by focal lesions in the posterior brain is highlighted to help situate various brain regions into an expanded [inter]action pathway.

Social gaze cueing elicits facilitatory and inhibitory effects on movement execution when the model might act on an object

Social cues, such as eye gaze and pointing fingers, can increase the prioritisation of specific locations for cognitive processing. A previous study using a manual reaching task showed that, although both gaze and pointing cues altered target prioritisation (reaction times [RTs]), only pointing cues affected action execution (trajectory deviations). These differential effects of gaze and pointing cues on action execution could be because the gaze cue was conveyed through a disembodied head; hence, the model lacked the potential for a body part (i.e., hands) to interact with the target. In the present study, the image of a male gaze model, whose gaze direction coincided with two potential target locations, was centrally presented. The model either had his arms and hands extended underneath the potential target locations, indicating the potential to act on the targets (Experiment 1), or had his arms crossed in front of his chest, indicating the absence of potential to act (Experiment 2). Participants reached to a target that followed a nonpredictive gaze cue at one of three stimulus onset asynchronies. RTs and reach trajectories of the movements to cued and uncued targets were analysed. RTs showed a facilitation effect for both experiments, whereas trajectory analysis revealed facilitatory and inhibitory effects, but only in Experiment 1 when the model could potentially act on the targets. The results of this study suggested that when the gaze model had the potential to interact with the cued target location, the model's gaze affected not only target prioritisation but also movement execution.

Representation of motion concepts in occipitotemporal cortex: fMRI activation, decoding and connectivity analyses

Embodied theories of semantic cognition predict that brain regions involved in motion perception are engaged when people comprehend motion concepts expressed in language. Left lateral occipitotemporal cortex (LOTC) is implicated in both motion perception and motion concept processing but prior studies have produced mixed findings on which parts of this region are engaged by motion language. We scanned participants performing semantic judgements about sentences describing motion events and static events. We performed univariate analyses, multivariate pattern analyses (MVPA) and psychophysiological interaction (PPI) analyses to investigate the effect of motion on activity and connectivity in different parts of LOTC. In multivariate analyses that decoded whether a sentence described motion or not, the middle and posterior parts of LOTC showed above-chance level performance, with performance exceeding that of other brain regions. Univariate ROI analyses found the middle part of LOTC was more active for motion events than static ones. Finally, PPI analyses found that when processing motion events, the middle and posterior parts of LOTC (overlapping with motion perception regions), increased their connectivity with cognitive control regions. Taken together, these results indicate that the more posterior parts of LOTC, including motion perception cortex, respond differently to motion vs. static events. These findings are consistent with embodiment accounts of semantic processing, and suggest that understanding verbal descriptions of motion engages areas of the occipitotemporal cortex involved in perceiving motion.

Perceiving social injustice during arrests of Black and White civilians by White police officers: An fMRI investigation

From social media to courts of law, recordings of interracial police officer-civilian interactions are now widespread and publicly available. People may be motivated to preferentially understand the dynamics of these interactions when they perceive injustice towards those whose communities experience disproportionate policing relative to others (e.g., non-White racial/ethnic groups). To explore these questions, two studies were conducted (study 1 neuroimaging n = 69 and study 2 behavioral n = 58). The fMRI study examined White participants' neural activity when viewing real-world videos with varying degrees of aggression or conflict of White officers arresting a Black or White civilian. Activity in brain regions supporting social cognition was greater when viewing Black (vs. White) civilians involved in more aggressive police encounters. Additionally, although an independent sample of perceivers rated videos featuring Black and White civilians as similar in overall levels of aggression when civilian race was obscured, participants in the fMRI study (where race was not obscured) rated officers as more aggressive and their use of force as less legitimate when the civilian was Black. In study 2, participants who had not viewed the videos also reported that they believe police are generally more unjustly aggressive towards Black compared with White civilians. These findings inform our understanding of how perceptions of conflict with the potential for injustice shape social cognitive engagement when viewing arrests of Black and White individuals by White police officers.

Social Action Effects: Representing Predicted Partner Responses in Social Interactions

The sociomotor framework outlines a possible role of social action effects on human action control, suggesting that anticipated partner reactions are a major cue to represent, select, and initiate own body movements. Here, we review studies that elucidate the actual content of social action representations and that explore factors that can distinguish action control processes involving social and inanimate action effects. Specifically, we address two hypotheses on how the social context can influence effect-based action control: first, by providing unique social features such as body-related, anatomical codes, and second, by orienting attention towards any relevant feature dimensions of the action effects. The reviewed empirical work presents a surprisingly mixed picture: while there is indirect evidence for both accounts, previous studies that directly addressed the anatomical account showed no signs of the involvement of genuinely social features in sociomotor action control. Furthermore, several studies show evidence against the differentiation of social and non-social action effect processing, portraying sociomotor action representations as remarkably non-social. A focus on enhancing the social experience in future studies should, therefore, complement the current database to establish whether such settings give rise to the hypothesized influence of social context.

More Than the Face: Representations of Bodies in the Inferior Temporal Cortex

Visual representations of bodies, in addition to those of faces, contribute to the recognition of con- and heterospecifics, to action recognition, and to nonverbal communication. Despite its importance, the neural basis of the visual analysis of bodies has been less studied than that of faces. In this article, I review what is known about the neural processing of bodies, focusing on the macaque temporal visual cortex. Early single-unit recording work suggested that the temporal visual cortex contains representations of body parts and bodies, with the dorsal bank of the superior temporal sulcus representing bodily actions. Subsequent functional magnetic resonance imaging studies in both humans and monkeys showed several temporal cortical regions that are strongly activated by bodies. Single-unit recordings in the macaque body patches suggest that these represent mainly body shape features. More anterior patches show a greater viewpoint-tolerant selectivity for body features, which may reflect a processing principle shared with other object categories, including faces. Expected final online publication date for the Annual Review of Vision Science, Volume 8 is September 2022. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Three Months-Old' Preferences for Biological Motion Configuration and Its Subsequent Decline

To perceive, identify and understand the action of others, it is essential to perceptually organize individual and local moving body parts (such as limbs) into the whole configuration of a human body in action. Configural processing-processing the relations among features or parts of a stimulus-is a fundamental ability in the perception of several important social stimuli, such as faces or biological motion. Despite this, we know very little about how human infants develop the ability to perceive and prefer configural relations in biological motion. We present two preferential looking experiments (one cross-sectional and one longitudinal) measuring infants' preferential attention between a coherent motion configuration of a person walking vs. a scrambled point-light walker (i.e., a stimulus in which all configural relations were removed, thus, in which the perception of a person is impossible). We found that three-month-old infants prefer a coherent point-light walker in relation to a scrambled display, but both five- and seven-month-old infants do not show any preference. We discuss our findings in terms of the different perceptual, attentional, motor, and brain processes available at each age group, and how they dynamically interact with selective attention toward the coherent and socially relevant motion of a person walking during our first year of life.

The role of temporal cortex in the control of attention

Attention is an indispensable component of active vision. Contrary to the widely accepted notion that temporal cortex processing primarily focusses on passive object recognition, a series of very recent studies emphasize the role of temporal cortex structures, specifically the superior temporal sulcus (STS) and inferotemporal (IT) cortex, in guiding attention and implementing cognitive programs relevant for behavioral tasks. The goal of this theoretical paper is to advance the hypothesis that the temporal cortex attention network (TAN) entails necessary components to actively participate in attentional control in a flexible task-dependent manner. First, we will briefly discuss the general architecture of the temporal cortex with a focus on the STS and IT cortex of monkeys and their modulation with attention. Then we will review evidence from behavioral and neurophysiological studies that support their guidance of attention in the presence of cognitive control signals. Next, we propose a mechanistic framework for executive control of attention in the temporal cortex. Finally, we summarize the role of temporal cortex in implementing cognitive programs and discuss how they contribute to the dynamic nature of visual attention to ensure flexible behavior.

Infant electroencephalogram coherence and early childhood inhibitory control: Foundations for social cognition in late childhood

Social cognition is a set of complex processes that mediate much of human behavior. The development of these skills is related to and interdependent on other cognitive processes, particularly inhibitory control. Brain regions associated with inhibitory control and social cognition overlap functionally and structurally, especially with respect to frontal brain areas. We proposed that the neural foundations of inhibitory control and social cognition are measurable in infancy. We used structural equation modeling and showed that 10-month frontotemporal neuroconnectivity measured using electroencephalogram coherence predicts social cognition at 9 years of age through age-4 inhibitory control. These findings provide insight into the neurodevelopmental trajectory of cognition and suggest that connectivity from frontal regions to other parts of the brain is a foundation for the development of these skills. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Cortical networks of dynamic scene category representation in the human brain

Humans have an impressive ability to rapidly process global information in natural scenes to infer their category. Yet, it remains unclear whether and how scene categories observed dynamically in the natural world are represented in cerebral cortex beyond few canonical scene-selective areas. To address this question, here we examined the representation of dynamic visual scenes by recording whole-brain blood oxygenation level-dependent (BOLD) responses while subjects viewed natural movies. We fit voxelwise encoding models to estimate tuning for scene categories that reflect statistical ensembles of objects and actions in the natural world. We find that this scene-category model explains a significant portion of the response variance broadly across cerebral cortex. Cluster analysis of scene-category tuning profiles across cortex reveals nine spatially-segregated networks of brain regions consistently across subjects. These networks show heterogeneous tuning for a diverse set of dynamic scene categories related to navigation, human activity, social interaction, civilization, natural environment, non-human animals, motion-energy, and texture, suggesting that the organization of scene category representation is quite complex.

Neural interactions in occipitotemporal cortex during basic human movement perception by dynamic causal modeling

Action recognition is an essential component of our daily life. The occipitotemporal cortex (OTC) is an important area in human movement perception. The previous studies have revealed that three vital regions including the extrastriate body area (EBA), human middle temporal complex (hMT+), and posterior superior temporal sulcus (pSTS) in OTC play an important role in motion perception. The aim of the current study is to explore the neural interactions between these three regions during basic human movement perception. Functional magnetic resonance imaging data were acquired when participants viewed dynamic videos depicting basic human movements. By the dynamic causal modeling analysis, a model space consisting of 576 models was constructed and evaluated to select the optimal model given the data. The information of the visual movement was found to enter the system through hMT+. We speculated that hMT+ would be the region to show sensitivity to the presence of motion and it subsequently influence and be influenced by the other two regions. Our results also revealed the manner in which the three regions interact during action recognition. Furthermore, We found significantly enhanced modulated connectivity from hMT+ to both EBA and pSTS, as well as from EBA to both hMT+ and pSTS. We inferred that there may be multiple routes for human action perception. One responsible route for processing motion signals is through hMT+ to pSTS, and the other projects information to pSTS may be via the form-processing route. In addition, pSTS may integrate and mediate visual signals and possibly convey them to distributed areas to maintain high-order cognitive tasks.

People accurately predict the transition probabilities between actions

Social life is a complex dance. To coordinate gracefully with one's partners, one must predict their actions. Here, we investigated how people predict others' actions. We hypothesized that people can accurately predict others' future actions based on knowledge of their current actions, coupled with knowledge of action transitions. To test whether people have accurate knowledge of the transition probabilities between actions, we compared actual rates of action transitions-calculated from four large naturalistic datasets-to participants' ratings of the transition probabilities between corresponding sets of actions. In five preregistered studies, participants demonstrated accurate mental models of action transitions. Furthermore, we found that people drew upon conceptual knowledge of actions-described by the six-dimensional ACT-FASTaxonomy-to guide their accurate predictions. Together, these results indicate that people can accurately anticipate other people's moves in the dance of social life and that the structure of action knowledge may be tailored to making these predictions.

Right STS responses to biological motion in infancy - An fNIRS study using point-light walkers

Biological motion perception-our capacity to perceive the intrinsic motion of humans and animals-has been implicated as a precursor of social development in infancy. In the adult brain, several biological motion neural correlates have been identified; of particular importance, the right posterior superior temporal sulcus (rpSTS). We present a study, conducted with fNIRS, which measured brain activations in infants' right posterior temporal region to point-light walkers, a standard stimulus category of biological motion perception studies. Seven-month-old infants (n = 23) participated in a within-subject blocked design with three experimental conditions and one baseline. Infants viewed: an intact upright point-light walker of a person approaching the observer; the same point-light walker stimulus but inverted; and a selected frame from the point-light walker stimulus, approaching the viewer at constant velocity with no articulated motion, close to object motion. We found activations for both the upright and the inverted point-light walkers. The rigid moving point-light walker frame did not elicit any response consistent with a functional activation in this region. Our results suggest that biological motion is processed differently in the right middle posterior temporal cortex in infancy, and that articulated motion is a critical feature in biological motion processing at this early age.

Children and adolescents' neural response to emotional faces and voices: Age-related changes in common regions of activation

The perception of facial and vocal emotional expressions engages overlapping regions of the brain. However, at a behavioral level, the ability to recognize the intended emotion in both types of nonverbal cues follows a divergent developmental trajectory throughout childhood and adolescence. The current study a) identified regions of common neural activation to facial and vocal stimuli in 8- to 19-year-old typically-developing adolescents, and b) examined age-related changes in blood-oxygen-level dependent (BOLD) response within these areas. Both modalities elicited activation in an overlapping network of subcortical regions (insula, thalamus, dorsal striatum), visual-motor association areas, prefrontal regions (inferior frontal cortex, dorsomedial prefrontal cortex), and the right superior temporal gyrus. Within these regions, increased age was associated with greater frontal activation to voices, but not faces. Results suggest that processing facial and vocal stimuli elicits activation in common areas of the brain in adolescents, but that age-related changes in response within these regions may vary by modality.

Anticipation in sociomotor actions: Similar effects for in- and outgroup interactions

In social interactions, own actions often trigger a particular response from another person. The sociomotor framework proposes that this consistent behavior of others can become incorporated into own action control. In line with this idea, recent studies have shown that own motor actions are facilitated if they are predictably being imitated rather than counterimitated by a social interaction partner. In the present study, we investigated whether this finding is influenced by the relationship between the interacting persons. To that end, we manipulated whether a participant was being imitated and counterimitated by an ingroup or by an outgroup member. In two experiments, we found a beneficial influence of being imitated irrespective of group membership. The results suggest that, while people incorporated their partner's behavior into own action control, this was not further qualified by group membership as a higher-order social variable. This finding points to a universal account of action control for actions with social action effects and actions with inanimate action effects alike.

Muscular effort coding in action representation in ballet dancers and controls: Electrophysiological evidence

The present electrophysiological (EEG) study investigated the neural correlates of perceiving effortful vs. effortless movements belonging to a specific repertoire (ballet). Previous evidence has shown an increased heart and respiratory rate during the observation and imagination of human actions that require a great muscular effort. In addition, TMS (transcranial magnetic stimulation) and EEG studies have evidenced a greater muscle-specific cortical excitability and an increase in late event-related potentials during the observation of effortful actions. In this investigation, fifteen professional female ballet dancers and 15 controls with no experience whatsoever with dance, gymnastics, or martial arts were recruited. They were shown 326 short videos displaying a male dancer performing standard ballet steps that could be either effortful or relatively effortless. Participants were instructed to observe each clip and imagine themselves physically executing the same movement. Importantly, they were blinded to the stimuli properties. The observation of effortful compared with effortless movements resulted in a larger P300 over frontal sites in dancers only, likely because of their visuomotor expertise with the specific steps. Moreover, an enhanced Late Positivity was identified over posterior sites in response to effortful stimuli in both groups, possibly reflecting the processing of larger quantities of visual kinematic information. The source reconstruction swLORETA performed on the Late Positivity component showed greater engagement of frontoparietal regions in dancers, while task-related frontal and occipitotemporal visual regions were more active in controls. It, therefore, appears that, in dancers, effort information was encoded in a more refined manner during action observation and in the absence of explicit instruction. Acquired motor knowledge seems to result in visuomotor resonance processes, which, in turn, underlies enhanced action representation of the observed movements.

The two-process theory of biological motion processing

Perception, identification, and understanding of others' actions from motion information are vital for our survival in the social world. A breakthrough in the understanding of action perception was the discovery that our visual system is sensitive to human action from the sparse motion input of only a dozen point lights, a phenomenon known as biological motion (BM) processing. Previous psychological and computational models cannot fully explain the emerging evidence for the existence of BM processing during early ontogeny. Here, we propose a two-process model of the mechanisms underlying BM processing. We hypothesize that the first system, the 'Step Detector,' rapidly processes the local foot motion and feet-below-the-body information that is specific to vertebrates, is less dependent on postnatal learning, and involves subcortical networks. The second system, the 'Bodily Action Evaluator,' slowly processes the fine global structure-from-motion, is specific to conspecific, and dependent on gradual learning processed in cortical networks. This proposed model provides new insight into research on the development of BM processing.

Emotion recognition and theory of mind in schizophrenia: A meta-analysis of neuroimaging studies

OBJECTIVES

Patients with schizophrenia have difficulties processing the emotional and cognitive states of others. Neuroimaging studies show inconsistent findings.

METHODS

We used a Seed-based d Mapping meta-analytic method to explore brain activation during facial emotion recognition and theory of mind tasks in schizophrenia patients.

RESULTS

The patients showed lesser recruitment of the facial emotion processing network; behavioural performance was associated with the activation of the precentral gyrus. We found abnormal activation of the mentalising network in schizophrenia patients during reasoning about other people's mental states; patients with worse performances showed lesser activation in the right insula and superior temporal gyrus. Multimodal meta-analysis showed overlaps of brain-related abnormalities for both modalities in schizophrenia, with reduced recruitment of the right insula, anterior cingulate and medial prefrontal cortex and increased activation in the bilateral parietal cortex. Meta-regression results indicate that illness duration, medication and symptomatology might influence social-cognitive network disruptions in schizophrenia.

CONCLUSIONS

These findings suggest the complex impairment of social cognition, as demonstrated by neural-related circuit disruptions during facial emotion processing and theory of mind tasks in schizophrenia.

Perceiving animacy purely from visual motion cues involves intraparietal sulcus

Distinguishing animate from inanimate objects is fundamental for social perception in humans and animals. Visual motion cues indicative of self-propelled object motion are useful for animacy perception: they can be detected over a wide expanse of visual field, at distance and in low visibility conditions, can attract attention and provide clues about object behaviour. However, the neural correlates of animacy perception evoked exclusively by visual motion cues, i.e. not relying on form, background or visual context, are unclear. We aimed to address this question in four psychophysical experiments in humans, two of which performed during neuroimaging. The stimulus was a single dot with constant form that moved on a blank background and evoked controlled degrees of perceived animacy through parametric variations of self-propelled motion cues. BOLD signals reflecting perceived animacy in a graded manner irrespective of eye movements were found in one intraparietal region. Additional whole-brain and region-of-interest analyses revealed no comparable effects in brain regions associated with social processing or other areas. Our study shows that animacy perception evoked solely by visual motion cues, a basic perceptual process in social cognition, engages brain regions not primarily associated with social cognition.

Face recognition ability does not predict person identification performance: using individual data in the interpretation of group results

There are large individual differences in people's face recognition ability. These individual differences provide an opportunity to recruit the best face-recognisers into jobs that require accurate person identification, through the implementation of ability-screening tasks. To date, screening has focused exclusively on face recognition ability; however real-world identifications can involve the use of other person-recognition cues. Here we incorporate body and biological motion recognition as relevant skills for person identification. We test whether performance on a standardised face-matching task (the Glasgow Face Matching Test) predicts performance on three other identity-matching tasks, based on faces, bodies, and biological motion. We examine the results from group versus individual analyses. We found stark differences between the conclusions one would make from group analyses versus analyses that retain information about individual differences. Specifically, tests of correlation and analysis of variance suggested that face recognition ability was related to performance for all person identification tasks. These analyses were strikingly inconsistent with the individual differences data, which suggested that the screening task was related only to performance on the face task. This study highlights the importance of individual data in the interpretation of results of person identification ability.

Computational Feature Analysis of Body Movements Reveals Hierarchical Brain Organization

Social species spend considerable time observing the body movements of others to understand their actions, predict their emotions, watch their games, or enjoy their dance movements. Given the important information obtained from body movements, we still know surprisingly little about the details of brain mechanisms underlying movement perception. In this fMRI study, we investigated the relations between movement features obtained from automated computational analyses of video clips and the corresponding brain activity. Our results show that low-level computational features map to specific brain areas related to early visual- and motion-sensitive regions, while mid-level computational features are related to dynamic aspects of posture encoded in occipital–temporal cortex, posterior superior temporal sulcus and superior parietal lobe. Furthermore, behavioral features obtained from subjective ratings correlated with activity in higher action observation regions. Our computational feature-based analysis suggests that the neural mechanism of movement encoding is organized in the brain not so much by semantic categories than by feature statistics of the body movements.

Fronto-parietal coding of goal-directed actions performed by artificial agents

With advances in technology, artificial agents such as humanoid robots will soon become a part of our daily lives. For safe and intuitive collaboration, it is important to understand the goals behind their motor actions. In humans, this process is mediated by changes in activity in fronto-parietal brain areas. The extent to which these areas are activated when observing artificial agents indicates the naturalness and easiness of interaction. Previous studies indicated that fronto-parietal activity does not depend on whether the agent is human or artificial. However, it is unknown whether this activity is modulated by observing grasping (self-related action) and pointing actions (other-related action) performed by an artificial agent depending on the action goal. Therefore, we designed an experiment in which subjects observed human and artificial agents perform pointing and grasping actions aimed at two different object categories suggesting different goals. We found a signal increase in the bilateral inferior parietal lobule and the premotor cortex when tool versus food items were pointed to or grasped by both agents, probably reflecting the association of hand actions with the functional use of tools. Our results show that goal attribution engages the fronto-parietal network not only for observing a human but also a robotic agent for both self-related and social actions. The debriefing after the experiment has shown that actions of human-like artificial agents can be perceived as being goal-directed. Therefore, humans will be able to interact with service robots intuitively in various domains such as education, healthcare, public service, and entertainment.

A Causal Role of the Right Superior Temporal Sulcus in Emotion Recognition From Biological Motion

Understanding the emotions of others through nonverbal cues is critical for successful social interactions. The right posterior superior temporal sulcus (pSTS) is one brain region thought to be key in the recognition of the mental states of others based on body language and facial expression. In the present study, we temporarily disrupted functional activity of the right pSTS by using continuous, theta-burst transcranial magnetic stimulation (cTBS) to test the hypothesis that the right pSTS plays a causal role in emotion recognition from body movements. Participants ( N = 23) received cTBS to the right pSTS, which was individually localized using fMRI, and a vertex control site. Before and after cTBS, we tested participants’ ability to identify emotions from point-light displays (PLDs) of biological motion stimuli and a nonbiological global motion identification task. Results revealed that accurate identification of emotional states from biological motion was reduced following cTBS to the right pSTS, but accuracy was not impaired following vertex stimulation. Accuracy on the global motion task was unaffected by cTBS to either site. These results support the causal role of the right pSTS in decoding information about others’ emotional state from their body movements and gestures.

Auditory object perception: A neurobiological model and prospective review

Interaction with the world is a multisensory experience, but most of what is known about the neural correlates of perception comes from studying vision. Auditory inputs enter cortex with its own set of unique qualities, and leads to use in oral communication, speech, music, and the understanding of emotional and intentional states of others, all of which are central to the human experience. To better understand how the auditory system develops, recovers after injury, and how it may have transitioned in its functions over the course of hominin evolution, advances are needed in models of how the human brain is organized to process real-world natural sounds and "auditory objects". This review presents a simple fundamental neurobiological model of hearing perception at a category level that incorporates principles of bottom-up signal processing together with top-down constraints of grounded cognition theories of knowledge representation. Though mostly derived from human neuroimaging literature, this theoretical framework highlights rudimentary principles of real-world sound processing that may apply to most if not all mammalian species with hearing and acoustic communication abilities. The model encompasses three basic categories of sound-source: (1) action sounds (non-vocalizations) produced by 'living things', with human (conspecific) and non-human animal sources representing two subcategories; (2) action sounds produced by 'non-living things', including environmental sources and human-made machinery; and (3) vocalizations ('living things'), with human versus non-human animals as two subcategories therein. The model is presented in the context of cognitive architectures relating to multisensory, sensory-motor, and spoken language organizations. The models' predictive values are further discussed in the context of anthropological theories of oral communication evolution and the neurodevelopment of spoken language proto-networks in infants/toddlers. These phylogenetic and ontogenetic frameworks both entail cortical network maturations that are proposed to at least in part be organized around a number of universal acoustic-semantic signal attributes of natural sounds, which are addressed herein.

Where You Look Matters for Body Perception: Preferred Gaze Location Contributes to the Body Inversion Effect

The Body Inversion Effect (BIE; reduced visual discrimination performance for inverted compared to upright bodies) suggests that bodies are visually processed configurally; however, the specific importance of head posture information in the BIE has been indicated in reports of BIE reduction for whole bodies with fixed head position and for headless bodies. Through measurement of gaze patterns and investigation of the causal relation of fixation location to visual body discrimination performance, the present study reveals joint contributions of feature and configuration processing to visual body discrimination. Participants predominantly gazed at the (body-centric) upper body for upright bodies and the lower body for inverted bodies in the context of an experimental paradigm directly comparable to that of prior studies of the BIE. Subsequent manipulation of fixation location indicates that these preferential gaze locations causally contributed to the BIE for whole bodies largely due to the informative nature of gazing at or near the head. Also, a BIE was detected for both whole and headless bodies even when fixation location on the body was held constant, indicating a role of configural processing in body discrimination, though inclusion of the head posture information was still highly discriminative in the context of such processing. Interestingly, the impact of configuration (upright and inverted) to the BIE appears greater than that of differential preferred gaze locations.

Human Action Perception is Consistent, Flexible, and Orientation Dependent

Previous research has found that observers of object-directed human action pay more attention to information regarding hand contact over information regarding spatial trajectories in action, and that processing of trajectory information is disrupted by inversion. However, observers can also flexibly modulate their attention to spatial trajectory depending on the goal or context of the actor. In Experiments 1(a) and 1b of the current research, we directly compared attention with hand and trajectory information across placing and dropping actions in order to determine whether the hand bias is always present or whether flexibility in action perception can attenuate this bias. Results demonstrated that observers attend more to hand information for placing, but attend equally to hand and trajectory information for dropping. Experiment 2 explored the role of the actor’s goal in processing spatial trajectory for mimed dropping actions and non-human control stimuli, and the role of goals in the inversion effect. Results indicated that goal relevance increases processing of trajectory information, and that processing of all spatial trajectories in human action is disrupted by inversion, regardless of the actor’s goal. The present findings highlight the role of prediction in action perception, and suggest that human action is processed with expertise.

Holding Biological Motion in Working Memory: An fMRI Study

Holding biological motion (BM), the movements of animate entities, in working memory (WM) is important to our daily life activities. However, the neural substrates underlying the WM processing of BM remain largely unknown. Employing the functional magnetic resonance imaging (fMRI) technique, the current study directly investigated this issue. We used point-light BM animations as the tested stimuli, and explored the neural substrates involved in encoding and retaining BM information in WM. Participants were required to remember two or four BM stimuli in a change-detection task. We first defined a set of potential brain regions devoted to the BM processing in WM in one experiment. We then conducted the second fMRI experiment, and performed time-course analysis over the pre-defined regions, which allowed us to differentiate the encoding and maintenance phases of WM. The results showed that a set of brain regions were involved in encoding BM into WM, including the middle frontal gyrus, inferior frontal gyrus, superior parietal lobule, inferior parietal lobule, superior temporal sulcus, fusiform gyrus, and middle occipital gyrus. However, only the middle frontal gyrus, inferior frontal gyrus, superior parietal lobule, and inferior parietal lobule were involved in retaining BM into WM. These results suggest that an overlapped network exists between the WM encoding and maintenance for BM; however, retaining BM in WM predominately relies on the mirror neuron system.

Human, Nature, Dynamism: The Effects of Content and Movement Perception on Brain Activations during the Aesthetic Judgment of Representational Paintings

Movement perception and its role in aesthetic experience have been often studied, within empirical aesthetics, in relation to the human body. No such specificity has been defined in neuroimaging studies with respect to contents lacking a human form. The aim of this work was to explore, through functional magnetic imaging (f MRI), how perceived movement is processed during the aesthetic judgment of paintings using two types of content: human subjects and scenes of nature. Participants, untutored in the arts, were shown the stimuli and asked to make aesthetic judgments. Additionally, they were instructed to observe the paintings and to rate their perceived movement in separate blocks. Observation highlighted spontaneous processes associated with aesthetic experience, whereas movement judgment outlined activations specifically related to movement processing. The ratings recorded during aesthetic judgment revealed that nature scenes received higher scored than human content paintings. The imaging data showed similar activation, relative to baseline, for all stimuli in the three tasks, including activation of occipito-temporal areas, posterior parietal, and premotor cortices. Contrast analyses within aesthetic judgment task showed that human content activated, relative to nature, precuneus, fusiform gyrus, and posterior temporal areas, whose activation was prominent for dynamic human paintings. In contrast, nature scenes activated, relative to human stimuli, occipital and posterior parietal cortex/precuneus, involved in visuospatial exploration and pragmatic coding of movement, as well as central insula. Static nature paintings further activated, relative to dynamic nature stimuli, central and posterior insula. Besides insular activation, which was specific for aesthetic judgment, we found a large overlap in the activation pattern characterizing each stimulus dimension (content and dynamism) across observation, aesthetic judgment, and movement judgment tasks. These findings support the idea that the aesthetic evaluation of artworks depicting both human subjects and nature scenes involves a motor component, and that the associated neural processes occur quite spontaneously in the viewer. Furthermore, considering the functional roles of posterior and central insula, we suggest that nature paintings may evoke aesthetic processes requiring an additional proprioceptive and sensori-motor component implemented by “motor accessibility” to the represented scenario, which is needed to judge the aesthetic value of the observed painting.

Connectivity of Superior Temporal Sulcus During Target Detection

Abstract. The aim of the current research was to study functional connectivity (FC) of the right superior temporal sulcus (rSTS) during visual target stimulus processing. This structure is presumed to be crucial in social cognition, but evidently participates in target detection as well. Twenty subjects participated in functional magnetic resonance examination for studying FC. We used psychophysiological interaction (PPI) analysis of data acquired during the visual oddball task. During the visual oddball task rSTS had increased connectivity bilaterally with structures involved in memory operations (mesiotemporal cortices and basal ganglia) and evaluative processing related to decision making (left anterior cingulate cortex). Moreover, we revealed decreased connectivity of rSTS with structures involved in attentional processes (right dorsolateral prefrontal cortex (DLPFC) and the posterior area with bilateral parietal cortex). Based on our results we hypothesize that in the detection of rare events, during visual information processing, rSTS is involved within neuronal networks related to attention, but also at later stages of stimuli processing.

A comparison of form processing involved in the perception of biological and nonbiological movements

Although there is evidence for specialization in the human brain for processing biological motion per se, few studies have directly examined the specialization of form processing in biological motion perception. The current study was designed to systematically compare form processing in perception of biological (human walkers) to nonbiological (rotating squares) stimuli. Dynamic form-based stimuli were constructed with conflicting form cues (position and orientation), such that the objects were perceived to be moving ambiguously in two directions at once. In Experiment 1, we used the classification image technique to examine how local form cues are integrated across space and time in a bottom-up manner. By comparing with a Bayesian observer model that embodies generic principles of form analysis (e.g., template matching) and integrates form information according to cue reliability, we found that human observers employ domain-general processes to recognize both human actions and nonbiological object movements. Experiments 2 and 3 found differential top-down effects of spatial context on perception of biological and nonbiological forms. When a background does not involve social information, observers are biased to perceive foreground object movements in the direction opposite to surrounding motion. However, when a background involves social cues, such as a crowd of similar objects, perception is biased toward the same direction as the crowd for biological walking stimuli, but not for rotating nonbiological stimuli. The model provided an accurate account of top-down modulations by adjusting the prior probabilities associated with the internal templates, demonstrating the power and flexibility of the Bayesian approach for visual form perception.

Biological motion perception links diverse facets of theory of mind during middle childhood

Two cornerstones of social development--social perception and theory of mind--undergo brain and behavioral changes during middle childhood, but the link between these developing domains is unclear. One theoretical perspective argues that these skills represent domain-specific areas of social development, whereas other perspectives suggest that both skills may reflect a more integrated social system. Given recent evidence from adults that these superficially different domains may be related, the current study examined the developmental relation between these social processes in 52 children aged 7 to 12 years. Controlling for age and IQ, social perception (perception of biological motion in noise) was significantly correlated with two measures of theory of mind: one in which children made mental state inferences based on photographs of the eye region of the face and another in which children made mental state inferences based on stories. Social perception, however, was not correlated with children's ability to make physical inferences from stories about people. Furthermore, the mental state inference tasks were not correlated with each other, suggesting a role for social perception in linking various facets of theory of mind.

Seeing biological actions in 3D: An fMRI study

Precise kinematics or body configuration cannot be recovered from visual input without disparity information. Yet, no imaging study has investigated the role of disparity on action observation. Here, we investigated the interaction between disparity and the main cues of biological motion, kinematics and configuration, in two fMRI experiments. Stimuli were presented as point‐light figures, depicting complex action sequences lasting 21 s. We hypothesized that interactions could occur at any of the three levels of the action observation network, comprising occipitotemporal, parietal and premotor cortex, with premotor cortex being the most likely location. The main effects of kinematics and configuration confirmed that the biological motion sequences activated all three levels of the action observation network, validating our approach. The interaction between configuration and disparity activated only premotor cortex, whereas interactions between kinematics and disparity occurred at all levels of the action observation network but were strongest at the premotor level. Control experiments demonstrated that these interactions could not be accounted for by low level motion in depth, task effects, spatial attention, or eye movements, including vergence. These results underscore the role of premotor cortex in action observation, and in imitating others or responding to their actions. Hum Brain Mapp 37:203–219, 2016. © 2015 The Authors Human Brain Mapping Published by Wiley Periodicals, Inc.

Exploring Biological Motion Processing in Parkinson's Disease Using Temporal Dilation

Biological motion (BM) perception is the compelling ability of the visual system to perceive complex animated movements effortlessly and promptly. A recent study has shown that BM can automatically lengthen perceived temporal duration independent of global configuration. The present study aimed mainly to investigate this temporal dilation effect of BM signals in Parkinson’s disease (PD) patients. We used the temporal dilation effect as an implicit measure of visual processing of BM. In all, 32 PD patients (under off-therapy conditions) and 32 healthy controls (HCs) participated in our study. In each trial, an upright BM sequence and an inverted BM sequence were presented within an interval in the center of the screen. We tested both canonical and scrambled BM sequences; the scrambled ones were generated by disturbing the global configuration of the canonical ones but preserving exactly the same local motion components. Observers were required to make a verbal two-alternative forced choice response to indicate which interval (the first or the second) appeared longer. Statistical analyses were conducted on the points of subjective equality (PSEs). We found that the temporal dilation effect was significantly reduced for PD patients compared with HCs in both canonical and scrambled BM conditions. Moreover, no temporal dilation effects of scrambled BM were shown in both early- and late-stage PD patients, while the temporal dilation effect of canonical BM was relatively preserved in the early stages.

Neuronal bases of structural coherence in contemporary dance observation

The neuronal processes underlying dance observation have been the focus of an increasing number of brain imaging studies over the past decade. However, the existing literature mainly dealt with effects of motor and visual expertise, whereas the neural and cognitive mechanisms that underlie the interpretation of dance choreographies remained unexplored. Hence, much attention has been given to the action observation network (AON) whereas the role of other potentially relevant neuro-cognitive mechanisms such as mentalizing (theory of mind) or language (narrative comprehension) in dance understanding is yet to be elucidated. We report the results of an fMRI study where the structural coherence of short contemporary dance choreographies was manipulated parametrically using the same taped movement material. Our participants were all trained dancers. The whole-brain analysis argues that the interpretation of structurally coherent dance phrases involves a subpart (superior parietal) of the AON as well as mentalizing regions in the dorsomedial prefrontal cortex. An ROI analysis based on a similar study using linguistic materials (Pallier et al., 2011) suggests that structural processing in language and dance might share certain neural mechanisms.

The contribution of dynamic visual cues to audiovisual speech perception

Seeing a speaker's facial gestures can significantly improve speech comprehension, especially in noisy environments. However, the nature of the visual information from the speaker's facial movements that is relevant for this enhancement is still unclear. Like auditory speech signals, visual speech signals unfold over time and contain both dynamic configural information and luminance-defined local motion cues; two information sources that are thought to engage anatomically and functionally separate visual systems. Whereas, some past studies have highlighted the importance of local, luminance-defined motion cues in audiovisual speech perception, the contribution of dynamic configural information signalling changes in form over time has not yet been assessed. We therefore attempted to single out the contribution of dynamic configural information to audiovisual speech processing. To this aim, we measured word identification performance in noise using unimodal auditory stimuli, and with audiovisual stimuli. In the audiovisual condition, speaking faces were presented as point light displays achieved via motion capture of the original talker. Point light displays could be isoluminant, to minimise the contribution of effective luminance-defined local motion information, or with added luminance contrast, allowing the combined effect of dynamic configural cues and local motion cues. Audiovisual enhancement was found in both the isoluminant and contrast-based luminance conditions compared to an auditory-only condition, demonstrating, for the first time the specific contribution of dynamic configural cues to audiovisual speech improvement. These findings imply that globally processed changes in a speaker's facial shape contribute significantly towards the perception of articulatory gestures and the analysis of audiovisual speech.

Social visual engagement in infants and toddlers with autism: early developmental transitions and a model of pathogenesis

Efforts to determine and understand the causes of autism are currently hampered by a large disconnect between recent molecular genetics findings that are associated with the condition and the core behavioral symptoms that define the condition. In this perspective piece, we propose a systems biology framework to bridge that gap between genes and symptoms. The framework focuses on basic mechanisms of socialization that are highly-conserved in evolution and are early-emerging in development. By conceiving of these basic mechanisms of socialization as quantitative endophenotypes, we hope to connect genes and behavior in autism through integrative studies of neurodevelopmental, behavioral, and epigenetic changes. These changes both lead to and are led by the accomplishment of specific social adaptive tasks in a typical infant's life. However, based on recent research that indicates that infants later diagnosed with autism fail to accomplish at least some of these tasks, we suggest that a narrow developmental period, spanning critical transitions from reflexive, subcortically-controlled visual behavior to interactional, cortically-controlled and social visual behavior be prioritized for future study. Mapping epigenetic, neural, and behavioral changes that both drive and are driven by these early transitions may shed a bright light on the pathogenesis of autism.

Ventral aspect of the visual form pathway is not critical for the perception of biological motion

Identifying the movements of those around us is fundamental for many daily activities, such as recognizing actions, detecting predators, and interacting with others socially. A key question concerns the neurobiological substrates underlying biological motion perception. Although the ventral "form" visual cortex is standardly activated by biologically moving stimuli, whether these activations are functionally critical for biological motion perception or are epiphenomenal remains unknown. To address this question, we examined whether focal damage to regions of the ventral visual cortex, resulting in significant deficits in form perception, adversely affects biological motion perception. Six patients with damage to the ventral cortex were tested with sensitive point-light display paradigms. All patients were able to recognize unmasked point-light displays and their perceptual thresholds were not significantly different from those of three different control groups, one of which comprised brain-damaged patients with spared ventral cortex (n > 50). Importantly, these six patients performed significantly better than patients with damage to regions critical for biological motion perception. To assess the necessary contribution of different regions in the ventral pathway to biological motion perception, we complement the behavioral findings with a fine-grained comparison between the lesion location and extent, and the cortical regions standardly implicated in biological motion processing. This analysis revealed that the ventral aspects of the form pathway (e.g., fusiform regions, ventral extrastriate body area) are not critical for biological motion perception. We hypothesize that the role of these ventral regions is to provide enhanced multiview/posture representations of the moving person rather than to represent biological motion perception per se.

Absent activation in medial prefrontal cortex and temporoparietal junction but not superior temporal sulcus during the perception of biological motion in schizophrenia: a functional MRI study

BACKGROUND

Patients with schizophrenia show disturbances in both visual perception and social cognition. Perception of biological motion (BM) is a higher-level visual process, and is known to be associated with social cognition. BM induces activation in the "social brain network", including the superior temporal sulcus (STS). Although deficits in the detection of BM and atypical activation in the STS have been reported in patients with schizophrenia, it remains unclear whether other nodes of the "social brain network" are also atypical in patients with schizophrenia.

PURPOSE

We aimed to explore whether brain regions other than STS were involved during BM perception in patients with schizophrenia, using functional magnetic resonance imaging (fMRI).

METHODS AND PATIENTS

Seventeen patients with schizophrenia, and 17 age- and sex- matched healthy controls, underwent fMRI scanning during a one-back visual task, containing three experimental conditions: (1) BM, (2) scrambled motion (SM), and (3) static condition. We used one-sample t-tests to examine neural responses selective to BM versus SM within each group, and two-sample t-tests to directly compare neural patterns to BM versus SM in schizophrenics versus controls.

RESULTS

We found significant activation in the STS region when BM was contrasted with SM in both groups, with no significant difference between groups. On the contrary, significant activation in the medial prefrontal cortex (MPFC) and bilateral temporoparietal junction (TPJ) was found only in the control group. When we directly compared the two groups, the healthy controls showed significant greater activation in left MPFC and TPJ to BM versus SM than patients with schizophrenia.

CONCLUSION

Our findings suggest that patients with schizophrenia show normal activation to biologically and socially relevant motion stimuli in the STS, but atypical activation in other regions of the social brain network, specifically MPFC and TPJ. Moreover, these results were not due to atypical processing of motion, suggesting that patients with schizophrenia lack in the recruitment of neural circuits needed for the visual perception of social cognition.

Sex differences in the neural basis of false-belief and pragmatic language comprehension

Increasing research evidence suggests that women are more advanced than men in pragmatic language comprehension and Theory of Mind (ToM), which is a cognitive component of empathy. We measured the hemodynamic responses of men and women while they performed a second-order false-belief (FB) task and a coherent story (CS) task. During the FB condition relative to the baseline (unlinked sentences [US]), we found convergent activity in ToM network regions, such as the temporoparietal junction (TPJ) bilaterally and precuneus, in both sexes. We also found a greater activity in the left medial prefrontal cortex (mPFC) and a greater deactivation in the ventromedial prefrontal cortex (vmPFC)/orbitofrontal cortex (OFC) bilaterally in women compared to men. However, we did not find difference in the brain activity between the sexes during the FB condition relative to the CS condition. The results suggest a significant overlap between neural bases of pragmatic language comprehension and ToM in both men and women. Taken together, these results are in line with the extreme male brain (EMB) hypothesis by demonstrating sex difference in the neural basis of ToM and pragmatic language, both of which are found to be impaired in individuals with Autism Spectrum Conditions (ASC). In addition, the results also suggest that on average women use both cognitive empathy (dorsal mPFC) and affective empathy (vmPFC) networks more than men for false-belief reasoning.

Spontaneous mentalizing captures variability in the cortical thickness of social brain regions

Theory of mind (ToM)--or thinking about the mental states of others--is a cornerstone of successful everyday social interaction. However, the brain bases of ToM are most frequently measured via explicit laboratory tasks that pose direct questions about mental states (e.g. "In this story, what does Steve think Julia believes?"). Neuroanatomical measures may provide a way to explore the brain bases of individual differences in more naturalistic everyday mentalizing. In the current study, we examined the relation between cortical thickness and spontaneous ToM using the novel Spontaneous Theory of Mind Protocol (STOMP), which measures participants' spontaneous descriptions of the beliefs, emotions and goals of characters in naturalistic videos. We administered standard ToM tasks and the STOMP to young adults (aged 18-26 years) and collected structural magnetic resonance imaging data from a subset of these participants. The STOMP produced robust individual variability and was correlated with performance on traditional ToM tasks. Further, unlike the traditional ToM tasks, STOMP performance was related to cortical thickness for a set of brain regions that have been functionally linked to ToM processing. These findings offer novel insight into the brain bases of variability in naturalistic mentalizing performance, with implications for both typical and atypical populations.